ClosedChannelException

来源:互联网 发布:java public class读法 编辑:程序博客网 时间:2024/06/13 04:25

遗留问题:在secureCRT上跑spark代码时偶尔会报错:ClosedChannelException
错误信息:

17/05/26 17:34:09 INFO YarnClientSchedulerBackend: Stopped17/05/26 17:34:09 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!17/05/26 17:34:09 INFO BlockManagerInfo: Removed broadcast_10_piece0 on 192.168.16.100:36241 in memory (size: 598.0 KB, free: 12.6 GB)17/05/26 17:34:09 ERROR LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerBlockUpdated(BlockUpdatedInfo(BlockManagerId(driver, 192.168.16.100, 36241),broadcast_10_piece0,StorageLevel(1 replicas),0,0))17/05/26 17:34:09 ERROR TransportClient: Failed to send RPC 7502960340689747796 to /192.168.16.108:35336: java.nio.channels.ClosedChannelExceptionjava.nio.channels.ClosedChannelException17/05/26 17:34:09 ERROR TransportClient: Failed to send RPC 6241621360882774485 to /192.168.16.107:52929: java.nio.channels.ClosedChannelExceptionjava.nio.channels.ClosedChannelException17/05/26 17:34:09 WARN BlockManagerMaster: Failed to remove broadcast 10 with removeFromMaster = true - Failed to send RPC 7502960340689747796 to /192.168.16.108:35336: java.nio.channels.ClosedChannelExceptionjava.io.IOException: Failed to send RPC 7502960340689747796 to /192.168.16.108:35336: java.nio.channels.ClosedChannelException        at org.apache.spark.network.client.TransportClient$3.operationComplete(TransportClient.java:249)        at org.apache.spark.network.client.TransportClient$3.operationComplete(TransportClient.java:233)        at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)        at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:567)        at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:424)        at io.netty.channel.AbstractChannel$AbstractUnsafe.safeSetFailure(AbstractChannel.java:801)        at io.netty.channel.AbstractChannel$AbstractUnsafe.write(AbstractChannel.java:699)        at io.netty.channel.DefaultChannelPipeline$HeadContext.write(DefaultChannelPipeline.java:1122)        at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:633)        at io.netty.channel.AbstractChannelHandlerContext.access$1900(AbstractChannelHandlerContext.java:32)        at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.write(AbstractChannelHandlerContext.java:908)        at io.netty.channel.AbstractChannelHandlerContext$WriteAndFlushTask.write(AbstractChannelHandlerContext.java:960)        at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:893)        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)        at java.lang.Thread.run(Thread.java:745)Caused by: java.nio.channels.ClosedChannelException17/05/26 17:34:09 INFO MemoryStore: MemoryStore cleared17/05/26 17:34:09 INFO BlockManager: BlockManager stopped17/05/26 17:34:09 INFO BlockManagerMaster: BlockManagerMaster stopped17/05/26 17:34:09 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!17/05/26 17:34:09 INFO SparkContext: Successfully stopped SparkContext17/05/26 17:34:09 INFO ShutdownHookManager: Shutdown hook called17/05/26 17:34:09 INFO ShutdownHookManager: Deleting directory /tmp/spark-26f7a564-81b5-4df0-8228-f4cc7f4a96bf