java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read
来源:互联网 发布:mac截图 编辑:程序博客网 时间:2024/06/04 17:55
环境:
- Spark 2.11-2.0.2
- Hadoop 2.6.5
使用Spark连接HDFS,在高并发的时候,偶尔会遇到以下异常:
2017-01-26 10:30:29,079 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: xxx:50010:DataXceiver error processing WRITE_BLOCK operation src: /xxx:52827 dst: /xxx:50010java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/xxx:50010 remote=/xxx:52827] at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:164) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.read1(BufferedInputStream.java:256) at java.io.BufferedInputStream.read(BufferedInputStream.java:317) at java.io.DataInputStream.read(DataInputStream.java:132) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:192) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:213) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:171) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:496) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:891) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:758) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:137) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:74) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:239) at java.lang.Thread.run(Thread.java:662)
出现以上错误是因为datanode的服务线程连接数都被占用,导致Spark等待超时。可以通过以下方法解决:
- 根据datanode的资源情况适当增加的服务线程数:在hdfs-default.xml配置文件里面新增或修改dfs.datanode.handler.count,默认是10
- 增加客户端的超时时间dfs.client.socket-timeout,默认是60000ms,注意,旧版本的Hadoop用的是dfs.socket.timeout,可参考http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/DeprecatedProperties.html
try (JavaSparkContext context = new JavaSparkContext(new SparkConf().setMaster("local[*]").setAppName("test"))) { context.sc().hadoopConfiguration().set("dfs.client.socket-timeout", "300000"); ......
0 0
- java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read
- 480000 millis timeout while waiting for channel to be ready for write异常处理
- How to solve java.net.SocketTimeoutException:60000millis problem in HDFS
- net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting head
- IIC 读写时候提示 timeout waiting for bus ready
- Timeout waiting for Tomcat
- Zookeeper:Timeout while waiting for epoch from quorum解决方案
- Waiting for core to be armed!
- Timeout waiting to lock cp_proj class cache for build file
- ds_ipcgetnext() - timeout waiting for mutex
- AxisFault: Timeout waiting for connection
- Timeout waiting for idle object
- AxisFault: Timeout waiting for connection
- AxisFault: Timeout waiting for connection
- ConnectionPoolTimeoutException: Timeout waiting for connection
- java.util.NoSuchElementException: Timeout waiting for idle object
- java.util.NoSuchElementException: Timeout waiting for idle object
- org.eclipse.jdi.TimeoutException:Timeout occurred while waiting for packet 1983.occurred
- QBXT DAY1
- 1038. Recover the Smallest Number (30)
- 心得 - 面向对象和面向过程的区别
- picasso使用
- SQL Server数据类型转换方法
- java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read
- webservice--SOAP
- PAT BASIC LEVEL 1036. 跟奥巴马一起编程(15)
- zndle的测试博客
- --wps知识点--不断总结
- 关于Android的.so文件你所需要知道的 - 简书
- PAT BASIC LEVEL 1037. 在霍格沃茨找零钱(20)
- MySQL 使用数据库(三)
- MVC 绑定数据