Reduce运行到99.99%到100%，最后出现了错误，导致任务失败

来源：互联网发布：如何做一名程序员编辑：程序博客网时间：2024/06/05 00:11

Error: java.io.IOException: Failing write. Tried pipeline recovery 5 times without success.
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.processDatanodeError(DFSOutputStream.java:1113)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:622)
attempt_1501681898109_0016_r_000006_0 100.00FAILEDreduce > reducehadoopserver04:8042logs
Thu Aug 3 12:44:22 +0800 2017 Thu Aug 3 18:31:01 +0800 20175hrs, 46mins, 39sec

Error: java.io.IOException: Failing write. Tried pipeline recovery 5 times without success.
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.processDatanodeError(DFSOutputStream.java:1113)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:622)

attempt_1501681898109_0016_r_000015_0 100.00FAILEDreduce > reducehadoopserver13:8042logs
Thu Aug 3 12:44:29 +0800 2017 Thu Aug 3 18:29:31 +0800 20175hrs, 45mins, 1sec
Error: java.io.IOException: Failing write. Tried pipeline recovery 5 times without success.
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.processDatanodeError(DFSOutputStream.java:1113)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:622)
attempt_1501681898109_0016_r_000019_0 100.00FAILEDreduce > reducehadoopserver13:8042logs
Thu Aug 3 12:44:31 +0800 2017 Thu Aug 3 18:31:20 +0800 20175hrs, 46mins, 49sec
Error: java.io.IOException: Failing write. Tried pipeline recovery 5 times without success.
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.processDatanodeError(DFSOutputStream.java:1113)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:622)

attempt_1501681898109_0016_r_000032_0 100.00FAILEDreduce > reducehadoopserver07:8042logs
Thu Aug 3 12:45:18 +0800 2017 Thu Aug 3 18:15:00 +0800 20175hrs, 29mins, 41sec

Error: java.io.IOException: Failing write. Tried pipeline recovery 5 times without success. a

Mmeory CPU--IO

why？？？

阅读全文

0 0