hadoop报错: java.lang.ArrayIndexOutOfBoundsException
来源:互联网 发布:hadoop python书籍 编辑:程序博客网 时间:2024/06/13 22:43
每次都是跑到map 99% reduce 33%的时候挂掉
2013-12-04 10:40:45,128 INFO org.apache.hadoop.mapred.Merger: Merging 16 sorted segments2013-12-04 10:40:45,172 INFO org.apache.hadoop.mapred.Merger: Merging 2 intermediate segments out of a total of 142013-12-04 10:40:45,314 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 8 segments left of total size: 255981 bytes2013-12-04 10:40:45,324 INFO org.apache.hadoop.mapred.Merger: Merging 16 sorted segments2013-12-04 10:40:45,385 INFO org.apache.hadoop.mapred.Merger: Merging 2 intermediate segments out of a total of 142013-12-04 10:40:45,527 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 8 segments left of total size: 163242 bytes2013-12-04 10:40:45,532 INFO org.apache.hadoop.mapred.Merger: Merging 16 sorted segments2013-12-04 10:40:45,577 INFO org.apache.hadoop.mapred.Merger: Merging 2 intermediate segments out of a total of 162013-12-04 10:40:46,984 INFO org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-12013-12-04 10:40:46,986 WARN org.apache.hadoop.mapred.Child: Error running childjava.lang.ArrayIndexOutOfBoundsExceptionat java.lang.System.arraycopy(Native Method)at org.apache.hadoop.mapred.IFile$Reader.rejigDataOffset(IFile.java:460)at org.apache.hadoop.mapred.IFile$Reader.readNextBlockAndReserveKey(IFile.java:424)at org.apache.hadoop.mapred.IFile$Reader.nextValue(IFile.java:599)at org.apache.hadoop.mapred.Merger$Segment.nextValue(Merger.java:243)at org.apache.hadoop.mapred.Merger$MergeQueue.next(Merger.java:388)at org.apache.hadoop.mapred.Merger.writeFile(Merger.java:157)at org.apache.hadoop.mapred.Merger$MergeQueue.merge(Merger.java:534)at org.apache.hadoop.mapred.Merger$MergeQueue.merge(Merger.java:414)at org.apache.hadoop.mapred.Merger.merge(Merger.java:78)at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:1563)at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1180)at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:392)at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)at org.apache.hadoop.mapred.Child$4.run(Child.java:266)at java.security.AccessController.doPrivileged(Native Method)at javax.security.auth.Subject.doAs(Subject.java:396)at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278)at org.apache.hadoop.mapred.Child.main(Child.java:260)2013-12-04 10:40:46,989 INFO org.apache.hadoop.mapred.Task: Runnning cleanup for the task
解决方法:
custom param中加入 mapred.ifile.buffer.reset.size.mb:256
- hadoop报错: java.lang.ArrayIndexOutOfBoundsException
- java split 报错java.lang.ArrayIndexOutOfBoundsException
- Webshpere启动报 java.lang.ArrayIndexOutOfBoundsException 错
- kylin报错 java.lang.ArrayIndexOutOfBoundsException: -1
- (数组角标异常)报错java.lang.ArrayIndexOutOfBoundsException..
- Spark On CDH5.6.0 报错java.lang.ArrayIndexOutOfBoundsException: 1
- jetty 启动项目报错:java.lang.ArrayIndexOutOfBoundsException: 52264
- MemCachedClient调用,报java.lang.ArrayIndexOutOfBoundsException错误
- fastjson报java.lang.ArrayIndexOutOfBoundsException: 160
- 报错:07-28 11:02:19.121: E/AndroidRuntime(5867): Caused by: java.lang.ArrayIndexOutOfBoundsException:
- Android-71-解决报错: java.lang.ArrayIndexOutOfBoundsException: length=2; index=2
- JBOSS报错:com.google.common.collect.ComputationException: java.lang.ArrayIndexOutOfBoundsException: 3
- eclipse用maven运行项目报错java.lang.ArrayIndexOutOfBoundsException: 28054
- jxl在linux环境下报java.lang.ArrayIndexOutOfBoundsException
- jsp页面引入jstl标签后台报java.lang.ArrayIndexOutOfBoundsException
- jsp页面引入jstl标签后台报java.lang.ArrayIndexOutOfBoundsException
- java.lang.ArrayIndexOutOfBoundsException 翻译
- java.lang.ArrayIndexOutOfBoundsException
- 云三兄弟的9k offer面试经验贴,给我一个不顶的理由!
- No rule to make target `include/config/auto.conf'
- Commons CLI
- C# 设计模式之原型模式(Prototype)
- CString char* Time转换 char* 连接操作
- hadoop报错: java.lang.ArrayIndexOutOfBoundsException
- Java中的一些关键字
- 静态代理
- shell脚本问题&技巧
- Android 4.2版本 华为EmotionUI1.6 开发者选项 开启
- 光明顶短信支付:BASE64编码
- Web 服务器初级介绍及应用
- HDU 3530 Subsequence(STL)
- apk 调用 第三方包