hive报错整理之Malformed ORC file 、Invalid postscript.
来源:互联网 发布:windows 10 恢复桌面 编辑:程序博客网 时间:2024/04/28 22:10
Caused by: java.io.IOException: Malformed ORC file
将本地文件的数据加载到hive的ORC格式表时,出现以下报错:
Diagnostic Messages for this Task:Error: java.io.IOException: java.lang.reflect.InvocationTargetException at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97) at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57) at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:295) at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.<init>(HadoopShimsSecure.java:242) at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getRecordReader(HadoopShimsSecure.java:356) at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:591) at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:168) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:409) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:281) ... 11 moreCaused by: java.io.IOException: Malformed ORC file hdfs://bigdata01.ibeifeng.com:8020/user/hive/warehouse/track_logs.db/track_log/date=20160323/20160323. Invalid postscript. at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.ensureOrcFooter(ReaderImpl.java:226) at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.extractMetaInfoFromFooter(ReaderImpl.java:336) at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.<init>(ReaderImpl.java:292) at org.apache.hadoop.hive.ql.io.orc.OrcFile.createReader(OrcFile.java:197) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:997) at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.<init>(CombineHiveRecordReader.java:65) ... 16 moreFAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTaskMapReduce Jobs Launched: Job 0: Map: 1 Reduce: 1 HDFS Read: 0 HDFS Write: 0 FAILTotal MapReduce CPU Time Spent: 0 msec
原因:
ORC格式是列式存储的表,不能直接从本地文件导入数据,只有当数据源表也是ORC格式存储时,才可以直接加载,否则会出现上述报错。
解决办法:
要么将数据源表改为以ORC格式存储的表,要么新建一个以textfile格式的临时表先将源文件数据加载到该表,然后在从textfile表中insert数据到ORC目标表中。
0 0
- hive报错整理之Malformed ORC file 、Invalid postscript.
- Hive ORC文件格式存储与测试(Malformed ORC file解决)
- hive:orc file
- Hive 之 ORC
- Hive:ORC File Format存储格式详解
- Hive:ORC File Format存储格式详解
- Hive ORC file实践与优势
- Hive语言手册之五:ORC文件格式
- Hive报错整理02
- IDL save postscript file
- Hive运行报错整理01
- Hive ORC和Parquet
- MapReduce基础开发之十读写ORC File
- 启动HIVE 服务报错 HWI WAR file not found
- hive ORC 文件存储格式
- Hive-ORC文件存储格式
- Hive Streaming 追加 ORC 文件
- WebLogic启动Node Manager报错:Invalid state file format. State file contents
- 流量单位的计算
- javarebel配置
- JSPatch-iOS热补丁方式快速修复bug
- CircleProgress
- 诡异的楼梯
- hive报错整理之Malformed ORC file 、Invalid postscript.
- Volley+OkHttp的整合
- ios本地通知(二)
- IOS中UITableViewCell使用详解
- error opening trace file: No such file or directory (2)
- 自定义DialogFrament弹框方式(底部,居中等)
- Windows 7如何建立一个FTP的快捷方式
- 为什么Android的点9图四周的颜色只能是白、黑或透明
- 粒子群算法Particle Swarm Optimization(PSO)