hive和hbase集成异常
来源:互联网 发布:java 实参 编辑:程序博客网 时间:2024/05/30 21:58
一、偶尔出现两个异常
Error: java.lang.IllegalArgumentException: Illegal character code:-1, <> at 0. User-space table qualifiers can only contain 'alphanumeric characters': i.e. [a-zA-Z_0-9-.]: � at org.apache.hadoop.hbase.TableName.isLegalTableQualifierName(TableName.java:195) at org.apache.hadoop.hbase.TableName.isLegalTableQualifierName(TableName.java:149) at org.apache.hadoop.hbase.TableName.<init>(TableName.java:303) at org.apache.hadoop.hbase.TableName.createTableNameIfNecessary(TableName.java:339) at org.apache.hadoop.hbase.TableName.valueOf(TableName.java:399) at org.apache.hadoop.hbase.mapreduce.TableSplit.readFields(TableSplit.java:282) at org.apache.hadoop.hive.hbase.HBaseSplit.readFields(HBaseSplit.java:53) at org.apache.hadoop.hive.ql.io.HiveInputFormat$HiveInputSplit.readFields(HiveInputFormat.java:156) at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSeri
Error: java.io.IOException: java.io.IOException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:246)
at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:574)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.io.IOException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:457)
at org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:434)
at org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:315)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:198)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:175)
at org.apache.hadoop.hive.hbase.HiveHBaseInputFormatUtil.getTable(HiveHBaseInputFormatUtil.java:44)
at org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat.getRecordReader(HiveHBaseTableInputFormat.java:93)
at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:244)
... 9 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:455)
... 16 more
Caused by: java.lang.NoSuchFieldError: HBASE_CLIENT_PREFETCH
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:763)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:689)
... 21 more
上面两个异常,是由于hive和hbase的lib中都放入了hive-hbase-handler.jar,而且两个版本一样
二、第二个现象是从hbase端写数据,在hive中查询异常,如下:
2016-07-19 14:26:09,605 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1468754987139_0086_m_000000_3: Error: java.io.EOFException at java.io.DataInputStream.readFully(DataInputStream.java:197) at java.io.DataInputStream.readUTF(DataInputStream.java:609) at java.io.DataInputStream.readUTF(DataInputStream.java:564) at org.apache.hadoop.hive.ql.io.HiveInputFormat$HiveInputSplit.readFields(HiveInputFormat.java:157) at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:71) at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42) at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:423) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
原来hive引入了hbase的所有jar,包括 phoenix,删除所有相关jar,只留了:
hbase-server-0.98.1-cdh5.1.0.jar
hbase-common-0.98.1-cdh5.1.0.jar
hbase-protocol-0.98.1-cdh5.1.0.jar
htrace-core-2.04.jar
最后程序正常
0 0
- hive和hbase集成异常
- hive和hbase集成
- hive和hbase集成应用
- Hive Hbase Integration(hive和habse的集成)
- Hive学习总结之五:HBase和Hive的集成
- Hive与HBase集成
- hive 集成hbase 笔记
- hive集成hbase笔记
- Hive与HBase集成
- hive与hbase集成
- hive集成hbase
- hive与hbase集成
- hive与hbase集成
- hive与hbase集成
- Hive HBase集成
- Hive集成HBase详解
- Hive集成HBase详解
- HBase与hive集成
- BSON与JSON的区别
- 使用PickView实现日期选择器
- sort冒泡
- DenseCap解读
- Git 学习笔记二
- hive和hbase集成异常
- 字符串反转
- Leveldb使用手册
- vim使用技巧篇
- CSS3嵌入字体@font-face
- SolrCloud之Sharding路由介绍
- PREROUTING 和 POSTROUTING, SNAT 和 DNAT图文解析(非常清淅)
- XSS跨站脚本攻击与CSRF跨站请求伪造攻击的学习总结。
- iOS-AFNetworking 2.0 跟踪文件上传进度