HIVE使用CTAS出现void类型字段报错
来源:互联网 发布:股票数据统计分析软件 编辑:程序博客网 时间:2024/06/03 14:08
最近使用hive CTAS语法时候遇到一个语义级的异常,
将问题的RC: java.lang.RuntimeException: Internal error: no LazyObject for VOID 在google上搜索可以看到
https://issues.apache.org/jira/browse/HIVE-2615,这个上已经有patch在fix了
原因就是使用CTAS的时候,创建出来的表字段类型有为VOID的(对应值为null),
desc了下table:
create table temp.ldc_aaa_
as
select cast(null as string) from default.dual;
Hive版本0.13
操作如下:
--结果查询的时候出错
--后台log4j信息:
2014-04-28 16:36:41,212 ERROR CliDriver (SessionState.java:printError(386)) - Failed with exception java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating module_namejava.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating module_name at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:150) at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1413) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:271) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:208)Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating module_name at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:80) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832) at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:90) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502) at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:490) at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136) ... 11 moreCaused by: java.lang.RuntimeException: Internal error: no LazyObject for VOID at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyPrimitiveClass(LazyFactory.java:119) at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObject(LazyFactory.java:155) at org.apache.hadoop.hive.serde2.lazy.LazyStruct.parse(LazyStruct.java:108) at org.apache.hadoop.hive.serde2.lazy.LazyStruct.getField(LazyStruct.java:190) at org.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector.getStructFieldData(LazySimpleStructObjectInspector.java:188) at org.apache.hadoop.hive.serde2.objectinspector.DelegatedStructObjectInspector.getStructFieldData(DelegatedStructObjectInspector.java:79) at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.evaluate(ExprNodeColumnEvaluator.java:98) at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:76) ... 17 more
create table temp.ldc_test_basselectcase when length(regexp_extract(event_obj_name,'^([A-Za-z_]+)([0-9]+)$',1))=0 then event_obj_name else regexp_extract(event_obj_name,'^([A-Za-z_]+)([0-9]+)$',1) end as module_name,get_urlgroup(url) as page_id,case when length(regexp_extract(event_obj_name,'^([A-Za-z_]+)([0-9]+)$',1))=0 then '-99' else regexp_extract(event_obj_name,'^([A-Za-z_]+)([0-9]+)$',2) end as position,null as page_on_time,null as is_bounce,null as is_enterfrom vipdw.dw_log_corritor_ds where dt='20140427' and event_obj_name not like '%|%|%|%'limit 10;
--结果查询的时候出错
hive (vipdw)> select * from temp.ldc_test_b;OKFailed with exception java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating module_name
--后台log4j信息:
2014-04-28 16:36:41,212 ERROR CliDriver (SessionState.java:printError(386)) - Failed with exception java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating module_namejava.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating module_name at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:150) at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1413) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:271) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:208)Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating module_name at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:80) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832) at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:90) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502) at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:490) at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136) ... 11 moreCaused by: java.lang.RuntimeException: Internal error: no LazyObject for VOID at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyPrimitiveClass(LazyFactory.java:119) at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObject(LazyFactory.java:155) at org.apache.hadoop.hive.serde2.lazy.LazyStruct.parse(LazyStruct.java:108) at org.apache.hadoop.hive.serde2.lazy.LazyStruct.getField(LazyStruct.java:190) at org.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector.getStructFieldData(LazySimpleStructObjectInspector.java:188) at org.apache.hadoop.hive.serde2.objectinspector.DelegatedStructObjectInspector.getStructFieldData(DelegatedStructObjectInspector.java:79) at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.evaluate(ExprNodeColumnEvaluator.java:98) at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:76) ... 17 more
https://issues.apache.org/jira/browse/HIVE-2615,这个上已经有patch在fix了
原因就是使用CTAS的时候,创建出来的表字段类型有为VOID的(对应值为null),
desc了下table:
hive (default)> desc temp.ldc_test_b;OKmodule_name string None page_id int None position string None sub_url string None refer_page_id int None type string None page_param string None cookie_id string None page_on_time void None is_bounce void None is_enter void None解决的办法也很简单只需要手动cast就可以,eg:
create table temp.ldc_aaa_
as
select cast(null as string) from default.dual;
0 0
- HIVE使用CTAS出现void类型字段报错
- hive void类型
- hive 字段类型
- HIVE的CTAS用法探究
- CTAS 使用一
- hive初次使用报错
- 使用hive查询hdfs数据时报字段类型异常
- void类型指针使用
- hive 表修改字段类型
- CTAS
- CDH中Hive使用函数报错
- Hive报错 类型WritableStringObjectInspector无法转换BooleanObjectInspector
- oracle,db2,teradata,hive字段类型汇总
- String类型字段,hibernate报错,超出长度
- 解决从mysql端使用sqoop导入hive端字段类型不匹配问题
- 使用 BLOB 类型字段
- hive报错
- Hive UDF 报错
- 反汇编(Disassembler) iPhone 好东西
- the connection to adb is down ,and a severe error has occured
- POJ 1176 Party Lamps (DFS)
- 第十二周项目三(3)最大公约数
- 《android编程权威指南》学习笔记之第四章 Activity应用的调试
- HIVE使用CTAS出现void类型字段报错
- stretchableImageWithLeftCapWidth:topCapHeight:函数用法
- 后缀表达式
- IOS的图标设置以及遇到的一些问题
- Error: Cannot retrieve metalink for repository: epel. Please verify its path and try again
- KVO--NSArray
- cocos2dx TiledMap图块之间有缝隙
- 【web++_第二阶段】关于mysql与JDBC的那点经验
- poj 3261 Milk Patterns(可重叠的k 次最长重复子串)