HIVE使用CTAS出现void类型字段报错

来源:互联网 发布:股票数据统计分析软件 编辑:程序博客网 时间:2024/06/03 14:08
最近使用hive CTAS语法时候遇到一个语义级的异常,
Hive版本0.13
操作如下:
create table temp.ldc_test_basselectcase when length(regexp_extract(event_obj_name,'^([A-Za-z_]+)([0-9]+)$',1))=0 then event_obj_name else regexp_extract(event_obj_name,'^([A-Za-z_]+)([0-9]+)$',1) end as module_name,get_urlgroup(url)  as page_id,case  when length(regexp_extract(event_obj_name,'^([A-Za-z_]+)([0-9]+)$',1))=0 then '-99' else regexp_extract(event_obj_name,'^([A-Za-z_]+)([0-9]+)$',2) end as position,null as page_on_time,null as is_bounce,null as is_enterfrom  vipdw.dw_log_corritor_ds where dt='20140427' and event_obj_name not like '%|%|%|%'limit 10; 

 
--结果查询的时候出错
hive (vipdw)> select * from temp.ldc_test_b;OKFailed with exception java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating module_name


--后台log4j信息:
2014-04-28 16:36:41,212 ERROR CliDriver (SessionState.java:printError(386)) - Failed with exception java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating module_namejava.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating module_name at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:150) at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1413) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:271) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:208)Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating module_name at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:80) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832) at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:90) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502) at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:490) at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136) ... 11 moreCaused by: java.lang.RuntimeException: Internal error: no LazyObject for VOID at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyPrimitiveClass(LazyFactory.java:119) at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObject(LazyFactory.java:155) at org.apache.hadoop.hive.serde2.lazy.LazyStruct.parse(LazyStruct.java:108) at org.apache.hadoop.hive.serde2.lazy.LazyStruct.getField(LazyStruct.java:190) at org.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector.getStructFieldData(LazySimpleStructObjectInspector.java:188) at org.apache.hadoop.hive.serde2.objectinspector.DelegatedStructObjectInspector.getStructFieldData(DelegatedStructObjectInspector.java:79) at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.evaluate(ExprNodeColumnEvaluator.java:98) at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:76) ... 17 more  

将问题的RC: java.lang.RuntimeException: Internal error: no LazyObject for VOID 在google上搜索可以看到
https://issues.apache.org/jira/browse/HIVE-2615,这个上已经有patch在fix了
原因就是使用CTAS的时候,创建出来的表字段类型有为VOID的(对应值为null),
desc了下table:
hive (default)> desc temp.ldc_test_b;OKmodule_name             string                  None               page_id                 int                     None               position                string                  None               sub_url                 string                  None               refer_page_id           int                     None               type                    string                  None               page_param              string                  None               cookie_id               string                  None               page_on_time            void                    None               is_bounce               void                    None               is_enter                void                    None        
解决的办法也很简单只需要手动cast就可以,eg:
create table temp.ldc_aaa_
as
select cast(null as string) from default.dual;

0 0
原创粉丝点击