Hive ClassNotFoundException: org.apache.hadoop.hive.contrib.serde2.RegexSerDe
来源:互联网 发布:淘宝服务市场 第三方 编辑:程序博客网 时间:2024/04/30 09:22
create table mylog2(time string,clause string) row format serde 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe' with serdeproperties ("input.regex" = "(\\d{4}-\\d{2}-\\d{2} \\d{2}:\\d{2}:\\d{2},\\d{3}) - (.*) -(.*)");
通过查看Tracking URL,发现抛出类找不到的异常,类
org.apache.hadoop.hive.contrib.serde2.RegexSerDe没有找到。
解决方法:
将下面的配置语句,加在配置文件: $HIVE_INSTALL/conf/hive-site.xml中,value中hive-contrib-*.jar的路径为你机器上实际的放置,在$HIVE_INSTALL/lib目录下寻找。
<property>
<name>hive.aux.jars.path</name>
<value>file:///home/develop/hive-0.8.0/lib/hive-contrib-0.8.0.jar</value>
<description>Added by tiger.zeng on 20120202.These JAR file are available to all users for all jobs</description>
</property>
退出hive shell, 重新启动hive shell, 可运行正常。
运行结果如下:hive> select time,clause from mylog2;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201202021642_0003, Tracking URL = http://hdp0:50030/jobdetails.jsp?jobid=job_201202021642_0003
Kill Command = /home/hadoop/hadoop-0.20.203.0/bin/hadoop job -Dmapred.job.tracker=hdp0:9001 -kill job_201202021642_0003
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2012-02-02 16:58:43,327 Stage-1 map = 0%, reduce = 0%
2012-02-02 16:58:49,375 Stage-1 map = 100%, reduce = 0%
2012-02-02 16:58:55,437 Stage-1 map = 100%, reduce = 100%
Ended Job = job_201202021642_0003
MapReduce Jobs Launched:
Job 0: Map: 1 HDFS Read: 1286 HDFS Write: 564 SUCESS
Total MapReduce CPU Time Spent: 0 msec
OK
NULL NULL
NULL NULL
NULL NULL
NULL NULL
2012-01-17 00:14:19,301 org.apache.hadoop.conf.Configuration -0 [main] DEBUG org.apache.hadoop.conf.Configuration
NULL NULL
NULL NULL
NULL NULL
NULL NULL
NULL NULL
2012-01-17 00:14:19,314 org.apache.hadoop.util.NativeCodeLoader -13 [main] DEBUG org.apache.hadoop.util.NativeCodeLoader
2012-01-17 00:14:19,314 org.apache.hadoop.util.NativeCodeLoader -13 [main] INFO org.apache.hadoop.util.NativeCodeLoader
2012-01-17 00:14:19,316 org.apache.hadoop.io.compress.zlib.ZlibFactory -15 [main] INFO org.apache.hadoop.io.compress.zlib.ZlibFactory
NULL NULL
Time taken: 21.28 seconds
hive>
参考:
Jasper Knulst 13020438802011年04月06日 星期三 06时51分20秒Hi(ve),I created a table like this;create table testtable (veld1 STRING,veld2 STRING,veld3 STRING) ROW FORMATSERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'> WITH SERDEPROPERTIES ("input.regex" ="([a-z]{4}[0-9])þ([a-z]{4}[0-9])þ([a-z]{4}[0-9])") STORED AS TEXTFILE;The table is OK, select * from testtable shows the contents of theunderlying HDFS file.However when I invoke a MR job by select veld2 from testtable, the MR jobstarts but I get mapper errors saying:"Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:java.lang.ClassNotFoundException:org.apache.hadoop.hive.contrib.serde2.RegexSerDe"I already copied the hive serde jar (in my case hive-serde-0.7.0-CDH3B4.jar)to $HADOOP_HOME/lib and restarted jobtracker/tasktrackers but that doesn'thelp.Cheers Jasper
You need to tell Hive about the JAR. This is how I do it in hive-site.xml:<property> <name>hive.aux.jars.path</name> <value>file:///usr/lib/hive/lib/hive-contrib-0.7.0-CDH3B4.jar</value> <description>These JAR file are available to all users for all jobs</description></property>Show quoted text
- Hive ClassNotFoundException: org.apache.hadoop.hive.contrib.serde2.RegexSerDe
- Hive ClassNotFoundException: Class org.apache.hadoop.hive.contrib.serde2.RegexSerDe not found
- Class org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe not found
- Kylin报错classnotfound:org.apache.hadoop.hive.serde2.typeinfo.typeInfo
- org.apache.hadoop.hive.serde2.SerDeException: Row is not a valid JSON Object
- java.lang.ClassNotFoundException: org.apache.hadoop.hive.hbase.HBaseSplit
- hive利用jdbc连接时报错:java.lang.ClassNotFoundException:org.apache.hadoop.hive.jdbc.HiveDriver
- Hive 启动报错 ClassNotFoundException: org.apache.hadoop.hive.service.HiveServer
- Hive1.2.1 启动报错 ClassNotFoundException: org.apache.hadoop.hive.service.HiveServer
- Maven连接Hive:java.lang.ClassNotFoundException: org.apache.hadoop.io.Writable
- HIVE RegexSerDe使用详解
- hive RegexSerDe View
- Hive 使用RegexSerDe来处理标准格式Apache Web日志
- Hive 使用RegexSerDe来处理标准格式Apache Web日志
- Hive 使用RegexSerDe来处理标准格式Apache Web日志
- hive中RegexSerDe的使用
- hive中RegexSerDe的使用
- org.apache.hadoop.hive.ql.exec.DDLTask. MetaException
- Linux 中config命令的使用
- 我认为电子商务只有一种死法——不重视数据冤死
- 使用ViewHolder模式来提高自定义adapter的效率
- cl_simple_tree_model -- 2
- #ifdef的用法
- Hive ClassNotFoundException: org.apache.hadoop.hive.contrib.serde2.RegexSerDe
- WordPress代码和分析-页面模板是从哪里来的
- Response.write后马上运行Response.redirect,Response.write没反应
- Android平台一日游
- cl_simple_tree_model -- 3
- 移动广告的爆炸式发展(信息图)
- android 好的开源项目
- ftp的使用tip
- 利用Page事件进行统一身份验证