oozie配置sharelib,java.lang.NoClassDefFoundError: org/apache/pig/Main
来源:互联网 发布:淘宝改中差评 编辑:程序博客网 时间:2024/06/08 09:21
作为笔记:
sharelib的配置
在用oozie调用pig时,用到sharelib,当时是直接运行的,然后就报错了.
2013-07-03 20:44:48,504 WARN PigActionExecutor:542 - USER[root] GROUP[-] TOKEN[] APP[pig-wf] JOB[0000004-130703183035408-oozie-root-W] ACTION[0000004-130703183035408-oozie-root-W@pig-node] Launcher exception: org/apache/pig/Main
java.lang.NoClassDefFoundError: org/apache/pig/Main
at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:323)
at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:218)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)
at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:467)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.ClassNotFoundException: org.apache.pig.Main
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 17 more
oozie提供一个sharelib:oozie-sharelib-**-incubating.tar.gz。解压即可。
然后需要把sharelib放到hdfs上,hdfs的sharelib路径发现也是个问题。
这个需要在oozie安装目录下conf/oozie-default.xml中查找:
<property>
<name>oozie.service.WorkflowAppService.system.libpath</name>
<value>/user/${user.name}/share/lib</value>
<description>
System library path to use for workflow applications.
This path is added to workflow application if their job properties sets
the property 'oozie.use.system.libpath' to true.
</description>
</property>
“/user/${user.name}/share/lib”就是我们需要放sharelib的路径。
- oozie配置sharelib,java.lang.NoClassDefFoundError: org/apache/pig/Main
- Exceptioninthread"main"java.lang.NoClassDefFoundError:org/apache/hadoop/util/ProgramDriver
- Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/lang/exception/Nestabl
- Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/lang/exception/Nestabl
- Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/axis/client/AdminClient
- Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/collections/map/LRUMap
- Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/xml/serializer/TreeWalker
- Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/xml/serializer/TreeWalker
- Exception in thread "main " java.lang.NoClassDefFoundError: org/apache/commons/logging/Log
- Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
- 解决Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
- Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/xmlbeans/XmlObject
- Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
- HIVE Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hive/ql/CommandNee
- Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/juli/l
- Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/juli/logging/LogFactory
- Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/juli/l
- Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hive/ql/CommandNeedRetr
- Extjs4 MVC Area面积图Numeric类型y轴不动态变化的方法
- STL中list,vector,deque,map,set区别、联系和使用场景
- Sources for Android SDK 源码关联 以及第三方jar包源码关联
- UNIX环境高级编程——文件和目录
- linux和windows下mysql忘记密码的情况下修改密码
- oozie配置sharelib,java.lang.NoClassDefFoundError: org/apache/pig/Main
- UICollectionView_1
- phpcms的栏目缓存无限级罗列展示处理方法
- Android开发当中Parcelable接口的使用
- experiment : 为IDA6.1产生mfc90u.dll的符号
- Ehcache详细解读
- 第十五章 Sqoop(1)
- [Wince6.0]Windows Embedded CE 6.0 安装过程,序列号和使用初步(转)
- apache和tomcat区别