resourcemanager java.lang.ClassNotFoundException: javax.activation.DataSource

来源:互联网 发布:淘宝联盟-阿里妈妈 编辑:程序博客网 时间:2024/05/20 20:17

hadoop无法启动resourcemanager和nodeManager ,
查看log发现java.lang.ClassNotFoundException: javax.activation.DataSource错误。

我出现这种情况是因为使用了jdk9,jdk9默认禁用访问许多javax. * API,可以更换为jdk8

还有一种方案不用修改jdk9

修改$HADOOP_HOME/etc/hadoop/yarn-env.sh文件

export YARN_RESOURCEMANAGER_OPTS="--add-modules=ALL-SYSTEM"export YARN_NODEMANAGER_OPTS="--add-modules=ALL-SYSTEM

参考: https://issues.apache.org/jira/browse/HADOOP-14978

第二天。。。又是一段惨痛的经历

使用jdk9之后修改了yarn-env.sh后resourcemanager和nodemanager。虽然服务成功启动了,但是在wordcount测试 中这两货还是执行失败了,由于菜鸟一枚,几番周折才发现错误原因,结果还是由于java.lang.ClassNotFoundException: javax.activation.DataSource的错误造成的。因此我还是用回jdk8,避免各种乱七八糟的原因

以下是resourcemanager中的日志输出

[2017-12-21 20:59:13.006]Container exited with a non-zero exit code 1. Error file: prelaunch.err.Last 4096 bytes of prelaunch.err :Last 4096 bytes of stderr :WARNING: An illegal reflective access operation has occurredWARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$2 (file:/home/hadoop/hadoop/hadoop-3.0.0/share/hadoop/yarn/lib/guice-4.0.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$2WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operationsWARNING: All illegal access operations will be denied in a future releaseDec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory registerINFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver as a provider classDec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory registerINFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider classDec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory registerINFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices as a root resource classDec 21, 2017 8:59:11 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiateINFO: Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25 AM'Dec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProviderINFO: Binding org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton"log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).log4j:WARN Please initialize the log4j system properly.log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.[2017-12-21 20:59:13.006]Container exited with a non-zero exit code 1. Error file: prelaunch.err.Last 4096 bytes of prelaunch.err :Last 4096 bytes of stderr :WARNING: An illegal reflective access operation has occurredWARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$2 (file:/home/hadoop/hadoop/hadoop-3.0.0/share/hadoop/yarn/lib/guice-4.0.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$2WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operationsWARNING: All illegal access operations will be denied in a future releaseDec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory registerINFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver as a provider classDec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory registerINFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider classDec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory registerINFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices as a root resource classDec 21, 2017 8:59:11 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiateINFO: Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25 AM'Dec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProviderINFO: Binding org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton"log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).log4j:WARN Please initialize the log4j system properly.log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.For more detailed output, check the application tracking page: http://master:8088/cluster/app/application_1513860808177_0001 Then click on links to logs of each attempt.. Failing the application.2017-12-21 20:59:13,063 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1513860808177_0001 State change from FINAL_SAVING to FAILED on event = APP_UPDATE_SAVED2017-12-21 20:59:13,063 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: Application removed - appId: application_1513860808177_0001 user: hadoop leaf-queue of parent: root #applications: 02017-12-21 20:59:13,072 WARN org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hadoop   OPERATION=Application Finished - Failed TARGET=RMAppManager RESULT=FAILURE  DESCRIPTION=App failed with state: FAILED   PERMISSIONS=Application application_1513860808177_0001 failed 2 times due to AM Container for appattempt_1513860808177_0001_000002 exited with  exitCode: 1Failing this attempt.Diagnostics: [2017-12-21 20:59:12.978]Exception from container-launch.Container id: container_1513860808177_0001_02_000001Exit code: 1

最后在logs/userlogs/好几级目录下找到了找到了application_1513860808177_0001日志,里面出现了
java.lang.ClassNotFoundException: javax.activation.DataSource错误

阅读全文
0 0
原创粉丝点击