java.lang.RuntimeException: java.lang.ClassNotFoundException

来源:互联网 发布:lil pump 知乎 编辑:程序博客网 时间:2024/05/17 01:38
15/04/22 15:01:50 INFO mapred.JobClient: Running job: job_201504211850_0019
15/04/22 15:01:51 INFO mapred.JobClient:  map 0% reduce 0%
15/04/22 15:02:06 INFO mapred.JobClient: Task Id : attempt_201504211850_0019_m_000000_0, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: myback.Testmapreduce$testMap
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:867)
at org.apache.hadoop.mapreduce.JobContext.getMapperClass(JobContext.java:199)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:719)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.ClassNotFoundException: myback.Testmapreduce$testMap
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:865)

... 8 more

通过阅读以上错误,我们发现主要是自己定义的testmap类寻找不到,这是为什么?

首先,我们是在windows下通过Eclipse连接Linux下的Hadoop,当选择将程序运行在Hadoop的时候,虽然本工程下有testmap类,但是run on Hadoop的时候,由于配置环境中没有testmap这个类,故而寻找不到。

解决办法是将工程达成jar包,加入到配置环境中。运行就不会再出现以上错误了。

15/04/22 15:11:10 WARN snappy.LoadSnappy: Snappy native library not loaded
15/04/22 15:11:10 INFO mapred.JobClient: Running job: job_201504211850_0020
15/04/22 15:11:11 INFO mapred.JobClient:  map 0% reduce 0%
15/04/22 15:11:35 INFO mapred.JobClient:  map 100% reduce 0%
15/04/22 15:11:51 INFO mapred.JobClient:  map 100% reduce 100%
15/04/22 15:11:56 INFO mapred.JobClient: Job complete: job_201504211850_0020
15/04/22 15:11:56 INFO mapred.JobClient: Counters: 29
15/04/22 15:11:56 INFO mapred.JobClient:   Job Counters 
15/04/22 15:11:56 INFO mapred.JobClient:     Launched reduce tasks=1
15/04/22 15:11:56 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=21618
15/04/22 15:11:56 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
15/04/22 15:11:56 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
15/04/22 15:11:56 INFO mapred.JobClient:     Launched map tasks=1
15/04/22 15:11:56 INFO mapred.JobClient:     Data-local map tasks=1
15/04/22 15:11:56 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=15858
15/04/22 15:11:56 INFO mapred.JobClient:   File Output Format Counters 
15/04/22 15:11:56 INFO mapred.JobClient:     Bytes Written=71
15/04/22 15:11:56 INFO mapred.JobClient:   FileSystemCounters
15/04/22 15:11:56 INFO mapred.JobClient:     FILE_BYTES_READ=116
15/04/22 15:11:56 INFO mapred.JobClient:     HDFS_BYTES_READ=184
15/04/22 15:11:56 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=43503
15/04/22 15:11:56 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=71
15/04/22 15:11:56 INFO mapred.JobClient:   File Input Format Counters 
15/04/22 15:11:56 INFO mapred.JobClient:     Bytes Read=78
15/04/22 15:11:56 INFO mapred.JobClient:   Map-Reduce Framework
15/04/22 15:11:56 INFO mapred.JobClient:     Map output materialized bytes=116
15/04/22 15:11:56 INFO mapred.JobClient:     Map input records=5
15/04/22 15:11:56 INFO mapred.JobClient:     Reduce shuffle bytes=116
15/04/22 15:11:56 INFO mapred.JobClient:     Spilled Records=10
15/04/22 15:11:56 INFO mapred.JobClient:     Map output bytes=100
15/04/22 15:11:56 INFO mapred.JobClient:     CPU time spent (ms)=3730
15/04/22 15:11:56 INFO mapred.JobClient:     Total committed heap usage (bytes)=131796992
15/04/22 15:11:56 INFO mapred.JobClient:     Combine input records=0
15/04/22 15:11:56 INFO mapred.JobClient:     SPLIT_RAW_BYTES=106
15/04/22 15:11:56 INFO mapred.JobClient:     Reduce input records=5
15/04/22 15:11:56 INFO mapred.JobClient:     Reduce input groups=1
15/04/22 15:11:56 INFO mapred.JobClient:     Combine output records=0
15/04/22 15:11:56 INFO mapred.JobClient:     Physical memory (bytes) snapshot=182493184
15/04/22 15:11:56 INFO mapred.JobClient:     Reduce output records=5
15/04/22 15:11:56 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=747270144
15/04/22 15:11:56 INFO mapred.JobClient:     Map output records=5

0 0