hadoop运行java类常见错误修改
来源:互联网 发布:淘宝怎么搜vpn2016 编辑:程序博客网 时间:2024/06/07 10:02
使用hadoop运行自己写的java类报错:
1
2
3
4
5
6
7
8
9
10
/home/hadoop/bin/hadoop MaxTemperature /home/hadoop/input/sample.txt output
Exception in thread
"main"
java.lang.NoClassDefFoundError: MaxTemperature
Caused by: java.lang.ClassNotFoundException: MaxTemperature
at java.net.URLClassLoader$
1
.run(URLClassLoader.java:
202
)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:
190
)
at java.lang.ClassLoader.loadClass(ClassLoader.java:
306
)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:
301
)
at java.lang.ClassLoader.loadClass(ClassLoader.java:
247
)
Could not find the main
class
: MaxTemperature. Program will exit.
步骤1:设置java环境变量,编辑 conf/hadoop-env.sh文件,至少需要将JAVA_HOME设置为Java安装根路径。
步骤2:行java程序时要先编译成class文件,才能用hadoop命令,但是编译的时候会有如下错误:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
axTemperature.java:
5
: 软件包 org.apache.hadoop.fs 不存在
import
org.apache.hadoop.fs.Path;
^
MaxTemperature.java:
6
: 软件包 org.apache.hadoop.io 不存在
import
org.apache.hadoop.io.IntWritable;
^
MaxTemperature.java:
7
: 软件包 org.apache.hadoop.io 不存在
import
org.apache.hadoop.io.Text;
^
MaxTemperature.java:
8
: 软件包 org.apache.hadoop.mapred 不存在
import
org.apache.hadoop.mapred.FileInputFormat;
^
MaxTemperature.java:
9
: 软件包 org.apache.hadoop.mapred 不存在
import
org.apache.hadoop.mapred.FileOutputFormat;
....
1
export CLASSPATH=.:/home/hadoop/hadoop-
0.20
.
2
-core.jar:$CLASSPATH
有的时候报错:
1
2
3
4
5
6
7
8
9
10
root
@tianbaoxing
-virtual-machine:/home/hadoop/java# /home/hadoop/bin/hadoop MaxTemperature /home/hadoop/input/sample.txt output
Exception in thread
"main"
java.lang.NoClassDefFoundError: MaxTemperature
Caused by: java.lang.ClassNotFoundException: MaxTemperature
at java.net.URLClassLoader$
1
.run(URLClassLoader.java:
202
)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:
190
)
at java.lang.ClassLoader.loadClass(ClassLoader.java:
306
)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:
301
)
at java.lang.ClassLoader.loadClass(ClassLoader.java:
247
)
Could not find the main
class
: MaxTemperature. Program will exit.
1
export HADOOP_CLASSPATH=/home/hadoop/java
最后正确的运行结果:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
/home/hadoop/bin/hadoop MaxTemperature /home/hadoop/input/sample.txt output
13
/
04
/
10
11
:
00
:
31
INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
13
/
04
/
10
11
:
00
:
31
WARN mapred.JobClient: Use GenericOptionsParser
for
parsing the arguments. Applications should implement Tool
for
the same.
13
/
04
/
10
11
:
00
:
32
WARN mapred.JobClient: No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
13
/
04
/
10
11
:
00
:
32
INFO mapred.FileInputFormat: Total input paths to process :
1
13
/
04
/
10
11
:
00
:
33
INFO mapred.JobClient: Running job: job_local_0001
13
/
04
/
10
11
:
00
:
33
INFO mapred.FileInputFormat: Total input paths to process :
1
13
/
04
/
10
11
:
00
:
33
INFO mapred.MapTask: numReduceTasks:
1
13
/
04
/
10
11
:
00
:
33
INFO mapred.MapTask: io.sort.mb =
100
13
/
04
/
10
11
:
02
:
48
INFO mapred.MapTask: data buffer =
79691776
/
99614720
13
/
04
/
10
11
:
02
:
48
INFO mapred.MapTask: record buffer =
262144
/
327680
13
/
04
/
10
11
:
02
:
48
INFO mapred.LocalJobRunner: file:/home/hadoop/input/sample.txt:
0
+
2336
13
/
04
/
10
11
:
02
:
48
INFO mapred.JobClient: map
0
% reduce
0
%
13
/
04
/
10
11
:
02
:
48
INFO mapred.MapTask: Starting flush of map output
13
/
04
/
10
11
:
02
:
52
INFO mapred.MapTask: Finished spill
0
13
/
04
/
10
11
:
02
:
52
INFO mapred.TaskRunner: Task:attempt_local_0001_m_000000_0 is done. And is in the process of commiting
13
/
04
/
10
11
:
02
:
52
INFO mapred.LocalJobRunner: file:/home/hadoop/input/sample.txt:
0
+
2336
13
/
04
/
10
11
:
02
:
52
INFO mapred.TaskRunner: Task
'attempt_local_0001_m_000000_0'
done.
13
/
04
/
10
11
:
02
:
52
INFO mapred.LocalJobRunner:
13
/
04
/
10
11
:
02
:
52
INFO mapred.Merger: Merging
1
sorted segments
13
/
04
/
10
11
:
02
:
52
INFO mapred.JobClient: map
100
% reduce
0
%
13
/
04
/
10
11
:
02
:
52
INFO mapred.Merger: Down to the last merge-pass, with
1
segments left of total size:
189
bytes
13
/
04
/
10
11
:
02
:
52
INFO mapred.LocalJobRunner:
13
/
04
/
10
11
:
02
:
53
INFO mapred.TaskRunner: Task:attempt_local_0001_r_000000_0 is done. And is in the process of commiting
13
/
04
/
10
11
:
02
:
53
INFO mapred.LocalJobRunner:
13
/
04
/
10
11
:
02
:
53
INFO mapred.TaskRunner: Task attempt_local_0001_r_000000_0 is allowed to commit now
13
/
04
/
10
11
:
02
:
53
INFO mapred.FileOutputCommitter: Saved output of task
'attempt_local_0001_r_000000_0'
to file:/home/hadoop/java/output
13
/
04
/
10
11
:
02
:
53
INFO mapred.LocalJobRunner: reduce > reduce
13
/
04
/
10
11
:
02
:
53
INFO mapred.TaskRunner: Task
'attempt_local_0001_r_000000_0'
done.
13
/
04
/
10
11
:
02
:
53
INFO mapred.JobClient: map
100
% reduce
100
%
13
/
04
/
10
11
:
02
:
53
INFO mapred.JobClient: Job complete: job_local_0001
13
/
04
/
10
11
:
02
:
53
INFO mapred.JobClient: Counters:
13
13
/
04
/
10
11
:
02
:
53
INFO mapred.JobClient: FileSystemCounters
13
/
04
/
10
11
:
02
:
53
INFO mapred.JobClient: FILE_BYTES_READ=
30485
13
/
04
/
10
11
:
02
:
53
INFO mapred.JobClient: FILE_BYTES_WRITTEN=
52400
13
/
04
/
10
11
:
02
:
53
INFO mapred.JobClient: Map-Reduce Framework
13
/
04
/
10
11
:
02
:
53
INFO mapred.JobClient: Reduce input groups=
2
13
/
04
/
10
11
:
02
:
53
INFO mapred.JobClient: Combine output records=
0
13
/
04
/
10
11
:
02
:
53
INFO mapred.JobClient: Map input records=
17
13
/
04
/
10
11
:
02
:
53
INFO mapred.JobClient: Reduce shuffle bytes=
0
13
/
04
/
10
11
:
02
:
53
INFO mapred.JobClient: Reduce output records=
2
13
/
04
/
10
11
:
02
:
53
INFO mapred.JobClient: Spilled Records=
34
13
/
04
/
10
11
:
02
:
53
INFO mapred.JobClient: Map output bytes=
153
13
/
04
/
10
11
:
02
:
53
INFO mapred.JobClient: Map input bytes=
2336
13
/
04
/
10
11
:
02
:
53
INFO mapred.JobClient: Combine input records=
0
13
/
04
/
10
11
:
02
:
53
INFO mapred.JobClient: Map output records=
17
13
/
04
/
10
11
:
02
:
53
INFO mapred.JobClient: Reduce input record=
17
0 0
- hadoop运行java类常见错误修改
- hadoop运行常见错误
- hadoop运行常见的错误
- java常见运行时错误
- 运行hadoop MapReduce程序常见错误及解决方法整理
- 运行hadoop MapReduce程序常见错误及解决方法整理
- 运行hadoop MapReduce程序常见错误及解决方法整理
- hadoop运行错误 java.io.FileNotFoundException QuasiMonteCarlo
- Eclipse中运行java常见的错误
- hadoop常见错误
- Hadoop 集群常见错误
- hadoop常见错误
- hadoop常见错误
- hadoop常见错误
- hadoop 常见错误
- Hadoop常见错误总结
- Hadoop 常见错误处理
- Hadoop常见错误
- hive与mysql安装 以及使用
- 局域网内不能访问SQL Server 2005 的解决办法
- 每日一道算法题:求N个整数的最大公约数
- Calendar详解
- Android 4.0 事件输入(Event Input)系统
- hadoop运行java类常见错误修改
- spring+jasperReport 异常
- 昆山电池片回收排名问题?
- 关于Dede后台你指定的文件名…
- DEDECMS在linux服务器&权限设置
- div+css命名规范
- Dedecms 数据库 结构分析
- dedecms 目录结构
- DEDEcms的tag标签修改长度、语法及调用方法