hadoop常见错误(转载)
来源:互联网 发布:php任务注册系统源码 编辑:程序博客网 时间:2024/05/22 09:06
hadoop常见异常
错误解决:
1.如果在bin/hadoop dfs -put /home/test-in input该过程中出现"can onlybe
2.如果put时出现java.io.IOException:Not a file: hdfs://localhost:9000/user/icymary/input/test-in
解决办法是bin/hadoopdfs -rmr input
bin/hadoop dfs -put /home/test-in input
原因是,当执行了多次put之后,就会在分布式文件系统中生成子目录,删除重新put即可。
3.如果在
4.如果
INFO mapred.JobClient: map 0% reduce 0%
且一直卡住,在log日志中也没有出现异样,那么解决办法是,把/etc/hosts里面多余的机器名删掉,即可。
-----------------------
异常分析
在进行上述实践过程中,可能会遇到某种异常情况,大致分析如下:
1、Call to localhost/127.0.0.1:9000 failed on localexception异常
(1)异常描述
可能你会在执行如下命令行的时候出现:
[root@localhost hadoop-0.19.2]# bin/hadoop jarhadoop-0.19.0-examples.jar wordcount input output
出错异常信息如下所示:
10/08/01 19:50:55 INFO ipc.Client: Retrying connect to server:localhost/127.0.0.1:9000. Already tried 0 time(s).
10/08/01 19:50:56 INFO ipc.Client: Retrying connect to server:localhost/127.0.0.1:9000. Already tried 1 time(s).
10/08/01 19:50:57 INFO ipc.Client: Retrying connect to server:localhost/127.0.0.1:9000. Already tried 2 time(s).
10/08/01 19:50:58 INFO ipc.Client: Retrying connect to server:localhost/127.0.0.1:9000. Already tried 3 time(s).
10/08/01 19:50:59 INFO ipc.Client: Retrying connect to server:localhost/127.0.0.1:9000. Already tried 4 time(s).
10/08/01 19:51:00 INFO ipc.Client: Retrying connect to server:localhost/127.0.0.1:9000. Already tried 5 time(s).
10/08/01 19:51:01 INFO ipc.Client: Retrying connect to server:localhost/127.0.0.1:9000. Already tried 6 time(s).
10/08/01 19:51:02 INFO ipc.Client: Retrying connect to server:localhost/127.0.0.1:9000. Already tried 7 time(s).
10/08/01 19:51:03 INFO ipc.Client: Retrying connect to server:localhost/127.0.0.1:9000. Already tried 8 time(s).
10/08/01 19:51:04 INFO ipc.Client: Retrying connect to server:localhost/127.0.0.1:9000. Already tried 9 time(s).
java.lang.RuntimeException: java.io.IOException: Call tolocalhost/127.0.0.1:9000 failed on local exception: Connectionrefused
atorg.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:323)
atorg.apache.hadoop.mapred.FileInputFormat.setInputPaths(FileInputFormat.java:295)
atorg.apache.hadoop.mapred.FileInputFormat.setInputPaths(FileInputFormat.java:268)
atorg.apache.hadoop.examples.WordCount.run(WordCount.java:146)
atorg.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
atorg.apache.hadoop.examples.WordCount.main(WordCount.java:155)
atsun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethod)
atsun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
atsun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
atjava.lang.reflect.Method.invoke(Method.java:597)
atorg.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
atorg.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:141)
atorg.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:61)
atsun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethod)
atsun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
atsun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
atjava.lang.reflect.Method.invoke(Method.java:597)
atorg.apache.hadoop.util.RunJar.main(RunJar.java:165)
atorg.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
atorg.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
atorg.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
atorg.apache.hadoop.mapred.JobShell.main(JobShell.java:68)
Caused by: java.io.IOException: Call to localhost/127.0.0.1:9000failed on local exception: Connection refused
atorg.apache.hadoop.ipc.Client.call(Client.java:699)
atorg.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:216)
at$Proxy0.getProtocolVersion(Unknown Source)
atorg.apache.hadoop.ipc.RPC.getProxy(RPC.java:319)
atorg.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:104)
atorg.apache.hadoop.hdfs.DFSClient.(DFSClient.java:177)
atorg.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:74)
在进行上述实践过程中,可能会遇到某种异常情况,大致分析如下:
1、Call to localhost/127.0.0.1:9000 failed on localexception异常
(1)异常描述
可能你会在执行如下命令行的时候出现:
[root@localhost hadoop-0.19.2]# bin/hadoop jarhadoop-0.19.0-examples.jar wordcount input output
出错异常信息如下所示:
10/08/01 19:50:55 INFO ipc.Client: Retrying connect to server:localhost/127.0.0.1:9000. Already tried 0 time(s).
10/08/01 19:50:56 INFO ipc.Client: Retrying connect to server:localhost/127.0.0.1:9000. Already tried 1 time(s).
10/08/01 19:50:57 INFO ipc.Client: Retrying connect to server:localhost/127.0.0.1:9000. Already tried 2 time(s).
10/08/01 19:50:58 INFO ipc.Client: Retrying connect to server:localhost/127.0.0.1:9000. Already tried 3 time(s).
10/08/01 19:50:59 INFO ipc.Client: Retrying connect to server:localhost/127.0.0.1:9000. Already tried 4 time(s).
10/08/01 19:51:00 INFO ipc.Client: Retrying connect to server:localhost/127.0.0.1:9000. Already tried 5 time(s).
10/08/01 19:51:01 INFO ipc.Client: Retrying connect to server:localhost/127.0.0.1:9000. Already tried 6 time(s).
10/08/01 19:51:02 INFO ipc.Client: Retrying connect to server:localhost/127.0.0.1:9000. Already tried 7 time(s).
10/08/01 19:51:03 INFO ipc.Client: Retrying connect to server:localhost/127.0.0.1:9000. Already tried 8 time(s).
10/08/01 19:51:04 INFO ipc.Client: Retrying connect to server:localhost/127.0.0.1:9000. Already tried 9 time(s).
java.lang.RuntimeException: java.io.IOException: Call tolocalhost/127.0.0.1:9000 failed on local exception: Connectionrefused
Caused by: java.io.IOException: Call to localhost/127.0.0.1:9000failed on local exception: Connection refused