Hadoop运行成功但最后显示Redirecting to job history server...Client: Retrying connect to server:

来源:互联网 发布:数组指定位置添加元素 编辑:程序博客网 时间:2024/06/10 04:13

运行环境:ubuntu

自己生成了个jar包,用hadoop jar xxx.jar ./input ./output,运行成功,可以输出结果,但是最后提示Redirecting to job history server...Client: Retrying connect to server:

具体如下:

16/06/22 21:51:23 INFO mapred.ClientServiceDelegate: Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server16/06/22 21:51:24 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:25 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:26 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:27 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:28 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:29 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:30 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:31 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:32 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:33 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:34 INFO mapred.ClientServiceDelegate: Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server16/06/22 21:51:35 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:36 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:37 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:38 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:39 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:40 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:41 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:42 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:43 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:44 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:44 INFO mapred.ClientServiceDelegate: Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server16/06/22 21:51:45 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:46 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:47 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:48 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:49 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:50 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:51 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:52 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:53 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)16/06/22 21:51:54 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)Exception in thread "main" java.io.IOException: java.net.ConnectException: Call From backup01/192.168.137.200 to backup01:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefusedat org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:338)at org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:423)at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:576)at org.apache.hadoop.mapreduce.Job$1.run(Job.java:326)at org.apache.hadoop.mapreduce.Job$1.run(Job.java:323)at java.security.AccessController.doPrivileged(Native Method)at javax.security.auth.Subject.doAs(Subject.java:422)at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)at org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323)at org.apache.hadoop.mapreduce.Job.isSuccessful(Job.java:623)at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1330)at org.apache.hadoop.examples.WordCount.main(WordCount.java:87)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at org.apache.hadoop.util.RunJar.run(RunJar.java:221)at org.apache.hadoop.util.RunJar.main(RunJar.java:136)Caused by: java.net.ConnectException: Call From backup01/192.168.137.200 to backup01:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefusedat sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)at java.lang.reflect.Constructor.newInstance(Constructor.java:423)at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:791)at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:731)at org.apache.hadoop.ipc.Client.call(Client.java:1473)at org.apache.hadoop.ipc.Client.call(Client.java:1400)at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)at com.sun.proxy.$Proxy14.getJobReport(Unknown Source)at org.apache.hadoop.mapreduce.v2.api.impl.pb.client.MRClientProtocolPBClientImpl.getJobReport(MRClientProtocolPBClientImpl.java:133)at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:324)... 17 moreCaused by: java.net.ConnectException: Connection refusedat sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530)at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:494)at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:608)at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:706)at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:369)at org.apache.hadoop.ipc.Client.getConnection(Client.java:1522)at org.apache.hadoop.ipc.Client.call(Client.java:1439)... 25 more

我们可以看到,连接job history server失败。

输入jps,看有哪些java进程:

23923 QuorumPeerMain65477 Jps28614 ResourceManager28456 SecondaryNameNode28239 NameNode

真的没有启动job history service。

解决方案:

启动historyserver:

sbin/mr-jobhistory-daemon.sh start historyserver


0 0