MRv2.0(YARN)作业完成后提示jobhistory server拒绝链接,解决方案

来源:互联网 发布:玉米投手记账软件 编辑:程序博客网 时间:2024/06/05 15:10

系统环境:ubuntu14.04

hadoop环境:hadoop-2.2.0

问题描述:最近运行一个MapReduce作业,作业完成后提示如下错误:

 Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server
2015-01-04 18:33:02INFO:[main] - Retrying connect to server: master/192.168.1.100:10020. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2015-01-04 18:33:03INFO:[main] - Retrying connect to server: master/192.168.1.100:10020. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2015-01-04 18:33:04INFO:[main] - Retrying connect to server: master/192.168.1.100:10020. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2015-01-04 18:33:05INFO:[main] - Retrying connect to server: master/192.168.1.100:10020. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2015-01-04 18:33:06INFO:[main] - Retrying connect to server: master/192.168.1.100:10020. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2015-01-04 18:33:07INFO:[main] - Retrying connect to server: master/192.168.1.100:10020. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2015-01-04 18:33:08INFO:[main] - Retrying connect to server: master/192.168.1.100:10020. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2015-01-04 18:33:09INFO:[main] - Retrying connect to server: master/192.168.1.100:10020. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2015-01-04 18:33:10INFO:[main] - Retrying connect to server: master/192.168.1.100:10020. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2015-01-04 18:33:11INFO:[main] - Retrying connect to server: master/192.168.1.100:10020. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)

CallFrom master/192.168.1.100 to master:10020 failed on connectionexception: java.net.ConnectException:拒绝连接....

出现上述错误的原因是jobhistory server没有启动,首先介绍一下什么是jobhistory server:

Hadoop自带了一个历史服务器,可以通过历史服务器查看已经运行完的Mapreduce作业记录,比如用了多少个Map、用了多少个Reduce、作业提交时间、作业启动时间、作业完成时间等信息。默认情况下,Hadoop历史服务器是没有启动的,我们可以通过下面的命令来启动Hadoop历史服务器

$ sbin/mr-jobhistory-daemon.sh start historyserver

这样我们就可以在相应机器的19888端口上打开历史服务器的WEB UI界面。可以查看已经运行完的作业情况。历史服务器可以单独在一台机器上启动,主要是通过在mapred-site.xml文件中配置如下参数:
<property>
<name>mapreduce.jobhistory.address</name>
<!-- 配置实际的主机名和端口-->
<value>master:10020</value>
</property>

<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>master:19888</value>
</property>

0 0