MRv2.0(YARN)作业完成后提示jobhistory server拒绝链接,解决方案
来源:互联网 发布:玉米投手记账软件 编辑:程序博客网 时间:2024/06/05 15:10
系统环境:ubuntu14.04
hadoop环境:hadoop-2.2.0
问题描述:最近运行一个MapReduce作业,作业完成后提示如下错误:
Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server
2015-01-04 18:33:02INFO:[main] - Retrying connect to server: master/192.168.1.100:10020. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2015-01-04 18:33:03INFO:[main] - Retrying connect to server: master/192.168.1.100:10020. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2015-01-04 18:33:04INFO:[main] - Retrying connect to server: master/192.168.1.100:10020. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2015-01-04 18:33:05INFO:[main] - Retrying connect to server: master/192.168.1.100:10020. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2015-01-04 18:33:06INFO:[main] - Retrying connect to server: master/192.168.1.100:10020. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2015-01-04 18:33:07INFO:[main] - Retrying connect to server: master/192.168.1.100:10020. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2015-01-04 18:33:08INFO:[main] - Retrying connect to server: master/192.168.1.100:10020. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2015-01-04 18:33:09INFO:[main] - Retrying connect to server: master/192.168.1.100:10020. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2015-01-04 18:33:10INFO:[main] - Retrying connect to server: master/192.168.1.100:10020. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2015-01-04 18:33:11INFO:[main] - Retrying connect to server: master/192.168.1.100:10020. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
CallFrom master/192.168.1.100 to master:10020 failed on connectionexception: java.net.ConnectException:拒绝连接....
出现上述错误的原因是jobhistory server没有启动,首先介绍一下什么是jobhistory server:
Hadoop自带了一个历史服务器,可以通过历史服务器查看已经运行完的Mapreduce作业记录,比如用了多少个Map、用了多少个Reduce、作业提交时间、作业启动时间、作业完成时间等信息。默认情况下,Hadoop历史服务器是没有启动的,我们可以通过下面的命令来启动Hadoop历史服务器
$ sbin/mr-jobhistory-daemon.sh start historyserver
这样我们就可以在相应机器的19888端口上打开历史服务器的WEB UI界面。可以查看已经运行完的作业情况。历史服务器可以单独在一台机器上启动,主要是通过在mapred-site.xml文件中配置如下参数:
<property>
<name>mapreduce.jobhistory.address</name>
<!-- 配置实际的主机名和端口-->
<value>master:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>master:19888</value>
</property>
- MRv2.0(YARN)作业完成后提示jobhistory server拒绝链接,解决方案
- MRv2下作业成功完成后,jobhistory server拒绝链接,解决方案
- YARN/MRv2 MRAppMaster深入剖析—作业生命周期
- YARN/MRv2 MRAppMaster深入剖析—作业恢复
- YARN/MRv2 MRAppMaster深入剖析—作业恢复
- YARN/MRv2 MRAppMaster深入剖析—作业生命周期
- YARN/MRv2 ResourceManager代码分析
- YARN/MRv2 ResourceManager代码分析
- 下一代hadoop(Mapreduce),YARN/MRv2
- YARN (MRv2) ResourceManager High Availability
- hadoop2 YARN/MRv2 Node Manager
- YARN/MRv2异步处理编程模型介绍
- YARN/MRv2 MRAppMaster深入剖析—概述
- YARN/MRv2 中基本术语介绍
- hadoop2/MRv2 YARN整体了解 ResourceManager
- 简述下一代mapreduce,yarn或者说是mrv2
- Hadoop MR &MRv2(YARN)编程模型
- SQL Server不存在或拒绝访问解决方案
- 嵌入式经常用到的移位,赋值,表示问题
- 有些东西,只要你做就是成功
- 3DES加密解密调用示例
- 自动寻径
- Java中的软引用,弱引用和虚引用
- MRv2.0(YARN)作业完成后提示jobhistory server拒绝链接,解决方案
- 球虫机器人
- 服务器端文件分片合并的思考和实践
- 工业4.0特征
- 极客Web前端开发资源大荟萃#006
- 极客Web前端开发资源大荟萃#007
- 在Activity的onCreate方法中显示PopupWindow导致异常的原因分析及解决方案
- 费马小定理
- 校招感悟