hadoop2.2.0 unhelthy nodes:log-dirs turned bad
来源:互联网 发布:有淘宝账号就能贷款 编辑:程序博客网 时间:2024/06/03 15:56
问题:遇到一个mapreduce job运行迟迟不出结果,到8088端口看,发现nodemanager检测到一个unhealthy node
google之,得到如下信息:
hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/ClusterSetup.html#Monitoring_Health_of_NodeManagers
Hadoop provides a mechanism by which administrators can configure the NodeManager to run an administrator supplied script periodically to determine if a node is healthy or not.
Administrators can determine if the node is in a healthy state by performing any checks of their choice in the script. If the script detects the node to be in an unhealthy state, it must print a line to standard output beginning with the string ERROR. The NodeManager spawns the script periodically and checks its output. If the script's output contains the string ERROR, as described above, the node's status is reported asunhealthy and the node is black-listed by the ResourceManager. No further tasks will be assigned to this node. However, the NodeManager continues to run the script, so that if the node becomes healthy again, it will be removed from the blacklisted nodes on the ResourceManager automatically. The node's health along with the output of the script, if it is unhealthy, is available to the administrator in the ResourceManager web interface. The time since the node was healthy is also displayed on the web interface.
The following parameters can be used to control the node health monitoring script inconf/yarn-site.xml.
The health checker script is not supposed to give ERROR if only some of the local disks become bad.NodeManager has the ability to periodically check the health of the local disks (specifically checks nodemanager-local-dirs and nodemanager-log-dirs) and after reaching the threshold of number of bad directories based on the value set for the config property yarn.nodemanager.disk-health-checker.min-healthy-disks, the whole node is marked unhealthy and this info is sent to resource manager also. The boot disk is either raided or a failure in the boot disk is identified by the health checker script.
由于是单机环境,只有一个node可用,而这个node被标记为unhealthy后,便不再接受任务,因此任务卡住,无法得到执行。
www.cs.colostate.edu/~koontz/me/wiki/doku.php?id=hadoop
This happens when you don't have write access to the folder listed (in this case,/tmp/nodemanagerLog
). To fix the problem, change this directory in your -site.xml file.
看来是因为我是以root用户在R上运行mapreduce,对log路径没有写权限,所以才会导致这个错误。。。
修改权限为777后重启hadoop,问题解决!
hadoop2.2.0 web interface可用端口:(转自http://blog.csdn.net/twlkyao/article/details/17317157)
1.ResourceManage
2.NameNode
3.NodeManager
4.FileSystem
- hadoop2.2.0 unhelthy nodes:log-dirs turned bad
- Hadoop YARN ERROR 1/1 local-dirs are bad *, 1/1 log-dirs are bad *
- Hadoop YARN ERROR 1/1 local-dirs are bad *, 1/1 log-dirs are bad *
- Why does Hadoop report “Unhealthy Node local-dirs and log-dirs are bad”?
- Hadoop YARN: 1/1 local-dirs are bad: /var/lib/hadoop-yarn/cache/yarn/nm-local-dir; 1/1 log-dirs are
- dirs
- dirs
- hbase-0.96.2-hadoop2 config log
- Hadoop2.6 HDFS EDIT LOG分析
- dirs文件
- git log 错误解决 bad default revision 'HEAD'
- hadoop 执行的时候查看log 出现 bad state: BETWEEN_LOG_SEGMENTS
- 农民企业家 Farmer-turned-entrepreneur
- bad ()
- Hadoop2动态调整Log级别-以datanode的heartbeat log为例
- hadoop2.2.0 安装
- hadoop2.2.0 搭建
- 编译hadoop2.2.0源码
- android获取手机应用之后放入gridview之后的点击事件
- Servlet
- 火狐和IE的window.event对象详解
- Hibernate具体参数说明
- URL编码与解码
- hadoop2.2.0 unhelthy nodes:log-dirs turned bad
- Eclipse快捷键大全
- Real User Experience Insight (RUEI)安装操作手册
- 学习笔记----扩展欧几里德
- poj-2482-Stars in Your Window-线段树
- 谈谈ASP.net的GridView控件
- hive Lateral View语法
- 监听---共享连接参数配置介绍
- 制作speex库