Hadoop的datanode无法启动

来源：互联网发布：mysql front怎么用编辑：程序博客网时间：2024/05/20 16:09

1、用bin/hadoop start-all.sh启动Hdoop集群，然后用jps命令查看守护进程时，没有DataNode

[root@master hadoop_data]# jps7349 JobTracker7242 SecondaryNameNode7496 TaskTracker6986 NameNode7698 Jps

2、查看日志： tail -100 hadoop-root-datanode-master.log，发现有“ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: All directories in dfs.data.dir are invalid”

2014-05-08 11:03:09,320 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: STARTUP_MSG: /************************************************************STARTUP_MSG: Starting DataNodeSTARTUP_MSG:   host = master/10.71.196.43STARTUP_MSG:   args = []STARTUP_MSG:   version = 1.1.2STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1 -r 1440782; compiled by 'hortonfo' on Thu Jan 31 02:03:24 UTC 2013************************************************************/2014-05-08 11:03:09,515 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties2014-05-08 11:03:09,530 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered.2014-05-08 11:03:09,532 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).2014-05-08 11:03:09,532 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system started2014-05-08 11:03:09,646 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered.2014-05-08 11:03:09,698 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Invalid directory in dfs.data.dir: Incorrect permission for /root/hadoop_data/data, expected: rwxr-xr-x, while actual: rwxrwxr-x2014-05-08 11:03:09,699 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: All directories in dfs.data.dir are invalid.2014-05-08 11:03:09,699 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Exiting Datanode2014-05-08 11:03:09,699 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG: /************************************************************SHUTDOWN_MSG: Shutting down DataNode at master/10.71.196.43************************************************************/

3、查看一下 dfs.data.dir路径下的权限

/root/hadoop_data/data目录的用户组多了一个写的权限

[root@master logs]# ls -l /root/hadoop_data/datatotal 28drwxr-xr-x  2 root root  4096 May  8 10:49 blocksBeingWrittendrwxr-xr-x 66 root root 12288 May  8 10:49 currentdrwxr-xr-x  2 root root  4096 Apr 22 15:38 detach-rw-r--r--  1 root root   157 Apr 22 15:38 storagedrwxr-xr-x  2 root root  4096 May  8 10:48 tmp[root@master logs]# cd /root/hadoop_data/[root@master hadoop_data]# ls -ltotal 12drwxrwxr-x 6 root root 4096 May  8 10:57 datadrwxr-xr-x 5 root root 4096 May  8 11:03 namedrwxr-xr-x 4 root root 4096 May  8 10:28 tmp

（4）解决办法：将该目录的用户组的写权限去掉：chmod g-w data

[root@master hadoop_data]# chmod g-w data[root@master hadoop_data]# ls -ltotal 12drwxr-xr-x 6 root root 4096 May  8 10:57 datadrwxr-xr-x 5 root root 4096 May  8 11:03 namedrwxr-xr-x 4 root root 4096 May  8 10:28 tmp

（5）停止hadoop服务：stop-all.sh
（6）重新启动hadoop服务：start-all.sh

[root@master hadoop_data]# start-all.sh Warning: $HADOOP_HOME is deprecated.starting namenode, logging to /usr/lib/hadoop/libexec/../logs/hadoop-root-namenode-master.outslave01: starting datanode, logging to /usr/lib/hadoop/libexec/../logs/hadoop-root-datanode-master.outmaster: starting secondarynamenode, logging to /usr/lib/hadoop/libexec/../logs/hadoop-root-secondarynamenode-master.outstarting jobtracker, logging to /usr/lib/hadoop/libexec/../logs/hadoop-root-jobtracker-master.outslave01: starting tasktracker, logging to /usr/lib/hadoop/libexec/../logs/hadoop-root-tasktracker-master.out[root@master hadoop_data]# jps8444 JobTracker8335 SecondaryNameNode8078 NameNode8203 DataNode8593 TaskTracker8712 Jps

此时datanode进程能正常启来了

0 0