Hadoop Hdfs DataNode 启动失败

来源：互联网发布：什么叫大数据思维编辑：程序博客网时间：2024/05/17 07:20

FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for block pool Block pool BP-510292565-192.168.246.128-1480349157516 (storage id DS-1215049814-192.168.88.130-50010-1455139414431) service to node2/192.168.246.131:8020
java.io.IOException: Incompatible clusterIDs in /opt/hadoop2/dfs/data: namenode clusterID = CID-14cfdb37-d378-4aa5-815b-2c84417ddbf9; datanode clusterID = CID-584a9b34-9a97-4155-8373-8b285674e9f3
at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:391)
at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:191)
at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:219)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:837)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:808)
at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:280)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:222)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:664)

at java.lang.Thread.run(Thread.java:745)

出现这个问题的原因是DataNode的clusterID和NameNode的clusterID不匹配. --clusterID的作用是唯一标识一个集群.

解决办法是,将name/current中的VERSION中的clusterID 覆盖data/current下的VERSION中的clusterID

-rw-r--r--. 1 root root      30 Nov 29 00:26 edits_0000000000000000014-0000000000000000015-rw-r--r--. 1 root root      30 Nov 29 00:28 edits_0000000000000000016-0000000000000000017-rw-r--r--. 1 root root      30 Nov 29 00:30 edits_0000000000000000018-0000000000000000019-rw-r--r--. 1 root root      30 Nov 29 00:32 edits_0000000000000000020-0000000000000000021-rw-r--r--. 1 root root      30 Nov 29 00:34 edits_0000000000000000022-0000000000000000023-rw-r--r--. 1 root root      30 Nov 29 00:36 edits_0000000000000000024-0000000000000000025-rw-r--r--. 1 root root      30 Nov 29 00:38 edits_0000000000000000026-0000000000000000027-rw-r--r--. 1 root root      30 Nov 29 00:40 edits_0000000000000000028-0000000000000000029-rw-r--r--. 1 root root      30 Nov 29 00:42 edits_0000000000000000030-0000000000000000031-rw-r--r--. 1 root root      30 Nov 29 00:44 edits_0000000000000000032-0000000000000000033-rw-r--r--. 1 root root      30 Nov 29 00:46 edits_0000000000000000034-0000000000000000035-rw-r--r--. 1 root root      30 Nov 29 00:48 edits_0000000000000000036-0000000000000000037-rw-r--r--. 1 root root      30 Nov 29 00:50 edits_0000000000000000038-0000000000000000039-rw-r--r--. 1 root root      30 Nov 29 00:52 edits_0000000000000000040-0000000000000000041-rw-r--r--. 1 root root      30 Nov 29 00:54 edits_0000000000000000042-0000000000000000043-rw-r--r--. 1 root root 1048576 Nov 29 00:54 edits_inprogress_0000000000000000044-rw-r--r--. 1 root root     196 Nov 29 00:05 fsimage_0000000000000000000-rw-r--r--. 1 root root      62 Nov 29 00:05 fsimage_0000000000000000000.md5-rw-r--r--. 1 root root       3 Nov 29 00:54 seen_txid-rw-r--r--. 1 root root     207 Nov 29 00:05 VERSION[root@node1 current]# pwd/opt/hadoop2/dfs/name/current

然后重启hdfs 可以看到没有启动的datanode 已经启动.

出现这个问题的原因是,在第一次格式化hdfs后,启动并使用了hadoop,在使用一段时间之后,又执行了 hdfs namenode -format命令,在这个过程中,出现流程上的错误,导致

namenode的clusterID重新生成了,但是datanode的clusterID没有重新生成

踩

0 0