HADOOP报错Incompatible namespaceIDs
来源:互联网 发布:centos 7 nodejs 安装 编辑:程序博客网 时间:2024/06/06 20:07
今早一来,突然发现使用-put命令往HDFS里传数据传不上去了,抱一大堆错误,然后我使用bin/hadoop dfsadmin -report查看系统状态
admin@adw1:/home/admin/joe.wangh/hadoop-0.19.2>bin/hadoop
Configured Capacity: 0 (0 KB)
Present Capacity: 0 (0 KB)
DFS Remaining: 0 (0 KB)
DFS Used: 0 (0 KB)
DFS Used%: ?%
-------------------------------------------------
Datanodes available: 0 (0 total, 0 dead)
使用bin/stop-all.sh关闭HADOOP
admin@adw1:/home/admin/joe.wangh/hadoop-0.19.2>bin/stop-all.sh
stopping jobtracker
172.16.197.192: stopping tasktracker
172.16.197.193: stopping tasktracker
stopping namenode
172.16.197.193: no datanode to stop
172.16.197.192: no datanode to stop
172.16.197.191: stopping secondarynamenode
哦,看到了吧,发现datanode前面并没有启动起来。去DATANODE上查看一下日志
admin@adw2:/home/admin/joe.wangh/hadoop-0.19.2/logs>vi
************************************************************/
2010-07-21 10:12:11,987 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Incompatible namespaceIDs in /home/admin/joe.wangh/hadoop/data/dfs.data.dir: namenode namespaceID = 898136669; datanode namespaceID = 2127444065
......
错误提示namespaceIDs不一致。
下面给出两种解决办法,我使用的是第二种。
Workaround 1: Start from scratch
I can testify that the following steps solve this error, but the side effects won't make you happy (me neither). The crude workaround I have found is to:
1.
2.
3.
4.
When deleting all the HDFS data and starting from scratch does not sound like a good idea (it might be ok during the initial setup/testing), you might give the second approach a try.
Workaround 2: Updating namespaceID of problematic datanodes
Big thanks to Jared Stehler for the following suggestion. I have not tested it myself yet, but feel free to try it out and send me your feedback. This workaround is "minimally invasive" as you only have to edit one file on the problematic datanodes:
1.
2.
3.
If you followed the instructions in my tutorials, the full path of the relevant file is /usr/local/hadoop-datastore/hadoop-hadoop/dfs/data/current/VERSION (background: dfs.data.dir is by default set to ${hadoop.tmp.dir}/dfs/data, and we set hadoop.tmp.dir to /usr/local/hadoop-datastore/hadoop-hadoop).
If you wonder how the contents of VERSION look like, here's one of mine:
#contents of <dfs.data.dir>/current/VERSION
namespaceID=393514426
storageID=DS-1706792599-10.10.10.1-50010-1204306713481
cTime=1215607609074
storageType=DATA_NODE
layoutVersion=-13
原因:每次namenode format会重新创建一个namenodeId,而tmp/dfs/data下包含了上次format下的id,namenode format清空了namenode下的数据,但是没有晴空datanode下的数据,导致启动时失败,所要做的就是每次fotmat前,清空tmp一下的所有目录.
- HADOOP报错Incompatible namespaceIDs
- HADOOP报错Incompatible namespaceIDs
- HADOOP报错Incompatible namespaceIDs .
- HADOOP报错Incompatible namespaceIDs
- HADOOP报错Incompatible namespaceIDs
- Incompatible namespaceIDs
- Incompatible namespaceIDs
- Hadoop name -format后Incompatible namespaceIDS 错误解决办法
- ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Incompatible namespaceIDs
- hadoop安装过程出现Incompatible namespaceIDs 错误解决方案
- hadoop启动出现异常 java.io.IOException: Incompatible namespaceIDs in
- Hadoop name -format后Incompatible namespaceIDS 错误解决办法
- 遇到问题---Hadoop---java.io.IOException: Incompatible namespaceIDs
- Incompatible namespaceIDS 错误解决办法
- hadoop启动jobtracker时错误java.io.IOException: Incompatible namespaceIDs的解决方法
- org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Incompatible namespaceIDs in /
- java.io.IOException: Incompatible namespaceIDs in /usr/hadoop/tmp/dfs/data: namenode namespaceID = 6
- org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Incompatible namespaceIDs
- [LeetCode] Container With Most Water
- Google搜索语法
- HDU 1214 圆桌会议
- 抽象类和接口(备忘)
- .Net WebService 远程访问配置
- HADOOP报错Incompatible namespaceIDs
- Teched 2012 Azure Lab实验手册中文版
- WEB项目启动加载的实现方式整理
- 关键业务的商用服务器
- replace和replaceAll
- 鱼和熊掌兼得--既实现AJAX又实现SEO
- 帮助你生成分享和显示社交媒体网络按钮的jQuery插件 - #50C1AL
- 我的助理辞职了,献给那些浮躁的职场人
- Store 加载数据小方法 备查