hadoop启动过程中遇到的问题

http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-user/201207.mbox/[e-mail]%3E

http://blog.csdn.net/huoyunshen88/article/details/9089797

2013-06-08 15:04:15,622 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: com.slave1/10.132.30.249:9000. Already tried 7 time(s).

namenode 的错误信息如下：
2013-06-08 14:53:37,733 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:hadoop cause:java.io.IOException: File /data/hdfs/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
2013-06-08 14:53:37,734 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 9000, call addBlock(/data/hdfs/tmp/mapred/system/jobtracker.info, DFSClient_1426524007, null) from 10.132.30.48:56119: error: java.io.IOException: File /data/hdfs/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
java.io.IOException: File /data/hdfs/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
2013-06-08 14:53:47,758 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:hadoop cause:java.io.IOException: File /data/hdfs/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
2013-06-08 14:53:47,758 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 9000, call addBlock(/data/hdfs/tmp/mapred/system/jobtracker.info, DFSClient_1426524007, null) from 10.132.30.48:56119: error: java.io.IOException: File /data/hdfs/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
java.io.IOException: File /data/hdfs/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
2013-06-08 14:53:57,783 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of transactions: 49 Total time for transactions(ms): 2Number of transactions batched in Syncs: 0 Number of syncs: 36 SyncTimes(ms): 1021 
2013-06-08 14:53:58,067 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:hadoop cause:java.io.IOException: File /data/hdfs/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
2013-06-08 14:53:58,068 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 9000, call addBlock(/data/hdfs/tmp/mapred/system/jobtracker.info, DFSClient_1426524007, null) from 10.132.30.48:56120: error: java.io.IOException: File /data/hdfs/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
java.io.IOException: File /data/hdfs/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
2013-06-08 14:54:08,181 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:hadoop cause:java.io.IOException: File /data/hdfs/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
2013-06-08 14:54:08,181 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 9000, call addBlock(/data/hdfs/tmp/mapred/system/jobtracker.info, DFSClient_1426524007, null) from 10.132.30.48:56120: error: java.io.IOException: File /data/hdfs/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
java.io.IOException: File /data/hdfs/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)


通过查询网络解决方法如下：
1.修改了之前的hosts配置文件，重启系统，reboot。
[root@master opt]# vi /etc/hosts
127.0.0.1      localhost.localdomain localhost
::1     localhost6.localdomain6 localhost6
10.132.30.249  com.slave1        slave1
10.132.30.48   com.master       master
10.132.30.250  com.slave2       slave2

修改后：
127.0.0.1      localhost.localdomain localhost
::1     localhost6.localdomain6 localhost6
10.132.30.48   master.localdomain          master  #master后的域名信息要和localhost的一致。
10.132.30.249  slave1
10.132.30.250  slave2
2.停掉hadoop集群
hadoop@master bin]$ ./stop-all.sh 
3.格式化整个namenode
[hadoop@master bin]$ hadoop namenode -format


4. 删除namedoe 和datanode 的 hdfs 临时文件 tmp目录。
[hadoop@master bin]$ rm -fr /data/hdfs/tmp/

5.重启后查看namenode 的master日志发现如下错误依旧存在。

Hadoop 配置常见问题

Q1: could only be replicated to 0 nodes, instead of 1
A: sudo /etc/init.d/iptables stop  Or 格式化namenode

在Hadoop的环境搭建过程中，常常会遇到类似这样的错误信息提示：“could only be replicated to 0 nodes, instead of 1”，产生这样的错误原因有多种，这里列举出以下四种常用的解决方法以供参考：

确保master（namenode） 、slaves（datanode）的防火墙已经关闭
确保DFS空间的使用情况
Hadoop默认的hadoop.tmp.dir的路径为/tmp/hadoop-${user.name},而有的linux系统的/tmp目录文件系统的类型往往是Hadoop不支持的。（本人在Ubuntu 10.10下测试伪分布式时就遇到了这种情况）

Q2: save mode
A:    hadoop dfsadmin -safemode leave


@: tow machine ssh set
A: hduser@master:~$ ssh-copy-id -i $HOME/.ssh/id_rsa.pub hduser@slave 
    hduser@master:~$ ssh master
    hduser@master:~$ ssh slave
    

Q: command
A:  bin/hadoop-daemon.sh start [namenode | secondarynamenode | datanode | jobtracker | tasktracker],

Q: Incompatible namespaceIDs
A:原因是格式化namenode后会重新创建一个新的namespaceID,以至于和datanode上原有的不一致。
       解决方法：    （1）删除datanode    dfs.data.dir目录（默认为tmp/dfs/data）下的数据文件，
                           （2）修改<dfs.data.dir>/current/VERSION 文件，把namespaceID修成与namenode上相同即可（log错误里会有提示）
                           （3）重新指定新的<dfs.data.dir>目录

问题总于解决了，网上很多说是/etc/hosts文件配置的问题，这么说太绝对了，很多时候具体问题需要具体分析。而我的问题主要是空间不足导致的，

参加http://blog.csdn.net/huoyunshen88/article/details/9089797。