报错Cannot set permission for /hdata/hdfs/system. Name node is in safe mode.

来源:互联网 发布:数控车床g73编程实例 编辑:程序博客网 时间:2024/04/30 03:09

错误现象:hadoop集群重启时报如下错误:
2013-07-05 14:41:50,074 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:hadoop (auth:SIMPLE) cause:org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot set permission for /hdata/hdfs/system. Name node is in safe mode.The reported blocks 0 needs additional 92 blocks to reach the threshold 0.9990 of total blocks 93. Safe mode will be turned off automatically.2013-07-05 14:41:50,075 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 9000, call setPermission(/hdata/hdfs/system, rwx------) from 10.132.30.48:47969: error: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot set permission for /hdata/hdfs/system. Name node is in safe mode.The reported blocks 0 needs additional 92 blocks to reach the threshold 0.9990 of total blocks 93. Safe mode will be turned off automatically.org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot set permission for /hdata/hdfs/system. Name node is in safe mode.The reported blocks 0 needs additional 92 blocks to reach the threshold 0.9990 of total blocks 93. Safe mode will be turned off automatically.at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setPermission(FSNamesystem.java:856)at org.apache.hadoop.hdfs.server.namenode.NameNode.setPermission(NameNode.java:739)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:601)at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1439)at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1435)at java.security.AccessController.doPrivileged(Native Method)at javax.security.auth.Subject.doAs(Subject.java:415)at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278)at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1433)



错误原因:/hdata/hdfs/system这个目录是配置在mapred-site.xml 中的,原因是这个目录配置的有问题。
[hadoop@master bin]$ cat ../conf/mapred-site.xml <?xml version="1.0"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!-- Put site-specific property overrides in this file. --><configuration><property>        <name>mapred.job.tracker</name>        <value>10.132.30.48:9001</value> </property> <property>       <name>mapred.system.dir</name>        <value>/hdata/hdfs/system</value></property></configuration>



解决办法: 查找资料发现,mapred.system.dir 这个配置参数值为mapreduce共享目录,不能为本地目录,只能是hdfs目录。
所以创建HDFS 文件目录/hdata/hdfs/hdfs_system,修改mapred-site.xml配置文件。


[hadoop@master bin]$ hadoop fs -mkdir /hdata/hdfs/hdfs_system[hadoop@master bin]$ vi ../conf/mapred-site.xml <?xml version="1.0"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!-- Put site-specific property overrides in this file. --><configuration><property>        <name>mapred.job.tracker</name>        <value>10.132.30.48:9001</value> </property> <property>       <name>mapred.system.dir</name>        <value>/hdata/hdfs/hdfs_system</value></property></configuration>


重启hadoop,发现报如下错误.
org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot set permission for /hdata/hdfs/hdfs_system. Name node is in safe mode.The reported blocks 0 needs additional 94 blocks to reach the threshold 0.9990 of total blocks 95. Safe mode will be turned off automatically.at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setPermission(FSNamesystem.java:856)at org.apache.hadoop.hdfs.server.namenode.NameNode.setPermission(NameNode.java:739)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:601)at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1439)at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1435)at java.security.AccessController.doPrivileged(Native Method)at javax.security.auth.Subject.doAs(Subject.java:415)at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278)at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1433)



再次查找资料发下有如下解决方法的:
[hadoop@master bin]$ hadoop fs -chown hadoop:hadoop /hdata/hdfs/hdfs_system


扩展理解本地目录和HDFS文件目录:
1.hdfs-site.xml文件中的dfs.name.dir为元数据存放目录,应当设置为本地目录


2.mapred-site.xml文件中mapred.local.dir应当设置为本地目录,
[hadoop@master bin]$ mkdir /hdata/hdfs/local_system[hadoop@master bin]$ ll /hdata/hdfs/总计 12drwxr-xr-x 2 hadoop hadoop 4096 06-14 09:36 datadrwxr-xr-x 5 hadoop hadoop 4096 07-05 17:07 namedrwxr-xr-x 4 hadoop hadoop 4096 07-03 11:52 tmp



3.core-site.xml文件中的hadoop.tmp.dir为临时文件(/hdata/hdfs/tmp)存放目录,mapreduce在执行一个task时,需要将task的JAR包
和配置文件等下载到本地,然后才能执行,而提交job时,只是将JAR等上传到了HDFS上。


4.mapred.system.dir 这个配置参数值为mapreduce共享目录,不能为本地目录,只能是hdfs目录。
原创粉丝点击