DATANODE目录挂载情况:dfs_data_dir_mount.hist

来源:互联网 发布:人员信息管理系统源码 编辑:程序博客网 时间:2024/06/16 13:24

在企业级的hadoop集群中,数据节点的数据一般都是有专门的数据硬盘进行存储,有的时候可能因为配置问题,

数据的目录个数比硬盘挂载个数要多,目录挂载情况文件:dfs_data_dir_mount.hist


例如:你挂载有10个硬盘,

但是存在目录有 :

/hdfs/data1/hadoop/hdfs/data,

 .... ,

/hdfs/data9/hadoop/hdfs/data,

/hdfs/data10/hadoop/hdfs/data,

/hdfs/data11/hadoop/hdfs/data,

......

/hdfs/data6/hadoop/hdfs/data,


这种情况的话,你的/hdfs/data{1..10}/hadoop/data都是正常挂载在对应的硬盘上,而多出的/hdfs/data{11..16}就会默认挂载到 根目录( / ) 下,

一般来说,hadoop集群的数据量都会比较大,这样就会很容易撑爆DATANODE的根目录(/),一旦根目录的使用达到98% 或者撑爆,数据节

点就会反应慢甚至宕机,从而影响集群的正常运行。


dfs_data_dir_mount.hist 文件记录了数据目录的挂载情况。dfs_data_dir_mount.hist所在的目录针对不同的集群在不同的位置,可以使用

搜索命令 find / -name dfs_data_dir_mount.hist 查找一下文件的位置.

我的集群的情况

 #sudo cat /var/lib/ambari-agent/data/datanode/dfs_data_dir_mount.hist



# This file keeps track of the last known mount-point for each dir.
# It is safe to delete, since it will get regenerated the next time that the component of the service starts. #该文件可以安全删除,不影响节点运行
# However, it is not advised to delete this file since Ambari may
# re-create a dir that used to be mounted on a drive but is now mounted on the root.
# Comments begin with a hash (#) symbol
# dir,mount_point

/hdfs/data12/hadoop/hdfs/data,/
/hdfs/data17/hadoop/hdfs/data,/
/hdfs/data8/hadoop/hdfs/data,/
/hdfs/data11/hadoop/hdfs/data,/
/hdfs/data22/hadoop/hdfs/data,/
/hdfs/data13/hadoop/hdfs/data,/
/hdfs/data2/hadoop/hdfs/data,/hdfs/data2
/hdfs/data7/hadoop/hdfs/data,/hdfs/data7
/hdfs/data10/hadoop/hdfs/data,/
/hdfs/data4/hadoop/hdfs/data,/hdfs/data4
/hdfs/data6/hadoop/hdfs/data,/hdfs/data6
/hdfs/data5/hadoop/hdfs/data,/hdfs/data5
/hdfs/data16/hadoop/hdfs/data,/
/hdfs/data14/hadoop/hdfs/data,/
/hdfs/data15/hadoop/hdfs/data,/
/hdfs/data21/hadoop/hdfs/data,/
/hdfs/data19/hadoop/hdfs/data,/
/hdfs/data20/hadoop/hdfs/data,/
/hdfs/data1/hadoop/hdfs/data,/hdfs/data1
/hdfs/data18/hadoop/hdfs/data,/
/hdfs/data9/hadoop/hdfs/data,/
/hdfs/data3/hadoop/hdfs/data,/hdfs/data3



从我的dfs_data_dir_mount.hist文件可以看出,/hdfs/data{11..22}都挂载到了根目录下。从而很多的集群数据被放在了根目录下,根目录的使用率达到了98%。


解决方案:

1.进入到ambari管理界面。

2.点击HDFS --> 配置 --> 管理配置组

3.新建一个组:组名为 机器厂商+系列+硬盘数 ,例如 DELL-X3650-14DISK

4.配置DELL-X3650-14DISK的目录个数:/hdfs/data1/hadoop/hdfs/data,......,/hdfs/data10/hadoop/hdfs/data

5.删除文件dfs_data_dir_mount.hist,重启DATANODE。

6.查看dfs_data_dir_mount.hist文件:

# This file keeps track of the last known mount-point for each dir.
# It is safe to delete, since it will get regenerated the next time that the component of the service starts. #该文件可以安全删除,不影响节点运行
# However, it is not advised to delete this file since Ambari may
# re-create a dir that used to be mounted on a drive but is now mounted on the root.
# Comments begin with a hash (#) symbol
# dir,mount_point

/hdfs/data8/hadoop/hdfs/data,/hdfs/data8
/hdfs/data2/hadoop/hdfs/data,/hdfs/data2
/hdfs/data7/hadoop/hdfs/data,/hdfs/datat10
/hdfs/data4/hadoop/hdfs/data,/hdfs/data4
/hdfs/data6/hadoop/hdfs/data,/hdfs/data6
/hdfs/data5/hadoop/hdfs/data,/hdfs/data5
/hdfs/data1/hadoop/hdfs/data,/hdfs/data1
/hdfs/data9/hadoop/hdfs/data,/hdfs/data9
/hdfs/data3/hadoop/hdfs/data,/hdfs/data3
/hdfs/data3/hadoop/hdfs/data,/hdfs/data3

正常情况应该是这样的,问题解决。


原创粉丝点击