hadoop2.6.0删除集群节点（三）

一、修改master节点hdfs-site.xml，增加dfs.hosts.exclude参数

[hadoop@master hadoop]$ vi hdfs-site.xml

        <property>
                 <name>dfs.hosts.exclude</name>
                  <value>/opt/hadoop/etc/hadoop/excludes</value>
         </property>
 

二、修改master节点的yarn-site.xml,增加yarn.resourcemanager.nodes.exclude-path参数

[hadoop@master hadoop]$ vi yarn-site.xml
        <property>
                 <name>yarn.resourcemanager.nodes.exclude-path</name>
                  <value>/opt/hadoop/etc/hadoop/excludes</value>
         </property>

三、修改master节点的mapred-site.xml，增加mapreduce.jobtracker.hosts.exclude.filename

[hadoop@master hadoop]$ vi mapred-site.xml
        <property>
                 <name>mapreduce.jobtracker.hosts.exclude.filename</name>
                  <value>/opt/hadoop/etc/hadoop/excludes</value>
         </property>

四、 新建excludes文件，添加需要删除的主机名

[hadoop@master hadoop]$ vi excludes
hadoop04 

五、执行refreshNodes使配置生效

[hadoop@master hadoop]$ hdfs dfsadmin -refreshNodes
Refresh nodes successful

[hadoop@master hadoop]$ yarn rmadmin -refreshNodes
17/03/17 03:15:01 INFO client.RMProxy: Connecting to ResourceManager at master/192.168.123.10:8033

六、检查集群状态

[hadoop@master hadoop]$ hdfs dfsadmin -report
Configured Capacity: 50395512832 (46.93 GB)
Present Capacity: 36941504512 (34.40 GB)
DFS Remaining: 36941332480 (34.40 GB)
DFS Used: 172032 (168 KB)
DFS Used%: 0.00%
Under replicated blocks: 2
Blocks with corrupt replicas: 0
Missing blocks: 0


-------------------------------------------------
Live datanodes (3):


Name: 192.168.123.11:50010 (slave1)
Hostname: slave1
Decommission Status : Normal
Configured Capacity: 25197727744 (23.47 GB)
DFS Used: 57344 (56 KB)
Non DFS Used: 6726893568 (6.26 GB)
DFS Remaining: 18470776832 (17.20 GB)
DFS Used%: 0.00%
DFS Remaining%: 73.30%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Fri Mar 17 03:15:55 EDT 2017




Name: 192.168.123.13:50010 (hadoop04)
Hostname: hadoop04
Decommission Status : Decommission in progress        《======当前状态为Decommission 
Configured Capacity: 25197727744 (23.47 GB)
DFS Used: 57344 (56 KB)
Non DFS Used: 6727172096 (6.27 GB)
DFS Remaining: 18470498304 (17.20 GB)
DFS Used%: 0.00%
DFS Remaining%: 73.30%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Fri Mar 17 03:15:55 EDT 2017




Name: 192.168.123.12:50010 (slave2)
Hostname: slave2
Decommission Status : Normal
Configured Capacity: 25197727744 (23.47 GB)
DFS Used: 57344 (56 KB)
Non DFS Used: 6727114752 (6.27 GB)
DFS Remaining: 18470555648 (17.20 GB)
DFS Used%: 0.00%
DFS Remaining%: 73.30%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Fri Mar 17 03:15:55 EDT 2017




Decommissioning datanodes (1):


Name: 192.168.123.13:50010 (hadoop04)
Hostname: hadoop04
Decommission Status : Decommission in progress
Configured Capacity: 25197727744 (23.47 GB)
DFS Used: 57344 (56 KB)
Non DFS Used: 6727172096 (6.27 GB)
DFS Remaining: 18470498304 (17.20 GB)
DFS Used%: 0.00%
DFS Remaining%: 73.30%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Fri Mar 17 03:15:55 EDT 2017

七、过一段时间重新检查集群状态

[hadoop@master ~]$ hdfs dfsadmin -report
Configured Capacity: 50395512832 (46.93 GB)
Present Capacity: 36941496320 (34.40 GB)
DFS Remaining: 36941324288 (34.40 GB)
DFS Used: 172032 (168 KB)
DFS Used%: 0.00%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0


-------------------------------------------------
Live datanodes (3):


Name: 192.168.123.11:50010 (slave1)
Hostname: slave1
Decommission Status : Normal
Configured Capacity: 25197727744 (23.47 GB)
DFS Used: 57344 (56 KB)
Non DFS Used: 6726897664 (6.26 GB)
DFS Remaining: 18470772736 (17.20 GB)
DFS Used%: 0.00%
DFS Remaining%: 73.30%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Fri Mar 17 03:39:35 EDT 2017




Name: 192.168.123.13:50010 (hadoop04)
Hostname: hadoop04
Decommission Status : Decommissioned                《=====说明节点已经剔除
Configured Capacity: 25197727744 (23.47 GB)
DFS Used: 57344 (56 KB)
Non DFS Used: 6727401472 (6.27 GB)
DFS Remaining: 18470268928 (17.20 GB)
DFS Used%: 0.00%
DFS Remaining%: 73.30%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Fri Mar 17 03:39:35 EDT 2017




Name: 192.168.123.12:50010 (slave2)
Hostname: slave2
Decommission Status : Normal
Configured Capacity: 25197727744 (23.47 GB)
DFS Used: 57344 (56 KB)
Non DFS Used: 6727118848 (6.27 GB)
DFS Remaining: 18470551552 (17.20 GB)
DFS Used%: 0.00%
DFS Remaining%: 73.30%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Fri Mar 17 03:39:35 EDT 2017

八、hadoop04节点jps进程检查

[hadoop@hadoop04 ~]$ jps
3507 DataNode        《====进程还是存在
3969 Jps

我们可以手工停掉该进程

[hadoop@hadoop04 ~]$ hadoop-daemon.sh stop datanode
stopping datanode
[hadoop@hadoop04 ~]$ jps
4012 Jps
[hadoop@hadoop04 ~]$

九、更新slaves文件和excludes文件将里面的hadoop04记录删除

[hadoop@master hadoop]$ cat slaves 
slave1
slave2
[hadoop@master hadoop]$ cat excludes 
[hadoop@master hadoop]$ 

删除节点完成

在执行hdfs dfsadmin -report时，仍然可以看见hadoop04的信息，hdfs重启后记录不见了

注意，在剔除节点的过程中尽量不要有作业运行，否则集群剔除节点的过程非常缓慢