cdh5.3.10_hadoop环境迁移之主机IP修改

来源：互联网发布：tflearn windows 编辑：程序博客网时间：2024/05/20 21:43

cdh5.3.10_hadoop环境迁移
主机迁移后hadoop集群调整：
1、修改/etc/hosts（三台主机均需要修改）
修改前：
[root@hadoop-master ~]# cat /etc/hosts
127.0.0.1 localhost.localdomain localhost.localdomain localhost4 localhost4.localdomain4 localhost
::1 localhost.localdomain localhost.localdomain localhost6 localhost6.localdomain6 localhost

10.117.130.207 hadoop-master
10.117.130.208 hadoop-slave1
10.117.130.209 hadoop-slave2
[root@hadoop-master ~]#
修改后：
[root@hadoop-master ~]# cat /etc/hosts
127.0.0.1 localhost.localdomain localhost.localdomain localhost4 localhost4.localdomain4 localhost
::1 localhost.localdomain localhost.localdomain localhost6 localhost6.localdomain6 localhost

10.117.130.110 hadoop-master
10.117.130.111 hadoop-slave1
10.117.130.112 hadoop-slave2
[root@hadoop-master ~]#

2、修改ntp时钟同步（三台主机均需要修改）
修改前：
[root@hadoop-master ~]# cat /etc/ntp.conf
server 10.117.130.207 prefer
修改后：
[root@hadoop-master ~]# cat /etc/ntp.conf
server 10.117.130.110 prefer

3、修改mysql中hadoop相关配置（仅在master节点修改）
修改前：
[root@hadoop-master data]# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 58
Server version: 5.6.29 MySQL Community Server (GPL)

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> mysql> select * from HOSTS;
+---------+-------------------------+--------------------------------------+---------------+----------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+
| HOST_ID | OPTIMISTIC_LOCK_VERSION | HOST_IDENTIFIER | NAME | IP_ADDRESS | RACK_ID | STATUS | CONFIG_CONTAINER_ID | MAINTENANCE_COUNT | DECOMMISSION_COUNT | CLUSTER_ID | NUM_CORES | TOTAL_PHYS_MEM_BYTES | PUBLIC_NAME | PUBLIC_IP_ADDRESS | CLOUD_PROVIDER |
+---------+-------------------------+--------------------------------------+---------------+----------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+
| 1 | 14 | 135caa69-b742-467b-bf08-6742781a1b2c | hadoop-master | 10.117.130.207 | /default | NA | 1 | 0 | 0 | 1 | 4 | 12626055168 | NULL | NULL | NULL |
| 2 | 13 | de481baa-e125-4c35-8538-aa3bf7229f33 | hadoop-slave1 | 10.117.130.208| /default | NA | 1 | 0 | 0 | 1 | 4 | 12626055168 | NULL | NULL | NULL |
| 3 | 11 | 6af8fa7f-1ce1-47cd-a1a6-6d8ef91c77f2 | hadoop-slave2 | 10.117.130.209 | /default | NA | 1 | 0 | 0 | 1 | 4 | 12626055168 | NULL | NULL | NULL |
+---------+-------------------------+--------------------------------------+---------------+----------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+
3 rows in set (0.00 sec)
mysql> update HOSTS set ip_address= '10.117.130.110' where host_id=1;
Query OK, 1 row affected (0.00 sec)
mysql> update HOSTS set ip_address= '10.117.130.111' where host_id=2;
Query OK, 1 row affected (0.00 sec)
mysql> update HOSTS set ip_address= '10.117.130.112' where host_id=3;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0

mysql> select * from HOSTS;
+---------+-------------------------+--------------------------------------+---------------+----------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+
| HOST_ID | OPTIMISTIC_LOCK_VERSION | HOST_IDENTIFIER | NAME | IP_ADDRESS | RACK_ID | STATUS | CONFIG_CONTAINER_ID | MAINTENANCE_COUNT | DECOMMISSION_COUNT | CLUSTER_ID | NUM_CORES | TOTAL_PHYS_MEM_BYTES | PUBLIC_NAME | PUBLIC_IP_ADDRESS | CLOUD_PROVIDER |
+---------+-------------------------+--------------------------------------+---------------+----------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+
| 1 | 14 | 135caa69-b742-467b-bf08-6742781a1b2c | hadoop-master | 10.117.130.110 | /default | NA | 1 | 0 | 0 | 1 | 4 | 12626055168 | NULL | NULL | NULL |
| 2 | 13 | de481baa-e125-4c35-8538-aa3bf7229f33 | hadoop-slave1 | 10.117.130.111 | /default | NA | 1 | 0 | 0 | 1 | 4 | 12626055168 | NULL | NULL | NULL |
| 3 | 11 | 6af8fa7f-1ce1-47cd-a1a6-6d8ef91c77f2 | hadoop-slave2 | 10.117.130.112 | /default | NA | 1 | 0 | 0 | 1 | 4 | 12626055168 | NULL | NULL | NULL |
+---------+-------------------------+--------------------------------------+---------------+----------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+
3 rows in set (0.00 sec)

mysql>

mysql> commit;
Query OK, 0 rows affected (0.00 sec)
mysql>

4、重启cdh-hadoop的服务
[root@hadoop-master data]# /opt/cloudera-manager/cm-5.3.10/etc/init.d/cloudera-scm-server restart
鍋滄 cloudera-scm-server锛 [纭畾]
Starting cloudera-scm-server: [纭畾]
[root@hadoop-master data]# /opt/cloudera-manager/cm-5.3.10/etc/init.d/cloudera-scm-agent restart
Stopping cloudera-scm-agent: [纭畾]
姝ｅ湪鍚姩 cloudera-scm-agent锛 [纭畾]
[root@hadoop-master data]
[root@hadoop-slave1 ~]# /opt/cloudera-manager/cm-5.3.10/etc/init.d/cloudera-scm-agent start
姝ｅ湪鍚姩 cloudera-scm-agent锛 [纭畾]
[root@hadoop-slave1 ~]#
[root@hadoop-slave2 ~]# /opt/cloudera-manager/cm-5.3.10/etc/init.d/cloudera-scm-agent start
姝ｅ湪鍚姩 cloudera-scm-agent锛 [纭畾]
[root@hadoop-slave2 ~]#

5、登录cdh集群web管理页面
http://10.117.130.110:7180/cmf/home
首先保证CDH第一个服务即主机一项是正常的
看到部分服务提示配置信息过旧：需要重新部署客户端配置此时点击链接按照页面提示重新部署客户端配置即可

6、重新部署cdh部分服务的客户端配置后重启cdh服务，cdh恢复正常

注意：由于cdh三台虚拟主机是镜像过来的，cdh相关服务文件或进程有状态保留，需要查杀相关进程
本次处理中，碰到节点2重启agent客户端进程一直在运行，问题出在9000端口相应的进程占用，导致cdh识别节点2出现问题
[root@hadoop-slave1 ~]# /opt/cloudera-manager/cm-5.3.10/etc/init.d/cloudera-scm-agent start
cloudera-scm-agent is already running
[root@hadoop-slave1 ~]# netstat -nap | grep 9000
tcp 0 0 10.117.130.111:9000 0.0.0.0:* LISTEN 12510/python
tcp 0 0 10.117.130.111:44223 10.117.130.111:9000 TIME_WAIT -
[root@hadoop-slave1 ~]# kill -9 0
Connection to hadoop-slave1 closed.
[root@hadoop-slave2 ~]# netstat -nap | grep 9000
[root@hadoop-slave2 ~]# /opt/cloudera-manager/cm-5.3.10/etc/init.d/cloudera-scm-agent start
姝ｅ湪鍚姩 cloudera-scm-agent锛 [纭畾]
[root@hadoop-slave2 ~]# netstat -nap | grep 9000
tcp 0 0 10.117.130.112:9000 0.0.0.0:* LISTEN 3689/python
tcp 0 0 10.117.130.112:50759 10.117.130.112:9000 TIME_WAIT -
[root@hadoop-slave2 ~]#

0 0