【伊利丹】Hadoop-2.5.0-CDH5.2.0 RM HA实验记录

来源：互联网发布：如何自学java 编辑：程序博客网时间：2024/05/01 21:30

引言

以前写了一个关于Hadoop2.0 NN HA实验记录博客，其中记录了hadoop中NameNode出现故障时可以自动切换主备节点，现在hadoop2.5.0版本中实现了RescoureMangager可以自动切换功能，因此下面写的博客是关于 Hadoop RM HA 。

正文

四个节点：SY-0217 SY-0355 SY-0225 SY-0226

SY-0217 : NN RM（主）

SY-0355：NN RM （备）

SY-0225,SY-0226：DN NM （数据节点）

第一步：需要准备好关于hadoop相关的安装包

例如有如下的安装包，以我的集群为例：

Hadoop：hadoop-2.5.0-cdh5.2.0.tar.gz

Zookeeper：zookeeper-1.4.5-cdh5.2.0.tar.gz

Jdk：jdk-7u71-linux-i586.tar.gz （CDH5不再支持JDK1.6，必须按照JDK1.7.（1.7.0_45）且只有64位包，无32位包）

假如你需要关于Hadoop CDH版本的安装包，其中包括 Hbase，hive，sqoop，mathout等安装包，你可以去下面地址去下载：

CDH5：http://archive.cloudera.com/cdh5/cdh/5/

CDH4：http://archive.cloudera.com/cdh4/cdh/4/

如果有问题，你也可去CDH官网去查找问题，因为上面又比较完善的官网文档做参考。

第二步：安装Hadoop CDH版本

把安装包上传到服务器或者虚拟机上面，并解压。

这里我是采用以前安装版本hadoop2.2.0的虚拟机，并且主备NameNode出现故障时可以自动切换，在上面解压后，并修改环境变量就OK了（修改环境变量就再重复了）。

如果你要从零开始的话，你需要去下面链接博客去学习搭建新的集群：Apache Hadoop 2.2.0 HDFS HA +YARN多机部署

博客地址：

http://blog.csdn.net/u010967382/article/details/20380387

四台虚拟机都要安装，并且有奇数个安装zookeeper的，我这里是3个（SY-0217 SY-0355 SY-0225）,其中的配置，可以参考我的博客：Hadoop2.0 NN HA实验记录

博客地址：

http://blog.csdn.net/fulongfbi/article/details/28418393

到这里准备工作都做好了，下面开始配置hadoop文件。

第三步：配置$HADOOP_HOME/etc/hadoop/core-stie.xml文件

按照如下配置（和我以前Hadoop NN HA 中的文件配置一致）：

<name>fs.defaultFS</name>

<value>hdfs://fulonghadoop</value>

</property>

<name>hadoop.tmp.dir</name>

<value>/home/kkzhangtao/hadoopcdh/hdfs/tmp</value>

</property>

<name>ha.zookeeper.quorum</name>

</property>

<value>org.apache.hadoop.hdfs.DistributedFileSystem</value>

<description>The FileSystem for hdfs: uris.</description>

</property>

</configuration>

第四步：配置$HADOOP_HOME/etc/hadoop/hdfs-stie.xml文件

按照如下配置（和我以前Hadoop NN HA 中的文件配置一致, 但是需要修改标记的地方）：

<name>dfs.nameservices</name>

<value>fulonghadoop</value>

Comma-separated list of nameservices.

as same as fs.defaultFS in core-site.xml.

</description>

</property>

<name>dfs.ha.namenodes.fulonghadoop</name>

The prefix for a given nameservice, contains a comma-separated

list of namenodes for a given nameservice (eg EXAMPLENAMESERVICE).

</description>

</property>

<name>dfs.namenode.rpc-address.fulonghadoop.nn1</name>

RPC address for nomenode1 of hadoop-test

</description>

</property>

<name>dfs.namenode.rpc-address.fulonghadoop.nn2</name>

RPC address for nomenode2 of hadoop-test

</description>

</property>

<name>dfs.namenode.http-address.fulonghadoop.nn1</name>

The address and the base port where the dfs namenode1 web ui will listen on.

</description>

</property>

<name>dfs.namenode.http-address.fulonghadoop.nn2</name>

The address and the base port where the dfs namenode2 web ui will listen on.

</description>

</property>

<name>dfs.namenode.servicerpc-address.fulonghadoop.n1</name>

</property>

<name>dfs.namenode.servicerpc-address.fulonghadoop.n2</name>

</property>

<name>dfs.namenode.name.dir</name>

<value>file:///home/kkzhangtao/hadoop-2.5.0-cdh5.2.0/hdfs/name</value>

<description>Determines where on the local filesystem the DFS name node

should store the name table(fsimage). If this is a comma-delimited list

of directories then the name table is replicated in all of the

directories, for redundancy. </description>

</property>

<name>dfs.namenode.shared.edits.dir</name>

<value>qjournal://SY-0355:8485;SY-0225:8485;SY-0226:8485/hadoop-test</value>

<description>A directory on shared storage between the multiple namenodes

in an HA cluster. This directory will be written by the active and read

by the standby in order to keep the namespaces synchronized. This directory

does not need to be listed in dfs.namenode.edits.dir above. It should be

left empty in a non-HA cluster.

</description>

</property>

<name>dfs.datanode.data.dir</name>

<value>file:///home/kkzhangtao/hadoop-2.5.0-cdh5.2.0/hdfs/data</value>

<description>Determines where on the local filesystem an DFS data node

should store its blocks. If this is a comma-delimited

list of directories, then data will be stored in all named

directories, typically on different devices.

Directories that do not exist are ignored.

</description>

</property>

<name>dfs.ha.automatic-failover.enabled</name>

Whether automatic failover is enabled. See the HDFS High

Availability documentation for details on automatic HA

configuration.

</description>

</property>

<name>dfs.journalnode.edits.dir</name>

<value>/home/kkzhangtao/hadoop-2.5.0-cdh5.2.0/hdfs/journal/</value>

</property>

<name>dfs.client.failover.proxy.provider.fulonghadoop</name>

<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>

<description>Configure the name of the Java class which will be used by the DFS Client to determine which NameNode is the current Active, and therefore which NameNode is currently serving client requests.

</description>

</property>

<name>dfs.ha.fencing.methods</name>

<value>sshfence</value>

<description>how to communicate in the switch process</description>

</property>

<name>dfs.ha.fencing.ssh.private-key-files</name>

<value>/home/kkzhangtao/.ssh/id_rsa</value>

<description>the location stored ssh key</description>

</property>

<name>dfs.ha.fencing.ssh.connect-timeout</name>

</property>

<name>dfs.namenode.handler.count</name>

</property>

</configuration>

第五步：配置$HADOOP_HOME/etc/hadoop/mapred-stie.xml文件

按照如下配置（和我以前Hadoop NN HA 中的文件配置一致）：

<name>mapreduce.framework.name</name>

<description>The runtime framework for executing MapReduce jobs.

Can be one of local, classic or yarn.

</description>

</property>

<name>mapreduce.jobhistory.address</name>

<description>MapReduce JobHistory Server IPC host:port</description>

</property>

<name>mapreduce.jobhistory.webapp.address</name>

<description>MapReduce JobHistory Server Web UI host:port</description>

</property>

</configuration>

第六步：配置$HADOOP_HOME/etc/hadoop/yarn-stie.xml文件

按照如下配置（需要重新配置）：

<name>yarn.nodemanager.aux-services</name>

<value>mapreduce_shuffle</value>

</property>

<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>

<value>org.apache.hadoop.mapred.ShuffleHandler</value>

</property>

<description>Address where the localizer IPC is.</description>

<name>yarn.nodemanager.localizer.address</name>

</property>

<description>NM Webapp address.</description>

<name>yarn.nodemanager.webapp.address</name>

</property>

<name>yarn.nodemanager.local-dirs</name>

<value>/home/kkzhangtao/RMHA/nodemanager/yarn/local</value>

</property>

<name>yarn.nodemanager.log-dirs</name>

<value>/home/kkzhangtao/RMHA/nodemanager/yarn/logs</value>

</property>

<name>mapreduce.shuffle.port</name>

</property>

<name>yarn.resourcemanager.connect.retry-interval.ms</name>

</property>

<name>yarn.resourcemanager.ha.enabled</name>

</property>

<name>yarn.resourcemanager.ha.automatic-failover.enabled</name>

</property>

<name>yarn.resourcemanager.ha.automatic-failover.embedded</name>

</property>

<name>yarn.resourcemanager.cluster-id</name>

<value>yarn-cluster</value>

</property>

<name>yarn.resourcemanager.ha.rm-ids</name>

</property>

<name>yarn.resourcemanager.ha.id</name>

</property>

<name>yarn.resourcemanager.scheduler.class</name>

<value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</value>

</property>

<name>yarn.resourcemanager.recovery.enabled</name>

</property>

<name>yarn.app.mapreduce.am.scheduler.connection.wait.interval-ms</name>

</property>

<name>yarn.resourcemanager.store.class</name>

<value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore</value>

</property>

<property><name>yarn.resourcemanager.zk-address</name>

</property>

<name>yarn.resourcemanager.zk.state-store.address</name>

</property>

<name>yarn.resourcemanager.address.rm1</name>

</property>

<name>yarn.resourcemanager.address.rm2</name>

</property>

<name>yarn.resourcemanager.scheduler.address.rm1</name>

</property>

<name>yarn.resourcemanager.scheduler.address.rm2</name>

</property>

<name>yarn.resourcemanager.admin.address.rm1</name>

</property>

<name>yarn.resourcemanager.admin.address.rm2</name>

</property>

<name>yarn.resourcemanager.resource-tracker.address.rm1</name>

</property>

<name>yarn.resourcemanager.resource-tracker.address.rm2</name>

</property>

<name>yarn.resourcemanager.webapp.address.rm1</name>

</property>

<name>yarn.resourcemanager.webapp.address.rm2</name>

</property>

<name>yarn.resourcemanager.webapp.https.address.rm1</name>

</property>

<name>yarn.resourcemanager.webapp.https.address.rm2</name>

</property>

</configuration>

第七步：配置$HADOOP_HOME/etc/hadoop/slaves,hadoop-env.sh,farscheduler.xml文件

slaves:

SY-0226

SY-0225

hadoop-env.sh:

export JAVA_HOME=/home/kkzhangtao/jdk1.7.0_71

farscheduler.xml:

本集群中的配置文件拷贝以前的Hadoop2.2.0的你也可以自己配置。

第八步：开始测试

下面是我按照测试需求写的启动和关闭脚本：

Startup.sh

#!/bin/bash

echo "-------------------------Step1----------------------------"

echo "Now,starting Zookeeper."

ssh SY-0217 $ZOOKEEPER_HOME/bin/zkServer.sh start

ssh SY-0355 $ZOOKEEPER_HOME/bin/zkServer.sh start

ssh SY-0225 $ZOOKEEPER_HOME/bin/zkServer.sh start

echo "Now,Zookeeper is started."

echo "-------------------------Step2----------------------------"

echo "Now,starting ZKFC.

ssh SY-0217 $HADOOP_HOME/bin/hdfs zkfc -formatZK"

ssh SY-0217 $HADOOP_HOME/sbin/hadoop-daemon.sh start zkfc

ssh SY-0355 $HADOOP_HOME/sbin/hadoop-daemon.sh start zkfc

echo "Now,ZKFC started."

echo "-------------------------Step3----------------------------"

echo "Now,starting JN."

ssh SY-0355 $HADOOP_HOME/sbin/hadoop-daemon.sh start journalnode

ssh SY-0225 $HADOOP_HOME/sbin/hadoop-daemon.sh start journalnode

ssh SY-0226 $HADOOP_HOME/sbin/hadoop-daemon.sh start journalnode

echo "Now,JN started."

echo "-------------------------Step4----------------------------"

echo "Now,starting Active,Standby NN."

ssh SY-0217 $HADOOP_HOME/sbin/hadoop-daemon.sh start namenode

ssh SY-0355 $HADOOP_HOME/sbin/hadoop-daemon.sh start namenode

echo "Now,Active,Standby NN started."

echo "-------------------------Step5----------------------------"

echo "Now,starting DataNodes."

ssh SY-0217 $HADOOP_HOME/sbin/hadoop-daemons.sh start datanode

echo "Now,DataNodes started."

echo "-------------------------Step6----------------------------"

echo "Now,starting Active,Standby RM."

ssh SY-0217 $HADOOP_HOME/sbin/yarn-daemon.sh start resourcemanager

ssh SY-0355 $HADOOP_HOME/sbin/yarn-daemon.sh start resourcemanager

echo "Now,Active,Standby RM started."

echo "-------------------------Step7----------------------------"

echo "Now,starting Nodemanagers."

ssh SY-0217 $HADOOP_HOME/sbin/yarn-daemons.sh start nodemanager

echo "Now,Nodemanagers started."

echo "-------------------------Step8----------------------------"

echo "Now,starting MR JobHistory Server."

ssh SY-0355 $HADOOP_HOME/sbin/mr-jobhistory-daemon.sh start historyserver

echo "Now,MR JobHistory Server started."

StopAll.sh

#!/bin/bash

echo "-------------------------Step1----------------------------"

echo "Now,stop JN."

ssh SY-0355 $HADOOP_HOME/sbin/hadoop-daemon.sh stop journalnode

ssh SY-0225 $HADOOP_HOME/sbin/hadoop-daemon.sh stop journalnode

ssh SY-0226 $HADOOP_HOME/sbin/hadoop-daemon.sh stop journalnode

echo "Now,JN stoped."

echo "-------------------------Step2----------------------------"

echo "Now,stoping Active,Standby NN."

ssh SY-0217 $HADOOP_HOME/sbin/hadoop-daemon.sh stop namenode

ssh SY-0355 $HADOOP_HOME/sbin/hadoop-daemon.sh stop namenode

echo "Now,Active,Standby NN stoped."

echo "-------------------------Step3----------------------------"

echo "Now,stop DataNodes."

ssh SY-0217 $HADOOP_HOME/sbin/hadoop-daemons.sh stop datanode

echo "Now,DataNodes stoped."

echo "-------------------------Step4----------------------------"

echo "Now,stop Active,Standby RM."

ssh SY-0217 $HADOOP_HOME/sbin/yarn-daemon.sh stop resourcemanager

ssh SY-0355 $HADOOP_HOME/sbin/yarn-daemon.sh stop resourcemanager

echo "Now,Active,Standby RM stoped."

echo "-------------------------Step5----------------------------"

echo "Now,stop NodeManagers."

ssh SY-0217 $HADOOP_HOME/sbin/yarn-daemons.sh stop nodemanager

echo "Now,NodeManagers stoped."

echo "-------------------------Step6----------------------------"

echo "Now,stop MR JobHistory Server."

ssh SY-0355 $HADOOP_HOME/sbin/mr-jobhistory-daemon.sh stop historyserver

echo "Now,MR stop Server stoped."

第九步：测试结果截图

1.通过在终端采用JSP命令查看，或者通过访问页面来查看。

采用JSP命令查看SY-0217和SY-0355截图：

采用访问页面查看RescourceManager的截图：( SY-0217:23188 这里端口号是自己修改的在yarn-site.xml配置中)

SY-0217访问页面：

SY-0355访问页面：

在hadoop集群中采用如下命令来运行一个MapReduce程序（我这里是自己写的一个MapReduce程序，并打好Jar包上传到HDFS上面，同时数据也准备好了）。

在HADOOP_HOME目录下面执行： hadoop jar /home/kkzhangtao/Videos/WordCount.jar WordCount /input /output

你也可跑hadoop自带的测试程序： hadoop jar share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.5.0-cdh5.2.0.jar pi 2 2

通过 Kill -9 14671 命令来杀死主RescourceManager。

再通过访问监控页面 SY-0355:23188 来查看任务进度，最后并完成，效果截图如下：

SY-0217访问页面：

SY-0355访问页面：

在运行的集群中跑个mapreduce程序来测试RM的切换时候保证任务不会中断，并能得到执行结果。

到此RescourceManager主备切换已经成功完成。

第十步：实验总结和目的

本实验的目的主要是来测试hadoop cdh （5.2.0）版本中的RescourceManager的主备切换。次要目的是了解CDH的版本的好处，以后变用于开发中去。

同时这里也涉及到了Hadoop 版本升级的问题，因此下片博客将会写到hadoop 版本之间升级问题和解决hadoop集群因增加节点导致的数据部均衡的问题。

如果有需要请关注............谢谢

0 0