Hadoop2搭建可手工配置的HA

-----------------------------
1.搭建手工切换的ha(比hadoop1集群搭建多了journalnode集群)
-----------------------------
namenode：haoop0和hadoop1
datanode：hadoop2、hadoop3、hadoop4
journalnode：haoop0、hadoop1、hadoop2（必须是奇数个节点）
resourcemanager：hadoop0
nodemanager：hadoop2、hadoop3、hadoop4


1.1 配置文件(hadoop-env.sh、core-site.xml、hdfs-site.xml、yarn-site.xml、mapred-site.xml、slaves)
1.1.1 hadoop-env.sh
  export JAVA_HOME=/usr/local/jdk
1.1.2 core-site.xml


<property>
<name>fs.defaultFS</name>
<value>hdfs://cluster1</value>
</property>


<property>
<name>hadoop.tmp.dir</name>
<value>/usr/local/hadoop/tmp</value>
</property>


1.1.3 hdfs-site.xml


<property>
<name>dfs.replication</name>
<value>2</value>
</property>


<property>
<name>dfs.nameservices</name>
<value>cluster1</value>
</property>


<property>
<name>dfs.ha.namenodes.cluster1</name>
<value>hadoop101,hadoop102</value>
</property>


<property>
<name>dfs.namenode.rpc-address.cluster1.hadoop101</name>
<value>hadoop0:9000</value>
</property>


<property>
<name>dfs.namenode.http-address.cluster1.hadoop101</name>
<value>hadoop0:50070</value>
</property>


<property>
<name>dfs.namenode.rpc-address.cluster1.hadoop102</name>
<value>hadoop1:9000</value>
</property>


<property>
<name>dfs.namenode.http-address.cluster1.hadoop102</name>
<value>hadoop1:50070</value>
</property>


<property>
<name>dfs.ha.automatic-failover.enabled.cluster1</name>
<value>false</value>
</property>


<property>
<name>dfs.namenode.shared.edits.dir</name>
<value>qjournal://hadoop0:8485;hadoop1:8485;hadoop2:8485/cluster1</value>
</property>


<property>
<name>dfs.journalnode.edits.dir</name>
<value>/usr/local/hadoop/tmp/journal</value>
</property>


<property>
<name>dfs.ha.fencing.methods</name>
<value>sshfence</value>
</property>


<property>
<name>dfs.ha.fencing.ssh.private-key-files</name>
<value>/root/.ssh/id_rsa</value>
</property>


<property>
<name>dfs.client.failover.proxy.provider.cluster1</name>
<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>


1.1.4 yarn-site.xml


<property>
<name>yarn.resourcemanager.hostname</name>
<value>hadoop0</value>
</property>


<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>


1.1.5 mapred-site.xml


<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>


1.1.6 slaves
hadoop2
hadoop3
hadoop4


1.1.7 把hadoop0上的hadoop文件夹复制到hadoop1、hadoop2、hadoop3、hadoop4节点
  scp -rq  hadoop hadoop1:/usr/local
  scp -rq  hadoop hadoop1:/usr/loca2
  scp -rq  hadoop hadoop1:/usr/loca3
  scp -rq  hadoop hadoop1:/usr/loca4




1.2 启动journalnode集群
  在hadoop0、hadoop1、hadoop2上分别执行hadoop/sbin/hadoop-daemon.sh start journalnode
1.3 格式化namenode、启动namenode
  在hadoop0上执行hadoop/bin/hdfs namenode -format
  在hadoop0上分别执行hadoop/sbin/hadoop-daemon.sh start namenode
  在hadoop1上执行hadoop/bin/hdfs namenode -bootstrapStandby
  在hadoop1上分别执行hadoop/sbin/hadoop-daemon.sh start namenode
  在hadoop0上执行hadoop/bin/hdfs haadmin -failover --forceactive hadoop101 hadoop102
  (通过机制保证只有一个为active，另一个为standby，可以手动进行切换)
1.4 启动datanode
  在hadoop0上分别执行hadoop/sbin/hadoop-daemons.sh start datanode(注意是hadoop-daemons.sh)
1.5 启动resourcemanager和nodemanager
  在hadoop0上执行 hadoop/sbin/start-yarn.sh start resourcemanager

   hadoop/sbin/start-yarn.sh start nodemanager

参考博客：http://www.superwu.cn/2014/02/12/1094/