YARN的HA搭建
来源:互联网 发布:手机制作软件视频教程 编辑:程序博客网 时间:2024/04/26 10:29
YARN的HA搭建
选择192.168.230.10(node1),192.168.230.14(node5)两台主机作为ResourceManagerdatanode与nodemanager一定要放在一个机器上
YARN的体系架构
1、配置/opt/modules/hadoop-2.5.1/etc/hadoop/mapred-site.xml:
<configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property></configuration>
2、配置/opt/modules/hadoop-2.5.1/etc/hadoop/yarn-site.xml:
<configuration> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property></configuration>
<!-- 启用 --> <property> <name>yarn.resourcemanager.ha.enabled</name> <value>true</value> </property> <!-- 集群ID号 --> <property> <name>yarn.resourcemanager.cluster-id</name> <value>cluster1</value> </property> <property> <name>yarn.resourcemanager.ha.rm-ids</name> <value>rm1,rm2</value> </property> <property> <name>yarn.resourcemanager.hostname.rm1</name> <value>node1</value> </property> <property> <name>yarn.resourcemanager.hostname.rm2</name> <value>node5</value> </property> <property> <name>yarn.resourcemanager.zk-address</name> <value>node2:2181,node3:2181,node5:2181</value> </property>
3、将在node1主机上配置好的yarn-site.xml,mapred-site.xml文件
scp /opt/modules/hadoop-2.5.1/etc/hadoop/yarn-site.xml root@node2:/opt/modules/hadoop-2.5.1/etc/hadoop/scp /opt/modules/hadoop-2.5.1/etc/hadoop/mapred-site.xml root@node2:/opt/modules/hadoop-2.5.1/etc/hadoop/scp /opt/modules/hadoop-2.5.1/etc/hadoop/yarn-site.xml root@node3:/opt/modules/hadoop-2.5.1/etc/hadoop/scp /opt/modules/hadoop-2.5.1/etc/hadoop/mapred-site.xml root@node3:/opt/modules/hadoop-2.5.1/etc/hadoop/scp /opt/modules/hadoop-2.5.1/etc/hadoop/yarn-site.xml root@node5:/opt/modules/hadoop-2.5.1/etc/hadoop/scp /opt/modules/hadoop-2.5.1/etc/hadoop/mapred-site.xml root@node5:/opt/modules/hadoop-2.5.1/etc/hadoop/
4、在node1主机上启动yarn
[root@node1 hadoop-2.5.1]# ./sbin/start-yarn.sh
[root@node1 hadoop-2.5.1]# jps
5、启动单节点ResourceManager
[root@node5 hadoop-2.5.1]# ./sbin/yarn-daemon.sh start resourcemanager
[root@node5 hadoop-2.5.1]# jps
6、在浏览器地址栏上访问
http://node1:8088/cluster/nodes
http://node5:8088/cluster/nodes,页面会直接跳转到node1的页面
6.1、杀死node1主机的resourcemanager进程,http://node1:8088/cluster访问不成功
[root@node1 hadoop-2.5.1]# kill -9 8145
6.2、查看http://node5:8088/cluster发现node5节点的ResourceManager成为Active
6.3、在node1主机重新单节点启动ResourceManager,发现node1的ResourceManager变成了Standby
[root@node1 hadoop-2.5.1]# ./sbin/yarn-daemon.sh start resourcemanager
0 0
- YARN的HA搭建
- yarn的HA搭建
- 搭建hadoop2.6.0 HA及YARN HA
- 搭建hadoop2.6.0 HA及YARN HA
- 搭建hadoop2.6.0 HA及YARN HA
- 搭建hadoop2.6.0 HA及YARN HA
- Hadoop2+HA+YARN环境搭建
- 搭建hadoop2.6.0 HA及YARN HA (很不错的博文)
- Hadoop2.2.0集群搭建(HA,联邦,yarn)
- hadoop集群搭建HDFS、HA、 YARN
- hadoop集群搭建HDFS、HA、 YARN
- hadoop2.7.3 HA YARN 环境搭建
- hadoop2.x新特性HDFS HA、YARN HA,分布式搭建
- 搭建hadoop2.6.0 HDFS HA及YARN HA
- yarn ha
- Yarn HA
- 配置hadoop2.X的namenode HA及Yarn HA
- Hadoop2的automatic HA+Federation+Yarn
- 移动前端开发之viewport的深入理解
- hadoop Shell命令详解
- java IO之字节流和字符流比较
- 字符串模式匹配算法(JAVA实现)
- BP
- YARN的HA搭建
- willMoveToParentViewController和didMoveToParentViewController
- 剖析淘宝TDDL
- XmlPullParser.nextText() bug 好文章,使用pull解析xml注意
- 1.单例模式
- hadoop无法关闭hdfs
- Gradle的简介与安装
- C++中vector容器的基本用法总结
- [LeetCode]49. Group Anagrams