How-to: Enable yarn ResourceManager HA
来源:互联网 发布:程序员的数学怎么样 编辑:程序博客网 时间:2024/05/16 06:56
- Update yarn-site.xml. Add following in yarn-site.xml(master.chff.dc and slave01.chff.dc are resourcemanager nodes):
<property>
<name>yarn.resourcemanager.ha.enabled</name>
<value>true</value>
</property>
<property>
<name>yarn.resourcemanager.ha.automatic-failover.enabled</name>
<value>true</value>
</property>
<!-- 使嵌入式自动故障转移。HA环境启动,与 ZKRMStateStore 配合 处理fencing -->
<property>
<name>yarn.resourcemanager.ha.automatic-failover.embedded</name>
<value>true</value>
</property>
<!-- 集群名称,确保HA选举时对应的集群 -->
<property>
<name>yarn.resourcemanager.cluster-id</name>
<value>yarn-cluster</value>
</property>
<property>
<name>yarn.resourcemanager.ha.rm-ids</name>
<value>rm1,rm2</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm1</name>
<value>master.chff.dc</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm2</name>
<value>slave01.chff.dc</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.class</name>
<value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</value>
</property>
<property>
<name>yarn.resourcemanager.recovery.enabled</name>
<value>true</value>
</property>
<property>
<name>yarn.app.mapreduce.am.scheduler.connection.wait.interval-ms</name>
<value>5000</value>
</property>
<!-- ZKRMStateStore 配置 -->
<property>
<name>yarn.resourcemanager.store.class</name>
<value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore</value>
</property>
<property>
<name>yarn.resourcemanager.zk-address</name>
<value>master.chff.dc:2183,slave01.chff.dc:2183,slave.chff.dc:2183</value>
</property>
<property>
<name>yarn.resourcemanager.zk.state-store.address</name>
<value>master.chff.dc:2183,slave01.chff.dc:2183,slave.chff.dc:2183</value>
</property>
<!-- Client访问RM的RPC地址 (applications manager interface) -->
<property>
<name>yarn.resourcemanager.address.rm1</name>
<value>master.chff.dc:23140</value>
</property>
<property>
<name>yarn.resourcemanager.address.rm2</name>
<value>slave01.chff.dc:23140</value>
</property>
<!-- AM访问RM的RPC地址(scheduler interface) -->
<property>
<name>yarn.resourcemanager.scheduler.address.rm1</name>
<value>master.chff.dc:23130</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address.rm2</name>
<value>slave01.chff.dc:23130</value>
</property>
<!-- RM admin interface -->
<property>
<name>yarn.resourcemanager.admin.address.rm1</name>
<value>master.chff.dc:23141</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address.rm2</name>
<value>slave01.chff.dc:23141</value>
</property>
<!--NM访问RM的RPC端口 -->
<property>
<name>yarn.resourcemanager.resource-tracker.address.rm1</name>
<value>master.chff.dc:23125</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address.rm2</name>
<value>slave01.chff.dc:23125</value>
</property>
<!-- RM web application 地址 -->
<property>
<name>yarn.resourcemanager.webapp.address.rm1</name>
<value>master.chff.dc:8088</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address.rm2</name>
<value>slave01.chff.dc:8088</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.https.address.rm1</name>
<value>master.chff.dc:8089</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.https.address.rm2</name>
<value>slave01.chff.dc:8089</value>
</property> - start yarn on two nodes:
start-yarn.sh - Check status:
[hadoop@master deploy_scripts]$ yarn rmadmin -getServiceState rm1
active
[hadoop@master deploy_scripts]$ yarn rmadmin -getServiceState rm2
standby
0 0
- How-to: Enable yarn ResourceManager HA
- How-to: enable HDFS HA at a new cluster
- hadoop集群之YARN`s ResourceManager HA(三)
- ResourceManager HA
- ResourceManager HA
- Resourcemanager HA
- yarn ResourceManager
- How to enable web condor?
- How to enable navigation arrows
- How to enable the debugfs
- How to Enable Multi-Touch
- 重启yarn服务报错no ResourceManager to stop
- YARN源码分析(三)-----ResourceManager HA之应用状态存储与恢复
- YARN源码分析(三)-----ResourceManager HA之应用状态存储与恢复
- Hadoop2.5.1 ResourceManager HA
- resourcemanager ha 配置
- ResourceManager HA 配置
- YARN ResourceManager failover机制
- JavaScript & jQuery交互式Web前端开发
- 多线程简单介绍(摘记)
- Java Reflection(十):数组
- 【笔记】Linux驱动学习第一章
- 尽管关于Java中文问题的讨论已经相当多了,但由于Java的相关技术标准繁多,面向Java的Web服务器、应用服务器以及JDBC数据库驱动等都没有官方的标准,所以Java应用在处理中文时所存在的问
- How-to: Enable yarn ResourceManager HA
- SAT语法三大从句分析
- mysql分表方法-----MRG_MyISAM引擎分表法
- 机器学习中的范数规则化之(一)L0、L1与L2范数
- Spring MVC之@RequestBody, @ResponseBody 详解
- #1 Two Sum
- UIWebView --- 内置的浏览器控件 加载网页
- yii2实现邮件发送
- Android项目结构