How-to: Enable yarn ResourceManager HA

来源:互联网 发布:程序员的数学怎么样 编辑:程序博客网 时间:2024/05/16 06:56
  1. Update yarn-site.xml. Add following in yarn-site.xml(master.chff.dc and slave01.chff.dc are resourcemanager nodes):
            <property>
                    <name>yarn.resourcemanager.ha.enabled</name>
                    <value>true</value>
            </property>
            <property>
                    <name>yarn.resourcemanager.ha.automatic-failover.enabled</name>
                    <value>true</value>
            </property>
            <!-- 使嵌入式自动故障转移。HA环境启动,与 ZKRMStateStore 配合 处理fencing -->
            <property>
                    <name>yarn.resourcemanager.ha.automatic-failover.embedded</name>
                    <value>true</value>
            </property>
            <!-- 集群名称,确保HA选举时对应的集群 -->
            <property>
            <name>yarn.resourcemanager.cluster-id</name>
            <value>yarn-cluster</value>
            </property>
            <property>
                    <name>yarn.resourcemanager.ha.rm-ids</name>
                    <value>rm1,rm2</value>
            </property>
            <property>
                    <name>yarn.resourcemanager.hostname.rm1</name>
                    <value>master.chff.dc</value>
            </property>
            <property>
                    <name>yarn.resourcemanager.hostname.rm2</name>
                    <value>slave01.chff.dc</value>
            </property>
            <property>
                    <name>yarn.resourcemanager.scheduler.class</name>
                    <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</value>
            </property>
            <property>
                    <name>yarn.resourcemanager.recovery.enabled</name>
                    <value>true</value>
            </property>
            <property>
                    <name>yarn.app.mapreduce.am.scheduler.connection.wait.interval-ms</name>
                    <value>5000</value>
            </property>
            <!-- ZKRMStateStore 配置 -->
            <property>
                    <name>yarn.resourcemanager.store.class</name>
                    <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore</value>
            </property>
            <property>
                    <name>yarn.resourcemanager.zk-address</name>
                    <value>master.chff.dc:2183,slave01.chff.dc:2183,slave.chff.dc:2183</value>
            </property>
            <property>
                    <name>yarn.resourcemanager.zk.state-store.address</name>
                    <value>master.chff.dc:2183,slave01.chff.dc:2183,slave.chff.dc:2183</value>
            </property>
            <!-- Client访问RM的RPC地址 (applications manager interface) -->
            <property>
                    <name>yarn.resourcemanager.address.rm1</name>
                    <value>master.chff.dc:23140</value>
            </property>
            <property>
                    <name>yarn.resourcemanager.address.rm2</name>
                    <value>slave01.chff.dc:23140</value>
            </property>
            <!-- AM访问RM的RPC地址(scheduler interface) -->
            <property>
                    <name>yarn.resourcemanager.scheduler.address.rm1</name>
                    <value>master.chff.dc:23130</value>
            </property>
            <property>
                    <name>yarn.resourcemanager.scheduler.address.rm2</name>
                    <value>slave01.chff.dc:23130</value>
            </property>
            <!-- RM admin interface -->
            <property>
                    <name>yarn.resourcemanager.admin.address.rm1</name>
                    <value>master.chff.dc:23141</value>
            </property>
            <property>
                    <name>yarn.resourcemanager.admin.address.rm2</name>
                    <value>slave01.chff.dc:23141</value>
            </property>
            <!--NM访问RM的RPC端口 -->
            <property>
                    <name>yarn.resourcemanager.resource-tracker.address.rm1</name>
                    <value>master.chff.dc:23125</value>
            </property>
            <property>
                    <name>yarn.resourcemanager.resource-tracker.address.rm2</name>
                    <value>slave01.chff.dc:23125</value>
            </property>
            <!-- RM web application 地址 -->
            <property>
                    <name>yarn.resourcemanager.webapp.address.rm1</name>
                    <value>master.chff.dc:8088</value>
            </property>
            <property>
                    <name>yarn.resourcemanager.webapp.address.rm2</name>
                    <value>slave01.chff.dc:8088</value>
            </property>
            <property>
                    <name>yarn.resourcemanager.webapp.https.address.rm1</name>
                    <value>master.chff.dc:8089</value>
            </property>
            <property>
                    <name>yarn.resourcemanager.webapp.https.address.rm2</name>
                    <value>slave01.chff.dc:8089</value>
            </property>
  2. start yarn on two nodes:
    start-yarn.sh
  3. Check status:
    [hadoop@master deploy_scripts]$ yarn rmadmin -getServiceState rm1
    active
    [hadoop@master deploy_scripts]$ yarn rmadmin -getServiceState rm2
    standby
0 0
原创粉丝点击