配置yarn集群

yarn源配置：

单台yarn源配置，使用资源较少，速度快

core.site.xml文件：

<configuration><!-- 指定hadoop的hdfs的namenode的访问路径. -->     <property>        <name>fs.defaultFS</name>        <value>hdfs://hadoop01:9000</value>    </property><!-- 指定hadoop临时目录的路径  这个路径是放元数据的.必须要先创建 -->     <property>        <name>hadoop.tmp.dir</name>        <value>/root/hpdata</value>     </property></configuration>

hdfs.site.xml文件：

<configuration><!-- 配置ScondaryNameNode的位置 -->    <property>      <name>dfs.namenode.secondary.http-address</name> 配置SN的主机名      <value>hadoop04:50090</value>    </property><!-- 配置ScondaryNameNode的位置 -->    <property>      <name>dfs.namenode.secondary.https-address</name>      <value>hadoop04:50091</value>    </property></configuration>yarn.site.xml文件：<configuration><!-- 指定集群中最大的管理者 --><property><name>yarn.resourcemanager.hostname</name><value>hadoop01</value></property><!-- 指定集群中管理者的版本 --><property><name>yarn.nodemanager.aux-services</name><value>mapreduce_shuffle</value></property></configuration>mapred.site.xml文件：<configuration><!-- 使用yar运行maperreduce. --><property><name>mapreduce.framework.name</name><value>yarn</value></property></configuration>

启动zkServer.sh start(三台机器上面启动)

然后start-all.sh启动

如果有单独的节点没有启动，则使用yarn-daemon.sh start DataNode

---

高可用yarn集群：

core.site.xml文件：

<configuration>    <property>        <name>fs.defaultFS</name>        <value>hdfs://bx</value>    </property>    <property>        <name>ha.zookeeper.quorum</name>        <value>CentOS8,CentOS9,CentOS10</value>    </property>    <property>        <name>hadoop.tmp.dir</name>        <value>/opt/hadoop</value>    </property></configuration>

hdfs.site.xml文件：

<configuration>    <property>        <name>dfs.nameservices</name>        <value>bx</value>    </property>    <property>        <name>dfs.ha.namenodes.bx</name>        <value>nn1,nn2</value>    </property>    <property>        <name>dfs.namenode.rpc-address.bx.nn1</name>        <value>CentOS8:8020</value>    </property>    <property>        <name>dfs.namenode.rpc-address.bx.nn2</name>        <value>CentOS9:8020</value>    </property>    <property>        <name>dfs.namenode.http-address.bx.nn1</name>        <value>CentOS8:50070</value>    </property>    <property>        <name>dfs.namenode.http-address.bx.nn2</name>        <value>CentOS9:50070</value>    </property>    <property>        <name>dfs.namenode.shared.edits.dir</name>        <value>qjournal://CentOS8:8485;CentOS10:8485;CentOS9:8485/abc</value>    </property>    <property>        <name>dfs.client.failover.proxy.provider.bx</name>        <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>    </property>    <property>        <name>dfs.ha.fencing.methods</name>        <value>sshfence</value>    </property>    <property>        <name>dfs.ha.fencing.ssh.private-key-files</name>        <value>/root/.ssh/id_rsa</value>    </property>    <property>        <name>dfs.journalnode.edits.dir</name>        <value>/opt/journalnode/</value>    </property>    <property>        <name>dfs.ha.automatic-failover.enabled</name>        <value>true</value>    </property></configuration>

yarn.site.xml文件：

<configuration><!-- Site specific YARN configuration properties --><property>        <name>yarn.nodemanager.aux-services</name>        <value>mapreduce_shuffle</value>    </property><property>   <name>yarn.resourcemanager.ha.enabled</name>   <value>true</value> </property> <property>   <name>yarn.resourcemanager.cluster-id</name>   <value>cluster1</value> </property> <property>   <name>yarn.resourcemanager.ha.rm-ids</name>   <value>rm1,rm2</value> </property> <property>   <name>yarn.resourcemanager.hostname.rm1</name>   <value>CentOS8</value> </property> <property>   <name>yarn.resourcemanager.hostname.rm2</name>   <value>CentOS9</value> </property> <property>   <name>yarn.resourcemanager.zk-address</name>   <value>CentOS8,CentOS9,CentOS10</value> </property></configuration>

配置mapred-site.xml文件：

<configuration><!-- 使用yar运行maperreduce. --><property><name>mapreduce.framework.name</name><value>yarn</value></property></configuration>

---

然后首先启动所有节点上面的zookeeper
命令：zkServer.sh start
然后在namenode节点上面启动yarn集群
start-all.sh