hadoop2 yarn 集群配置

一、配置文件配置

进入hadoop_home/etc/Hadoop目录

二、配置core_site.xml

<configuration>        <property>        <name>fs.defaultFS</name>                <value>hdfs://xxxx:8020</value>        </property>        <property>                <name>hadoop.tmp.dir</name>                <value>/spark/data/hadoop/tmp</value>                <description>A base for other temporary directories.</description>        </property>        <property>                <name>io.file.buffer.size</name>                <value>65535</value>                <description>Size of read/write buffer used in SequenceFiles.</description>        </property>        <property>                <name>io.native.lib.available</name>                <value>true</value>                <description>Should native hadoop libraries, if present, be used.</description>        </property>        <property>                <name>fs.hdfs.impl</name>                <value>org.apache.hadoop.hdfs.DistributedFileSystem</value>                <description>The FileSystem for hdfs: uris.</description>        </property>  </configuration>

三、配置hadoop_env.sh

注意把里面的JAVA_HOME改为自已配的*注意要使用绝对路径

四、配置hdfs-site.xml

<configuration><property>      <name>dfs.http.address</name>            <value>xxxx:10075</value>                <description>                            The address and the base port where the dfs namenode web ui will listen on.                            If the port is 0 then the server will start on a free port.                </description>        </property>        <property>            <name>dfs.namenode.secondary.http-address</name>                <value>xxxx:10090</value>        </property>        <property>            <name>dfs.namenode.name.dir</name>                <value>/spark/data/hadoop/dfs/name</value>        </property>        <property>            <name>dfs.datanode.data.dir</name>                <value>/spark/data/hadoop/dfs/data</value>        </property>        <property>            <name>dfs.replication</name>                <value>3</value>        </property>        <property>                <name>dfs.namenode.handler.count</name>                <value>100</value>        </property></configuration>

五、配置mapred-env.sh

注意把JAVA_HOME配置成自已的

六、配置slaves，即加入slave地址

七、配置yarn-env.sh

注意把JAVA_HOME配置成自已的

八、配置yarn-site.xml

<configuration><!-- Site specific YARN configuration properties --><property><name>yarn.resourcemanager.hostname</name><value>xxxx</value></property></configuration>

八、分发文件夹到其他主机上

九、格式化集群

bin/hadoop namenode -format

十、启动

十一、查看进程、查看日志