hadoop简单集群搭建

来源：互联网发布：剑灵天族女身体数据图编辑：程序博客网时间：2024/06/05 08:33

之前已经实现单台的伪分布集群。现在通过四台服务器来实现集群分布式

1.ssh-keygen -t rsa
2.使用ssh-copy-id -i ~/.ssh/id_dsa.pub hadoopX分发到各个主机上
3.配置hadoop-env.sh
export JAVA_HOME =
4.配置core-site.xml

<property>    <name>fs.defaultFS</name>    <value>hdfs://hadoop0:8020</value></property>

5.配置hdsf-site.xml

<configuration>    <property>            <name>dfs.namenode.data.dir</name>            <value>/home/hadoop/bda/tmp/dfs/name</value>    </property>     <property>            <name>dfs.datanode.data.dir</name>            <value>/home/hadoop/bda/tmp/dfs/data</value>    </property></configuration>

6.配置yarn-site.xml

<configuration>    <!-- Site specific YARN configuration properties -->    <property>            <name>yarn.resourcemanager.hostname</name>            <value>hadoop0</value>    </propertry>    <property>            <name>yarn.nodemanager.aux-services</name>            <value>mapreduce_shuffle</value>    </propertry></configuration>

7.配置mapreduce-site.xml

 <configuration>    <property>            <name>mapreduce.framework.name</name>            <value>yarn<value>    </property></configuration>

8.配置slave
hadoop1
hadoop2
hadoop3

9.分发hadoop 及配置文件
scp -r ~/bda hadoop@hadoop1:~/
scp -r ~/bda hadoop@hadoop2:~/
scp -r ~/bda hadoop@hadoop3:~/

scp ~/.bashrc hadoop@hadoop1:~/
scp ~/.bashrc hadoop@hadoop2:~/
scp ~/.bashrc hadoop@hadoop3:~/

10.格式化
hdfs namenode -format

11.启动
/sbin/start-all.sh

阅读全文

0 0