搭建Hadoop前期配置---HADOOP系列（一）

来源：互联网发布：数据挖掘算法与应用编辑：程序博客网时间：2024/06/03 16:42

搭建Hadoop完全分布式最少需要三个节点，即一个主节点，三个数据节点（其中一个数据节点可搭建在主节点上），在创建虚拟机创建一个非root账户，下面为Hadoop搭建工作的前期配置。

1.修改主机名，搭建Hadoop时经常采用复制的方式创建虚拟机，方便识别需要修改主机的名字

[hadoop@master Desktop]$ su rootPassword: [root@master Desktop]# cd /etc/sysconfig[root@master sysconfig]# vi network[root@master sysconfig]# vi network

打开的文件如下：HOSTNAME为主机名配置，修改后重启即可。

NETWORKING=yesHOSTNAME=master

2.安装java，rpm -ivh + 文件名为安装命令

[hadoop@master Desktop]$ su rootPassword: [root@master Desktop]# cp jdk-8u131-linux-x64.rpm /usr/[root@master Desktop]# cd /usr[root@master usr]# rpm -ivh jdk-8u131-linux-x64.rpm Preparing...                ########################################### [100%]   1:jdk1.8.0_131           ########################################### [100%]Unpacking JAR files...tools.jar...plugin.jar...javaws.jar...deploy.jar...rt.jar...jsse.jar...charsets.jar...localedata.jar...[root@master usr]# java -versionjava version "1.8.0_131"Java(TM) SE Runtime Environment (build 1.8.0_131-b11)Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)[root@master usr]#

然后在环境变量文件/etc/profile文件中配置JAVA

[root@master hadoop]# vi /etc/profile

在这个文件中加入下面的内容，JAVA_HOME为JAVA的安装路径，其他不用变

export JAVA_HOME=/usr/java/jdk1.8.0_131export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jarexport PATH=$PATH:$JAVA_HOME/bin

3.配置SSH免密码登录，authorized_keys存储所有节点的公钥，id_rsa为私钥，id_rsa.pub为公钥

[hadoop@master ~]$ pwd/home/hadoop[hadoop@master ~]$ ssh-keygen -t rsaGenerating public/private rsa key pair.Enter file in which to save the key (/home/hadoop/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/hadoop/.ssh/id_rsa.Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.The key fingerprint is:a1:27:98:d5:b7:a5:a4:c1:6b:a4:a8:b6:74:97:75:b8 hadoop@masterThe key's randomart image is:+--[ RSA 2048]----+|                 ||       o         ||      . * o .    ||     = + O +     ||    + + S +      ||   .   * o       ||  + . o E        || o o .           ||  .              |+-----------------+[hadoop@master ~]$ cd .ssh[hadoop@master .ssh]$ lsid_rsa  id_rsa.pub[hadoop@master .ssh]$ cp id_rsa.pub authorized_keys[hadoop@master .ssh]$ lsauthorized_keys  id_rsa  id_rsa.pub

将authorized_keys文件传给其他节点

[hadoop@master ~]$ scp -r authorized_keys node1:/home/hadoop[hadoop@master ~]$ scp -r authorized_keys node2:/home/hadoop

将authorized_keys文件合并

[hadoop@master ~]$ cat authorized_keys >> .ssh/authorized_keys

SSH免密登录验证

[hadoop@master ~]$ ssh node1Last login: Sun May  7 06:46:56 2017 from node2[hadoop@node1 ~]$ exitlogoutConnection to node1 closed.[hadoop@master ~]$ ssh node2Last login: Sun May  7 06:46:56 2017 from node1[hadoop@node2 ~]$ exitlogoutConnection to node2 closed.[hadoop@master ~]$

至此，搭建Hadoop的前期配置完成，下一步准备搭建Hadoop。

0 0