Spark修炼之道（进阶篇）——Spark入门到精通：第一节 Spark 1.5.0集群搭建

本节主要内容

1. 操作系统环境准备
2. Hadoop 2.4.1集群搭建
3. Spark 1.5.0 集群部署

注：在利用CentOS 6.5操作系统安装spark 1.5集群过程中，本人发现Hadoop 2.4.1集群可以顺利搭建，但在Spark 1.5.0集群启动时出现了问题（可能原因是64位操作系统原因，源码需要重新编译，但本人没经过测试），经本人测试在ubuntu 10.04 操作系统上可以顺利成功搭建。大家可以利用CentOS 6.5进行尝试，如果有问题，再利用ubuntu 10.04搭建，所有步骤基本一致

1. 操作系统环境准备

（1）安装VMWare

  下载地址：http://pan.baidu.com/s/1bniBipD  密码：pbdw  安装过程略

（2）下载操作系统并安装

Ubuntu 10.04操作系统下载地址：

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">链接：http://pan<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.baidu</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>/s/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>kTy9Umj 密码：<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>w5b</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

CentOS 6.5下载地址：

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">下载地址：http://pan<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.baidu</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>/s/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>mgkuKdi密码：xtm5</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

本实验要求装三台：CentOS 6.5，可以分别安装，也可以安装完一台后克隆两台，具体过程略。初学者，建议三台分别安装。安装后如下图所示： 

（3）CentOS 6.5网络配置

安装好的虚拟机一般默认使用的是NAT（关于NAT、桥接等虚拟机网络连接方式参见本人博客：http://blog.csdn.net/lovehuangjiaju/article/details/48183485），由于三台机器之间需要互通之外，还需要与本机连通，因此采用将网络连接方式设置为Bridged（三台机器相同的设置），如下图所法： 

修改主机名

（1）修改centos_salve01虚拟机主机名：

<code class="hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">vim /etc/sysconfig/network</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

/etc/sysconfig/network修改后的内容如下： 

（2）vim /etc/sysconfig/network命令修改centos_slave02虚拟机主机名 
/etc/sysconfig/network修改后的内容如下： 

（3）vim /etc/sysconfig/network命令修改centos_slave03虚拟机主机名 
/etc/sysconfig/network修改后的内容如下： 

修改主机IP地址

在大家在配置时，修改/etc/sysconfig/network-scripts/ifcfg-eth0文件对应的BOOTPROT=static、IPADDR、NETMASK、GATEWAY及DNS1信息即可

（1）修改centos_salve01虚拟机主机IP地址：

<code class="hljs lasso has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">vim /etc/sysconfig/network<span class="hljs-attribute" style="box-sizing: border-box;">-scripts</span>/ifcfg<span class="hljs-attribute" style="box-sizing: border-box;">-eth0</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

修改后内容如下：

<code class="hljs ini has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-setting" style="box-sizing: border-box;">DEVICE=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"eth0"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">BOOTPROTO=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"static"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">HWADDR=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"00:0c:29:3f:69:4d"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">IPV6INIT=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"yes"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">NM_CONTROLLED=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"yes"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">ONBOOT=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"yes"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">TYPE=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Ethernet"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">UUID=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"5315276c-db0d-4061-9c76-9ea86ba9758e"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">IPADDR=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"192.168.1.111"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">NETMASK=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"255.255.255.0"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">GATEWAY=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"192.168.1.1"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">DNS1=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"8.8.8.8"</span></span></span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li></ul>

 
（2）修改centos_salve02虚拟机主机IP地址：

<code class="hljs lasso has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">vim /etc/sysconfig/network<span class="hljs-attribute" style="box-sizing: border-box;">-scripts</span>/ifcfg<span class="hljs-attribute" style="box-sizing: border-box;">-eth0</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

修改后内容如下：

<code class="hljs ini has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-setting" style="box-sizing: border-box;">DEVICE=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"eth0"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">BOOTPROTO=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"static"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">HWADDR=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"00:0c:29:64:f9:80"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">IPV6INIT=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"yes"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">NM_CONTROLLED=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"yes"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">ONBOOT=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"yes"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">TYPE=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Ethernet"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">UUID=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"5315276c-db0d-4061-9c76-9ea86ba9758e"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">IPADDR=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"192.168.1.112"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">NETMASK=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"255.255.255.0"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">GATEWAY=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"192.168.1.1"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">DNS1=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"8.8.8.8"</span></span></span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li></ul>

（3）修改centos_salve03虚拟机主机IP地址：

<code class="hljs lasso has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">vim /etc/sysconfig/network<span class="hljs-attribute" style="box-sizing: border-box;">-scripts</span>/ifcfg<span class="hljs-attribute" style="box-sizing: border-box;">-eth0</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

修改后内容如下：

<code class="hljs ini has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-setting" style="box-sizing: border-box;">DEVICE=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"eth0"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">BOOTPROTO=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"static"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">HWADDR=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"00:0c:29:1e:80:b1"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">IPV6INIT=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"yes"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">NM_CONTROLLED=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"yes"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">ONBOOT=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"yes"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">TYPE=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Ethernet"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">UUID=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"5315276c-db0d-4061-9c76-9ea86ba9758e"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">IPADDR=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"192.168.1.113"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">NETMASK=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"255.255.255.0"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">GATEWAY=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"192.168.1.1"</span></span></span><span class="hljs-setting" style="box-sizing: border-box;">DNS1=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"8.8.8.8"</span></span></span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li></ul>

/etc/sysconfig/network-scripts/ifcfg-eth0文件内容解析：

<code class="hljs delphi has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">DEVICE=eth0                 <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//指出设备名称</span>BOOTPROT=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">static</span>             <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//启动类型 dhcp|static，使用桥接模式，必须是static</span>HWADDR=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">06</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>B:FE:DF:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7</span>C    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//硬件Mac地址</span>IPADDR=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.2</span>          <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//IP地址</span>NETMASK=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">255.255</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">255.0</span>       <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//子网掩码</span>NETWORK=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.0</span>         <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//网络地址</span>GATEWAY=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.1</span>         <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//网关地址</span>ONBOOT=yes                  <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//是否启动应用</span><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">TYPE</span>=Ethernet               <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//网络类型</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li></ul>

设置完成后，使用

<code class="hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">service network restart</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

命令重新启动网络，配置即可生效。

设置主机名与IP地址映射

（1）修改centos_salve01主机名与IP地址映射

<code class="hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">vim /etc/hosts</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

设置内容如下：

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">127.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span> slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>  localhost localhost<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.localdomain</span> localhost4 localhost4<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.localdomain</span>4::<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>       slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.111</span> slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.112</span> slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.113</span> slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>具体如下图：</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>

（2）修改centos_salve02主机名与IP地址映射

<code class="hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">vim /etc/hosts</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

设置内容如下：

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">127.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span> slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>  localhost localhost<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.localdomain</span> localhost4 localhost4<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.localdomain</span>4::<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>       slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.111</span> slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.112</span> slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.113</span> slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li></ul>

具体如下图： 

（3）修改centos_salve03主机名与IP地址映射

<code class="hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">vim /etc/hosts</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

设置内容如下：

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">127.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span> slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>  localhost localhost<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.localdomain</span> localhost4 localhost4<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.localdomain</span>4::<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>       slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.111</span> slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.112</span> slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.113</span> slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>

修改主机DNS

采用下列命令设置各主机DNS（三台机器进行相同的设置)

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">vim /etc/resolv<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.conf</span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

设置后的内容：

<code class="hljs vala has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># Generated by NetworkManager</span>search example.comnameserver <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8.8</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.8</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.8</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>

8.8.8.8为Google提供的DNS服务器

网络连通测试

前面所有的配置完成后，重启centos_salve01、centos_salve02、centos_salve03使主机名设置生效，然后分别在三台机器上作如下测试命令： 
下面只给出在centos_salve01虚拟机上的测试

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root@slave01 ~]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># ping slave02.example.com</span><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">PING</span> slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.112</span>) <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">84</span>) bytes of data.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> bytes from slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.112</span>): icmp_seq=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> ttl=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> time=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.417</span> ms<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> bytes from slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.112</span>): icmp_seq=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> ttl=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> time=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.355</span> ms<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> bytes from slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.112</span>): icmp_seq=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span> ttl=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> time=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.363</span> ms^C--- slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">ping</span> statistics ---<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span> packets transmitted, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span> received, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>% packet loss, time <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2719</span>msrtt min/avg/max/mdev = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.355</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.378</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.417</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.031</span> ms[root@slave01 ~]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># ping slave03.example.com</span><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">PING</span> slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.113</span>) <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">84</span>) bytes of data.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> bytes from slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.113</span>): icmp_seq=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> ttl=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> time=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.386</span> ms<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> bytes from slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.113</span>): icmp_seq=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> ttl=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> time=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.281</span> ms^C--- slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">ping</span> statistics ---<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> packets transmitted, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> received, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>% packet loss, time <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1799</span>msrtt min/avg/max/mdev = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.281</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.333</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.386</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.055</span> ms</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li></ul>

测试外网的连通性（我在装的时候，8.8.8.8，已经被禁用….心中一万头cnm）：

<code class="hljs vbnet has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root@slave01 ~]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># ping www.baidu.com</span>ping: unknown host www.baidu.com[root@slave01 ~]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># ping 8.8.8.8</span>PING <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8.8</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.8</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.8</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8.8</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.8</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.8</span>) <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">84</span>) bytes <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">of</span> data.<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">From</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.111</span> icmp_seq=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> Destination Host Unreachable<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">From</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.111</span> icmp_seq=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span> Destination Host Unreachable<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">From</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.111</span> icmp_seq=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span> Destination Host Unreachable<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">From</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.111</span> icmp_seq=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span> Destination Host Unreachable<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">From</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.111</span> icmp_seq=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7</span> Destination Host Unreachable<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">From</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.111</span> icmp_seq=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span> Destination Host Unreachable</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li></ul>

（4）SSH完密码登录

#### (1) OpenSSH安装 
如果大家在配置时，ping 8.8.8.8能够ping通，则主机能够正常上网；如果不能上网，则将网络连接方式重新设置为NAT，并修改网络配置文件为dhcp方式。在保证网络连通的情况下执行下列命令：

<code class="hljs axapta has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">yum install openssh-<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">server</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

#### (2) 无密码登录实现

使用以下命令生成相应的密钥（三台机器进行相同的操作）

<code class="hljs lasso has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">ssh<span class="hljs-attribute" style="box-sizing: border-box;">-keygen</span> <span class="hljs-attribute" style="box-sizing: border-box;">-t</span> rsa </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

执行过程一直回车即可

<code class="hljs vbnet has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root@slave01 ~]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># ssh-keygen -t rsa</span>Generating <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span>/<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">private</span> rsa <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">key</span> pair.Enter file <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> which <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">to</span> save the <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">key</span> (/root/.ssh/id_rsa): Enter passphrase (empty <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> no passphrase): Enter same passphrase again: Your identification has been saved <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> /root/.ssh/id_rsa.Your <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">key</span> has been saved <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> /root/.ssh/id_rsa.pub.The <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">key</span> fingerprint <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">is</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span>e:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>f:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">39</span>:ed:f4:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>e:a3:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">55</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">62</span>:f5:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span>a:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>d:c5:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>c:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span> [e-mail] <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">key</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">'s randomart image is:</span>+--[ RSA <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2048</span>]----+|        E        ||         +       ||        o =      ||       . + .     ||        S . .    ||       + X .     ||        B *      ||       .o=o.     ||      .. +oo.    |+-----------------+</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li></ul>

生成的文件分别为/root/.ssh/id_rsa（私钥）、/root/.ssh/id_rsa.pub（公钥）

完成后将公钥拷贝到要免登陆的机器上（三台可进行相同操作）：

<code class="hljs lasso has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">ssh<span class="hljs-attribute" style="box-sizing: border-box;">-copy</span><span class="hljs-attribute" style="box-sizing: border-box;">-id</span> <span class="hljs-attribute" style="box-sizing: border-box;">-i</span> slave01<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">.</span>example<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">.</span>comssh<span class="hljs-attribute" style="box-sizing: border-box;">-copy</span><span class="hljs-attribute" style="box-sizing: border-box;">-id</span> <span class="hljs-attribute" style="box-sizing: border-box;">-i</span> slave02<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">.</span>example<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">.</span>comssh<span class="hljs-attribute" style="box-sizing: border-box;">-copy</span><span class="hljs-attribute" style="box-sizing: border-box;">-id</span> <span class="hljs-attribute" style="box-sizing: border-box;">-i</span> slave03<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">.</span>example<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">.</span>com</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>

2. Hadoop 2.4.1集群搭建

集群搭建相关软件下载地址：

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">链接：http://pan<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.baidu</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>/s/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>sjIG3b3 密码：<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">38</span>gh</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

下载后将所有软件都放置在E盘的share目录下: 

设置share文件夹为虚拟机的共享目录，如下图所示： 

在linux系统中，采用

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> /]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cd /mnt/hgfs/share</span>[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> share]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ls</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

命令可以切换到该目录下，如下图 

Spark官方要求的JDK、Scala版本

<code class="hljs applescript has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">Spark runs <span class="hljs-function_start" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">on</span></span> Java <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7</span>+, Python <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.6</span>+ <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">and</span> R <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3.1</span>+. For <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">the</span> Scala API, Spark <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.5</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span> uses Scala <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.10</span>. You will need <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">to</span> use a compatible Scala <span class="hljs-property" style="box-sizing: border-box;">version</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.10</span>.x).</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

（1）JDK 1.8 安装

在根目录下创建sparkLearning目前，后续所有相关软件都放置在该目录下，代码如下：

<code class="hljs coffeescript has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-property" style="box-sizing: border-box;">@slave01</span> /]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># mkdir /sparkLearning</span>[root<span class="hljs-property" style="box-sizing: border-box;">@slave01</span> /]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ls</span>bin   etc             lib         media  proc  selinux        sys  <span class="hljs-reserved" style="box-sizing: border-box;">var</span>boot  hadoopLearning  lib64       mnt    root  sparkLearning  tmpdev   home            lost+found  opt    sbin  srv            usr</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>

将共享目录中的jdk安装包复制到/sparkLearning目录

<code class="hljs perl has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> share]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cp /mnt/hgfs/share/jdk-8u40-linux-x64.gz /sparkLearning/</span>[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> share]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cd /sparkLearning/</span>//解压[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># tar -zxvf jdk-8u40-linux-x64.gz </span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul>

设置环境变量：

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># vim /etc/profile</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

在文件最后添加：

<code class="hljs bash has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> JAVA_HOME=/sparkLearning/jdk1.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8.0</span>_40<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> PATH=<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${JAVA_HOME}</span>/bin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">$PATH</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

如下图： 

测试配置是否成功：

<code class="hljs d has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//使修改后的配置生效</span>[root<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">@slave01</span> sparkLearning]# source /etc/profile<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//环境变量是否已经设置</span>[root<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">@slave01</span> sparkLearning]# $JAVA_HOMEbash: /sparkLearning/jdk1.8.0_40: <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">is</span> a directory<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//测试java是否安装配置成功</span>[root<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">@slave01</span> sparkLearning]# java -<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">version</span>java <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">version</span> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"1.8.0_40"</span>Java(TM) SE Runtime Environment (build <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.8</span>.0_40-b25)Java HotSpot(TM) <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span>-Bit Server VM (build <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">25.40</span>-b25, mixed mode)</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li></ul>

（2）Scala 2.10.4 安装

<code class="hljs java has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//复制文件到sparkLearning目录下</span>[root<span class="hljs-annotation" style="color: rgb(155, 133, 157); box-sizing: border-box;">@slave</span>01 sparkLearning]# cp /mnt/hgfs/share/scala-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.10</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>.tgz  .<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//解压</span>[root<span class="hljs-annotation" style="color: rgb(155, 133, 157); box-sizing: border-box;">@slave</span>01 sparkLearning]# tar -zxvf scala-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.10</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>.tgz > /dev/<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">null</span>[root<span class="hljs-annotation" style="color: rgb(155, 133, 157); box-sizing: border-box;">@slave</span>01 sparkLearning]# vim /etc/profile</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li></ul>

将/etc/profile文件末尾内容修改如下：

<code class="hljs bash has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> JAVA_HOME=/sparkLearning/jdk1.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8.0</span>_40<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> SCALA_HOME=/sparkLearning/scala-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.10</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> PATH=<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${JAVA_HOME}</span>/bin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${SCALA_HOME}</span>/bin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">$PATH</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>

测试Scala是否安装成功

<code class="hljs coffeescript has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-property" style="box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># source /etc/profile</span>[root<span class="hljs-property" style="box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># $SCALA_HOME</span><span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">bash</span>: /sparkLearning/scala-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.10</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>: <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">is</span> a directory[root<span class="hljs-property" style="box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># scala -version</span>Scala code runner version <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.10</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span> -- Copyright <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2002</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2013</span>, LAMP/EPFL</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>

（3）Zookeeper-3.4.5 集群搭建

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cp /mnt/hgfs/share/zookeeper-3.4.5.tar.gz .</span>[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># tar -zxvf zookeeper-3.4.5.tar.gz > /dev/null</span>[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cp zookeeper-3.4.5/conf/zoo_sample.cfg zoo.cfg</span>[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># vim zookeeper-3.4.5/conf/zoo_sample.cfg zoo.cfg</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>

修改dataDir为：

<code class="hljs fix has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-attribute" style="box-sizing: border-box;">dataDir</span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">/sparkLearning/zookeeper-3.4.5/zookeeper_data</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

在文件末尾添加如下内容：

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">server<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>=slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2888</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3888</span>server<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.2</span>=slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2888</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3888</span>server<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.3</span>=slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2888</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3888</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>

如图所示： 

创建ZooKeeper集群数据保存目录

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cd zookeeper-3.4.5/</span>[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> zookeeper-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3.4</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># mkdir zookeeper_data</span>[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> zookeeper-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3.4</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cd zookeeper_data/</span>[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> zookeeper_data]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># touch myid </span>[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> zookeeper_data]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># echo 1 > myid </span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>

将slave01.example.com（centos_slave01）上的sparkLearning目录拷贝到另外两台服务器上：

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> /]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># scp -r /sparkLearning slave02.example.com:/</span>[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> /]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># scp -r /sparkLearning slave03.example.com:/</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

/etc/profile文件也进行覆盖

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> /]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># scp  /etc/profile slave02.example.com:/etc/profile</span>[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> /]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># scp  /etc/profile slave03.example.com:/etc/profile</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

修改zookeeper_data中的myid信息：

<code class="hljs perl has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">//配置slave02.example.com上的myid[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> /]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ssh salve02.example.com</span>[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave02</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># echo 2 > /sparkLearning/zookeeper-3.4.5/zookeeper_data/myid</span>[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave02</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># more /sparkLearning/zookeeper-3.4.5/zookeeper_data/myid</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>//配置slave03.example.com上的myid[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave02</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ssh slave03.example.com</span>Last login: Fri Sep <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">18</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">01</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">33</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">29</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2015</span> from slave01.example.com[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave03</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># echo 3 > /sparkLearning/zookeeper-3.4.5/zookeeper_data/myid</span>[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave03</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># more /sparkLearning/zookeeper-3.4.5/zookeeper_data/myid</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li></ul>

如此便完成配置，下面对集群进行测试：

<code class="hljs coffeescript has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">//</span>在slave03.example.com主机上[root<span class="hljs-property" style="box-sizing: border-box;">@slave03</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cd /sparkLearning/zookeeper-3.4.5/bin</span>[root<span class="hljs-property" style="box-sizing: border-box;">@slave03</span> bin]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ls</span>README.txt    zkCli.cmd  zkEnv.cmd  zkServer.cmdzkCleanup.sh  zkCli.sh   zkEnv.sh   zkServer.sh<span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">//</span>启动slave03.example.com上的ZooKeeper[root<span class="hljs-property" style="box-sizing: border-box;">@slave03</span> bin]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ./zkServer.sh start</span>JMX enabled <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">by</span> <span class="hljs-reserved" style="box-sizing: border-box;">default</span>Using <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">config</span>: <span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">/sparkLearning/zookeeper-3.4.5/bin/</span>../conf/zoo.cfgStarting zookeeper ... STARTED[root<span class="hljs-property" style="box-sizing: border-box;">@slave03</span> bin]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ./zkServer.sh status</span>JMX enabled <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">by</span> <span class="hljs-reserved" style="box-sizing: border-box;">default</span>Using <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">config</span>: <span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">/sparkLearning/zookeeper-3.4.5/bin/</span>../conf/zoo.cfgError contacting service. It <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">is</span> probably <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">not</span> running.<span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">//</span>在slave02.example.com主机上[root<span class="hljs-property" style="box-sizing: border-box;">@slave02</span> bin]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ./zkServer.sh start</span>JMX enabled <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">by</span> <span class="hljs-reserved" style="box-sizing: border-box;">default</span>Using <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">config</span>: <span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">/sparkLearning/zookeeper-3.4.5/bin/</span>../conf/zoo.cfgStarting zookeeper ... STARTED<span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">//</span>查看zookeeper集群状态，如果Mode显示为follower或leader则表明配置成功[root<span class="hljs-property" style="box-sizing: border-box;">@slave02</span> bin]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ./zkServer.sh status</span>JMX enabled <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">by</span> <span class="hljs-reserved" style="box-sizing: border-box;">default</span>Using <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">config</span>: <span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">/sparkLearning/zookeeper-3.4.5/bin/</span>../conf/zoo.cfg<span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">Mode</span>: follower<span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">//</span>在slave01.example.com主机上[root<span class="hljs-property" style="box-sizing: border-box;">@slave01</span> bin]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ./zkServer.sh start</span>JMX enabled <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">by</span> <span class="hljs-reserved" style="box-sizing: border-box;">default</span>Using <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">config</span>: <span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">/sparkLearning/zookeeper-3.4.5/bin/</span>../conf/zoo.cfgStarting zookeeper ... STARTED[root<span class="hljs-property" style="box-sizing: border-box;">@slave01</span> bin]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ./zkServer.sh status</span>JMX enabled <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">by</span> <span class="hljs-reserved" style="box-sizing: border-box;">default</span>Using <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">config</span>: <span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">/sparkLearning/zookeeper-3.4.5/bin/</span>../conf/zoo.cfg<span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">Mode</span>: follower<span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">//</span>在slave03.example.com主机上zookeeper状态[root<span class="hljs-property" style="box-sizing: border-box;">@slave03</span> bin]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ./zkServer.sh status</span>JMX enabled <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">by</span> <span class="hljs-reserved" style="box-sizing: border-box;">default</span>Using <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">config</span>: <span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">/sparkLearning/zookeeper-3.4.5/bin/</span>../conf/zoo.cfg<span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">Mode</span>: leader</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li></ul>

（4）Hadoop 2.4.1 集群搭建

（1）Hadoop 2.4.1基本目录浏览

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root@slave01 bin]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># cp /mnt/hgfs/share/hadoop-2.4.1.tar.gz /sparkLearning/</span>[root@slave01 bin]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># cd /sparkLearning/</span>[root@slave01 sparkLearning]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># tar -zxvf hadoop-2.4.1.tar.gz > /dev/null</span>[root@slave01 sparkLearning]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># cd hadoop-2.4.1</span>[root@slave01 hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># ls</span>bin  include  libexec      NOTICE<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.txt</span>  sbinetc  lib      LICENSE<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.txt</span>  README<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.txt</span>  sharecd [root@slave01 hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># cd etc/hadoop/</span>[root@slave01 hadoop]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># ls</span>capacity-scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xml</span>      hdfs-site<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xml</span>               mapred-site<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xml</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.template</span>configuration<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xsl</span>           httpfs-env<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.sh</span>               slavescontainer-executor<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.cfg</span>      httpfs-log4j<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.properties</span>     ssl-client<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xml</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span>core-site<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xml</span>               httpfs-signature<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.secret</span>     ssl-server<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xml</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span>hadoop-env<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.cmd</span>              httpfs-site<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xml</span>             yarn-env<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.cmd</span>hadoop-env<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.sh</span>               log4j<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.properties</span>            yarn-env<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.sh</span>hadoop-metrics2<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.properties</span>  mapred-env<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.cmd</span>              yarn-site<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xml</span>hadoop-metrics<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.properties</span>   mapred-env<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.sh</span>hadoop-policy<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xml</span>           mapred-queues<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xml</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.template</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li></ul>

（2）将Hadoop 2.4.1添加到环境变量

使用命令：vim /etc/profile 将环境变量信息修改如下：

<code class="hljs bash has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> JAVA_HOME=/sparkLearning/jdk1.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8.0</span>_40<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> SCALA_HOME=/sparkLearning/scala-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.10</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> HADOOP_HOME=/sparkLearning/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> PATH=<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${JAVA_HOME}</span>/bin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${SCALA_HOME}</span>/bin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${HADOOP_HOME}</span>/bin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${HADOOP_HOME}</span>/sbin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">$PATH</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul>

（3）将Hadoop 2.4.1添加到环境变量

使用命令：vim hadoop-env.sh 将环境变量信息修改如下，在export JAVA_HOME修改为：

<code class="hljs bash has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> JAVA_HOME=/sparkLearning/jdk1.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8.0</span>_40</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

（4）修改core-site.xml文件

利用vim core-site.xml命令，文件内容如下：

<code class="hljs xml has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">  <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 指定hdfs的nameservice为ns1 --></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>fs.defaultFS<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>hdfs://ns1<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 指定hadoop临时目录 --></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>hadoop.tmp.dir<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>/sparkLearning/hadoop-2.4.1/tmp<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 指定zookeeper地址 --></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>ha.zookeeper.quorum<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>slave01.example.com:2181,slave02.example.com:2181,slave03.example.com:2181<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span><span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li></ul>

（5）修改hdfs-site.xml文件

vim hdfs-site.xml内容如下：

<code class="hljs xml has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span>                <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!--指定hdfs的nameservice为ns1，需要和core-site.xml中的保持一致 --></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.nameservices<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>ns1<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- ns1下面有两个NameNode，分别是nn1，nn2 --></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.ha.namenodes.ns1<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>nn1,nn2<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- nn1的RPC通信地址 --></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.namenode.rpc-address.ns1.nn1<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>slave01.example.com:9000<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- nn1的http通信地址 --></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.namenode.http-address.ns1.nn1<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>slave01.example.com:50070<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- nn2的RPC通信地址 --></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.namenode.rpc-address.ns1.nn2<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>slave02.example.com:9000<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- nn2的http通信地址 --></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.namenode.http-address.ns1.nn2<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>slave02.example.com:50070<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 指定NameNode的元数据在JournalNode上的存放位置 --></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.namenode.shared.edits.dir<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>qjournal://slave01.example.com:8485;slave02.example.com:8485;slave03.example.com:8485/ns1<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 指定JournalNode在本地磁盘存放数据的位置 --></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.journalnode.edits.dir<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>/sparkLearning/hadoop-2.4.1/journal<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 开启NameNode失败自动切换 --></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.ha.automatic-failover.enabled<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>true<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 配置失败自动切换实现方式 --></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.client.failover.proxy.provider.ns1<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 配置隔离机制方法，多个机制用换行分割，即每个机制暂用一行--></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.ha.fencing.methods<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                        sshfence                        shell(/bin/true)                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 使用sshfence隔离机制时需要ssh免登陆 --></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.ha.fencing.ssh.private-key-files<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>/home/hadoop/.ssh/id_rsa<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 配置sshfence隔离机制超时时间 --></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.ha.fencing.ssh.connect-timeout<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>30000<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>            <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li><li style="box-sizing: border-box; padding: 0px 5px;">61</li><li style="box-sizing: border-box; padding: 0px 5px;">62</li><li style="box-sizing: border-box; padding: 0px 5px;">63</li><li style="box-sizing: border-box; padding: 0px 5px;">64</li><li style="box-sizing: border-box; padding: 0px 5px;">65</li><li style="box-sizing: border-box; padding: 0px 5px;">66</li><li style="box-sizing: border-box; padding: 0px 5px;">67</li><li style="box-sizing: border-box; padding: 0px 5px;">68</li><li style="box-sizing: border-box; padding: 0px 5px;">69</li><li style="box-sizing: border-box; padding: 0px 5px;">70</li></ul>

（4）修改mapred-site.xml文件

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> hadoop]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cp mapred-site.xml.template mapred-site.xml</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

vim mapred-site.xml修改文件内容如下:

<code class="hljs xml has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"> <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span>                    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 指定mr框架为yarn方式 --></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>mapreduce.framework.name<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>yarn<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span> <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span>    </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li></ul>

（6）修改yarn-site.xml文件

<code class="hljs xml has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-pi" style="color: rgb(0, 102, 102); box-sizing: border-box;"><?xml version="1.0"?></span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!--  Licensed under the Apache License, Version 2.0 (the "License");  you may not use this file except in compliance with the License.  You may obtain a copy of the License at    http://www.apache.org/licenses/LICENSE-2.0  Unless required by applicable law or agreed to in writing, software  distributed under the License is distributed on an "AS IS" BASIS,  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  See the License for the specific language governing permissions and  limitations under the License. See accompanying LICENSE file.--></span><span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span>                        <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 开启RM高可靠 --></span>                        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                           <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>yarn.resourcemanager.ha.enabled<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                           <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>true<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                        <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 指定RM的cluster id --></span>                        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                           <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>yarn.resourcemanager.cluster-id<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                           <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>SparkCluster<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                        <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 指定RM的名字 --></span>                        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                           <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>yarn.resourcemanager.ha.rm-ids<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                           <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>rm1,rm2<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                        <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 分别指定RM的地址 --></span>                        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                           <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>yarn.resourcemanager.hostname.rm1<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                           <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>slave01.example.com<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                           <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>yarn.resourcemanager.hostname.rm2<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                           <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>slave02.example.com<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                        <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 指定zk集群地址 --></span>                        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                           <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>yarn.resourcemanager.zk-address<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                           <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                           <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                           <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>yarn.nodemanager.aux-services<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>                           <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>mapreduce_shuffle<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>                        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li></ul>

（7）修改slaves文件

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>

（8）配置文件拷贝到其它服务器

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">//slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>上的配置文件拷贝到slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>[root@slave01 hadoop]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># scp -r /etc/profile slave02.example.com:/etc/profile</span>profile                                       <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2027</span>     <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.0</span>KB/s   <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>    [root@slave01 hadoop]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># scp -r /sparkLearning/hadoop-2.4.1 slave02.example.com:/sparkLearning/</span>//slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>上的配置文件拷贝到slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>[root@slave01 hadoop]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># scp -r /etc/profile slave03.example.com:/etc/profile</span>profile                                       <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2027</span>     <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.0</span>KB/s   <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>    [root@slave01 hadoop]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># scp -r /sparkLearning/hadoop-2.4.1 slave03.example.com:/sparkLearning/</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li></ul>

（9）启动journalnode

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">//使用下列命令启动journalnode[root@slave01 hadoop]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># hadoop-daemons.sh start journalnode</span><span class="hljs-label" style="box-sizing: border-box;">slave02.example.com:</span> starting journalnode, logging to /sparkLearning/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/logs/hadoop-root-journalnode-slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.out</span><span class="hljs-label" style="box-sizing: border-box;">slave03.example.com:</span> starting journalnode, logging to /sparkLearning/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/logs/hadoop-root-journalnode-slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.out</span><span class="hljs-label" style="box-sizing: border-box;">slave01.example.com:</span> starting journalnode, logging to /sparkLearning/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/logs/hadoop-root-journalnode-slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.out</span>//JournalNode进程存在，启动成功[root@slave01 hadoop]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># jps</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">11261</span> JournalNode<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">11295</span> Jps[root@slave01 hadoop]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># ssh slave02.example.com</span>Last login: Fri Sep <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">18</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">05</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">33</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">05</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2015</span> from slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>[root@slave02 ~]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># jps</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6598</span> JournalNode<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6795</span> Jps[root@slave02 ~]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># ssh slave03.example.com</span>Last login: Fri Sep <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">18</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">05</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">33</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">26</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2015</span> from slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>[root@slave03 ~]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># jps</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5876</span> JournalNode<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6047</span> Jps[root@slave03 ~]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># </span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li></ul>

（10）格式化HDFS

登录slave02.example.com服务器，执行下列命令

<code class="hljs markdown has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root@slave02 ~]#  hdfs namenode -format//下面是执行结果15/09/18 06:05:26 INFO namenode.NameNode: STARTUP_MSG: /<span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span>STARTUP_MSG: Starting NameNodeSTARTUP_MSG:   host = slave02.example.com/127.0.0.1STARTUP_MSG:   args = [-format]STARTUP_MSG:   version = 2.4.1STARTUP_MSG:   classpath = /sparkLearning/hadoop-2.4.1/etc/hadoop:/sparkLearning/hadoop-........省略无关信息...............STARTUP_MSG:   build = http://svn.apache.org/repos/asf/hadoop/common -r 1604318; compiled by 'jenkins' on 2014-06-21T05:43ZSTARTUP<span class="hljs-emphasis" style="box-sizing: border-box;">_MSG:   java = 1.8.0_</span>40.....................................................省略...../sparkLearning/hadoop-2.4.1/tmp/dfs/name has been successfully formatted.15/09/18 06:05:30 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 015/09/18 06:05:30 INFO util.ExitUtil: Exiting with status 015/09/18 06:05:30 INFO namenode.NameNode: SHUTDOWN_MSG: /<span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span>SHUTDOWN_MSG: Shutting down NameNode at slave02.example.com/127.0.0.1<span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span>/</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li></ul>

（11）格式化HDFS信息复制到slave03.example.com服务器

<code class="hljs erlang-repl has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[<span class="hljs-function_or_atom" style="box-sizing: border-box;">root</span>@<span class="hljs-function_or_atom" style="box-sizing: border-box;">slave02</span> ~]# <span class="hljs-function_or_atom" style="box-sizing: border-box;">scp</span> -<span class="hljs-function_or_atom" style="box-sizing: border-box;">r</span> /<span class="hljs-function_or_atom" style="box-sizing: border-box;">sparkLearning</span>/<span class="hljs-function_or_atom" style="box-sizing: border-box;">hadoop</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>/<span class="hljs-function_or_atom" style="box-sizing: border-box;">tmp</span>/ <span class="hljs-function_or_atom" style="box-sizing: border-box;">slave01</span>.<span class="hljs-function_or_atom" style="box-sizing: border-box;">example</span>.<span class="hljs-function_or_atom" style="box-sizing: border-box;">com</span>:/<span class="hljs-function_or_atom" style="box-sizing: border-box;">sparkLearning</span>/<span class="hljs-function_or_atom" style="box-sizing: border-box;">hadoop</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>/<span class="hljs-function_or_atom" style="box-sizing: border-box;">fsimage_0000000000000000000</span>.<span class="hljs-function_or_atom" style="box-sizing: border-box;">md5</span>               <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">%   62     0.1KB/s   00:00    </span><span class="hljs-function_or_atom" style="box-sizing: border-box;">seen_txid</span>                                     <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">%    2     0.0KB/s   00:00    </span><span class="hljs-function_or_atom" style="box-sizing: border-box;">fsimage_0000000000000000000</span>                   <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">%  350     0.3KB/s   00:00    </span><span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">VERSION</span>                                       <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">%  200     0.2KB/s   00:00   </span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul>

（12）格式化ZK(在slave02.example.com上执行即可)

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root@slave02 hadoop]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># hdfs zkfc -formatZK</span>Java HotSpot(TM) <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span>-Bit Server VM warning: You have loaded library /sparkLearning/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/lib/native/libhadoop<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.so</span> which might have disabled stack guard. The VM will try to fix the stack guard now.......省略无关信息...............//执行成功<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">09</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">18</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">06</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span> INFO ha<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ActiveStandbyElector</span>: Successfully created /hadoop-ha/ns1 <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> ZK.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">09</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">18</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">06</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span> INFO zookeeper<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ZooKeeper</span>: Session: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0x34fe096c3ca0000</span> closed<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">09</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">18</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">06</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span> INFO zookeeper<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ClientCnxn</span>: EventThread shut down</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li></ul>

（13）启动HDFS(在slave02.example.com上执行)

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave02</span> hadoop]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># start-dfs.sh</span>[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave02</span> hadoop]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># jps</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7714</span> <span class="hljs-constant" style="box-sizing: border-box;">QuorumPeerMain</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6598</span> <span class="hljs-constant" style="box-sizing: border-box;">JournalNode</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8295</span> <span class="hljs-constant" style="box-sizing: border-box;">DataNode</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8202</span> <span class="hljs-constant" style="box-sizing: border-box;">NameNode</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8716</span> <span class="hljs-constant" style="box-sizing: border-box;">Jps</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8574</span> <span class="hljs-constant" style="box-sizing: border-box;">DFSZKFailoverController</span>[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave02</span> hadoop]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ssh slave01.example.com</span><span class="hljs-constant" style="box-sizing: border-box;">Last</span> <span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">login:</span> <span class="hljs-constant" style="box-sizing: border-box;">Thu</span> <span class="hljs-constant" style="box-sizing: border-box;">Aug</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">06</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">24</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2015</span> from slave01.example.com[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># jps</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13744</span> <span class="hljs-constant" style="box-sizing: border-box;">DataNode</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13681</span> <span class="hljs-constant" style="box-sizing: border-box;">NameNode</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">11862</span> <span class="hljs-constant" style="box-sizing: border-box;">QuorumPeerMain</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14007</span> <span class="hljs-constant" style="box-sizing: border-box;">Jps</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13943</span> <span class="hljs-constant" style="box-sizing: border-box;">DFSZKFailoverController</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13851</span> <span class="hljs-constant" style="box-sizing: border-box;">JournalNode</span>[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave03</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># jps</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5876</span> <span class="hljs-constant" style="box-sizing: border-box;">JournalNode</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7652</span> <span class="hljs-constant" style="box-sizing: border-box;">Jps</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7068</span> <span class="hljs-constant" style="box-sizing: border-box;">DataNode</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6764</span> <span class="hljs-constant" style="box-sizing: border-box;">QuorumPeerMain</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li></ul>

（14）启动YARN(在slave01.example.com上执行)

<code class="hljs r has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">//slave01.example.com[root@slave01 ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># start-yarn.sh</span><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">...</span>输出省略.....[root@slave01 ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># jps</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14528</span> Jps<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13744</span> DataNode<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13681</span> NameNode<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14228</span> NodeManager<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">11862</span> QuorumPeerMain<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13943</span> DFSZKFailoverController<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14138</span> ResourceManager<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13851</span> JournalNode</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li></ul>

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">//slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>[root@slave02 ~]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># jps</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">11216</span> Jps<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10656</span> JournalNode<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7714</span> QuorumPeerMain<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">11010</span> NodeManager<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10427</span> DataNode<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10844</span> DFSZKFailoverController<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10334</span> NameNode</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li></ul>

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">//slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>[root@slave03 ~]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># jps</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8610</span> JournalNode<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8791</span> NodeManager<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8503</span> DataNode<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">9001</span> Jps<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6764</span> QuorumPeerMain</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li></ul>

（15）查看hadoop运行管理界面

打开浏览器，输入http://slave01.example.com:8088/，可以得到hadoop集群管理界面: 

输入http://slave01.example.com:50070 可以得到HDFS管理界面 

至此Hadoop集群配置成功

3. Spark 1.5.0 集群部署

（1）将Spark添加到环境变量

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> hadoop]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cp /mnt/hgfs/share/spark-1.5.0-bin-hadoop2.4.tgz /sparkLearning/</span>[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># tar -zxvf spark-1.5.0-bin-hadoop2.4.tgz > /dev/null</span>[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># vim /etc/profile</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul>

将/etc/profile内容修改如下：

<code class="hljs bash has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> JAVA_HOME=/sparkLearning/jdk1.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8.0</span>_40<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> SCALA_HOME=/sparkLearning/scala-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.10</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> HADOOP_HOME=/sparkLearning/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> SPARK_HOME=/sparkLearning/spark-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.5</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>-bin-hadoop2.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> PATH=<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${JAVA_HOME}</span>/bin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${SCALA_HOME}</span>/bin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${HADOOP_HOME}</span>/bin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${HADOOP_HOME}</span>/sbin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${SPARK_HOME}</span>/bin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${SPARK_HOME}</span>/sbin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">$PATH</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>

（2）将Spark添加到环境变量

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root@slave01 sparkLearning]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># cd spark-1.5.0-bin-hadoop2.4/conf</span>[root@slave01 conf]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># ls</span>docker<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.properties</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.template</span>  metrics<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.properties</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.template</span>   spark-env<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.sh</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.template</span>fairscheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xml</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.template</span>  slaves<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.template</span>log4j<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.properties</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.template</span>   spark-defaults<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.conf</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.template</span>//复制模板文件[root@slave01 conf]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># cp spark-env.sh.template spark-env.sh</span>[root@slave01 conf]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># vim spark-env.sh</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li></ul>

在spark-env.sh文件中添加如下内容：

<code class="hljs bash has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> JAVA_HOME=/sparkLearning/jdk1.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8.0</span>_40<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> SCALA_HOME=/sparkLearning/scala-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.10</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> HADOOP_CONF_DIR=/sparkLearning/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>/etc/hadoop</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> conf]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cp slaves.template slaves</span>[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> conf]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># vim slaves</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>

slaves文件内容如下：

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># A Spark Worker will be started on each of the machines listed below.</span>slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>

（3）将配置信息复制到其它服务器

<code class="hljs r has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root@slave01 sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># scp /etc/profile slave02.example.com:/etc/profile</span>profile                                       <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2123</span>     <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.</span>1KB/s   <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>    [root@slave01 sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># scp /etc/profile slave03.example.com:/etc/profile</span>profile                                       <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2123</span>     <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.</span>1KB/s   <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>    [root@slave01 sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># vim /etc/profile</span>[root@slave01 sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># scp -r spark-1.5.0-bin-hadoop2.4 slave02.example.com:/sparkLearning/</span><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">...</span>执行过程省略.....[root@slave01 sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># scp -r spark-1.5.0-bin-hadoop2.4 slave03.example.com:/sparkLearning/</span><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">...</span>执行过程省略.....</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li></ul>

（4）启动Spark集群

因为本人机器上装了Ambari Server，占用了8080端口，而Spark Master默认端是8080，因此将sbin/start-master.sh中的SPARK_MASTER_WEBUI_PORT修改为8888

<code class="hljs bash has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">if</span> [ <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">$SPARK_MASTER_WEBUI_PORT</span>"</span> = <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">""</span> ]; <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">then</span>  SPARK_MASTER_WEBUI_PORT=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8888</span><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">fi</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root@slave01 sbin]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># ./start-all.sh </span>starting org<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.apache</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.spark</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.deploy</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.master</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Master</span>, logging to /sparkLearning/spark-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.5</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span>-bin-hadoop2<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>/sbin/../logs/spark-root-org<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.apache</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.spark</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.deploy</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.master</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Master</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>-slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.out</span><span class="hljs-label" style="box-sizing: border-box;">slave03.example.com:</span> starting org<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.apache</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.spark</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.deploy</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.worker</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Worker</span>, logging to /sparkLearning/spark-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.5</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span>-bin-hadoop2<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>/sbin/../logs/spark-root-org<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.apache</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.spark</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.deploy</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.worker</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Worker</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>-slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.out</span><span class="hljs-label" style="box-sizing: border-box;">slave02.example.com:</span> starting org<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.apache</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.spark</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.deploy</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.worker</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Worker</span>, logging to /sparkLearning/spark-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.5</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span>-bin-hadoop2<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>/sbin/../logs/spark-root-org<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.apache</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.spark</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.deploy</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.worker</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Worker</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>-slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.out</span><span class="hljs-label" style="box-sizing: border-box;">slave01.example.com:</span> starting org<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.apache</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.spark</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.deploy</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.worker</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Worker</span>, logging to /sparkLearning/spark-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.5</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span>-bin-hadoop2<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>/sbin/../logs/spark-root-org<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.apache</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.spark</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.deploy</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.worker</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Worker</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>-slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.out</span>[root@slave01 sbin]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># jps</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13744</span> DataNode<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13681</span> NameNode<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14228</span> NodeManager<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16949</span> Master<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">11862</span> QuorumPeerMain<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13943</span> DFSZKFailoverController<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14138</span> ResourceManager<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13851</span> JournalNode<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">17179</span> Jps<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">17087</span> Worker</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li></ul>

浏览器中输入slave01.example.com:8888 
 
但是在启动过程中出现了错误，查看日志文件

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave02</span> logs]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># more spark-root-org.apache.spark.deploy.worker.Worker-1-slave02.example.com.out</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

日志内容中包括下列错误：

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-label" style="box-sizing: border-box;">akka.actor.ActorNotFound:</span> Actor not found for: ActorSelection[Anchor(akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.tcp</span>://sparkMaster@slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7077</span>/), Path(/user/Master)]    at akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.actor</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ActorSelection</span>$$anonfun$resolveOne$1<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.apply</span>(ActorSelection.<span class="hljs-label" style="box-sizing: border-box;">scala:</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">65</span>)at akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.actor</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ActorSelection</span>$$anonfun$resolveOne$1<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.apply</span>(ActorSelection.<span class="hljs-label" style="box-sizing: border-box;">scala:</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">63</span>)    at scala<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.concurrent</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.impl</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.CallbackRunnable</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.run</span>(Promise<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.scala</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span>)    at akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.dispatch</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.BatchingExecutor</span>$AbstractBatch<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.processBatch</span>(BatchingExe<span class="hljs-label" style="box-sizing: border-box;">cutor.scala:</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">55</span>)    at akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.dispatch</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.BatchingExecutor</span>$Batch<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.run</span>(BatchingExecutor<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.scala</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">73</span>)    at akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.dispatch</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ExecutionContexts</span>$sameThreadExecutionContext$<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.unbatched</span>Execute(Future<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.scala</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">74</span>)    at akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.dispatch</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.BatchingExecutor</span>$class<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.execute</span>(BatchingExecutor<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.scala</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20</span>)    at akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.dispatch</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ExecutionContexts</span>$sameThreadExecutionContext$<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.execute</span>(F<span class="hljs-label" style="box-sizing: border-box;">uture.scala:</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">73</span>)    at scala<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.concurrent</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.impl</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.CallbackRunnable</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.executeWithValue</span>(Promise<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.scala</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">40</span>)    at scala<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.concurrent</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.impl</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Promise</span>$DefaultPromise<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.tryComplete</span>(Promise<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.scal</span><span class="hljs-label" style="box-sizing: border-box;">a:</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">248</span>)    at akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.pattern</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.PromiseActorRef</span>.$bang(AskSupport<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.scala</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">266</span>)    at akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.actor</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.EmptyLocalActorRef</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.specialHandle</span>(ActorRef<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.scala</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">533</span>)    at akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.actor</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.DeadLetterActorRef</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.specialHandle</span>(ActorRef<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.scala</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">569</span>).....省略.....................</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li></ul>

没找到具体原因，在ubuntu 10.04服务器上进行相同的配置，集群搭建却成功了（心中一万头…..），运行界面如下: 

（5）测试Spark集群

采用下列命上传spark-1.5.0-bin-hadoop2.4目录下的README.md文件到相应的根目录。

<code class="hljs livecodeserver has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"> hadoop dfs -<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">put</span> README.md </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

如下图： 

进入/spark-1.5.0-bin-hadoop2.4/bin目录，启动./spark-shell，如下图所示： 

执行REDME.md文件的wordcount操作：

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">scala> val textCount = sc<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.textFile</span>(“README<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.md</span>”)<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.filter</span>(line => line<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.contains</span>(“Spark”))<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.count</span>()</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

如下图： 

执行结果如下图： 

至此，Spark 1.5集群搭建成功。

转载: http://blog.csdn.net/lovehuangjiaju/article/details/48494737