Hadoop学习笔记（三）---hadoop的安装

1.下载hadoop
2.通过ftp上传到linux中
3.把文件复制到 /usr/local目录下面，然后解压，重命名
4.跟jdk差不多，来配置环境变量：
1）vi /etc/profile
2.修改之前设置的jdk环境变量如下：

export JAVA_HOME=/usr/local/jdkexport HADOOP_HOME=/usr/local/hadoopexport PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin

5.保存退出，然后执行以下：source /etc/profile，使环境变量生效

6.下面修改hadoop的配置文件：

1).hadoop-env.sh

vim /usr/local/hadoop/conf/hadoop-env.sh

找到下面的位置，修改如下：

# The java implementation to use.  Required. export JAVA_HOME=/usr/local/jdk

2）.下面配置core-site.xml

[root@localhost conf]# vim /usr/local/hadoop/conf/core-site.xml

这是原始文件，可见configuration里面没有任何东西：

<?xml version="1.0"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!-- Put site-specific property overrides in this file. --><configuration></configuration>

把下面的配置加到configuration中：

<configuration>   <property>     <name>fs.default.name</name>     <value>hdfs://172.21.15.184:9000</value>   </property>   <property>     <name>hadoop.tmp.dir</name>     <value>/usr/local/hadoop/tmp</value>   </property></configuration>

3）.下面修改：hdfs-site.xml

vim /usr/local/hadoop/conf/hdfs-site.xml

然后你会发现这个文件中的configuration中也是什么也没有，添加下面的代码：

<configuration>  <property>    <name>dfs.replication</name>    <value>1</value>  </property>  <property>    <name>dfs.permissions</name>    <value>false</value>  </property></configuration>

4）下面修改：mapred-site.xml

vim /usr/local/hadoop/conf/mapred-site.xml

同样的道理，添加代码如下：

<configuration>   <property>     <name>mapred.job.tracker</name>     <value>172.21.15.189:9001</value>   </property></configuration>

下面格式化一下namenode：，输出下面的内容表示成功：

[root@localhost conf]# hadoop namenode -formatWarning: $HADOOP_HOME is deprecated.15/05/31 03:14:56 INFO namenode.NameNode: STARTUP_MSG: /************************************************************STARTUP_MSG: Starting NameNodeSTARTUP_MSG:   host = localhost.localdomain/127.0.0.1STARTUP_MSG:   args = [-format]STARTUP_MSG:   version = 1.2.1STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1503152; compiled by 'mattf' on Mon Jul 22 15:23:09 PDT 2013STARTUP_MSG:   java = 1.7.0_79************************************************************/15/05/31 03:14:56 INFO util.GSet: Computing capacity for map BlocksMap15/05/31 03:14:56 INFO util.GSet: VM type       = 64-bit15/05/31 03:14:56 INFO util.GSet: 2.0% max memory = 101364531215/05/31 03:14:56 INFO util.GSet: capacity      = 2^21 = 2097152 entries15/05/31 03:14:56 INFO util.GSet: recommended=2097152, actual=209715215/05/31 03:14:56 INFO namenode.FSNamesystem: fsOwner=root15/05/31 03:14:56 INFO namenode.FSNamesystem: supergroup=supergroup15/05/31 03:14:56 INFO namenode.FSNamesystem: isPermissionEnabled=false15/05/31 03:14:56 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=10015/05/31 03:14:56 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)15/05/31 03:14:56 INFO namenode.FSEditLog: dfs.namenode.edits.toleration.length = 015/05/31 03:14:56 INFO namenode.NameNode: Caching file names occuring more than 10 times 15/05/31 03:14:57 INFO common.Storage: Image file /usr/local/hadoop/tmp/dfs/name/current/fsimage of size 110 bytes saved in 0 seconds.15/05/31 03:14:57 INFO namenode.FSEditLog: closing edit log: position=4, editlog=/usr/local/hadoop/tmp/dfs/name/current/edits15/05/31 03:14:57 INFO namenode.FSEditLog: close success: truncate to 4, editlog=/usr/local/hadoop/tmp/dfs/name/current/edits15/05/31 03:14:57 INFO common.Storage: Storage directory /usr/local/hadoop/tmp/dfs/name has been successfully formatted.15/05/31 03:14:57 INFO namenode.NameNode: SHUTDOWN_MSG: /************************************************************SHUTDOWN_MSG: Shutting down NameNode at localhost.localdomain/127.0.0.1************************************************************/

在下面的启动hadoop中，我们先设置一下ssh免密码登陆，不然待会启动的时候会让你一直输密码，很麻烦：

[root@localhost hadoop]# vim /etc/selinux/config

下面是原文件：

# This file controls the state of SELinux on the system.# SELINUX= can take one of these three values:#     enforcing - SELinux security policy is enforced.#     permissive - SELinux prints warnings instead of enforcing.#     disabled - No SELinux policy is loaded.SELINUX=enforcing# SELINUXTYPE= can take one of these two values:#     targeted - Targeted processes are protected,#     mls - Multi Level Security protection.SELINUXTYPE=targeted

设置SELINUX=disabled

保存退出。

然后[root@localhost hadoop]# ssh-keygen -t dsa，一直回车便可

然后[root@localhost /]# cd ~/.ssh

你会看到里面有这么一个文件：id_dsa.pub

输入命令：[root@localhost .ssh]# cat id_dsa.pub >> authorized_keys

然后测试一下，如果出现下面的内容，没让你输密码就对了

[root@localhost ~]# ssh localhostLast login: Sun May 31 03:32:36 2015 from 172.21.15.189

下面就可以启动hadoop了：

切换到hadoop下面的bin目录下面，然后执行：

[root@localhost bin]# start-all.sh

[root@localhost bin]# start-all.shWarning: $HADOOP_HOME is deprecated.namenode running as process 24585. Stop it first.localhost: starting datanode, logging to /usr/local/hadoop/libexec/../logs/hadoop-root-datanode-localhost.localdomain.outlocalhost: secondarynamenode running as process 24746. Stop it first.jobtracker running as process 24820. Stop it first.localhost: starting tasktracker, logging to /usr/local/hadoop/libexec/../logs/hadoop-root-tasktracker-localhost.localdomain.out

下面查看一下进程：

[root@localhost bin]# jps26582 TaskTracker24746 SecondaryNameNode26669 Jps24820 JobTracker26373 DataNode24585 NameNode

显示上面的内容，恭喜你安装成功了