hadoopYanr2.3安装

环境配置

虚拟机：ubuntu12

java:jdk6

hadoop2.3: hadoop.2.3.0.tar.gz

1,java 安装路经

jdk-6u45-linux-i586.bin

/usr/lib/jdk/jdk1.6.0_45

配置环境变理

sudo  vi /etc/profile

export JAVA_HOME=/usr/lib/jdk/jdk1.6.0_45

export HADOOP_HOME=/home/lew/software/hadoop-2.3.0

export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin

export CLASS_PATH=$CLASS_PATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib

2，修改host

 vi /etc/hosts 

127.0.0.1 hadoop

lew@hadoop:~$ hostname

hadoop

3，安装hadoop (伪分步式安装)

tar -zxvf hadoop-2.3.0.tar.gz

 1)配置 hadoop-env.sh

 export JAVA_HOME=/usr/lib/jdk/jdk1.6.0_45

vi /etc/profile

export HADOOP_HOME=/home/lew/software/hadoop-2.3.0/

 2)mapred-site.xml

  <property>

     <name>mapreduce.framework.name</name>

     <value>yarn</value>

  </property>

 3)core-site.xml

 <property>

             <name>fs.default.name</name>

             <value>hdfs://yarn001:8020</value>

 </property>

<property>

                <name>hadoop.tmp.dir</name>

                <value>/home/lew/hadoop/tmp/</value>  注：默认、/tmp 目录下 机器每次重启都会此目录

        </property>

 4>yarn-site.xml

  <property>

 <name>yarn.nodemanager.aux-services</name>

 <value>mapreduce_shuffle</value>   注：如果配置有问题，启动nodamanager时，启动后会自动集合，

  </property>

5> hdfs-site.xml

<property>

 <name>dfs.repalcation</name>

 <value>1</value>

</property>

3，设置ssh 登录

 ssh-keygen -t rsa

 cd ~/.ssh/

 cat id_rsa.pub >>authorized_keys

4，启动hadoop

  bin/hadoop namenode -format

   sbin/hadoop-daemon.sh start namenode

   sbin/hadoop-daemon.sh start datanode

或 sbin/start-dfs.sh

   sbin/yarn-daemon.sh start resourcemanager

   sbin/yarn-daemon.sh start nodemanager

或 sbin/start-yarn.sh

或直接使用：sbin/start-all.sh

全部停用：sbin/stop-all.sh

5,测试hadoop 

lew@hadoop:~/software/hadoop-2.3.0$ jps
9258 NameNode
9560 ResourceManager
12309 Jps
9401 DataNode
9792 NodeManager

 http://localhost:8088/

http://localhost:8042

  

6，测试系统自带的worcount 列子

 1）准备数据：

 lew@hadoop:~/input$ echo "hello hadoop1" >test1.txt

lew@hadoop:~/input$ echo "hello lew" >test2.txt

lew@hadoop:~/input$ cat test*

hello hadoop1

hello lew

2）将数据上传hdfs 文件系统中

lew@hadoop:~/software/hadoop-2.3.0$ bin/hadoop fs -put /home/lew/input/ /input/

lew@hadoop:~/software/hadoop-2.3.0$ bin/hadoop fs -ls /input/input

Found 2 items

-rw-r--r-- 3 lew supergroup 14 2014-03-13 22:31 /input/input/test1.txt

-rw-r--r-- 3 lew supergroup 10 2014-03-13 22:31 /input/input/test2.txt

3）执行wordcout例子

bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.3.0.jar wordcount /input/input/ /output

信息如下：

lew@hadoop:~/software/hadoop-2.3.0$ bin/hadoop fs -cat /input/part-r*
hadoop  1
hello   2
world   1

请关注我的微信，阅读更多精彩文章：