Hadoop - single node setup

来源：互联网发布：2016淘宝开店得多少钱编辑：程序博客网时间：2024/06/06 02:48

1. softwares:

1. VMware_workstation_10.0.4.exe

{ HA4AM-AM38K-CZH39-XKC7K-23871 | 4A27Q-FHKEL-WZQH1-Y21QP-0A5H6 | HY21G-01J9P-GZWM8-R0976-0CJ65 }

2. ubuntu-12.04-desktop-i386.iso

3. jdk-7u75-linux-i586.tar.gz

4. hadoop-1.0.3.tar.gz

2. uncompress tar.gz:

tar -xvf jdk-7u75-linux-i586.tar.gz -----> jdk1.7.0_75

tar -xvf hadoop-1.0.3.tar.gz -----> hadoop-1.0.3

3. java environment configuration :

    export JAVA_HOME=/home/hadoop/java/jdk1.7.0_75
        export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
        export PATH=$JAVA_HOME/bin:$PATH

4. hadoop environment configuration :

sudo apt-get ssh

sudo apt-get rsync

$ ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

2.1 /conf/core-site.xml [ hdfs enter configuration ]

                  <configuration>
                              <property>
                                        <name>fs.default.name</name>
                                        <value>hdfs://localhost:9000</value>
                              </property>
                 </configuration>

2.2 /conf/hdfs-site.xml

                <configuration>
       <property>
                                        <name>dfs.replication</name>
                                        <value>1</value>
       </property>
                </configuration>

2.3 /conf/mapred-site.xml

                <configuration>
       <property>
                                        <name>mapred.job.tracker</name>
                                        <value>localhost:9001</value>
       </property>
                </configuration>

2.4 /conf/hadoop-env.sh

export JAVA_HOME=/home/hadoop/java/jdk1.7.0_75

3. start hadoop :

      ./hadoop namenode -format
     ./hadoop fs -mkdir /input
     ./hadoop fs -put ./bin/*.sh /input
       bin/hadoop jar hadoop-examples-1.0.3.jar wordcount /input /output

4. test hadoop:

http://localhost:50030 [Job Tracker Web Page]

http://localhost:50070 [HDFS Web Page]

0 0