Hadoop2.2.0伪分布式环境搭建（附：64位下编译Hadoop-2.2.0过程）
来源：互联网 发布：房产拍卖淘宝司法拍卖 编辑：程序博客网 时间：2024/04/30 13:41
Hadoop2.2.0伪分布式环境搭建：
写在前面：Hadoop2.2.0默认是支持32位的OS，如果想要在64位OS下运行的话，可以通过在64位OS下面编译Hadoop2.2.0来实现，编译的操作步骤在最后面呈现。
1：
操作：下载软件；
文件：Hadoop-2.2.0.tar.gz
步骤：
到http://mirrors.cnnic.cn/apache/hadoop/common/下面选择对应的版本号下载
或者使用编译过后的支持64位OS的包
hadoop-2.2.0.x86_64.tar.gz_免费高速下载|百度云网盘-分享无限制
http://pan.baidu.com/s/1eQAB4rg
2：
操作：设置环境变量；
文件：/etc/profile
步骤：
sudovim /etc/profile
向文件尾部添加以下内容：
<span style="font-size:18px;">export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-amd64export JRE_HOME=$JAVA_HOME/jreexport HADOOP_HOME2=/home/rocketeer/Hadoop/hadoop-2.2.0export PATH=.:$JAVA_HOME/bin:$HADOOP_HOME2/bin:$HADOOP_HOME2/sbin:$PATHexport CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/jre/lib:$CLASSPATH export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_PREFIX}/lib/nativeexport HADOOP_OPTS="-Djava.library.path=$HADOOP_PREFIX/lib"</span>
3：
操作：设置主机名（本条必须配置，不然在格式化namenode的时候会报不能识别主机名的错误）
文件：/etc/hostname
/etc/hosts
步骤：
sudo vim /etc/hostname
打开vim编辑器，把里面的内容修改为：
rocketeer
sudovim /etc/hosts
将开头两行改为以下内容
<span style="font-size:18px;"> 127.0.0.1 localhost.localdomain localhost 127.0.0.1 new-hostname.localdomain new-hostname</span>
或者改为
<span style="font-size:18px;"> 127.0.0.1 localhost 192.168.159.148 rocketeer</span>
验证：
然后重启虚拟机即可，重启之后输入命令查看是否修改成功：
Hostname
Hostname –f
输入以上两条命令，看是否都返回最新的主机名rocketeer
4：
操作：设置无密码登陆SSH
步骤：
sudo apt-getinstall ssh
安装完成后会在~目录（当前用户主目录，即这里的/home/hduser）下产生一个隐藏文件夹.ssh（ls -a 可以查看隐藏文件）。如果没有这个文件，自己新建即可：
mkdir .ssh
cd .ssh
ssh-keygen -t rsa之后一路回车（产生秘钥）
id_rsa.pub追加到授权的 key 里面去
cat id_rsa.pub >> authorized_keys
重启 SSH服务命令使其生效
service ssh restart
最后执行
ssh localhost
实现无密码连接localhost
5：
操作：hadoop配置文件修改
文件：/home/rocketeer/Hadoop/hadoop-2.2.0/etc/hadoop下面的文件
core-site.xml hadoop-env.sh hdfs-site.xml masters slaves mapred-site.xml
yarn-site.xml
配置文件的添加和修改都在<configuration></configuration>中
步骤：
cd Hadoop/hadoop-2.2.0/etc/hadoop
（1）通过core-site.xml配置namenode和tmp目录
sudo vim core-site.xml
添加以下内容
<span style="font-size:18px;"> <configuration> <property> <name>fs.defaultFS</name> <value>hdfs://localhost:9000</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/home/rocketeer/Hadoop/hadoop2_tmp</value> </property> </configuration></span>
（2）通过hadoop-env.sh配置java的路径（必须配置）
<span style="font-size:18px;"> export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-amd64</span>
（3）通过hdfs-site.xml配置namenode和 datanode存储路径
sudo vim hdfs-site.xml
添加以下内容
<span style="font-size:18px;"> <property> <name>dfs.namenode.name.dir</name> <value>${hadoop.tmp.dir}/namenode</value> </property> <property> <name>dfs.datanode.data.dir</name> <value>${hadoop.tmp.dir}/datanode</value> </property></span>
（4）通过mapred-site.xmlhadoop2.0有了yarn所以原来的mapred配置都转向yarn-site.xml文件中了，这里也就指定yarn。
mvmapred-site.xml.template mapred-site.xml
sudo vim mapred-site.xml
添加以下内容
<span style="font-size:18px;"> <property> <name>mapreduce.job.tracker</name> <value>http://127.0.0.1:9001</value> </property> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> <property> <name>mapreduce.system.dir</name> <value>/mapred/system</value> <final>true</final> </property> <property> <name>mapred.local.dir</name> <value>/mapred/local</value> <final>true</final> </property></span>
（5）yarn-site.xml为了简单，快速做测试，使用默认的即可。
（6）masters其实这里可以不配置，其实是指定secondenamenode
（7）slaves指定datanode和tasknode
（8）由于是伪分布式，所以master和slaves都是localhost，如果是真的分布式，这里只想的主机是不一样的，可以和链接里面的博客对比
6：
操作：格式化namenode
步骤：
Cd Hadoop/hadoop-2.2.0/bin
在这个目录下面输入命令：
hdfs namenode –format
显示下列信息，则说明格式化成功：
<span style="font-size:18px;"> rocketeer@rocketeer:~/Hadoop/hadoop-2.2.0/bin$ hdfs namenode -format14/07/12 23:02:20 INFOnamenode.NameNode: STARTUP_MSG: /************************************************************STARTUP_MSG: Starting NameNodeSTARTUP_MSG: host = rocketeer/202.106.199.36STARTUP_MSG: args = [-format]STARTUP_MSG: version = 2.2.0STARTUP_MSG: classpath =/home/rocketeer/Hadoop/hadoop-2.2.0/etc/hadoop:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/jets3t-0.6.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/jettison-1.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/hadoop-annotations-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/commons-lang-2.5.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/commons-el-1.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/commons-math-2.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/hadoop-auth-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/jackson-xc-1.8.8.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/commons-httpclient-3.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/servlet-api-2.5.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/commons-codec-1.4.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/mockito-all-1.8.5.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/commons-cli-1.2.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/zookeeper-3.4.5.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/activation-1.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/asm-3.2.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/jetty-util-6.1.26.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/commons-collections-3.2.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/commons-io-2.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/xz-1.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/jackson-core-asl-1.8.8.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/commons-compress-1.4.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/guava-11.0.2.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/jetty-6.1.26.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/avro-1.7.4.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/commons-digester-1.8.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/commons-net-3.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/commons-configuration-1.6.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/xmlenc-0.52.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/jsp-api-2.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/jersey-json-1.9.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/jersey-server-1.9.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/paranamer-2.3.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/netty-3.6.2.Final.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/jsr305-1.3.9.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/junit-4.8.2.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/stax-api-1.0.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/jsch-0.1.42.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/log4j-1.2.17.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/commons-logging-1.1.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/jackson-jaxrs-1.8.8.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/jersey-core-1.9.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0-tests.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-nfs-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/commons-lang-2.5.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/commons-el-1.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/asm-3.2.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/commons-io-2.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/jackson-core-asl-1.8.8.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/guava-11.0.2.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.8.8.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/commons-logging-1.1.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/hadoop-hdfs-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/hadoop-hdfs-nfs-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/hadoop-hdfs-2.2.0-tests.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/hadoop-annotations-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/snappy-java-1.0.4.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/javax.inject-1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/asm-3.2.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/junit-4.10.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/commons-io-2.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/hamcrest-core-1.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/xz-1.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/jackson-core-asl-1.8.8.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/avro-1.7.4.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.8.8.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/guice-3.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/jersey-server-1.9.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/paranamer-2.3.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/jersey-core-1.9.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-client-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-common-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-common-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-site-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-api-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/javax.inject-1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/asm-3.2.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/junit-4.10.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/commons-io-2.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/hamcrest-core-1.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/xz-1.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/jackson-core-asl-1.8.8.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/guice-3.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0-tests.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.2.0.jar:/home/rocketeer/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.2.0.jar:/contrib/capacity-scheduler/*.jarSTARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common-r 1529768; compiled by 'hortonmu' on 2013-10-07T06:28ZSTARTUP_MSG: java = 1.7.0_55************************************************************/14/07/12 23:02:20 INFOnamenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]OpenJDK 64-Bit Server VM warning:You have loaded library/home/rocketeer/Hadoop/hadoop-2.2.0/lib/native/libhadoop.so.1.0.0 which mighthave disabled stack guard. The VM will try to fix the stack guard now.It's highly recommended that youfix the library with 'execstack -c <libfile>', or link it with '-znoexecstack'.14/07/12 23:02:48 WARNutil.NativeCodeLoader: Unable to load native-hadoop library for yourplatform... using builtin-java classes where applicable14/07/12 23:02:49 WARN common.Util:Path $HADOOP_HDFS2/namenode should be specified as a URI in configurationfiles. Please update hdfs configuration.14/07/12 23:02:49 WARN common.Util:Path $HADOOP_HDFS2/namenode should be specified as a URI in configurationfiles. Please update hdfs configuration.Formatting using clusterid:CID-e6deb9bd-356a-4dea-ba0e-d3af695a6c5214/07/12 23:02:49 INFOnamenode.HostFileManager: read includes:HostSet()14/07/12 23:02:50 INFOnamenode.HostFileManager: read excludes:HostSet()14/07/12 23:02:50 INFOblockmanagement.DatanodeManager: dfs.block.invalidate.limit=100014/07/12 23:02:50 INFO util.GSet:Computing capacity for map BlocksMap14/07/12 23:02:50 INFO util.GSet:VM type = 64-bit14/07/12 23:02:50 INFO util.GSet:2.0% max memory = 966.7 MB14/07/12 23:02:50 INFO util.GSet:capacity = 2^21 = 2097152 entries14/07/12 23:02:50 INFOblockmanagement.BlockManager: dfs.block.access.token.enable=false14/07/12 23:02:50 INFOblockmanagement.BlockManager: defaultReplication = 314/07/12 23:02:50 INFOblockmanagement.BlockManager: maxReplication = 51214/07/12 23:02:50 INFOblockmanagement.BlockManager: minReplication = 114/07/12 23:02:50 INFOblockmanagement.BlockManager: maxReplicationStreams = 214/07/12 23:02:50 INFOblockmanagement.BlockManager: shouldCheckForEnoughRacks = false14/07/12 23:02:50 INFOblockmanagement.BlockManager: replicationRecheckInterval = 300014/07/12 23:02:50 INFOblockmanagement.BlockManager: encryptDataTransfer =false14/07/12 23:02:50 INFOnamenode.FSNamesystem: fsOwner = rocketeer (auth:SIMPLE)14/07/12 23:02:50 INFOnamenode.FSNamesystem: supergroup = supergroup14/07/12 23:02:50 INFOnamenode.FSNamesystem: isPermissionEnabled = true14/07/12 23:02:50 INFOnamenode.FSNamesystem: HA Enabled: false14/07/12 23:02:50 INFOnamenode.FSNamesystem: Append Enabled: true14/07/12 23:02:51 INFO util.GSet:Computing capacity for map INodeMap14/07/12 23:02:51 INFO util.GSet:VM type = 64-bit14/07/12 23:02:51 INFO util.GSet:1.0% max memory = 966.7 MB14/07/12 23:02:51 INFO util.GSet:capacity = 2^20 = 1048576 entries14/07/12 23:02:51 INFOnamenode.NameNode: Caching file names occuring more than 10 times14/07/12 23:02:51 INFO namenode.FSNamesystem:dfs.namenode.safemode.threshold-pct = 0.999000012874603314/07/12 23:02:51 INFOnamenode.FSNamesystem: dfs.namenode.safemode.min.datanodes = 014/07/12 23:02:51 INFOnamenode.FSNamesystem: dfs.namenode.safemode.extension = 3000014/07/12 23:02:51 INFOnamenode.FSNamesystem: Retry cache on namenode is enabled14/07/12 23:02:51 INFOnamenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cacheentry expiry time is 600000 millis14/07/12 23:02:51 INFO util.GSet:Computing capacity for map Namenode Retry Cache14/07/12 23:02:51 INFO util.GSet:VM type = 64-bit14/07/12 23:02:51 INFO util.GSet:0.029999999329447746% max memory = 966.7 MB14/07/12 23:02:51 INFO util.GSet:capacity = 2^15 = 32768 entriesRe-format filesystem in StorageDirectory /home/rocketeer/Hadoop/hadoop-2.2.0/bin/$HADOOP_HDFS2/namenode ? (Yor N) y14/07/12 23:03:46 INFOcommon.Storage: Storage directory/home/rocketeer/Hadoop/hadoop-2.2.0/bin/$HADOOP_HDFS2/namenode has beensuccessfully formatted.14/07/12 23:03:46 INFOnamenode.FSImage: Saving image file/home/rocketeer/Hadoop/hadoop-2.2.0/bin/$HADOOP_HDFS2/namenode/current/fsimage.ckpt_0000000000000000000using no compression14/07/12 23:03:46 INFOnamenode.FSImage: Image file /home/rocketeer/Hadoop/hadoop-2.2.0/bin/$HADOOP_HDFS2/namenode/current/fsimage.ckpt_0000000000000000000of size 201 bytes saved in 0 seconds.14/07/12 23:03:46 INFOnamenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 014/07/12 23:03:46 INFO util.ExitUtil:Exiting with status 014/07/12 23:03:46 INFOnamenode.NameNode: SHUTDOWN_MSG: /************************************************************SHUTDOWN_MSG: Shutting downNameNode at rocketeer/202.106.199.36************************************************************/</span>
7：
（1）启动hadoop-2.2.0集群
Start-all.sh
（2）分步启动：
启动namenode
1 sbin/hadoop-daemon.sh start namenode
2 sbin/hadoop-daemon.sh start datanode
运行测试
1 jps
出现：
12935NameNode
5309 Jps
13012DataNode
证明启动成功，如果没有出现DataNode或者NameNode，证明启动没有成功，可以查看hadoop安装目录下的logs下的日志记录。
可以使用sbin/hadoop-daemon.shstop datanode（或namenode）来关闭。
启动Manage管理
1 sbin/yarn-daemon.sh start resourcemanager
2 sbin/yarn-daemon.sh start nodemanager
启动mapreduce history查看进程
mr-jobhistory-daemon.shstart historyserver
运行测试
1 jps
出现：
13338NodeManager
13111ResourceManager
12935NameNode
5309 Jps
13012DataNode
证明启动成功同时也可以使用yarn-daemon.shstop resourcemanager（nodemanager）来关闭。
如果没有单独配置yarn-site.xml中的yarn.resourcemanager.webapp.address，默认的端口8088访问
http://127.0.0.1:8088/ 可以访问hadoop管理页面
如果没有单独配置hdfs-site.xml中的dfs.namenode.http-address,默认端口50070
http://127.0.0.1:50070可以访问namenode节点信息。
Historyserver访问
http://127.0.0.1:19888
64位下编译Hadoop-2.2.0过程：
参考文章：
在Ubuntu12.04 64位上编译hadoop2.2.0-你若幸福，便是晴天 -博客频道 - CSDN.NET
http://blog.csdn.net/tianfei_1984/article/details/20030383
hadoop前戏配置三：hadoop2.2.0重新编译为64位，个人测试成功
http://www.360doc.com/content/14/0625/11/7839419_389587052.shtml
sudoapt-get install g++ autoconf automake libtool make cmake zlib1g-dev pkg-configlibssl-dev
下载protobuf-2.5.0.tar.gz到虚拟机
解压，依次运行
$./configure --prefix=/usr
$ sudomake
$ sudomake check
$ sudomake install
protoc--version
检查一下版本
ubuntu下用apt-get安装maven
$ sudoapt-get install maven
编译 hadoop 2.2.0
下载 hadoop 2.2.0 http://www.apache.org/dyn/closer.cgi/hadoop/common/
解压到用户目录 /home/rocketeer/Downloads
目前的2.2.0的Source Code压缩包解压出来的code有个bug需要patch后才能编译。否则编译hadoop-auth会提示上面错误。
解决办法如下：
修改下面的pom文件。该文件在hadoop源码包下寻找：
hadoop-common-project/hadoop-auth/pom.xml
打开上面的的pom文件，在54行加入如下的依赖：
<dependency>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty-util</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty</artifactId>
<scope>test</scope>
</dependency>
进入 hadoop-2.2.0-src目录
因为已经安装了maven,protobuf, java环境也有了，compiler也有了所以直接运行
$ mvnpackage -Pdist,native -DskipTests -Dtar
然后重新运行编译指令即可。编译是一个缓慢的过程，耐心等待哦。
当看到下面的信息时，编译成功。
安装配置 hadoop 2.2.0
此时编译好的文件位于hadoop-2.2.0-src/hadoop-dist/target/hadoop-2.2.0/目录中
0 0
- Hadoop2.2.0伪分布式环境搭建（附：64位下编译Hadoop-2.2.0过程）
- 在ubuntu14.04环境下搭建伪分布式hadoop-2.2.0 64位系统
- Hadoop I 搭建Linux下Hadoop2.6.0伪分布式环境
- centos下搭建单机和伪分布式hadoop环境-（4）搭建过程中的问题汇总
- mac下搭建hadoop伪分布式环境
- linux平台下的hadoop环境搭建（伪分布式）
- Hadoop小兵笔记【四】hadoop2.2.0伪分布式环境搭建问题-ssh登录失败
- Hadoop小兵笔记【五】hadoop2.2.0伪分布式环境搭建疑难-第一个用例wordcount失败
- Hadoop实战（1）_阿里云搭建Hadoop2.x的伪分布式环境
- 在Win7虚拟机下搭建Hadoop2.6.0伪分布式环境
- centos下hadoop2.6伪分布式环境搭建
- CentOS6.7 下搭建 Hadoop2.7.1 单机伪分布式环境
- CentOS 64位系统环境下Hadoop2.6伪分布式安装笔记
- ubuntu16.04(64位)+Hadoop2.7.3伪分布式环境下部署Spark
- Hadoop之——hadoop2.2.0伪分布式搭建
- 【hadoop】 1002-hadoop2.4.1伪分布式搭建
- Hadoop 2.4.0 伪分布式环境搭建
- hadoop环境搭建（二）伪分布式
- C#如何判断一个字符串是16进制字符串
- E. Cubes codeforce 180/E
- POJ 3368 Frequent values
- Quartz Core 图层编程
- ArcGIS教程：河网分级
- Hadoop2.2.0伪分布式环境搭建（附：64位下编译Hadoop-2.2.0过程）
- 软件测试技术类面试题集锦
- URI和URL的区别
- 郑州市治疗精神抑郁症
- sublime text 编译lua（详细可靠）
- 多线程和多进程的区别（小结）
- POJ 2965 The Pilots Brothers' refrigerator 搜索
- 01---Java配置环境
- sql语句中where 1=1的作用
