Hadoop2.2版本编译运行

一 编译环境：
 centos5.10  64Bit， 
 jdk 1.6.0_45   
 maven 3.1.1
 protoc2.5.0


二 基础环境配置：


1、安装JDK




1.1:
    vi .bash_profile:


export PATH  
export JAVA_HOME=/data/server/1.6.0_45  
export PATH=$JAVA_HOME/bin:$PATH    #首先从JAVA_HOME下寻找可调用的java可执行程序，否则再调用centos自带的openjdk
  
    source .bash_profile


    测试下JDK是否安装成功: java -version
 
        java version "1.6.0_45"  
        Java(TM) SE Runtime Environment (build 1.6.0_45-b18)  
        Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)  


2、编译前的准备(maven)


maven官方下载地址，可以选择源码编码安装，这里就直接下载编译好的 就可以了


    wget http://mirror.bit.edu.cn/apache/maven/maven-3/3.1.1/binaries/apache-maven-3.1.1-bin.zip  
    解压文件后，同样在.bash_profile里配置环境变量
    export MAVEN_HOME=/opt/maven3.1.1  
    export PATH=$PATH:$MAVEN_HOME/bin  

验证配置是否成功: mvn -version:

Apache Maven 3.1.1 (0728685237757ffbf44136acec0402957f723d9a; 2013-09-17 23:22:22+0800)
Maven home: /data/server/apache-maven-3.1.1
Java version: 1.6.0_45, vendor: Sun Microsystems Inc.
Java home: /data/server/jdk1.6.0_45/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "2.6.18-371.11.1.el5", arch: "amd64", family: "unix"


3、编译hadoop




首先官方下载hadoop源码


wget http://mirrors.cnnic.cn/apache/hadoop/common/hadoop-2.2.0/hadoop-2.2.0-src.tar.gz  
如果是你32bit的机器，可以直接下载官方已经编译好的包，64bit的机子跑编译好的包跑不了。
    由于maven国外服务器可能连不上，先给maven配置一下国内镜像，在maven目录下，conf/settings.xml,在<mirrors></mirros>里添加，原本的不要动


<mirror>  
     <id>nexus-osc</id>  
      <mirrorOf>*</mirrorOf>  
  <name>Nexusosc</name>  
  <url>http://maven.oschina.net/content/groups/public/</url>  
</mirror>  


4.其他环境的配置；
hadoop2.2.0编译需要protoc2.5.0的支持，所以还要下载protoc，下载地址:https://code.google.com/p/protobuf/downloads/list，要下载2.5.0版本
对protoc进行编译安装前先要装几个依赖包:gcc,gcc-c++,make 如果已经安装的可以忽略


安装protoc
tar -xvf protobuf-2.5.0.tar.bz2  
cd protobuf  
make && make install  
安装完配置下环境变量，就不多说了，跟上面过程一样。


yum install gcc  
yum intall gcc-c++  
yum install make  


yum install cmake  
yum install openssl-devel  
yum install ncurses-devel  
    
5. 编译打包
   5.1 编译clean


cd hadoop2.2.0-src  
mvn clean install –DskipTests 



如果遇到以下报错:
[ERROR] class file for org.mortbay.component.AbstractLifeCycle not found
这次是遇到BUG了按照https://issues.apache.org/jira/browse/HADOOP-10110 
下载HADOOP-10110.patch，然后把该patch打上去，修正该bug


  5.2  如果mvn  clean  install  顺利完成，则进行打包：
mvn package -Pdist,native -DskipTests -Dtar 


如果顺利的话，如下显示：
[INFO] ------------------------------------------------------------------------  
[INFO] Reactor Summary:  
[INFO]   
[INFO] Apache Hadoop Main ................................ SUCCESS [3.709s]  
[INFO] Apache Hadoop Project POM ......................... SUCCESS [2.229s]  
[INFO] Apache Hadoop Annotations ......................... SUCCESS [5.270s]  
[INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.388s]  
[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [3.485s]  
[INFO] Apache Hadoop Maven Plugins ....................... SUCCESS [8.655s]  
[INFO] Apache Hadoop Auth ................................ SUCCESS [7.782s]  
[INFO] Apache Hadoop Auth Examples ....................... SUCCESS [5.731s]  
[INFO] Apache Hadoop Common .............................. SUCCESS [1:52.476s]  
[INFO] Apache Hadoop NFS ................................. SUCCESS [9.935s]  
[INFO] Apache Hadoop Common Project ...................... SUCCESS [0.110s]  
[INFO] Apache Hadoop HDFS ................................ SUCCESS [1:58.347s]  
[INFO] Apache Hadoop HttpFS .............................. SUCCESS [26.915s]  
[INFO] Apache Hadoop HDFS BookKeeper Journal ............. SUCCESS [17.002s]  
[INFO] Apache Hadoop HDFS-NFS ............................ SUCCESS [5.292s]  
[INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.073s]  
[INFO] hadoop-yarn ....................................... SUCCESS [0.335s]  
[INFO] hadoop-yarn-api ................................... SUCCESS [54.478s]  
[INFO] hadoop-yarn-common ................................ SUCCESS [39.215s]  
[INFO] hadoop-yarn-server ................................ SUCCESS [0.241s]  
[INFO] hadoop-yarn-server-common ......................... SUCCESS [15.601s]  
[INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [21.566s]  
[INFO] hadoop-yarn-server-web-proxy ...................... SUCCESS [4.754s]  
[INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [20.625s]  
[INFO] hadoop-yarn-server-tests .......................... SUCCESS [0.755s]  
[INFO] hadoop-yarn-client ................................ SUCCESS [6.748s]  
[INFO] hadoop-yarn-applications .......................... SUCCESS [0.155s]  
[INFO] hadoop-yarn-applications-distributedshell ......... SUCCESS [4.661s]  
[INFO] hadoop-mapreduce-client ........................... SUCCESS [0.160s]  
[INFO] hadoop-mapreduce-client-core ...................... SUCCESS [36.090s]  
[INFO] hadoop-yarn-applications-unmanaged-am-launcher .... SUCCESS [2.753s]  
[INFO] hadoop-yarn-site .................................. SUCCESS [0.151s]  
[INFO] hadoop-yarn-project ............................... SUCCESS [4.771s]  
[INFO] hadoop-mapreduce-client-common .................... SUCCESS [24.870s]  
[INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [3.812s]  
[INFO] hadoop-mapreduce-client-app ....................... SUCCESS [15.759s]  
[INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [6.831s]  
[INFO] hadoop-mapreduce-client-jobclient ................. SUCCESS [8.126s]  
[INFO] hadoop-mapreduce-client-hs-plugins ................ SUCCESS [2.320s]  
[INFO] Apache Hadoop MapReduce Examples .................. SUCCESS [9.596s]  
[INFO] hadoop-mapreduce .................................. SUCCESS [3.905s]  
[INFO] Apache Hadoop MapReduce Streaming ................. SUCCESS [7.118s]  
[INFO] Apache Hadoop Distributed Copy .................... SUCCESS [11.651s]  
[INFO] Apache Hadoop Archives ............................ SUCCESS [2.671s]  
[INFO] Apache Hadoop Rumen ............................... SUCCESS [10.038s]  
[INFO] Apache Hadoop Gridmix ............................. SUCCESS [6.062s]  
[INFO] Apache Hadoop Data Join ........................... SUCCESS [4.104s]  
[INFO] Apache Hadoop Extras .............................. SUCCESS [4.210s]  
[INFO] Apache Hadoop Pipes ............................... SUCCESS [9.419s]  
[INFO] Apache Hadoop Tools Dist .......................... SUCCESS [2.306s]  
[INFO] Apache Hadoop Tools ............................... SUCCESS [0.037s]  
[INFO] Apache Hadoop Distribution ........................ SUCCESS [21.579s]  
[INFO] Apache Hadoop Client .............................. SUCCESS [7.299s]  
[INFO] Apache Hadoop Mini-Cluster ........................ SUCCESS [7.347s]  
[INFO] ------------------------------------------------------------------------  
[INFO] BUILD SUCCESS  
[INFO] ------------------------------------------------------------------------  
[INFO] Total time: 11:53.144s  
[INFO] Finished at: Fri Nov 22 16:58:32 CST 2013  
[INFO] Final Memory: 70M/239M  
[INFO] ------------------------------------------------------------------------  


直到看到上面的内容那就说明编译完成了。
编译后的路径在:hadoop-2.2.0-src/hadoop-dist/target/hadoop-2.2.0


[root@localhost bin]# ./hadoop version  
Hadoop 2.2.0  
Subversion Unknown -r Unknown  
Compiled by root on 2013-11-22T08:47Z  
Compiled with protoc 2.5.0  
From source with checksum 79e53ce7994d1628b240f09af91e1af4  
This command was run using /data/hadoop-2.2.0-src/hadoop-dist/target/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0.jar  


可以看出hadoop的版本
[root@localhost hadoop-2.2.0]# file lib//native/*  
lib//native/libhadoop.a:        current ar archive  
lib//native/libhadooppipes.a:   current ar archive  
lib//native/libhadoop.so:       symbolic link to `libhadoop.so.1.0.0'  
lib//native/libhadoop.so.1.0.0: <span style="color:#ff0000;">ELF 64-bit LSB shared object, x86-64, version 1</span> (SYSV), dynamically linked, not stripped  
lib//native/libhadooputils.a:   current ar archive  
lib//native/libhdfs.a:          current ar archive  
lib//native/libhdfs.so:         symbolic link to `libhdfs.so.0.0.0'  
lib//native/libhdfs.so.0.0.0:   <span style="color:#ff0000;">ELF 64-bit LSB shared object, x86-64, version 1</span> (SYSV), dynamically linked, not stripped  


hadoop-2.2.0-src/hadoop-dist/target下的hadoop-2.2.0 则为最终自行编译生成的可执行的64bit 环境下的hadoop版本，能支持apache官方下载的hbase-0.98版本