编译hadoop-2.2.0源码，RHEL6.3x64

编译环境：

OS:   RHEL6.3.x86_64

Kernel:2.6.32-358.el6.x86_64

JDK: 1.6.0_35  x86_64

编译前准备：

1.安装依赖的rpm包

   yum -y install  lzo-devel  zlib-devel  gcc  autoconf  automake  libtool openssl-devel ncurses-devel cmake

2.安装maven

   下载地址:  http://mirror.bit.edu.cn/apache/maven/maven-3/3.1.1/binaries/apache-maven-3.1.1-bin.tar.gz

    解压到/usr/local/maven目录下，然后编辑/etc/profile，在其中编辑MAVEN_HOME的环境变量，保存后执行：

    source /etc/profile  或  .  /etc/profile 

    使之生效。

    验证生效与否：

  # mvn -v

  Apache Maven 3.1.1 (0728685237757ffbf44136acec0402957f723d9a; 2013-09-17 23:22:22+0800)

    Maven home: /usr/local/maven

    Java version: 1.6.0_35, vendor: Sun Microsystems Inc.

    Java home: /usr/java/jdk1.6.0_35/jre

    Default locale: zh_CN, platform encoding: UTF-8

    OS name: "linux", version: "2.6.32-358.el6.x86_64", arch: "amd64", family: “unix"

3.安装protobuf   http://protobuf.googlecode.com/files/protobuf-2.5.0.tar.gz

    tar xf protobuf-2.5.0.tar.gz

    cd protobuf-2.5.0

    ./configure

    make

    make install

   

开始编译Hadoop

1.下载hadoop-2.2.0 源码。

    地址:http://mirror.bit.edu.cn/apache/hadoop/common/hadoop-2.2.0/hadoop-2.2.0-src.tar.gz 

2.tar xf hadoop-2.2.0-src.tar.gz

   cd  hadoop-2.2.0-src

3.在编译前，首先修复一个2.2.0版本的一个bug，否则会在编译过程中遇到如下报错：

 

   [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-auth: Compilation failure: Compilation failure:

    [ERROR] /home/chuan/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[84,13] cannot access org.mortbay.component.AbstractLifeCycle

    [ERROR] class file for org.mortbay.component.AbstractLifeCycle not found

  

   bug patch地址：https://issues.apache.org/jira/browse/hadoop-10110

   方法：修改hadoop源码目录下的hadoop-common-project/hadoop-auth/pom.xml

   添加或修改如下部分内容：

       

      <dependency>

          <groupId>org.mortbay.jetty</groupId>

+        <artifactId>jetty-util</artifactId>

+        <scope>test</scope>

+    </dependency>

+    <dependency>

+       <groupId>org.mortbay.jetty</groupId>

         <artifactId>jetty</artifactId>

         <scope>test</scope>

     </dependency>

 

4.在hadoop源码目录下执行：

   

   # mvn package -Pdist,native -DskipTests -Dtar -e -X

 

[INFO] Apache Hadoop Main ................................ SUCCESS [4.419s]

[INFO] Apache Hadoop Project POM ......................... SUCCESS [3.358s]

[INFO] Apache Hadoop Annotations ......................... SUCCESS [4.488s]

[INFO] Apache Hadoop Assemblies ..........................  SUCCESS [0.640s]

[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [3.900s]

[INFO] Apache Hadoop Maven Plugins ....................... SUCCESS [6.072s]

[INFO] Apache Hadoop Auth ................................ SUCCESS [4.491s]

[INFO] Apache Hadoop Auth Examples ....................... SUCCESS [4.014s]

[INFO] Apache Hadoop Common .............................. SUCCESS [1:48.272s]

[INFO] Apache Hadoop NFS ................................. SUCCESS [10.646s]

[INFO] Apache Hadoop Common Project ...................... SUCCESS [0.066s]

[INFO] Apache Hadoop HDFS ................................ SUCCESS [1:54.818s]

[INFO] Apache Hadoop HttpFS .............................. SUCCESS [20.414s]

[INFO] Apache Hadoop HDFS BookKeeper Journal ............. SUCCESS [14.205s]

[INFO] Apache Hadoop HDFS-NFS ............................ SUCCESS [5.866s]

[INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.079s]

[INFO] hadoop-yarn ....................................... SUCCESS [0.361s]

[INFO] hadoop-yarn-api ................................... SUCCESS [51.017s]

[INFO] hadoop-yarn-common ................................ SUCCESS [33.649s]

[INFO] hadoop-yarn-server ................................ SUCCESS [0.230s]

[INFO] hadoop-yarn-server-common ......................... SUCCESS [13.812s]

[INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [19.117s]

[INFO] hadoop-yarn-server-web-proxy ...................... SUCCESS [4.240s]

[INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [19.048s]

[INFO] hadoop-yarn-server-tests .......................... SUCCESS [1.012s]

[INFO] hadoop-yarn-client ................................ SUCCESS [6.523s]

[INFO] hadoop-yarn-applications .......................... SUCCESS [0.174s]

[INFO] hadoop-yarn-applications-distributedshell ......... SUCCESS [3.049s]

[INFO] hadoop-mapreduce-client ........................... SUCCESS [0.169s]

[INFO] hadoop-mapreduce-client-core ...................... SUCCESS [29.535s]

[INFO] hadoop-yarn-applications-unmanaged-am-launcher .... SUCCESS [3.287s]

[INFO] hadoop-yarn-site .................................. SUCCESS [0.211s]

[INFO] hadoop-yarn-project ............................... SUCCESS [6.975s]

[INFO] hadoop-mapreduce-client-common .................... SUCCESS [24.512s]

[INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [3.984s]

[INFO] hadoop-mapreduce-client-app ....................... SUCCESS [13.070s]

[INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [6.213s]

[INFO] hadoop-mapreduce-client-jobclient ................. SUCCESS [5.803s]

[INFO] hadoop-mapreduce-client-hs-plugins ................ SUCCESS [2.423s]

[INFO] Apache Hadoop MapReduce Examples .................. SUCCESS [8.608s]

[INFO] hadoop-mapreduce .................................. SUCCESS [5.522s]

[INFO] Apache Hadoop MapReduce Streaming ................. SUCCESS [6.932s]

[INFO] Apache Hadoop Distributed Copy .................... SUCCESS [10.974s]

[INFO] Apache Hadoop Archives ............................ SUCCESS [3.047s]

[INFO] Apache Hadoop Rumen ............................... SUCCESS [8.680s]

[INFO] Apache Hadoop Gridmix ............................. SUCCESS [6.245s]

[INFO] Apache Hadoop Data Join ........................... SUCCESS [3.823s]

[INFO] Apache Hadoop Extras .............................. SUCCESS [4.784s]

[INFO] Apache Hadoop Pipes ............................... SUCCESS [7.803s]

[INFO] Apache Hadoop Tools Dist .......................... SUCCESS [2.759s]

[INFO] Apache Hadoop Tools ............................... SUCCESS [0.046s]

[INFO] Apache Hadoop Distribution ........................ SUCCESS [20.150s]

[INFO] Apache Hadoop Client .............................. SUCCESS [8.368s]

[INFO] Apache Hadoop Mini-Cluster ........................ SUCCESS [0.278s]

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 10:56.567s

[INFO] Finished at: Wed Dec 25 23:13:59 CST 2013

[INFO] Final Memory: 59M/121M

[INFO] ————————————————————————————————————

 

   直到最后出现BUILD SUCCESS，说明编译成功。

   源码目录下  hadoop-dist/target/hadoop-2.2.0.tar.gz 就是我们最终编译成功的hadoop包。