CentOS6.8编译Hadoop-2.4.1

来源:互联网 发布:淘宝网官网首页手机版 编辑:程序博客网 时间:2024/06/14 05:48

        前面我写过一篇怎样编译hadoop-2.2.0的博客,今天我又试着编译了一下hadoop-2.4.1版本,编译的步骤跟编译hadoop-2.2.0的时候基本一样,编译hadoop-2.2.0的博客地址是:

http://blog.csdn.net/u012453843/article/details/52903324

        我这里就只说下我遇到的问题

1.一定要确保服务器能够连网,因为要下载很多东西,如果不能连网,将无法成功编译

2.编译的过程中碰到了如下问题,后来从网上查找答案,发现是hadoop-2.4.1-src/hadoop-hdfs-project/hadoop-hdfs-httpfs/downloads目录下文件没有下载完全,缺少apache-tomcat-6.0.36.tar.gz,我们便下载一个并把它放到这个目录下,然后重新编译。

[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (dist) on project hadoop-hdfs-httpfs: An Ant BuildException has occured: Can't get http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.36/bin/apache-tomcat-6.0.36.tar.gz to /usr/local/hadoop/hadoop-2.4.1-src/hadoop-hdfs-project/hadoop-hdfs-httpfs/downloads/apache-tomcat-6.0.36.tar.gz
[ERROR] around Ant part ...<get dest="downloads/apache-tomcat-6.0.36.tar.gz" skipexisting="true" verbose="true" src="http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.36/bin/apache-tomcat-6.0.36.tar.gz"/>... @ 5:182 in /usr/local/hadoop/hadoop-2.4.1-src/hadoop-hdfs-project/hadoop-hdfs-httpfs/target/antrun/build-main.xml
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :hadoop-hdfs-httpfs
[root@test hadoop-2.4.1-src]# ^C
[root@test hadoop-2.4.1-src]# 

        经过了一段时间的编译终于编译成功

[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop Main ................................ SUCCESS [5.031s]
[INFO] Apache Hadoop Project POM ......................... SUCCESS [3.855s]
[INFO] Apache Hadoop Annotations ......................... SUCCESS [7.075s]
[INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.569s]
[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [3.354s]
[INFO] Apache Hadoop Maven Plugins ....................... SUCCESS [9.588s]
[INFO] Apache Hadoop MiniKDC ............................. SUCCESS [8.882s]
[INFO] Apache Hadoop Auth ................................ SUCCESS [9.300s]
[INFO] Apache Hadoop Auth Examples ....................... SUCCESS [5.570s]
[INFO] Apache Hadoop Common .............................. SUCCESS [3:25.938s]
[INFO] Apache Hadoop NFS ................................. SUCCESS [18.331s]
[INFO] Apache Hadoop Common Project ...................... SUCCESS [0.082s]
[INFO] Apache Hadoop HDFS ................................ SUCCESS [6:12.442s]
[INFO] Apache Hadoop HttpFS .............................. SUCCESS [40.572s]
[INFO] Apache Hadoop HDFS BookKeeper Journal ............. SUCCESS [22.507s]
[INFO] Apache Hadoop HDFS-NFS ............................ SUCCESS [10.303s]
[INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.143s]
[INFO] hadoop-yarn ....................................... SUCCESS [0.134s]
[INFO] hadoop-yarn-api ................................... SUCCESS [3:06.873s]
[INFO] hadoop-yarn-common ................................ SUCCESS [1:13.959s]
[INFO] hadoop-yarn-server ................................ SUCCESS [0.167s]
[INFO] hadoop-yarn-server-common ......................... SUCCESS [26.176s]
[INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [31.386s]
[INFO] hadoop-yarn-server-web-proxy ...................... SUCCESS [8.314s]
[INFO] hadoop-yarn-server-applicationhistoryservice ...... SUCCESS [15.003s]
[INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [34.895s]
[INFO] hadoop-yarn-server-tests .......................... SUCCESS [0.996s]
[INFO] hadoop-yarn-client ................................ SUCCESS [14.592s]
[INFO] hadoop-yarn-applications .......................... SUCCESS [0.129s]
[INFO] hadoop-yarn-applications-distributedshell ......... SUCCESS [7.437s]
[INFO] hadoop-yarn-applications-unmanaged-am-launcher .... SUCCESS [5.101s]
[INFO] hadoop-yarn-site .................................. SUCCESS [0.125s]
[INFO] hadoop-yarn-project ............................... SUCCESS [9.303s]
[INFO] hadoop-mapreduce-client ........................... SUCCESS [0.138s]
[INFO] hadoop-mapreduce-client-core ...................... SUCCESS [53.614s]
[INFO] hadoop-mapreduce-client-common .................... SUCCESS [50.496s]
[INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [7.640s]
[INFO] hadoop-mapreduce-client-app ....................... SUCCESS [23.926s]
[INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [23.072s]
[INFO] hadoop-mapreduce-client-jobclient ................. SUCCESS [2:02.021s]
[INFO] hadoop-mapreduce-client-hs-plugins ................ SUCCESS [6.489s]
[INFO] Apache Hadoop MapReduce Examples .................. SUCCESS [22.299s]
[INFO] hadoop-mapreduce .................................. SUCCESS [10.712s]
[INFO] Apache Hadoop MapReduce Streaming ................. SUCCESS [30.326s]
[INFO] Apache Hadoop Distributed Copy .................... SUCCESS [58.770s]
[INFO] Apache Hadoop Archives ............................ SUCCESS [6.264s]
[INFO] Apache Hadoop Rumen ............................... SUCCESS [16.981s]
[INFO] Apache Hadoop Gridmix ............................. SUCCESS [11.848s]
[INFO] Apache Hadoop Data Join ........................... SUCCESS [6.313s]
[INFO] Apache Hadoop Extras .............................. SUCCESS [6.905s]
[INFO] Apache Hadoop Pipes ............................... SUCCESS [13.115s]
[INFO] Apache Hadoop OpenStack support ................... SUCCESS [12.148s]
[INFO] Apache Hadoop Client .............................. SUCCESS [11.822s]
[INFO] Apache Hadoop Mini-Cluster ........................ SUCCESS [0.269s]
[INFO] Apache Hadoop Scheduler Load Simulator ............ SUCCESS [36.518s]
[INFO] Apache Hadoop Tools Dist .......................... SUCCESS [20.957s]
[INFO] Apache Hadoop Tools ............................... SUCCESS [0.067s]
[INFO] Apache Hadoop Distribution ........................ SUCCESS [56.226s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 29:14.596s
[INFO] Finished at: Tue Nov 15 00:22:34 PST 2016
[INFO] Final Memory: 77M/237M
[INFO] ------------------------------------------------------------------------
[root@test hadoop-2.4.1-src]# ls
BUILDING.txt       hadoop-client          hadoop-hdfs-project       hadoop-minicluster   hadoop-tools         NOTICE.txt
dev-support        hadoop-common-project  hadoop-mapreduce-project  hadoop-project       hadoop-yarn-project  pom.xml
hadoop-assemblies  hadoop-dist            hadoop-maven-plugins      hadoop-project-dist  LICENSE.txt          README.txt
[root@test hadoop-2.4.1-src]# cd hadoop-dist/
[root@test hadoop-dist]# ls
pom.xml  target
[root@test hadoop-dist]# cd target/
[root@test target]# ls
antrun       dist-tar-stitching.sh  hadoop-2.4.1.tar.gz    hadoop-dist-2.4.1-javadoc.jar  maven-archiver
dist-layout-stitching.sh  hadoop-2.4.1           hadoop-dist-2.4.1.jar  javadoc-bundle-options         test-dir
[root@test target]# 

0 0
原创粉丝点击