[大数据-hadoop基础]window下编译hadoop2.7.4-eclipse mar2插件

编译hadoop2.7.4-eclipse mar2插件
准备条件:
1.将自编译的hadoop-2.7.4.tar.gz解压到D:\BigData\hadoop-2.7.4(管理员运行winRAR，否则会解压失败）。
2.下载eclipse安装包

eclipse-jee-oxygen-1a-win32-x86_64.zip
eclipse-jee-mars-2-linux-gtk-x86_64.tar.gz
3.下载hadoop2x-eclipse-plugin源代码(以zip格式下载,然后解压到一个合适的路径)
hadoop2x-eclipse-plugin
4.下载及配置Ant（网上百度大把教程）.
5.解压后的路径.

D:\BigData\hadoop-2.7.4
E:\Workspaces\Ant\hadoop2x-eclipse-plugin-master

6.修改依赖包版本文件libraries.properties

E:\Workspaces\Ant\hadoop2x-eclipse-plugin-master\src\ivy\libraries.properties

# This is the version of hadoop we are generatinghadoop.version=2.7.4hadoop-gpl-compression.version=0.1.0#These are the versions of our dependencies (in alphabetical order)apacheant.version=1.7.0ant-task.version=2.0.10asm.version=3.2aspectj.version=1.6.5aspectj.version=1.6.11checkstyle.version=4.2commons-cli.version=1.2commons-codec.version=1.4commons-collections.version=3.2commons-configuration.version=1.6commons-daemon.version=1.0.13commons-httpclient.version=3.0.1commons-lang.version=2.6commons-logging.version=1.0.4commons-logging-api.version=1.0.4commons-math.version=2.1commons-el.version=1.0commons-fileupload.version=1.2commons-io.version=2.4commons-net.version=3.1core.version=3.1.1coreplugin.version=1.3.2# hsqldb.version=1.8.0.10hsqldb.version=2.0.0ivy.version=2.1.0jasper.version=5.5.12jackson.version=1.8.8#not able to figureout the version of jsp & jsp-api version to get it resolved throught ivy# but still declared here as we are going to have a local copy from the lib folderjsp.version=2.1jsp-api.version=5.5.12jsp-api-2.1.version=6.1.14jsp-2.1.version=6.1.14jets3t.version=0.6.1jetty.version=6.1.26jetty-util.version=6.1.26# jersey-core.version=1.8# jersey-json.version=1.8# jersey-server.version=1.8jersey-core.version=1.9jersey-json.version=1.9jersey-server.version=1.9# junit.version=4.5junit.version=4.11jdeb.version=0.8jdiff.version=1.0.9# json.version=1.0json.version=1.1kfs.version=0.1# log4j.version=1.2.15log4j.version=1.2.17lucene-core.version=2.3.1mockito-all.version=1.8.5# jsch.version=0.1.42jsch.version=0.1.54oro.version=2.0.8rats-lib.version=0.5.1servlet.version=4.0.6servlet-api.version=2.5# slf4j-api.version=1.4.3# slf4j-log4j12.version=1.4.3slf4j-api.version=1.7.10slf4j-log4j12.version=1.7.10wagon-http.version=1.0-beta-2xmlenc.version=0.52# xerces.version=1.4.4xerces.version=2.9.1htrace.version=3.1.0-incubating

7.修改依赖包版本文件libraries.properties

E:\Workspaces\Ant\hadoop2x-eclipse-plugin-master\ivy\libraries.properties
注意:添加了一项:htrace.version=3.1.0-incubating

# This is the version of hadoop we are generatinghadoop.version=2.7.4hadoop-gpl-compression.version=0.1.0#These are the versions of our dependencies (in alphabetical order)apacheant.version=1.7.0ant-task.version=2.0.10asm.version=3.2aspectj.version=1.6.5aspectj.version=1.6.11checkstyle.version=4.2commons-cli.version=1.2commons-codec.version=1.4commons-collections.version=3.2.2commons-configuration.version=1.6commons-daemon.version=1.0.13commons-httpclient.version=3.0.1commons-lang.version=2.6commons-logging.version=1.0.4commons-logging-api.version=1.0.4commons-math.version=2.1commons-el.version=1.0commons-fileupload.version=1.2commons-io.version=2.4commons-net.version=3.1core.version=3.1.1coreplugin.version=1.3.2# hsqldb.version=1.8.0.10hsqldb.version=2.0.0ivy.version=2.1.0jasper.version=5.5.12jackson.version=1.9.13#not able to figureout the version of jsp & jsp-api version to get it resolved throught ivy# but still declared here as we are going to have a local copy from the lib folderjsp.version=2.1jsp-api.version=5.5.12jsp-api-2.1.version=6.1.14jsp-2.1.version=6.1.14jets3t.version=0.6.1jetty.version=6.1.26jetty-util.version=6.1.26# jersey-core.version=1.8# jersey-json.version=1.8# jersey-server.version=1.8jersey-core.version=1.9jersey-json.version=1.9jersey-server.version=1.9# junit.version=4.5junit.version=4.11jdeb.version=0.8jdiff.version=1.0.9# json.version=1.0json.version=1.1kfs.version=0.1# log4j.version=1.2.15log4j.version=1.2.17lucene-core.version=2.3.1mockito-all.version=1.8.5# jsch.version=0.1.42jsch.version=0.1.54oro.version=2.0.8rats-lib.version=0.5.1servlet.version=4.0.6servlet-api.version=2.5# slf4j-api.version=1.4.3# slf4j-log4j12.version=1.4.3slf4j-api.version=1.7.10slf4j-log4j12.version=1.7.10wagon-http.version=1.0-beta-2xmlenc.version=0.52# xerces.version=1.4.4xerces.version=2.9.1protobuf.version=2.5.0guava.version=11.0.2netty.version=3.6.2.Finalhtrace.version=3.1.0-incubating

以上版本号：请自行与D:\BigData\hadoop-2.7.4\share\hadoop包含的版本进行核对并进行修改。

8.修改build.xml

E:\Workspaces\Ant\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin\build.xml

<copy file="${hadoop.home}/share/hadoop/common/lib/commons-httpclient-${commons-httpclient.version}.jar"  todir="${build.dir}/lib" verbose="true"/><copy file="${hadoop.home}/share/hadoop/common/lib/servlet-api-${servlet-api.version}.jar"  todir="${build.dir}/lib" verbose="true"/><copy file="${hadoop.home}/share/hadoop/common/lib/commons-io-${commons-io.version}.jar"  todir="${build.dir}/lib" verbose="true"/>

 lib/servlet-api-${servlet-api.version}.jar, lib/commons-io-${commons-io.version}.jar,

10.编译过程:

E:>cd E:\Workspaces\Ant\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin E:\Workspaces\Ant\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin>ant jar -Dversion=2.7.4 -Dhadoop.version=2.7.4 -Declipse.home=D:\Android\eclipse -Dhadoop.home=D:\BigData\hadoop-2.7.4 -debug

最后生成：hadoop-eclipse-plugin-2.7.4.jar

E:\Workspaces\Ant\hadoop2x-eclipse-plugin-master\build\contrib\eclipse-plugin/hadoop-eclipse-plugin-2.7.4.jar

11.可能出现的问题:（如下适用linux系统，window系统只要下载对应的jar，然后拷贝到错误提示中路径即可）。
11.1 问题1:缺少包commons-collections-3.2.1.jar

BUILD FAILED
/cloud/src/hadoop2.7/hadoop2x-eclipse-plugin-master/src/contrib/eclipse-plugin/build.xml:118: Warning: Could not find file /cloud/hadoop-2.7.4/share/hadoop/common/lib/commons-collections-3.2.1.jar to copy.

解决方法:
将commons-collections-3.2.1.jar下载后放入对应的目录。
commons-collections-3.2.1.jarx下载地址

11.2 问题2:缺少包slf4j-log4j12-1.7.5.jar

BUILD FAILED
/cloud/src/hadoop2.7/hadoop2x-eclipse-plugin-master/src/contrib/eclipse-plugin/build.xml:121: Warning: Could not find file /cloud/hadoop-2.7.4/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar to copy.

解决方法:

#cd /cloud/hadoop-2.7.4/share/hadoop/common/lib/#http://central.maven.org/maven2/org/slf4j/slf4j-log4j12/1.7.5/slf4j-log4j12-1.7.5.jar

11.3问题3:slf4j-api-1.7.5.jar

BUILD FAILED
/cloud/src/hadoop2.7/hadoop2x-eclipse-plugin-master/src/contrib/eclipse-plugin/build.xml:122: Warning: Could not find file /cloud/hadoop-2.7.4/share/hadoop/common/lib/slf4j-api-1.7.5.jar to copy.

解决方法:

wget http://central.maven.org/maven2/org/slf4j/slf4j-api/1.7.5/slf4j-api-1.7.5.jar

11.4 问题4:缺少htrace-core-3.0.4.jar

BUILD FAILED
/cloud/src/hadoop2.7/hadoop2x-eclipse-plugin-master/src/contrib/eclipse-plugin/build.xml:127: Warning: Could not find file /cloud/hadoop-2.7.4/share/hadoop/common/lib/htrace-core-3.0.4.jar to copy.

解决方法:

wget http://central.maven.org/maven2/org/htrace/htrace-core/3.0.4/htrace-core-3.0.4.jar