0-0 编译hadoop2.2.0
来源:互联网 发布:think in java 英文版 编辑:程序博客网 时间:2024/06/05 15:12
编译hadoop-2.2.0(ubuntux86_64)
###############################################
1.一些准备
sudo apt-get install cmake make -y
sudo apt-get install g++
sudo apt-get install gcc
sudo apt-get install build-essential
sudo apt-get install libssl-dev
sudo apt-get install zlib1g-dev
sudo apt-get install libglib2.0-dev
################################################
2.安装常用
(1)cp-r * /opt/,vi/etc/profile环境变量 (~/.bashrc)
/opt/jdk1.7.0_45
/opt/apache-maven-3.0.5
/opt/findbugs-2.0.2
####编译要求是2.5.0
cd protobuf-2.5.0
./configure
make
sudo make install
protoc –version
###增加国内maven 开源中国镜像
sudo vi /opt/apache-maven-3.0.5/conf/settings.xml
<mirror>
<id>nexus-osc</id>
<mirrorOf>*</mirrorOf>
<name>Nexus osc</name>
<url>http://maven.oschina.net/content/groups/public/</url>
</mirror>
#####################################################
3.解压源码包
(1) 打补丁
访问:https://issues.apache.org/jira/browse/HADOOP-10110
###########
hadoop@master1:~/下载/hadoop-2.2.0-src$
wget https://issues.apache.org/jira/secure/attachment/12614482/HADOOP-10110.patch
#######
ll -h hadoop-common-project/hadoop-auth/pom.xml
出现:
-rw-r--r-- 1 hadoop hadoop 6.9K 10月 7 13:52hadoop-common-project/hadoop-auth/pom.xml
#####
patch -p0 < ./HADOOP-10110.patch
出现:
patching file hadoop-common-project/hadoop-auth/pom.xml
Hunk #1 succeeded at 65 (offset 11 lines).
#####
ll -h hadoop-common-project/hadoop-auth/pom.xml
出现:( 发现时间变了)
-rw-r--r-- 1 hadoop hadoop 7.3K 3月 3 22:47hadoop-common-project/hadoop-auth/pom.xml
4.编译开始
mvn clean package -DskipTests -Pdist,native,docs -Dtar
编译成功之后,在hadoop-dist/target/目录下生成有hadoop-2.2.0-tar.gz文件,这个就是我们要的东东。
###############################################################################
5.常见问题
【问题1】
最新的2.2.0 的SourceCode 压缩包解压出来的code有个bug 需要patch后才能编译。否则编译hadoop-auth 会提示下面错误:
[ERROR] Failed to execute goalorg.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile(default-testCompile) on project hadoop-auth: Compilation failure: Compilationfailure:
[ERROR]/home/chuan/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[84,13]cannot access org.mortbay.component.AbstractLifeCycle
[ERROR] class file fororg.mortbay.component.AbstractLifeCycle not found
下载Patch :https://issues.apache.org/jira/browse/HADOOP-10110
【问题2】
[ERROR] Failed to execute goalorg.apache.hadoop:hadoop-maven-plugins:2.2.0:protoc (compile-protoc) on projecthadoop-common: org.apache.maven.plugin.MojoExecutionException: 'protoc --version'did not return a version -> [Help 1]
[ERROR]
[ERROR] To see the full stacktrace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the-X switch to enable full debug logging.
安装protobuf
export LD_LIBRARY_PATH=/usr/local/lib/
【问题2-1】
protoc --version
protoc: error while loadingshared libraries: libprotoc.so.8: cannot open shared object file: No such fileor directory
查看是否安装了
# ls /usr/local/lib
libhiredis.a libltdl.a libltdl.so.3.1.0 libprotobuf-lite.la libprotobuf.so libprotoc.la liby.alibhiredis.so libltdl.la libprotobuf.a libprotobuf-lite.so libprotobuf.so.8 libprotoc.so pkgconfiglibhiredis.so.0 libltdl.so libprotobuf.la libprotobuf-lite.so.8 libprotobuf.so.8.0.0 libprotoc.so.8libhiredis.so.0.10 libltdl.so.3 libprotobuf-lite.a libprotobuf-lite.so.8.0.0 libprotoc.a libprotoc.so.8.0.0
加入到/etc/ld.so.conf,动态装入器将在其中查找共享库。
# vi /etc/ld.so.conf
include /usr/local/lib
但是在动态装入器能“看到”这一信息之前,必须将它转换到 ld.so.cache 文件中。可以通过运行 ldconfig 命令做到这一点:
# ldconfig
(当 ldconfig 操作结束时,您会有一个最新的 /etc/ld.so.cache 文件,它反映您对 /etc/ld.so.conf 所做的更改。从这一刻起,动态装入器在寻找共享库时会查看您在 /etc/ld.so.conf 中指定的所有新目录。 )
# protoc --version
libprotoc 2.5.0
【问题3】
[ERROR] Failed to execute goalorg.apache.maven.plugins:maven-antrun-plugin:1.6:run (make) on projecthadoop-common: An Ant BuildException has occured: Execute failed:java.io.IOException: Cannot run program "cmake" (in directory"/home/l/software/hadoop-2.2.0-src/hadoop-common-project/hadoop-common/target/native"):error=2, No such file or directory -> [Help 1]
安装cmake
wget http://www.cmake.org/files/v2.8/cmake-2.8.12.tar.gz
tar zxvfcmake-2.8.12.tar.gz
cd cmake-2.8.12
./configure
make
sudo make install
【问题4】
configure: error:C++ preprocessor "/lib/cpp" fails sanity check
安装build-essential
[ERROR] Failed to execute goalorg.apache.maven.plugins:maven-antrun-plugin:1.6:run (make) on projecthadoop-hdfs: An Ant BuildException has occured: exec returned: 1 -> [Help 1]
安装libglib2.0-dev
[ERROR] Failed to execute goalorg.apache.maven.plugins:maven-antrun-plugin:1.6:run (make) on projecthadoop-pipes: An Ant BuildException has occured: exec returned: 1 -> [Help1]
安装libssl-dev
- 0-0 编译hadoop2.2.0
- oozie4.0 hadoop2.3编译
- hadoop2.0源码编译 错误及原因
- 在eclipse下编译hadoop2.0源码
- 在eclipse下编译hadoop2.0源码
- 在eclipse下编译hadoop2.0源码
- Linux编译Hadoop2.0生成Hadoop_X64
- 编译hadoop2.2.0源码
- hadoop2.2.0编译
- hadoop2.2.0源代码编译
- Hadoop2.2.0编译
- hadoop2.2.0编译
- hadoop2.2.0源码编译
- 动手编译hadoop2.2.0
- hadoop2.2.0源码编译
- Hadoop2.2.0源码编译
- CentOS6.4编译Hadoop2.2.0
- hadoop2.2.0编译--错误解决办法
- item7: 为多态基类声明virtual析构函数
- hdu 3333 Turing Tree ( 树状数组)
- 第四章作业
- C# 相对路径合集
- hdu2962 二分 + spfa
- 0-0 编译hadoop2.2.0
- Oracle 日期的处理
- C语言基础
- 关于子父类的调用问题
- C语言中 数据类型的转换
- Mahout安装
- 我想利用时钟写个无限弹窗的小程序 ,但是程序会自动停止
- 0-1 伪分布hadoop和hbase搭建
- 背包问题的一些想法