解决Hadoop0.21.0 HDFS/MapReduce编译错误(ivy-resolve-common does not find hadoop-commo)
来源:互联网 发布:兴趣部落签到软件 编辑:程序博客网 时间:2024/05/21 19:14
转自:http://junsky.iteye.com/blog/1070647
解决Hadoop0.21.0 HDFS/MapReduce编译错误(ivy-resolve-common does not find hadoop-commo)
最近老大安排,要跟踪HDFS ConcatenateFile的前提条件;于是尝试用ant/ivy编译了一下hdfs,提示ivy找不到hadoop-common组件;于是又编译了一下mapreduce,提示ivy找不到hadoop-comm/hadoop-hdfs;编译hadoop-common,提示编译成功;在编译mapreduce错误如下:
ivy-resolve-common:
[ivy:resolve]
[ivy:resolve] :: problems summary ::
[ivy:resolve] :::: WARNINGS
[ivy:resolve] module not found: org.apache.hadoop#hadoop-common;0.21.0
[ivy:resolve] ==== apache-snapshot: tried
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common/0.21.0/hadoop-common-0.21.0.pom
[ivy:resolve] -- artifact org.apache.hadoop#hadoop-common;0.21.0!hadoop-common.jar:
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common/0.21.0/hadoop-common-0.21.0.jar
[ivy:resolve] ==== maven2: tried
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.21.0/hadoop-common-0.21.0.pom
[ivy:resolve] -- artifact org.apache.hadoop#hadoop-common;0.21.0!hadoop-common.jar:
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.21.0/hadoop-common-0.21.0.jar
[ivy:resolve] module not found: org.apache.hadoop#hadoop-common-test;0.21.0
[ivy:resolve] ==== apache-snapshot: tried
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common-test/0.21.0/hadoop-common-test-0.21.0.pom
[ivy:resolve] -- artifact org.apache.hadoop#hadoop-common-test;0.21.0!hadoop-common-test.jar:
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common-test/0.21.0/hadoop-common-test-0.21.0.jar
[ivy:resolve] ==== maven2: tried
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common-test/0.21.0/hadoop-common-test-0.21.0.pom
[ivy:resolve] -- artifact org.apache.hadoop#hadoop-common-test;0.21.0!hadoop-common-test.jar:
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common-test/0.21.0/hadoop-common-test-0.21.0.jar
[ivy:resolve] module not found: org.apache.hadoop#hadoop-hdfs;0.21.0
[ivy:resolve] ==== apache-snapshot: tried
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-hdfs/0.21.0/hadoop-hdfs-0.21.0.pom
[ivy:resolve] -- artifact org.apache.hadoop#hadoop-hdfs;0.21.0!hadoop-hdfs.jar:
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-hdfs/0.21.0/hadoop-hdfs-0.21.0.jar
[ivy:resolve] ==== maven2: tried
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.21.0/hadoop-hdfs-0.21.0.pom
[ivy:resolve] -- artifact org.apache.hadoop#hadoop-hdfs;0.21.0!hadoop-hdfs.jar:
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.21.0/hadoop-hdfs-0.21.0.jar
[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve] :: UNRESOLVED DEPENDENCIES ::
[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve] :: org.apache.hadoop#hadoop-common;0.21.0: not found
[ivy:resolve] :: org.apache.hadoop#hadoop-common-test;0.21.0: not found
[ivy:resolve] :: org.apache.hadoop#hadoop-hdfs;0.21.0: not found
[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve]
[ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
BUILD FAILED
/opt/hadoop-0.21.0/mapred/build.xml:1861: impossible to resolve dependencies:
resolve failed - see output for details
通过Google,发现原来这是Apache-Hadoop的BUG:
https://issues.apache.org/jira/browse/HDFS-1519(hdfs ivy配置bug)
主要原因是ivy配置问题:
hdfs的ivy在配置关联类库hadoop-common时候位置指定错误,具体修改如下:
hdfs/ivy.xml
<dependency org="org.apache.hadoop" name="hadoop-common"
-----rev="${hadoop-common.version}"
+++ rev="${hadoop-common.version}-SNAPSHOT"
conf="common->default"/>
hdfs/src/contrib/hdfsproxy/ivy.xml
<dependency org="org.apache.hadoop"
name="hadoop-common"
---- rev="${hadoop-common.version}"
+++ rev="${hadoop-common.version}-SNAPSHOT"
conf="common->default"/>
<dependency org="org.apache.hadoop"
name="hadoop-common-test"
--- rev="${hadoop-common.version}"
+++ rev="${hadoop-common.version}-SNAPSHOT"
conf="common->default"/>
hdfs/src/contrib/thriftfs/ivy.xml
<dependency org="org.apache.hadoop"
name="hadoop-common"
--- rev="${hadoop-common.version}"
+++ rev="${hadoop-common.version}-SNAPSHOT"
conf="common->default"/>
mapreduce的ivy在配置关联类库hadoop-common/hadoop-dfs时候位置指定错误,修改方法与hdfs类似:
1.修改mapreduce project里的ivy.xml(多个)
在eclipse里search file(ivy.xml):
(1)rev="${hadoop-common.version}" replace rev="${hadoop-common.version}-SNAPSHOT"
(2)rev="${hadoop-hdfs.version}" replace rev="${hadoop-hdfs.version}-SNAPSHOT"
2.修改buid.xml
<unzip
-- -- src="${common.ivy.lib.dir}/hadoop-hdfs-${hadoop-hdfs.version}.jar"
+++ src="${common.ivy.lib.dir}/hadoop-hdfs-${hadoop-hdfs.version}-SNAPSHOT.jar"
dest="${build.dir}">
- 解决Hadoop0.21.0 HDFS/MapReduce编译错误(ivy-resolve-common does not find hadoop-commo)
- 解决Hadoop0.21.0 HDFS/MapReduce编译错误(impossible to resolve dependencies)
- 错误:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://localhost/
- eclipse编译hadoop错误:Input path does not exist: hdfs://192.168.184.21:9001/user/xxxx/input
- org.apache.hadoop.mapreduce.lib.input.InvalidInputException:input path does not exist:hdfs://10.0.2.
- 解决Xcode编译错误:does not contain bitcode
- Hadoop0.21.0编译手册
- Hadoop0.21.0编译方法
- Hadoop之MapReduce & HDFS
- hadoop hdfs , mapreduce 第一步
- 解决错误:build.properties does not exist
- MapReduce程序运行错误:The auxService: mapreduce_shuffle does not exist
- 解决NDK交叉编译 selected processor does not support ARM mode libtheora的错误
- 解决NDK交叉编译 selected processor does not support ARM mode libtheora的错误
- Bug之解决Xcode 7编译错误:* does not contain bitcode.
- 如何解决sql server2005的“不允许远程连接”错误.(how to resolve the error: "SQL Server does not allow remote connections" for SQL Server2005)
- maven 编译hadoop错误解决
- hadoop异常: org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exi
- 自动装箱和拆箱 枚举 反射 数组反射的应用。ArrayList和HashSet
- 关于VMware虚拟机下ubuntu的安装
- 编译Eclipse hadoop插件
- conv2,filter2,imfilter的区别
- Hdu--1556--Color the ball(树状数组最简单运用)
- 解决Hadoop0.21.0 HDFS/MapReduce编译错误(ivy-resolve-common does not find hadoop-commo)
- 替换Android主界面
- UVa 10050 Hartals(优先队列)
- Sprite Kit编程指南(2)-使用精灵
- RCP 视图间通信:SWT Canvas向JFace ViewPart发送消息
- Android平台Native开发与JNI机制详解
- tomcat下集成quartz任务调度
- ORACLE的锁机制
- 表的概念