Hadoop1.x程序升级到Hadoop2.x需要的依赖库,Cannot initialize Cluster.Please check your configuration for ma
来源:互联网 发布:广西移动重大故障知乎 编辑:程序博客网 时间:2024/06/06 03:56
根据官方文档(Apache Hadoop MapReduce - Migrating from ApacheHadoop 1.x to Apache Hadoop 2.x:http://hadoop.apache.org/docs/r2.2.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduce_Compatibility_Hadoop1_Hadoop2.html)所述,Hadoop2.x是对Hadoop1.x程序兼容的,由于Hadoop2.x对Hadoop1.x做了重大的结构调整,很多程序依赖库被拆分了,所以以前(Hadoop1.x)的依赖库不再可用(不是说API不可用,而是api存放的包不一样),如果需要在Hadoop2.X上面编译,请用下面的依赖库:
<dependencies>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-core</artifactId>
<version>
2.2
.
0
</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>
2.2
.
0
</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-common</artifactId>
<version>
2.2
.
0
</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-jobclient</artifactId>
<version>
2.2
.
0
</version>
</dependency>
</dependencies>
记得加上
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-common</artifactId>
<version>
2.2
.
0
</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-jobclient</artifactId>
<version>
2.2
.
0
</version>
</dependency>
否则运行程序的时候将会出现一下的异常:
Exception in thread
"main"
java.io.IOException: Cannot initialize Cluster.
Please check your configuration
for
mapreduce.framework.name and the
correspond server addresses.
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:
120
)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:
82
)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:
75
)
at org.apache.hadoop.mapred.JobClient.init(JobClient.java:
465
)
at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:
444
)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:
826
)
at com.wyp.hadoop.MaxTemperature.main(MaxTemperature.java:
41
)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:
57
)
at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:
43
)
at java.lang.reflect.Method.invoke(Method.java:
606
)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:
120
)
一个简单的Mapreduce程序请参见《编写简单的Mapreduce程序并部署在Hadoop2.2.0上运行》
如果你不是用Maven管理依赖的类库,你可以分别在${HADOOP_HOME}/share/hadoop/common和${HADOOP_HOME}/share/hadoop/mapreduce目录里面以下几个依赖库,分别是hadoop-mapreduce-client-core-2.2.0.jar、hadoop-common-2.2.0.jar、hadoop-mapreduce-client-common-2.2.0.jar、hadoop-mapreduce-client-jobclient-2.2.0.jar和上面的四个依赖正好对应。
转载自过往记忆(http://www.iteblog.com/)
本文链接地址:《Hadoop1.x程序升级到Hadoop2.x需要的依赖库》(http://www.iteblog.com/archives/840) 0 0
- Hadoop1.x程序升级到Hadoop2.x需要的依赖库,Cannot initialize Cluster.Please check your configuration for ma
- 解决Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the co
- hadoop出现Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name
- 【解决办法】Cannot initialize Cluster. Please check you..
- 从Hadoop1.x集群升级到Hadoop2.x步骤
- hadoop1.x和hadoop2.x的对比
- HADOOP1.X与HADOOP2.X的区别
- hadoop1.x和hadoop2.x的区别
- Hadoop2.x对Hadoop1.x的改进
- 从Hadoop1.x集群升级到Hadoop2.x失败回滚步骤
- Cannot load mcrypt extension. Please check your PHP configuration
- Hadoop1.x集群升级Hadoop2.x相关文章
- Hadoop1.x与Hadoop2的区别
- Hadoop1.x与Hadoop2的区别
- Hadoop1.x与Hadoop2的区别
- Hadoop1.x与Hadoop2的区别
- Hadoop1.x与Hadoop2的区别
- Hadoop1.x与Hadoop2的区别
- Android线程间通信(二):MessageQueue(上)
- 递归简论学习
- AJAX—核心XMLHttpRequest对象
- Struts2应用开发步骤
- QQ好友列表(处理头部控件的点击)viewForHeaderInSection:
- Hadoop1.x程序升级到Hadoop2.x需要的依赖库,Cannot initialize Cluster.Please check your configuration for ma
- Android LayoutAnimation
- Error While Loading Shared Libraries, Cannot Open Shared Object File
- 20151111,微软11月11日发布12个安全补丁
- hive学习笔记(1)
- Unity+NGUI性能优化方法总结
- OC入门-继承和多态
- struts2常量配置的方式
- 求最多可以灌溉多少块田地(CodeForce 66B)