Yarn 2.2.0安装
来源:互联网 发布:狗万万博软件 编辑:程序博客网 时间:2024/05/17 17:18
1. 版本库的问题:
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicableStarting namenodes on [Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /home/hchen/hadoop-2.2.0/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.namenode]sed: -e expression #1, char 6: unknown option to `s' have: ssh: Could not resolve hostname have: Name or service not knownHotSpot(TM): ssh: Could not resolve hostname HotSpot(TM): Name or service not known-c: Unknown cipher type 'cd'Java: ssh: Could not resolve hostname Java: Name or service not knownThe authenticity of host 'namenode (192.168.1.62)' can't be established.RSA key fingerprint is 65:f9:aa:7c:8f:fc:74:e4:c7:a2:f5:7f:d2:cd:55:d4.Are you sure you want to continue connecting (yes/no)? VM: ssh: Could not resolve hostname VM: Name or service not knownYou: ssh: Could not resolve hostname You: Name or service not knownwarning:: ssh: Could not resolve hostname warning:: Name or service not knownlibrary: ssh: Could not resolve hostname library: Name or service not knownhave: ssh: Could not resolve hostname have: Name or service not known64-Bit: ssh: Could not resolve hostname 64-Bit: Name or service not known
出现原因见该贴: http://m.blog.csdn.net/blog/haidao2009/14897813
http://blog.csdn.net/pelick/article/details/12065147
不过我修改了环境变量没起作用,索性直接在hadoop-config.sh文件中加了:
export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_PREFIX}/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_PREFIX/lib"
启动OK.
BTW, 如果要彻底根除这个
参考这贴WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
1) http://blog.csdn.net/lalaguozhe/article/details/10580727
2) http://blog.csdn.net/huanggang028/article/details/17411791
3) http://blog.csdn.net/jiedushi/article/details/7496327
2.
- <property>
- <name>yarn.nodemanager.aux-services</name>
- <value>mapreduce.shuffle</value>
- </property>
改为
不然会出现这种错误
- <property>
- <name>yarn.nodemanager.aux-services</name>
- <value>mapreduce_shuffle</value>
- </property>
java.lang.IllegalArgumentException: The ServiceName: mapreduce.shuffle set in yarn.nodemanager.aux-services is invalid.The valid service name should only contain a-zA-Z0-9_
3. 默认是ipv6, 在yarn 中exec后加入后改为:
exec "$JAVA" -Dproc_$COMMAND -Djava.net.preferIPv4Stack=true $JAVA_HEAP_MAX $HADOOP_OPTS -classpath "$CLASSPATH" $CLASS "$@"
4. 提交作业进行测试
hadoop jar hadoop-mapreduce-examples-2.2.0.jar pi 200 1000
OK.
0 0
- Yarn 2.2.0安装
- 安装yarn
- Yarn 安装
- Hadoop2.0 YARN cloudra4.4.0安装配置
- 在 Yarn 上 安装 Spark 0.9.0
- hive2.3-spark2.0-yarn 安装
- Hadoop2.0 YARN cloudra4.4.0安装配置
- hadoop yarn安装
- Hadoop YARN 安装
- Hadoop Yarn 安装
- Hadoop Yarn 安装
- Storm on yarn安装
- yarn常见安装问题
- Hadoop-Yarn安装部署
- Storm on yarn 安装
- 安装Hadoop YARN
- spark on yarn 安装
- YARN安装配置
- 解决STRUTS+AJAX+JSP 请求到后台乱码问题
- backbone.js使用心得
- 如何正确配置Nginx+PHP
- Struts2.1 annotation防止表单重复提交
- jquery-ajax-queue使用心得
- Yarn 2.2.0安装
- handlebars.js 使用心得(1)
- 【最终版 & 总结】自实现自旋锁 与 mutex,spinlock比较(结果令人吃惊)
- struts2中form提交到action中的中文参数乱码问题解决办法(包括取中文路径)
- Object-C内存管理
- Android应用开发SharedPreferences存储数据的使用方法
- 模式实践:观察者模式与Spring
- 用Struts2中的ActionSupport中的validate方法进行表单验证
- 第五项修炼之我见