Cloudera 版spark 解决spark_shuffle aux-service 缺失的问题

来源:互联网 发布:linux ll什么意思 编辑:程序博客网 时间:2024/05/16 02:26

CDHwebUi-> YARN (MR2 Included) -> Configuration -> NodeManager Default Group -> Advanced
在 “NodeManager Advanced Configuration Snippet (Safety Valve) for yarn-site.xml” 弹出框中粘贴如下:


yarn.nodemanager.aux-services
spark_shuffle,mapreduce_shuffle


yarn.nodemanager.aux-services.spark_shuffle.class
org.apache.spark.network.yarn.YarnShuffleService

Click on Save Changes. Actions -> Deploy client configuration Actions -> Restart

0 0
原创粉丝点击