WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster uito ensure
来源:互联网 发布:ecshop3源码下载 编辑:程序博客网 时间:2024/06/05 08:43
当运行Spark程序,出现这样的问题
WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster uito ensure that workers are registered and have sufficient memory
然后停下来,登录webUI看到的Status是Wait,分配的核是0,内存是参数 –executor-memory(或-Dspark.executor.memory)
这是因为当前的集群的可用资源不能满足应用程序所请求的资源。
这个首先检查一下spark-env.sh文件里面分给每个worker的executor的memory是多少。当app中上述的参数超过这个数值,那么app就不会找这个worker进行工作,而是去找满足数值的worker。
当然,一般来说每个节点的分配的都是一样,所以遇到这种情况基本是因为参数超过了分配给worker的executor的memory的数值。
到spark-env.sh文件修改executor(worker)的数值(当然前提是你有这么多内存)
问题就会得到解决。
0 0
- WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster uito ensure
- WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure
- Spark WARN cluster.ClusterScheduler: Initial job has not accepted any resources;check your cluster
- Spark WARN cluster.ClusterScheduler: Initial job has not accepted any resources;check your cluster
- spark-summit 中出现Initial job has not accepted any resources; check your cluster UI to ensure that wor
- Initial job has not accepted any resources; check your cluster UI to ensure that workers are registe
- erImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are
- 运行spark问题:Initial job has not accepted any resources; check your cluster UI to ensure that workers a
- Spark错误:WARN TaskSchedulerImpl: Initial job has not accepted any resources;
- Spark执行样例报警告:WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources
- Initial job has not accepted any resources;check your cluster All masters are unresponsive! Giving u
- spark WARNTaskSchedulerImpl:Initial job has not accepted any resources; check your cluster UI to
- initial job has not accepted any resources
- spark on yarn,Initial job has not accepted any resources
- Spark-sql执行部分语句是报错:WARN org.apache.spark.scheduler.TaskSchedulerImpl: Initial job has not accepted a
- spark-submit 报错 Initial job has not accepted any resources
- initial job has not accepted any resources的spark错误解决办法
- HiBench运行spark用例报Initial job has not accepted any resources错误
- qemu-2.4.0编译时出错:error: ‘XFS_IOC_ZERO_RANGE’ undeclared (first use in this function)
- c# XML和实体类之间相互转换(序列化和反序列化)
- Linux双网卡:一个配置外网,一个配置内网
- Windows与Linux程序的返回值
- C#内存优化
- WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster uito ensure
- js 页面数字倒计时
- 开源大数据处理工具汇总(下)
- Linux多线程基础学习(五)线程同步-条件变量
- Linux修改本机/etc/hosts的hostName后经常不生效
- Objective-C 程序设计 第四章
- The application could not be verified.
- 凶鬼恶灵.Supernatural
- 基本配置记录