Initial job has not accepted any resources; check your cluster UI to ensure that workers are registe
来源:互联网 发布:python ui界面开发 编辑:程序博客网 时间:2024/06/06 03:33
spark在提交任务时,出现如下错误:
15/03/26 22:29:36 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory15/03/26 22:29:51 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory15/03/26 22:30:06 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory15/03/26 22:30:21 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory
从警告信息上看,初始化job时没有获取到任何资源;提示检查集群,确保workers可以被注册并有足够的内存资源。
如上问题产生的原因是多方面的,可能原因如下:
1.因为提交任务的节点不能和spark工作节点交互,因为提交完任务后提交任务节点上会起一个进程,展示任务进度,大多端口为4044,工作节点需要反馈进度给该该端口,所以如果主机名或者IP在hosts中配置不正确。所以检查下主机名和ip是否配置正确
2.有可能是内存不足
检查内存
conf.set("spark.executor.memory", "3000m")
Make sure to set SPARK_LOCAL_IP
andSPARK_MASTER_IP
.
查看8080端口,确保一些workers保持Alive状态,确保 some cores 是可利用的。
- Initial job has not accepted any resources; check your cluster UI to ensure that workers are registe
- erImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are
- 运行spark问题:Initial job has not accepted any resources; check your cluster UI to ensure that workers a
- spark-summit 中出现Initial job has not accepted any resources; check your cluster UI to ensure that wor
- WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure
- spark WARNTaskSchedulerImpl:Initial job has not accepted any resources; check your cluster UI to
- WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster uito ensure
- Initial job has not accepted any resources;check your cluster All masters are unresponsive! Giving u
- Spark WARN cluster.ClusterScheduler: Initial job has not accepted any resources;check your cluster
- Spark WARN cluster.ClusterScheduler: Initial job has not accepted any resources;check your cluster
- spark运行报错:check your cluster UI to ensure that workers are registered and have sufficient resources
- spark 解决办法 check your cluster UI to ensure that workers are registered and have sufficient memory
- initial job has not accepted any resources
- spark on yarn,Initial job has not accepted any resources
- Spark错误:WARN TaskSchedulerImpl: Initial job has not accepted any resources;
- Spark执行样例报警告:WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources
- spark-submit 报错 Initial job has not accepted any resources
- initial job has not accepted any resources的spark错误解决办法
- Spring的核心技术(六)---基于Setter的依赖注入
- 15个值得开发人员关注的jQuery开发技巧和心得
- C#读取摄像头并对图像做简单处理--AVICap32篇
- Android 操作SQLite基本用法
- android adb 卸载内置系统apk方法
- Initial job has not accepted any resources; check your cluster UI to ensure that workers are registe
- UITextView 如何使显示不全的单词不折行显示
- 文章标题
- 基于Wolf32F031 自由评估板的Accelerometer(MMA7455)实验硬件原理及软件API使用方法(适用于STM32F030/031)
- 抽象工厂
- Spring的核心技术(七)---循环依赖
- python PIL 处理图片的模快 Image类
- maven安装 JDK问题
- Android百度云推送实现消息推送