spark 2.1 TaskSetManager
来源:互联网 发布:淘宝大金空调贴牌造假 编辑:程序博客网 时间:2024/06/07 12:04
/** * Schedules the tasks within a single TaskSet in the TaskSchedulerImpl. This class keeps track of * each task, retries tasks if they fail (up to a limited number of times), and * handles locality-aware scheduling for this TaskSet via delay scheduling. The main interfaces * to it are resourceOffer, which asks the TaskSet whether it wants to run a task on one node, * and statusUpdate, which tells it that one of its tasks changed state (e.g. finished). * * THREADING: This class is designed to only be called from code with a lock on the * TaskScheduler (e.g. its event handlers). It should not be called from other threads. * * @param sched the TaskSchedulerImpl associated with the TaskSetManager * @param taskSet the TaskSet to manage scheduling for * @param maxTaskFailures if any particular task fails this number of times, the entire * task set will be aborted */private[spark] class TaskSetManager( sched: TaskSchedulerImpl, val taskSet: TaskSet, val maxTaskFailures: Int, clock: Clock = new SystemClock()) extends Schedulable with Logging {
阅读全文
0 0
- spark 2.1 TaskSetManager
- spark 2.1 TaskSetManager, Schedulable And Pool
- Spark源码分析之TaskSetManager分析
- Spark的位置优先: TaskSetManager 的有效 Locality Levels
- TasksetManager冲突导致SparkContext异常关闭
- spark 2.1 spark.yarn.services
- Spark 2.1.0 -- Spark Streaming Programming Guide
- spark 2.1 spark-shell Startup Process
- spark 2.1 spark executor topology information
- spark-2.1.0安装
- spark(2.1.0)
- Spark 2.1 structured streaming
- Spark 2.1 CallSite
- spark 2.1 ConfigProvider
- Spark ML 2.1 --Pipelines
- spark-2.1.0
- spark 2.1 SparkContext postEnvironmentUpdate
- spark 2.1 StorageLevel
- Python网络编程(Socket Tcp Udp)
- android去掉手机号码中的空格和-
- Java多线程基础
- PHP实现二分查找
- git初次使用命令行总结
- spark 2.1 TaskSetManager
- springBoot启动错误 Your ApplicationContext is unlikely to start due to a @ComponentScan of the default
- Dubbo学习总结(5)——Dubbo服务架构及服务治理过程演进
- verilog 中 wire 和reg 的使用
- iOS UITextField中复制和粘贴为中文
- (93)蓝图接口用户界面
- dubbo源码解析(二): dubbo服务的启动
- TCP解析
- 1024. 科学计数法 (20)