spark 2.1 TaskSetManager

来源:互联网 发布:淘宝大金空调贴牌造假 编辑:程序博客网 时间:2024/06/07 12:04
/** * Schedules the tasks within a single TaskSet in the TaskSchedulerImpl. This class keeps track of * each task, retries tasks if they fail (up to a limited number of times), and * handles locality-aware scheduling for this TaskSet via delay scheduling. The main interfaces * to it are resourceOffer, which asks the TaskSet whether it wants to run a task on one node, * and statusUpdate, which tells it that one of its tasks changed state (e.g. finished). * * THREADING: This class is designed to only be called from code with a lock on the * TaskScheduler (e.g. its event handlers). It should not be called from other threads. * * @param sched           the TaskSchedulerImpl associated with the TaskSetManager * @param taskSet         the TaskSet to manage scheduling for * @param maxTaskFailures if any particular task fails this number of times, the entire *                        task set will be aborted */private[spark] class TaskSetManager(    sched: TaskSchedulerImpl,    val taskSet: TaskSet,    val maxTaskFailures: Int,    clock: Clock = new SystemClock()) extends Schedulable with Logging {
原创粉丝点击