Spark-futureAction

来源:互联网 发布:linux分页显示命令 编辑:程序博客网 时间:2024/06/03 13:33

Spark-futureAction

@(spark)[FutureAction]

FutureAction

 * A future for the result of an action to support cancellation. This is an extension of the                                                                             * Scala Future interface to support cancellation.                                                                                                                       */                                                                                                                                                                     trait FutureAction[T] extends Future[T] { 

SimpleFutureAction

/**                                                                                                                                                                      * A [[FutureAction]] holding the result of an action that triggers a single job. Examples include                                                                       * count, collect, reduce.                                                                                                                                               */                                                                                                                                                                     class SimpleFutureAction[T] private[spark](jobWaiter: JobWaiter[_], resultFunc: => T)   
  1. SimpleFutureAction的核心实际上是那个叫做JobWaiter,它本身是个listener。
  2. cancel也是通过JobWaiter实现的

JavaFutureActionWrapper

就是对JavaFutureAction的封装

ComplexFutureAction

/**                                                                                                                                                                      * A [[FutureAction]] for actions that could trigger multiple Spark jobs. Examples include take,                                                                         * takeSample. Cancellation works by setting the cancelled flag to true and interrupting the                                                                             * action thread if it is being blocked by a job.                                                                                                                        */                                                                                                                                                                     class ComplexFutureAction[T] extends FutureAction[T] {    

关于scala的promise和future,请参考
基本上就是runJob之后就一直block。

0 0
原创粉丝点击