spark 2.1 ExecutorData and ExecutorInfo
来源:互联网 发布:坚持30秒 h5 源码 编辑:程序博客网 时间:2024/06/03 17:10
- ExecutorData
/** * Grouping of data for an executor used by CoarseGrainedSchedulerBackend. * * @param executorEndpoint The RpcEndpointRef representing this executor * @param executorAddress The network address of this executor * @param executorHost The hostname that this executor is running on * @param freeCores The current number of cores available for work on the executor * @param totalCores The total number of cores available to the executor */private[cluster] class ExecutorData( val executorEndpoint: RpcEndpointRef, val executorAddress: RpcAddress, override val executorHost: String, var freeCores: Int, override val totalCores: Int, override val logUrlMap: Map[String, String]) extends ExecutorInfo(executorHost, totalCores, logUrlMap)
- ExecutorInfo
/** * :: DeveloperApi :: * Stores information about an executor to pass from the scheduler to SparkListeners. */@DeveloperApiclass ExecutorInfo( val executorHost: String, val totalCores: Int, val logUrlMap: Map[String, String]) { def canEqual(other: Any): Boolean = other.isInstanceOf[ExecutorInfo] override def equals(other: Any): Boolean = other match { case that: ExecutorInfo => (that canEqual this) && executorHost == that.executorHost && totalCores == that.totalCores && logUrlMap == that.logUrlMap case _ => false } override def hashCode(): Int = { val state = Seq(executorHost, totalCores, logUrlMap) state.map(_.hashCode()).foldLeft(0)((a, b) => 31 * a + b) }}
0 0
- spark 2.1 ExecutorData and ExecutorInfo
- spark 2.1 Encoders and UploadBlock
- spark 2.1 RDDCheckpointData and ReliableRDDCheckpointData
- spark 2.1 Stage and ResultStage and ShuffleMapStage
- spark 2.1 metrics Source and BlockManagerSource
- spark 2.1 TaskSetManager, Schedulable And Pool
- Hadoop and Hbase and Spark
- Spark, Shark, and RDDs
- Spark streaming and flume
- storm and spark
- Scala and Spark Begin
- Spark SQL and DataFrames
- spark configuration and monitor
- Spark ML 2.1 -- Extracting, transforming and selecting features (持续更新)
- Spark SQL and DataFrame for Spark 1.3
- Compile Spark and Build Spark Package
- Spark and SPARQL; RDF Graphs and GraphX
- Hadoop and Spark and Hive Installation
- Java String.compareTo()方法
- 4K标准---电视显示标准(ITU-R BT.2020)
- jq ajax 跨域请求
- Ionic实战项目-跨平台App开发
- CSS常见布局解决方案
- spark 2.1 ExecutorData and ExecutorInfo
- 如何添加防火墙允许端口
- JavaScript学习笔记4——DOM
- [转]Spring常用注解总结(3)
- 商城项目实战 | 4.2 RecyclerView 使用完全解析 体验艺术般的控件(二)
- Linux查看物理CPU个数、核数、逻辑CPU个数
- linux如何查看程序链接的动态库
- rabbitmq
- TCL学习之info命令