Master原理剖析与源码分析:注册机制原理剖析与源码分析
来源:互联网 发布:网络约车平台哪家好 编辑:程序博客网 时间:2024/05/16 19:22
1、主备切换机制原理剖析与源码分析
2、注册机制原理剖析与源码分析
3、状态改变处理机制源码分析
4、资源调度机制源码分析(schedule(),两种资源调度算法)
//master中注册RegisterApplication case RegisterApplication(description, driver) => { // TODO Prevent repeated registrations from some driver if (state == RecoveryState.STANDBY) { // ignore, don't send response } else { logInfo("Registering app " + description.name) //创建Application val app = createApplication(description, driver) //注册Application //将Application加入缓存,并加入等待调度的队列 registerApplication(app) logInfo("Registered app " + description.name + " with ID " + app.id) //使用持久化引擎持久Application persistenceEngine.addApplication(app) //反向向SparkDeploybackend的AppAppClient的Client的Actor发送信息。也就是的RegisteredApplication。 driver.send(RegisteredApplication(app.id, self)) //调用schedule()方法调度。 schedule() }private def registerApplication(app: ApplicationInfo): Unit = { val appAddress = app.driver.address if (addressToApp.contains(appAddress)) { logInfo("Attempted to re-register application at same address: " + appAddress) return } applicationMetricsSystem.registerSource(app.appSource) //这里将app的信息加入内存缓存中 apps += app idToApp(app.id) = app endpointToApp(app.driver) = app addressToApp(appAddress) = app //这里将app加入等待调度的队列 waitingApps += app }
阅读全文
0 0
- Master原理剖析与源码分析:注册机制原理剖析与源码分析
- Spark内核源码深度剖析:Master注册机制原理剖析与源码分析
- Master原理剖析与源码分析:Master状态改变处理机制原理剖析与源码分析
- Master原理剖析与源码分析:主备切换机制原理剖析与源码分析
- Spark内核源码深度剖析:Master主备切换机制原理剖析与源码分析
- Master原理剖析与源码分析:资源调度机制源码分析(schedule(),两种资源调度算法)
- Executor原理剖析与源码分析
- Worker原理剖析与源码分析
- DAGScheduler原理剖析与源码分析
- TaskScheduler原理剖析与源码分析
- Shuffle原理剖析与源码分析
- CacheManager原理剖析与源码分析
- BlockManager原理剖析与源码分析
- 6.job触发流程原理剖析与源码分析
- job触发流程原理剖析与源码分析
- Spark2.2 job触发流程原理剖析与源码分析
- Spark2.2 TaskScheduler原理剖析与源码分析
- Spark内核源码深度剖析:SparkContext原理剖析与源码分析
- CS231_A1:Softmax
- 第四章 ActiveMQ连接(Connector)配置 笔记
- Java 内存区域和GC机制
- Hibernate入门八(二级缓存)
- POJ
- Master原理剖析与源码分析:注册机制原理剖析与源码分析
- 关于mvvm的ComboBox绑定SelectedValue值不正确问题
- Redis笔记(1)- 安装
- AngularJs自定义过滤器
- 安装mysql服务出现Install/Remove of the Service Denied!错误
- angular部分指令的简单使用
- tensorboard的histogram数据解读
- 自定义RecyclerView之垂直和网格转换
- javascript停止冒泡和阻止浏览器默认行为