Hive Hook类型
来源:互联网 发布:手机卡能注册几个淘宝 编辑:程序博客网 时间:2024/05/22 06:10
Hive在程序启动不同阶段设置了hook供用户自定义不同的行为
一共有四个阶段提供Hook以供注入定制化代码
- Driver run hooks (Pre/Post)
- Semantic analyizer hooks (Pre/Post)
- Execution hooks (Pre/Failure/Post)
- Client statistics publisher
- Driver.run() takes the command
HiveDriverRunHook.preDriverRun()
(HiveConf.ConfVars.HIVE_DRIVER_RUN_HOOKS
)- Driver.compile() starts processing the command: creates the abstract syntax tree
AbstractSemanticAnalyzerHook.preAnalyze()
(HiveConf.ConfVars.SEMANTIC_ANALYZER_HOOK
)- Semantic analysis
AbstractSemanticAnalyzerHook.postAnalyze()
(HiveConf.ConfVars.SEMANTIC_ANALYZER_HOOK
)- Create and validate the query plan (physical plan)
- Driver.execute() : ready to run the jobs
ExecuteWithHookContext.run()
(HiveConf.ConfVars.PREEXECHOOKS
)- ExecDriver.execute() runs all the jobs
- For each job at every HiveConf.ConfVars.HIVECOUNTERSPULLINTERVAL interval:
ClientStatsPublisher.run()
is called to publish statistics
(HiveConf.ConfVars.CLIENTSTATSPUBLISHERS
)
If a task fails:ExecuteWithHookContext.run()
(HiveConf.ConfVars.ONFAILUREHOOKS
) - Finish all the tasks
ExecuteWithHookContext.run()
(HiveConf.ConfVars.POSTEXECHOOKS
)- Before returning the result
HiveDriverRunHook.postDriverRun()
(HiveConf.ConfVars.HIVE_DRIVER_RUN_HOOKS
) - Return the result.
For each of the hooks I indicated the interfaces you have to implement. In the brackets there's the corresponding conf. prop. key you have to set in order to register the class at the beginning of the script. E.g: setting the PreExecution hook (9th stage of the workflow)
HiveConf.ConfVars.PREEXECHOOKS -> hive.exec.pre.hooks :set hive.exec.pre.hooks=com.example.MyPreHook;
Unfortunately these features aren't really documented, but you can always look into theDriver class to see the evaluation order of the hooks.
Remark: I assumed here Hive 0.11.0, I don't think that the Cloudera distribution differs (too much
参考:
http://stackoverflow.com/questions/17461932/hive-execution-hook
0 0
- Hive Hook类型
- hive hook
- HOOK类型
- HOOK类型
- hook钩子类型
- hive Timestamp类型慎用
- hive void类型
- hive 类型转换
- hive 表类型
- Hive之复合类型
- HIVE自定义函数类型
- Hive之表类型
- hive 字段类型
- Hive类型专题
- hive的四种表类型
- Hive 表类型简述
- Hive连接的类型
- (轉)HOOK概念,运行机制及类型
- JqueryUI datagrid动态加载标题和数据
- Debian 7+ 一键部署anyconnect&&pac智能分流
- [matplotlib] 绘制Cross-Validation的误差图
- 杭电-1722 Cake (GCD)
- 【UVALive】2147 - Push!!(bfs+dfs+优先队列)
- Hive Hook类型
- java高并发解决方案
- 73. Set Matrix Zeroes
- AngularJS 全选框事件~
- RecyclerView(实现分割线)
- IE8兼容rgba
- 算法导论_第十一章_散列表
- 【bzoj1864 [Zjoi2006]】三色二叉树
- 自定义Html标签方式