Bagging, boosting and stacking in machine learning
来源:互联网 发布:软件过程规范概念 编辑:程序博客网 时间:2024/04/29 01:48
All three are so-called "meta-algorithms": approaches to combine several machine learning techniques into one predictive model in order to decrease the variance (bagging), bias (boosting) or improving the predictive force (stacking alias ensemble).
Every algorithm consists of two steps:
Producing a distribution of simple ML models on subsets of the original data.
Combining the distribution into one "aggregated" model.
Here is a short description of all three methods:
Bagging (stands for Bootstrap Aggregating) is a way to decrease the variance of your prediction by generating additional data for training from your original dataset using combinations with repetitions to produce multisets of the same cardinality/size as your original data. By increasing the size of your training set you can't improve the model predictive force, but just decrease the variance, narrowly tuning the prediction to expected outcome.
Boosting is a two-step approach, where one first uses subsets of the original data to produce a series of averagely performing models and then "boosts" their performance by combining them together using a particular cost function (=majority vote). Unlike bagging, in the classical boosting the subset creation is not random and depends upon the performance of the previous models: every new subsets contains the elements that were (likely to be) misclassified by previous models.
Stacking is a similar to boosting: you also apply several models to your original data. The difference here is, however, that you don't have just an empirical formula for your weight function, rather you introduce a meta-level and use another model/approach to estimate the input together with outputs of every model to estimate the weights or, in other words, to determine what models perform well and what badly given these input data.
Here is a comparison table:
As you see, these all are different approaches to combine several models into a better one, and there is no single winner here: everything depends upon your domain and what you're going to do. You can still treat stacking as a sort of more advances boosting, however, the difficulty of finding a good approach for your meta-level makes it difficult to apply this approach in practice.
Short examples of each:
- Bagging: Ozone data.
- Boosting: is used to improve optical character recognition (OCR) accuracy.
- Stacking: is used in classification of cancer microarrays in medicine.
- Bagging, boosting and stacking in machine learning
- Ensemble Learning: Bootstrap aggregating (Bagging) & Boosting & Stacked generalization (Stacking)
- 集成学习(Ensemble Learning)-bagging-boosting-stacking
- 机器学习-->集成学习-->Bagging,Boosting,Stacking
- 模型融合:bagging、Boosting、Blending、Stacking
- 机器学习-->集成学习-->Bagging,Boosting,Stacking
- 集成学习---bagging and boosting
- Bagging, Boosting, and Randomization对比
- Machine Learning with Scikit-Learn and Tensorflow 7.10 Stacking
- Quick Introduction to Boosting Algorithms in Machine Learning
- Machine Learning with Scikit-Learn and Tensorflow 7.9 Gradient Boosting
- Ensemble learning:Bagging,Random Forest,Boosting
- 用R建立集成学习模型(boosting、bagging、stacking)
- 集成学习——bagging and boosting
- Preliminary understanding of bagging and boosting
- Bagging and Boosting 概念和区别
- Kernels in Machine Learning and SVM
- boosting-bagging
- android 介绍Retrofit的简单使用
- 虚拟环境管理工具之: anaconda (安装-使用)-Mac OS
- 视频直播
- MarkdownPad 2 在win10下出错:HTML 渲染错误(This view has crashed) 的解决办法 + MarkdownPad2.5 注册码
- LintCode 买卖股票的最佳时机 III
- Bagging, boosting and stacking in machine learning
- java pdf转成一张图片
- ASP.NET 站点地图的使用
- 查看apk的签名信息和签名文件的信息
- 使用eclipse实现cocos2d-x微信登录
- QtCreator下用openpose搭建自己的QT工程
- Unity官方_第一人称游戏<恶魔射手>----学习笔记一(英雄脚本)
- 51Nod 1298 圆与三角形
- bzoj 4951: [Wf2017]Money for Nothing