stochastic noise and deterministic noise
来源:互联网 发布:潭州学院java vip视频 编辑:程序博客网 时间:2024/05/29 15:08
在机器学习中,导致overfitting的原因之一是noise,这个noise可以分为两种,即stochastic noise,随机噪声来自数据产生过程,比如测量误差等,和deterministic noise,确定性噪声来自added complexity,即model too complex。这两种类型的造成来源不同,但是对于学习的影响是相似的,large noise总会导致overfitting。
This is a very subtle question!
The most important thing to realize is that in learning,
i) If there is stochastic noise with ‘magnitude’
ii) If there deterministic noise then you are in trouble.
The stochastic noise can be viewed as one part of the data generation process (eg. measurement errors). The deterministic noise can similarly be viewed as another part of the data generation process, namely f. The deterministic and stochastic noise are fixed. In your analogy, you can increase the stochastic noise by increasing the noise variance and you get into deeper trouble. Similarly, you can increase the deterministic noise by making f more complex and you will get into deeper trouble.
I just need to tell you what ‘trouble’ means. Well, we actually use another word instead of ‘trouble’ - overfitting.
This means you may be likely to make an inferior choice over the superior choice because the inferior choice has lower in-sample error. Doing stuff that looks good in-sample that leads to disasters out-of-sample is the essence of overfitting. An example of this is trying to choose the regularization parameter. If you pick a lower regularization parameter, then you have lower in-sample error, but it leads to higher out-of-sample error - you picked the
Now let’s get back to the subtle part of your question. There is actually another way to decrease the deterministic noise - increase the complexity of
上面一段主要摘自《learning from data》一书,主要说明的内容是overfitting的含义以及noise对于overfitting的效用。
下面是对overfitting的很好的总结:
VC维大=>模型复杂度高=>error in sample 小=>模型不够平滑=>generalization能力弱=>error out of sample大=>overfitting=>模型并没有卵用。
总的来说,deterministic noise是由于你选择的
deterministic function可用来生成伪随机数(pseudo-random generator)。
详细的论述可以参看《learning from data》
2015-8-27
艺少
- stochastic noise and deterministic noise
- noise
- noise
- Receiver Noise and sensitivity
- Noise and Turbulence
- salt and pepper noise
- noise and error
- noise and filter
- 8.1 Noise and Probabilistic Target
- NTU-Coursera机器学习:Noise and Error
- Chapter 2 Statistics, Probability and Noise
- Softmax, Negative Sampling, and Noise Contrastive Estimation
- 机器学习基石-Noise and Error
- Noise教程
- 噪點(noise)
- white noise
- Perlin Noise
- Perlin Noise
- 表单的建立和PHP的交互
- 对于async的错误理解
- HDU 1595 find the longest of the shortest (最短路+记录路径+枚举删边)
- javascript中window对象及属性
- Runtime学习笔记
- stochastic noise and deterministic noise
- 通过工具来监控webService请求和返回时的数据
- java保留两位小数
- UTF-8简史
- [PHP] LAMP环境搭建
- iOS导航页
- Xamarin之TableView
- 使用Category 重写frame
- 5. jQuery 效果 - 隐藏和显示