Something About RMSEP
来源:互联网 发布:网络通信运营商 编辑:程序博客网 时间:2024/05/16 17:24
评价PLS的指标
They differ in the type of cases that are used to measure them:
- RMSEC: calibration error, i.e. the residuals of the calibration data.
(R)MSEC measures goodness of fit between your data and the calibration model. Depending on the type of data, model and application this can be subject to a huge optimistic bias due to overfitting compared to the (R)MSE observed for real cases when applying the calibration. If the model suffers from not being complex enough (underfitting), calibration error approximates prediction error. But it cannot indicate overfitting. - RMSECV: errors are calculated on test/train splits using a cross validation scheme for the splitting.
If the splitting of the data is done correctly, this gives a good estimate on how the model built on the data set at hand performs for unknown cases. However, due to the resampling nature of the approach, it actually measures performance for unknown cases that were obtained among the calibration cases. I.e. it does not measure how well the model works for cases that are measured months after calibration is done. For that, you need - MSEP/RMSEP: prediction error, i.e. measured on real cases and compared to reference values obtained for these.
- RMSEP can measure e.g. how performance deteriorates over time (e.g. due to instrument drift), but only if the validation experiments have a design that allows to measure these influences.
General remarks:
- I’d recommend to report for both cross validation and prediction errors in detail how the test cases are set apart, and for what factors independence was ensured.I regularly meet descriptions of “independent testing” (RMSEP) where acutally a single split of the calibration data was performed.
- A one-time split of the data obtained for calibration typically yields no better performance estimate than a cross validation. I claim this because in practice, most data leaks occur just as easily for one split as for the many splits in cross validation. Nevertheless, it may be easier to implement a protocol that in practice avoids these errors in a very transparent way for predicition error
0 0
- Something About RMSEP
- Something about ...
- Something about float divide
- something about Tomcat(zz)
- something about Listeners
- something about Macro
- Something about Music
- something about struts
- something about ado
- something about SEO
- Something About Pretect Mode
- something about this week
- Something about Ajax
- something about Ajax Ⅱ
- Something | About db-design
- Something About ! CSS
- Something About FLV视频
- Something about WM_PAINT
- Android中Spinner下拉列表(使用ArrayAdapter和自定义Adapter实现)
- 事件处理 onTouch onTouchEvent源码分析
- Qt之QParallelAnimationGroup
- 欢迎使用CSDN-markdown编辑器
- 1050. String Subtraction (20)
- Something About RMSEP
- NoSQL之Redis---主从复制
- UVA 1615 高速公路
- WPF动态添加按钮并相应事件。
- du 命令
- 投票的选择
- 初探Vue原理之view-model的数据动态双向绑定
- 文件上传之本地预览
- UGUI之ScrollRect添加响应事件后不能滑动