最小二乘曲线拟合matlab实现

曲线拟合[1]是一种常用的模型拟合方法，经常用在相机标定或其它传感器标定上。对于加性高斯噪声，最小二乘通常是一种最优的估计方法（PS:当外点存在时，通常可以采用其它robust estimator 或者采用ransac进行拟合）。本文演示最小二乘曲线拟合技术，所有代码可从点击此处下载。

对如下图所示的加噪声曲线，如何进行曲线拟合呢？

我们可以采用ｄ阶多项式去逼近它：

系数到则由最小二乘确定。在本文，d取1，３，５.

随机数据的产生如下：

function [y_truth,y_observed] = unknown_model1(x)

y_truth =  0.001*x.^4 + x.^3-5*x.^2+0.5*x;　　％4阶的数据
y_observed = y_truth + randn(length(x),1)';    ％观测到的数据包含单位高斯噪声


end

整个演示程序如下：

clc;

clear;
close all;
%% sample
x_all = 0:0.1:10;
[y_truth_all,y_all]=unknown_model1(x_all);
N = length(x_all);
%use the first half for training
x = x_all(1:N/2);
y = y_all(1:N/2);
y_truth = y_truth_all(1:N/2);


plot(x,y_truth,'g-x','LineWidth',1.5);
hold on;
plot(x,y,'m-x','LineWidth',1.5);
legend('model truth','observation');
title('training');


%% curve fitting, try linear model
X = [x;ones(1,length(x))]';
Y = y';
lambda1 = (X'*X)\X'*Y;　％最小二乘拟合 (least-square fitting)
% evaluate the esimated model
ye1 = X*lambda1;
hold on;
plot(x,ye1,'c-o','LineWidth',1.5);




%% curve fitting, try order-3 order polynomial
X = [x.^3;x.^2;x;ones(1,length(x))]';
Y = y';
lambda2 = (X'*X)\X'*Y;％最小二乘拟合
% evaluate the esimated model
ye2 = X*lambda2;
hold on;
plot(x,ye2,'r-o','LineWidth',1.5);




%% curve fitting, try order-5 order polynomial
X = [x.^5;x.^4;x.^3;x.^2;x;ones(1,length(x))]';
Y = y';
lambda3 = (X'*X)\X'*Y;％最小二乘拟合
% evaluate the esimated model
ye3 = X*lambda3;
hold on;
plot(x,ye3,'b-o','LineWidth',1.5);



%%
figure;
plot(x,y,'g-x',x,ye1,'c-o',x,ye2,'r-o',x,ye3,'b-o','LineWidth',1.5);
legend('model truth', 'order-1','order-3', 'order-5');

以上是训练的代码。由图可见，３阶和5阶与真实模型最为接近，一阶的误差较大。

下面根据估计的模型对未来未知的数据进行预测，比较各模型与真实模型的误差大小
%% show future trends
X = [x_all;ones(1,length(x_all))]';
ye1 = X*lambda1;
X = [x_all.^3;x_all.^2;x_all;ones(1,length(x_all))]';
ye2 = X*lambda2;
X = [x_all.^5;x_all.^4;x_all.^3;x_all.^2;x_all;ones(1,length(x_all))]';
ye3 = X*lambda3;
figure;
plot(x_all,y_truth_all,'g-x',x_all,y_all,'m-x',x_all,ye1,'c-o',x_all,ye2,'r-o',x_all,ye3,'b-o','LineWidth',1.5);
legend('model truth','observation', 'order-1 poly fitting','order-3 poly fitting', 'order-5 poly fitting');
title('testing');

由图可见，3阶模型与真实模型最为接近，而原来在训练中比较接近的5阶模型对未知数据有较大的误差，即存在所谓的过拟合现象(overfitting)[2].

参考资料：

[1]http://en.wikipedia.org/wiki/Curve_fitting

[2]http://en.wikipedia.org/wiki/Overfitting