UFLDL Exercise:PCA and Whitening

这个练习还是围绕着pca，pca白化，zca白化的，不过这里用的图像，而不再是简单的二维数据，能够让我们直观地看到这些预处理的作用

Step 0b: Zero-mean the data (by row)

%% Step 0b: Zero-mean the data (by row)%  You can make use of the mean and repmat/bsxfun functions.% -------------------- YOUR CODE HERE -------------------- avg = mean(x,1);x = x - repmat(avg,size(x,1),1);

Step 1a&1b: Implement PCA to obtain xRot & Check your implementation of PCA

%% Step 1a: Implement PCA to obtain xRot%  Implement PCA to obtain xRot, the matrix in which the data is expressed%  with respect to the eigenbasis of sigma, which is the matrix U.% -------------------- YOUR CODE HERE -------------------- xRot = zeros(size(x)); % You need to compute thissigma = x * x' / size(x,2);[u s v] = svd(sigma);xRot = u' * x;%%================================================================%% Step 1b: Check your implementation of PCA%  The covariance matrix for the data expressed with respect to the basis U%  should be a diagonal matrix with non-zero entries only along the main%  diagonal. We will verify this here.%  Write code to compute the covariance matrix, covar. %  When visualised as an image, you should see a straight line across the%  diagonal (non-zero entries) against a blue background (zero entries).% -------------------- YOUR CODE HERE -------------------- covar = zeros(size(x, 1)); % You need to compute thiscovar = xRot * xRot' / size(xRot,2);% Visualise the covariance matrix. You should see a line across the% diagonal against a blue background.figure('name','Visualisation of covariance matrix');imagesc(covar);

结果如下

Step 2: Find k, the number of components to retain

%% Step 2: Find k, the number of components to retain%  Write code to determine k, the number of components to retain in order%  to retain at least 99% of the variance.% -------------------- YOUR CODE HERE -------------------- k = 0; % Set k accordinglyall = sum(diag(s));for i=1:size(s,1)    if sum(diag(s(1:i,1:i))) / all >= 0.99        k = i;        break;    endend

Step 3: Implement PCA with dimension reduction

%% Step 3: Implement PCA with dimension reduction%  Now that you have found k, you can reduce the dimension of the data by%  discarding the remaining dimensions. In this way, you can represent the%  data in k dimensions instead of the original 144, which will save you%  computational time when running learning algorithms on the reduced%  representation.% %  Following the dimension reduction, invert the PCA transformation to produce %  the matrix xHat, the dimension-reduced data with respect to the original basis.%  Visualise the data and compare it to the raw data. You will observe that%  there is little loss due to throwing away the principal components that%  correspond to dimensions with low variation.% -------------------- YOUR CODE HERE -------------------- xHat = zeros(size(x));  % You need to compute thisxTilde = u(:,1:k)' * x;xHat = u(:,1:k) * xTilde;% Visualise the data, and compare it to the raw data% You should observe that the raw and processed data are of comparable quality.% For comparison, you may wish to generate a PCA reduced image which% retains only 90% of the variance.figure('name',['PCA processed images ',sprintf('(%d / %d dimensions)', k, size(x, 1)),'']);display_network(xHat(:,randsel));figure('name','Raw images');display_network(x(:,randsel));

对比图如下

原数据

保留90%的方差

保留99%的方差

Step 4a&4b: Implement PCA with whitening and regularisation & Check your implementation of PCA whitening

%% Step 4a: Implement PCA with whitening and regularisation%  Implement PCA with whitening and regularisation to produce the matrix%  xPCAWhite. epsilon = 0.1;xPCAWhite = zeros(size(x));xPCAWhite = diag(sqrt(1./(diag(s) + epsilon))) * xRot;% -------------------- YOUR CODE HERE -------------------- %%================================================================%% Step 4b: Check your implementation of PCA whitening %  Check your implementation of PCA whitening with and without regularisation. %  PCA whitening without regularisation results a covariance matrix %  that is equal to the identity matrix. PCA whitening with regularisation%  results in a covariance matrix with diagonal entries starting close to %  1 and gradually becoming smaller. We will verify these properties here.%  Write code to compute the covariance matrix, covar. %%  Without regularisation (set epsilon to 0 or close to 0), %  when visualised as an image, you should see a red line across the%  diagonal (one entries) against a blue background (zero entries).%  With regularisation, you should see a red line that slowly turns%  blue across the diagonal, corresponding to the one entries slowly%  becoming smaller.% -------------------- YOUR CODE HERE -------------------- % Visualise the covariance matrix. You should see a red line across the% diagonal against a blue background.covar = xPCAWhite * xPCAWhite' / size(xPCAWhite,2)figure('name','Visualisation of covariance matrix');imagesc(covar);

epsilon = 0.1

epsilon = 0

Step 5: Implement ZCA whitening

%% Step 5: Implement ZCA whitening%  Now implement ZCA whitening to produce the matrix xZCAWhite. %  Visualise the data and compare it to the raw data. You should observe%  that whitening results in, among other things, enhanced edges.xZCAWhite = zeros(size(x));% -------------------- YOUR CODE HERE -------------------- xZCAWhite = u * xPCAWhite;% Visualise the data, and compare it to the raw data.% You should observe that the whitened images have enhanced edges.figure('name','ZCA whitened images');display_network(xZCAWhite(:,randsel));figure('name','Raw images');display_network(x(:,randsel));

结果如下，强化了边缘