机器学习课程练习（四）——softmax

前言

斯坦福的UFLDL教程每一个章节都配有练习

本文是softmax这一章节的练习的解答

练习的目的是学习softmax算法的基本知识，并且训练一个softmax手写体数字识别器

具体内容可以浏览课程网页

注意事项

1.matlab是从1开始计数的所以为了方便起见，将所有数字0的标记变为10

2.注意利用稀疏矩阵函数sparse和full生成ground_truth

3.为了防止指数函数的上溢，将所有的指数项都减去其最大值，这并不影响分类的结果

4.善于利用bsxfun函数有助于减少代码量

5.需要利用到稀疏自动编码器一章中的l-bfgs函数，已经梯度函数验证函数和手写体数字的数据以及读取函数

softmaxCost

function [cost, grad] = softmaxCost(theta, numClasses, inputSize, lambda, data, labels)% numClasses - the number of classes % inputSize - the size N of the input vector% lambda - weight decay parameter% data - the N x M input matrix, where each column data(:, i) corresponds to%        a single test set% labels - an M x 1 matrix containing the labels corresponding for the input data%% Unroll the parameters from thetatheta = reshape(theta, numClasses, inputSize);numCases = size(data, 2);groundTruth = full(sparse(labels, 1:numCases, 1));cost = 0;thetagrad = zeros(numClasses, inputSize);%% ---------- YOUR CODE HERE --------------------------------------%  Instructions: Compute the cost and gradient for softmax regression.%                You need to compute thetagrad and cost.%                The groundTruth matrix might come in handy.%计算h(x)hypothesis = zeros(numClasses, inputSize);hypothesis = theta * data;hypothesis = bsxfun(@minus, hypothesis, max(hypothesis, [], 1));hypothesis = exp(hypothesis);hypothesis = bsxfun(@rdivide, hypothesis, sum(hypothesis));%计算代价函数theta2 = theta .^ 2;weight_decay = 0.5 * lambda * sum(theta2(:));log_h = groundTruth .* log(hypothesis);cost = - sum(log_h(:)) ./ numCases + weight_decay;%计算梯度thetagrad = - (groundTruth - hypothesis) * data' ./ numCases + lambda * theta;% ------------------------------------------------------------------% Unroll the gradient matrices into a vector for minFuncgrad = [thetagrad(:)];end

softmaxPredict函数

function [pred] = softmaxPredict(softmaxModel, data)% softmaxModel - model trained using softmaxTrain% data - the N x M input matrix, where each column data(:, i) corresponds to%        a single test set%% Your code should produce the prediction matrix % pred, where pred(i) is argmax_c P(y(c) | x(i)). % Unroll the parameters from thetatheta = softmaxModel.optTheta;  % this provides a numClasses x inputSize matrixpred = zeros(1, size(data, 2));%% ---------- YOUR CODE HERE --------------------------------------%  Instructions: Compute pred using theta assuming that the labels start %                from 1.hypothesis = exp(theta * data);[a,pred] = max(hypothesis);% ---------------------------------------------------------------------end