Coursera－Machine Learning（第二周 Linear Regression）

部分代码

第二周 Linear Regression

computeCost

predictions = X * theta;J = 1/(2*m)*(predictions - y)'*(predictions - y);

gradientDescent

temp1 = theta(1) - (alpha / m) * sum((X * theta - y).* X(:,1));temp2 = theta(2) - (alpha / m) * sum((X * theta - y).* X(:,2));theta(1) = temp1;theta(2) = temp2;J_history(iter) = computeCost(X, y, theta);

featureNormalize

mu = mean(X);sigma = std(X);for i=1:size(mu,2)    X_norm(:,i) = (X(:,i).-mu(i))./sigma(i);endcomputeCostMultipredictions = X * theta;J = 1/(2*m)*(predictions - y)' * (predictions - y);

gradientDescentMulti

for i=1:size(X,2)    temp(i) = theta(i) - (alpha / m) * sum((X * theta - y).* X(:,i));endfor j=1:size(X,2)    theta(j) = temp(j);end

Normal Equation

theta = pinv(X'*X)*X'*y;

Ex2

1.1 Visualizing the data

% Find Indices of Positive and Negative Examplespos = find(y==1); neg = find(y == 0);  % 对应0/1的相应地址向量% Plot Examplesplot(X(pos, 1), X(pos, 2), 'k+','LineWidth', 2, ...'MarkerSize', 7);plot(X(neg, 1), X(neg, 2), 'ko', 'MarkerFaceColor', 'y', ...'MarkerSize', 7);

SIGMOID function

g = 1./(ones(size(z))+e.^(-z));

1.2.2 Cost function and gradient

H=sigmoid(X*theta);J = 1/m * sum((-y.*log(H)).-((1- y).*log(1 - H)));grad = 1/m * ((H.-y)'*X)';

Evaluating logistic regression

p_predict = sigmoid(X * theta);p_point = find(p_predict >= 0.5);p(p_point,1) =1;

2 Regularized logistic regression

H = sigmoid(X*theta);theta2 = theta(2:length(theta),1);J = 1/m * sum((-y.*log(H)).-((1- y).*log(1 - H))) + lambda/(2*m) * sum(theta2.^2);grad = 1/m * ((H.-y)'*X)';grad = [grad(1,1);(grad + lambda/m*theta)(2:length(theta),1)];

2.5 Optional (ungraded) exercises

Notice the changes in the decision boundary as you vary lambda. With a small lambda, you should nd that the classier gets almost every training example correct, but draws a very complicated boundary, thus overtting the data.

作业:Multi-class Classification and Neural Networks

1.3.3 Vectorizing regularized logistic regression

H = sigmoid(X * theta);J = sum(-y.*log(H) - (1-y).*log(1-H))/m + lambda/(2*m) * sum(theta(2:end).^2);grad = X' * (H-y)/m;grad = [grad(1);(grad + lambda/m*theta)(2:length(theta))];1.4 One-vs-all Classicationfor k=1:num_labels    initial_theta = zeros(n + 1, 1);    options = optimset('GradObj', 'on', 'MaxIter', 50);    [theta] = fmincg (@(t)(lrCostFunction(t, X, (y == k), lambda)),initial_theta, options);    all_theta(k,:) = theta';end

1.4.1 One-vs-all Prediction

[c,i] = max(sigmoid(X * all_theta'), [], 2);p = i;

2 Neural Networks

X = [ones(m, 1) X];tem1 = Theta1 * X';ans1 = sigmoid(tem1);tem2 = Theta2 * [ones(1, m);ans1];ans2 = sigmoid(tem2);[c,i] = max(ans2', [], 2);p = i;

Programming Exercise 4: Neural Networks Learning

ans1 = [ones(m, 1) X];tem2 = ans1 * Theta1';ans2 = sigmoid(tem2);tem3 = [ones(m, 1) ans2] * Theta2';H = sigmoid(tem3);yy = zeros(m, num_labels);for i = 1:m    yy(i,y(i)) = 1;endJ = 1/m * sum(sum(-yy.*log(H)-(1-yy).*log(1-H))) + lambda/2/m * (sum(sum(Theta1(:,2:end).^2)) + sum(sum(Theta2(:,2:end).^2)));for row = 1:m    ans1 = [1 X(row,:)]';    tem2 = Theta1 * ans1;    ans2 = [1; sigmoid(tem2)];    tem3 = Theta2 * ans2;    ans3 = sigmoid(tem3);    delta3 = ans3 - yy'(:, row);    delta2 = (Theta2' * delta3) .* sigmoidGradient([1; tem2]);    delta2 = delta2(2:end);    Theta1_grad = Theta1_grad + delta2 * ans1';    Theta2_grad = Theta2_grad + delta3 * ans2';endTheta1_grad = Theta1_grad ./ m;Theta1_grad(:, 2:end) = Theta1_grad(:, 2:end) + (lambda/m) * Theta1(:, 2:end);Theta2_grad = Theta2_grad ./ m;