Coursera吴恩达机器学习课程 总结笔记及作业代码——第4周神经网络
来源:互联网 发布:闪恋 知乎 编辑:程序博客网 时间:2024/05/22 12:54
Neural Networks
1.1 Non-linear hypotheses
在课程的开头,提到了非线性假设,会因为特征量的增多导致二次项数的剧增。
举个例子,在图像识别中,一个50*50像素的图片,拥有的特征量为2500,那么它的二次项数为2500*2500/2,大约为3百万个。
1.2 Model representation
上层是一个三层神经网络,第一层为输入层,第二层为隐藏层,第三层为输出层。
每条边上有一个权值
下面是符号表示。
若第j层单元数为
在bp神经网络中,我们使用sigmoid函数作为激励函数。即
1.3 Examples and intuitions
通过构造一个2层神经网络我们实现了and逻辑。
通过构造一个2层神经网络我们实现了and逻辑。
我们通过组合的方式将and notand 和or逻辑组合在一起构造一个三层神经网络完成X XNOR Y 逻辑。
1.4 Multi-class classification
对于多分类问题,我们可以通过设置多个输出值来实现。
具体在程序中,可以通过每个输出值 取最大值来判断属于哪一类。
1.4 作业代码
这次的作业是用逻辑回归和神经网络来实现手写数字识别,比较下两者的准确性。
ex3.m
%% Machine Learning Online Class - Exercise 3 | Part 1: One-vs-all% Instructions% ------------%% This file contains code that helps you get started on the% linear exercise. You will need to complete the following functions% in this exericse:%% lrCostFunction.m (logistic regression cost function)% oneVsAll.m% predictOneVsAll.m% predict.m%% For this exercise, you will not need to change any code in this file,% or any other files other than those mentioned above.%%% Initializationclear ; close all; clc%% Setup the parameters you will use for this part of the exerciseinput_layer_size = 400; % 20x20 Input Images of Digitsnum_labels = 10; % 10 labels, from 1 to 10 % (note that we have mapped "0" to label 10)%% =========== Part 1: Loading and Visualizing Data =============% We start the exercise by first loading and visualizing the dataset.% You will be working with a dataset that contains handwritten digits.%% Load Training Datafprintf('Loading and Visualizing Data ...\n')load('ex3data1.mat'); % training data stored in arrays X, ym = size(X, 1);% Randomly select 100 data points to displayrand_indices = randperm(m);sel = X(rand_indices(1:100), :);displayData(sel);fprintf('Program paused. Press enter to continue.\n');pause;%% ============ Part 2a: Vectorize Logistic Regression ============% In this part of the exercise, you will reuse your logistic regression% code from the last exercise. You task here is to make sure that your% regularized logistic regression implementation is vectorized. After% that, you will implement one-vs-all classification for the handwritten% digit dataset.%% Test case for lrCostFunctionfprintf('\nTesting lrCostFunction() with regularization');theta_t = [-2; -1; 1; 2];X_t = [ones(5,1) reshape(1:15,5,3)/10];y_t = ([1;0;1;0;1] >= 0.5);lambda_t = 3;[J grad] = lrCostFunction(theta_t, X_t, y_t, lambda_t);fprintf('\nCost: %f\n', J);fprintf('Expected cost: 2.534819\n');fprintf('Gradients:\n');fprintf(' %f \n', grad);fprintf('Expected gradients:\n');fprintf(' 0.146561\n -0.548558\n 0.724722\n 1.398003\n');fprintf('Program paused. Press enter to continue.\n');pause;%% ============ Part 2b: One-vs-All Training ============fprintf('\nTraining One-vs-All Logistic Regression...\n')lambda = 0.1;[all_theta] = oneVsAll(X, y, num_labels, lambda);fprintf('Program paused. Press enter to continue.\n');pause;%% ================ Part 3: Predict for One-Vs-All ================pred = predictOneVsAll(all_theta, X);fprintf('\nTraining Set Accuracy: %f\n', mean(double(pred == y)) * 100);
lrCostFunction.m
function [J, grad] = lrCostFunction(theta, X, y, lambda)%LRCOSTFUNCTION Compute cost and gradient for logistic regression with %regularization% J = LRCOSTFUNCTION(theta, X, y, lambda) computes the cost of using% theta as the parameter for regularized logistic regression and the% gradient of the cost w.r.t. to the parameters. % Initialize some useful valuesm = length(y); % number of training examples% You need to return the following variables correctly J = 0;grad = zeros(size(theta));% ====================== YOUR CODE HERE ======================% Instructions: Compute the cost of a particular choice of theta.% You should set J to the cost.% Compute the partial derivatives and set grad to the partial% derivatives of the cost w.r.t. each parameter in theta%% Hint: The computation of the cost function and gradients can be% efficiently vectorized. For example, consider the computation%% sigmoid(X * theta)%% Each row of the resulting matrix will contain the value of the% prediction for that example. You can make use of this to vectorize% the cost function and gradient computations. %% Hint: When computing the gradient of the regularized cost function, % there're many possible vectorized solutions, but one solution% looks like:% grad = (unregularized gradient for logistic regression)% temp = theta; % temp(1) = 0; % because we don't add anything for j = 0 % grad = grad + YOUR_CODE_HERE (using the temp variable)%J = 1/m * (-y' * log(sigmoid(X*theta)) - (1 - y')* log(1-sigmoid(X*theta))) + lambda/2/m*sum(theta(2:end) .^ 2);grad(1, :) = 1/m * (X(:,1)'* (sigmoid(X*theta) - y));grad(2:end, :) = 1/m * (X(:,2:end)'* (sigmoid(X*theta) - y)) + lambda/m*theta(2:end, :);% =============================================================grad = grad(:);end
oneVsAll.m
function [all_theta] = oneVsAll(X, y, num_labels, lambda)%ONEVSALL trains multiple logistic regression classifiers and returns all%the classifiers in a matrix all_theta, where the i-th row of all_theta %corresponds to the classifier for label i% [all_theta] = ONEVSALL(X, y, num_labels, lambda) trains num_labels% logistic regression classifiers and returns each of these classifiers% in a matrix all_theta, where the i-th row of all_theta corresponds % to the classifier for label i% Some useful variablesm = size(X, 1);n = size(X, 2);% You need to return the following variables correctly all_theta = zeros(num_labels, n + 1);% Add ones to the X data matrixX = [ones(m, 1) X];% ====================== YOUR CODE HERE ======================% Instructions: You should complete the following code to train num_labels% logistic regression classifiers with regularization% parameter lambda. %% Hint: theta(:) will return a column vector.%% Hint: You can use y == c to obtain a vector of 1's and 0's that tell you% whether the ground truth is true/false for this class.%% Note: For this assignment, we recommend using fmincg to optimize the cost% function. It is okay to use a for-loop (for c = 1:num_labels) to% loop over the different classes.%% fmincg works similarly to fminunc, but is more efficient when we% are dealing with large number of parameters.%% Example Code for fmincg:%% % Set Initial theta% initial_theta = zeros(n + 1, 1);% % % Set options for fminunc% options = optimset('GradObj', 'on', 'MaxIter', 50);% % % Run fmincg to obtain the optimal theta% % This function will return theta and the cost % [theta] = ...% fmincg (@(t)(lrCostFunction(t, X, (y == c), lambda)), ...% initial_theta, options);%initial_theta = zeros(n+1, 1);options = optimset('GradObj', 'on', 'MaxIter', 50);for c=1:num_labels all_theta(c, :) = fmincg(@(t)(lrCostFunction(t, X, (y==c), lambda)), initial_theta, options);end% =========================================================================end
predictOneVsAll.m
function p = predictOneVsAll(all_theta, X)%PREDICT Predict the label for a trained one-vs-all classifier. The labels %are in the range 1..K, where K = size(all_theta, 1). % p = PREDICTONEVSALL(all_theta, X) will return a vector of predictions% for each example in the matrix X. Note that X contains the examples in% rows. all_theta is a matrix where the i-th row is a trained logistic% regression theta vector for the i-th class. You should set p to a vector% of values from 1..K (e.g., p = [1; 3; 1; 2] predicts classes 1, 3, 1, 2% for 4 examples) m = size(X, 1);num_labels = size(all_theta, 1);% You need to return the following variables correctly p = zeros(size(X, 1), 1);% Add ones to the X data matrixX = [ones(m, 1) X];% ====================== YOUR CODE HERE ======================% Instructions: Complete the following code to make predictions using% your learned logistic regression parameters (one-vs-all).% You should set p to a vector of predictions (from 1 to% num_labels).%% Hint: This code can be done all vectorized using the max function.% In particular, the max function can also return the index of the % max element, for more information see 'help max'. If your examples % are in rows, then, you can use max(A, [], 2) to obtain the max % for each row.% temp = all_theta * X';[maxx, pp] = max(temp);p = pp';% =========================================================================end
最终预测准确性:Training Set Accuracy: 95.120000
下面是以三层bp神经网络处理的手写数字识别,其中权重矩阵已给出,故少了反向传播的步骤。
ex3_nn.m
%% Machine Learning Online Class - Exercise 3 | Part 2: Neural Networks% Instructions% ------------% % This file contains code that helps you get started on the% linear exercise. You will need to complete the following functions % in this exericse:%% lrCostFunction.m (logistic regression cost function)% oneVsAll.m% predictOneVsAll.m% predict.m%% For this exercise, you will not need to change any code in this file,% or any other files other than those mentioned above.%%% Initializationclear ; close all; clc%% Setup the parameters you will use for this exerciseinput_layer_size = 400; % 20x20 Input Images of Digitshidden_layer_size = 25; % 25 hidden unitsnum_labels = 10; % 10 labels, from 1 to 10 % (note that we have mapped "0" to label 10)%% =========== Part 1: Loading and Visualizing Data =============% We start the exercise by first loading and visualizing the dataset. % You will be working with a dataset that contains handwritten digits.%% Load Training Datafprintf('Loading and Visualizing Data ...\n')load('ex3data1.mat');m = size(X, 1);% Randomly select 100 data points to displaysel = randperm(size(X, 1));sel = sel(1:100);displayData(X(sel, :));fprintf('Program paused. Press enter to continue.\n');pause;%% ================ Part 2: Loading Pameters ================% In this part of the exercise, we load some pre-initialized % neural network parameters.fprintf('\nLoading Saved Neural Network Parameters ...\n')% Load the weights into variables Theta1 and Theta2load('ex3weights.mat');%% ================= Part 3: Implement Predict =================% After training the neural network, we would like to use it to predict% the labels. You will now implement the "predict" function to use the% neural network to predict the labels of the training set. This lets% you compute the training set accuracy.pred = predict(Theta1, Theta2, X);fprintf('\nTraining Set Accuracy: %f\n', mean(double(pred == y)) * 100);fprintf('Program paused. Press enter to continue.\n');pause;% To give you an idea of the network's output, you can also run% through the examples one at the a time to see what it is predicting.% Randomly permute examplesrp = randperm(m);for i = 1:m % Display fprintf('\nDisplaying Example Image\n'); displayData(X(rp(i), :)); pred = predict(Theta1, Theta2, X(rp(i),:)); fprintf('\nNeural Network Prediction: %d (digit %d)\n', pred, mod(pred, 10)); % Pause with quit option s = input('Paused - press enter to continue, q to exit:','s'); if s == 'q' break endend
predict.m
function p = predict(Theta1, Theta2, X)%PREDICT Predict the label of an input given a trained neural network% p = PREDICT(Theta1, Theta2, X) outputs the predicted label of X given the% trained weights of a neural network (Theta1, Theta2)% Useful valuesm = size(X, 1);num_labels = size(Theta2, 1);% You need to return the following variables correctly p = zeros(size(X, 1), 1);% ====================== YOUR CODE HERE ======================% Instructions: Complete the following code to make predictions using% your learned neural network. You should set p to a % vector containing labels between 1 to num_labels.%% Hint: The max function might come in useful. In particular, the max% function can also return the index of the max element, for more% information see 'help max'. If your examples are in rows, then, you% can use max(A, [], 2) to obtain the max for each row.%X = [ones(m, 1) X];XX = sigmoid(X*Theta1');pp = sigmoid([ones(size(XX, 1), 1) XX] * Theta2');[a, p] = max(pp, [], 2);% =========================================================================end
最终预测准确性:Training Set Accuracy: 97.520000
- Coursera吴恩达机器学习课程 总结笔记及作业代码——第4周神经网络
- Coursera吴恩达机器学习课程 总结笔记及作业代码——第5周神经网络续
- Coursera吴恩达机器学习课程 总结笔记及作业代码——第1,2周
- Coursera吴恩达机器学习课程 总结笔记及作业代码——第3周逻辑回归
- Coursera吴恩达机器学习课程 总结笔记及作业代码——第7周支持向量机
- Coursera吴恩达机器学习课程 总结笔记及作业代码——第6周有关机器学习的小建议
- Coursera吴恩达机器学习课程 第2周作业代码
- coursera-斯坦福-机器学习-吴恩达-第4周笔记-神经网络
- coursera机器学习课程第五周——课程笔记
- coursera机器学习课程第六周——课程笔记
- coursera机器学习课程第七周——课程笔记
- Coursera机器学习课程笔记——第1周——Introduction引言
- Coursera吴恩达机器学习课程 编程作业
- Coursera机器学习 week5 神经网络的学习 编程作业代码
- Coursera机器学习 week4 神经网络的表示 编程作业代码
- Coursera机器学习课程笔记(4) Regularization
- Coursera机器学习第8周作业
- Coursera 机器学习(by Andrew Ng)课程学习笔记(四)——神经网络(一)
- HDOJ 2007平方和与立方和
- 文章标题
- AngularJs路由
- Struts2 java.io.FileNotFoundException
- redis 集群搭建过程及搭建问题处理
- Coursera吴恩达机器学习课程 总结笔记及作业代码——第4周神经网络
- J
- break跳出循环用法
- php trait 简明教程,多重继承的实现方案
- Java设计模式透析之 —— 策略(Strategy)
- 僵尸进程和孤儿进程
- angular-cli安装步骤
- Python基础五
- 基数排序之数组中缺失的数字