UFLDL教程Exercise答案(6)：Implement deep networks for digit classification
来源：互联网 发布：codeblocks c语言 编辑：程序博客网 时间：2024/05/21 10:50
教程地址：http://deeplearning.stanford.edu/wiki/index.php/UFLDL_Tutorial
Exercise地址：http://deeplearning.stanford.edu/wiki/index.php/Exercise:_Implement_deep_networks_for_digit_classification
代码
将sparseAutoencoderCost.m softmaxCost.m feedForwardAutoencoder.m initializeParameters.m loadMNISTImages.m loadMNISTLabels.m文件以及minFunc文件夹从以前的文件中拷贝到本节练习所在的工作路径中。
Step 0: Initialize constants and parameters ——代码已给
Step 00: Load data from the MNIST database——代码已给
根据自己的训练集所在的路径修改相应路径。
我的路径设置为：'F:/DeepLearning/UFLDL/mnist/train-images.idx3-ubyte' ；'F:/DeepLearning/UFLDL/mnist/train-labels.idx1-ubyte'
Step 1: Train the data on the first stacked autoencoder ——stackedAEExercise.m
%% ---------------------- YOUR CODE HERE ---------------------------------% Instructions: Train the first layer sparse autoencoder, this layer has% an hidden size of "hiddenSizeL1"% You should store the optimal parameters in sae1OptThetasae1OptTheta = sae1Theta;% Use minFunc to minimize the function addpath minFunc/ options.Method = 'lbfgs'; % Here, we use L-BFGS to optimize our cost function. options.maxIter = 400; % Maximum number of iterations of L-BFGS to run options.display = 'on'; [sae1OptTheta, cost] = minFunc( @(p) sparseAutoencoderCost(p, ... inputSize, hiddenSizeL1, ... lambda, sparsityParam, ... beta, trainData), ... sae1Theta, options); % -------------------------------------------------------------------------
Step 2: Train the data on the second stacked autoencoder ——stackedAEExercise.m
%% ---------------------- YOUR CODE HERE ---------------------------------% Instructions: Train the second layer sparse autoencoder, this layer has% an hidden size of "hiddenSizeL2" and an inputsize of% "hiddenSizeL1"%% You should store the optimal parameters in sae2OptThetasae2OptTheta = sae2Theta;% Use minFunc to minimize the function options.Method = 'lbfgs'; % Here, we use L-BFGS to optimize our cost function. options.maxIter = 400; % Maximum number of iterations of L-BFGS to run options.display = 'on'; [sae2OptTheta, cost] = minFunc( @(p) sparseAutoencoderCost(p, ... hiddenSizeL1, hiddenSizeL2, ... lambda, sparsityParam, ... beta, sae1Features), ... sae2Theta, options); % -------------------------------------------------------------------------
Step 3: Train the softmax classifier on the L2 features ——stackedAEExercise.m
%% ---------------------- YOUR CODE HERE ---------------------------------% Instructions: Train the softmax classifier, the classifier takes in% input of dimension "hiddenSizeL2" corresponding to the% hidden layer size of the 2nd layer.%% You should store the optimal parameters in saeSoftmaxOptTheta %% NOTE: If you used softmaxTrain to complete this part of the exercise,% set saeSoftmaxOptTheta = softmaxModel.optTheta(:);%%%use softmaxTrain.m options.maxIter = 100;softmaxModel = softmaxTrain(hiddenSizeL2, numClasses, lambda, ... sae2Features, trainLabels, options);saeSoftmaxOptTheta = softmaxModel.optTheta(:);%%%use softmaxCost.m % addpath minFunc/ % options.Method = 'lbfgs'; % Here, we use L-BFGS to optimize our cost function. % minFuncOptions.display = 'on'; % options.maxIter = 400; % [saeSoftmaxOptTheta, cost3] = minFunc( @(p) softmaxCost(p, ... % numClasses, hiddenSizeL2, lambda, ... % sae2Features, trainLabels), ... % saeSoftmaxTheta, options);% -------------------------------------------------------------------------
Step 4: Implement fine-tuning ——stackedAEExercise.m + stackedAECost.m
%%%stackedAEExercise.m
%% ---------------------- YOUR CODE HERE ---------------------------------% Instructions: Train the deep network, hidden size here refers to the '% dimension of the input to the classifier, which corresponds % to "hiddenSizeL2".%%options.Method = 'lbfgs'; options.maxIter = 400; options.display = 'on'; [stackedAEOptTheta, cost] = minFunc( @(p) stackedAECost(p, inputSize, hiddenSizeL2, ... numClasses, netconfig, ... lambda, trainData, trainLabels), ... stackedAETheta, options); % -------------------------------------------------------------------------
%%%stackedAECost.m%% --------------------------- YOUR CODE HERE -----------------------------% Instructions: Compute the cost function and gradient vector for % the stacked autoencoder.%% You are given a stack variable which is a cell-array of% the weights and biases for every layer. In particular, you% can refer to the weights of Layer d, using stack{d}.w and% the biases using stack{d}.b . To get the total number of% layers, you can use numel(stack).%% The last layer of the network is connected to the softmax% classification layer, softmaxTheta.%% You should compute the gradients for the softmaxTheta,% storing that in softmaxThetaGrad. Similarly, you should% compute the gradients for each layer in the stack, storing% the gradients in stackgrad{d}.w and stackgrad{d}.b% Note that the size of the matrices in stackgrad should% match exactly that of the size of the matrices in stack.%% %【1】进行前馈传递%计算hiddenL1和hiddenL2的神经元的激活值depth = numel(stack); %网络的hidden层的总层数 2层% stack{i}.z 存储第i层的值,% stack{i}.a %存储第i层的激活值stack{1}.a = data; %stack{1}.a中存储输入值for i = 1 : depth stack{i+1}.z = stack{i}.w * stack{i}.a + repmat(stack{i}.b,1,M); stack{i+1}.a = sigmoid(stack{i+1}.z);end%计算softmax层的输出p = softmaxTheta * stack{end}.a; p = bsxfun(@minus, p, max(p, [],1)); p = exp(p); % p = bsxfun(@rdivide, p, sum(p)); %【2】计算softmax的cost functioncost = -1/M * sum(sum(groundTruth .* log(p))) + lambda/2 * sum(sum(softmaxTheta.^2));%【3】计算误差项stack{depth+1}.delta = -(softmaxTheta' * (groundTruth - p)) .* stack{end}.a .* (1-stack{end}.a);for i= depth:-1:2 stack{i}.delta = stack{i}.w' * stack{i+1}.delta .* stack{i}.a .* (1 - stack{i}.a);end%【4】计算偏导数%计算隐藏层的偏导数for i=depth : -1 : 1 stackgrad{i}.w = stack{i+1}.delta * stack{i}.a' /M; stackgrad{i}.b = sum(stack{i+1}.delta,2) /M;end%计算softmax的偏导数softmaxThetaGrad = -1/M * (groundTruth - p) * stack{end}.a' + lambda * softmaxTheta;% -------------------------------------------------------------------------Step 5: Test the model ——stackedAEPredict.m
%% ---------- YOUR CODE HERE --------------------------------------% Instructions: Compute pred using theta assuming that the labels start % from 1.M = size(data,2); depth = numel(stack); stack{1}.a = data; for i = 1:depth stack{i+1}.z = stack{i}.w * stack{i}.a + repmat(stack{i}.b, 1, M); stack{i+1}.a = sigmoid(stack{i+1}.z); end softmaxOut = softmaxTheta * stack{end}.a; [~, pred] = max(softmaxOut);% -----------------------------------------------------------0 0
- UFLDL教程答案(6):Exercise:Implement deep networks for digit classification
- UFLDL教程Exercise答案(6)：Implement deep networks for digit classification
- Stanford UFLDL教程 Exercise: Implement deep networks for digit classification
- UFLDL教程: Exercise: Implement deep networks for digit classification
- UFLDL Exercise: Implement deep networks for digit classification
- UFLDL Exercise:Implement deep networks for digit classification
- UFLDL Exercise: Implement deep networks for digit classificationz
- Exercise: Implement deep networks for digit classification 代码示例
- Convolutional neural networks(CNN) (九) Implement deep networks for digit classification Exercise
- UFLDL Tutorial_Building Deep Networks for Classification
- UFLDL学习笔记5（Building Deep Networks for Classification）
- 【UFLDL-exercise7-stacked autoencoder for digit classification】
- UFLDL教程 Exercise:Sparse Autoencoder（答案）
- UFLDL教程 Exercise:Vectorization（答案）
- UFLDL教程答案(1):Exercise:Sparse_Autoencoder
- UFLDL教程答案(2):Exercise:Vectorization
- UFLDL教程答案(3):Exercise:PCA_in_2D&PCA_and_Whitening
- UFLDL教程答案(4):Exercise:Softmax Regression
- linux系统配置静态IP
- Express cookie-parser
- Android逆向之旅---动态方式破解apk进阶篇(IDA调试so源码)
- 第十三周 项目2最小生成树的克鲁斯卡尔算法
- 第14周项目1-（1）验证折半查找算法
- UFLDL教程Exercise答案(6)：Implement deep networks for digit classification
- kafka删除和创建topic
- centos下压缩文件7z解压
- Android逆向之旅---Android应用的安全的攻防之战
- python学习笔记（一）
- 第十四周-项目一 验证算法（3）
- 各种图片编码格式详解（bmp,jpg,png）
- Android程序之聚合数据全国天气预报查询API接口使用演示
- 第十四周项目1 - 验证二叉树相关算法
