UFLDL stackedae_exercise编程答案
来源:互联网 发布:网络流行用语2017 编辑:程序博客网 时间:2024/06/03 13:55
Step 0: Initialize constants and parameters
Open stackedAEExercise.m. In this step, we set meta-parameters to the same values that were used in previous exercise, which should produce reasonable results. You may to modify the meta-parameters if you wish.
Step 1: Train the data on the first stacked autoencoder
Train the first autoencoder on the training images to obtain its parameters. This step is identical to the corresponding step in the sparse autoencoder and STL assignments, complete this part of the code so as to learn a first layer of features using your sparseAutoencoderCost.m and minFunc.
Step 2: Train the data on the second stacked autoencoder
We first forward propagate the training set through the first autoencoder (using feedForwardAutoencoder.m that you completed in Exercise:Self-Taught_Learning) to obtain hidden unit activations. These activations are then used to train the second sparse autoencoder. Since this is just an adapted application of a standard autoencoder, it should run similarly with the first. Complete this part of the code so as to learn a first layer of features using your sparseAutoencoderCost.m and minFunc.
This part of the exercise demonstrates the idea of greedy layerwise training with the same learning algorithm reapplied multiple times.
代码实现:
addpath minFunc/options.Method = 'lbfgs'; % Here, we use L-BFGS to optimize our cost % function. Generally, for minFunc to work, you % need a function pointer with two outputs: the % function value and the gradient. In our problem, % sparseAutoencoderCost.m satisfies this.options.maxIter = 400; % Maximum number of iterations of L-BFGS to run options.display = 'on';% % [sae1OptTheta, cost] = minFunc( @(p) sparseAutoencoderCost(p, ... inputSize, hiddenSizeL1, ... lambda, sparsityParam, ... beta, trainData), ... sae1Theta, options);% save('sae1OptTheta','sae1OptTheta');% load sae1OptTheta.mat;
Step 3: Train the softmax classifier on the L2 features
Next, continue to forward propagate the L1 features through the second autoencoder (using feedForwardAutoencoder.m) to obtain the L2 hidden unit activations. These activations are then used to train the softmax classifier. You can either use softmaxTrain.m or directly use softmaxCost.m that you completed in Exercise:Softmax Regression to complete this part of the assignment.
代码实现:
[sae2OptTheta, cost] = minFunc( @(p) sparseAutoencoderCost(p, ... hiddenSizeL1, hiddenSizeL2, ... lambda, sparsityParam, ... beta, sae1Features), ... sae2Theta, options);% save('sae2OptTheta','sae2OptTheta'); % load sae2OptTheta.mat;
softmax训练代码实现:
% %train sftmax regressionoptions.maxIter = 100;softmaxModel = softmaxTrain(hiddenSizeL2, numClasses, lambda, ... sae2Features, trainLabels, options);saeSoftmaxOptTheta = softmaxModel.optTheta(:);% save('saeSoftmaxOptTheta','saeSoftmaxOptTheta');% load saeSoftmaxOptTheta.mat;
Step 4: Implement fine-tuning
To implement fine tuning, we need to consider all three layers as a single model. Implement stackedAECost.m to return the cost and gradient of the model. The cost function should be as defined as the log likelihood and a gradient decay term. The gradient should be computed using back-propagation as discussed earlier. The predictions should consist of the activations of the output layer of the softmax model.
To help you check that your implementation is correct, you should also check your gradients on a synthetic small dataset. We have implemented checkStackedAECost.m to help you check your gradients. If this checks passes, you will have implemented fine-tuning correctly.
Note: When adding the weight decay term to the cost, you should regularize only the softmax weights (do not regularize the weights that compute the hidden layer activations).
Implementation Tip: It is always a good idea to implement the code modularly and check (the gradient of) each part of the code before writing the more complicated parts.
stackedAECost.m中代码实现:
% feedforwardn=numel(stack);z=cell(n+1,1);a=cell(n+1,1);a{1}=data;for i=1:n z_temp=stack{i}.w*a{i}; z{i+1}=bsxfun(@plus,z_temp,stack{i}.b); a{i+1}=sigmoid(z{i+1});end%softmax outputH=softmaxTheta*a{i+1};H=bsxfun(@minus, H, max(H, [], 1)); % to prevent overflowExpM=exp(H);P=bsxfun(@rdivide,ExpM,sum(ExpM));%costcost=-1/M*sum(sum(groundTruth.*log(P)))+lambda/2*sum((softmaxThetaGrad(:)).^2);%the gradientdelta=cell(n+1,1);delta{n+1}=-softmaxTheta.'*(groundTruth-P).*a{n+1}.*(1-a{n+1}); %200*10softmaxThetaGrad=-1/M.*(groundTruth-P)*a{n+1}.'+lambda.*softmaxThetaGrad;%deltafor l=n:-1:2 delta{l}=stack{l}.w.'*delta{l+1}.*a{l}.*(1-a{l}); stackgrad{l}.w=delta{l+1}*(a{l}).'./M; stackgrad{l}.b=sum(delta{l+1},2)./M;endstackgrad{1}.w=delta{2}*(a{1}).'./M;stackgrad{1}.b=sum(delta{2},2)./M;主函数中代码实现:
%% ---------------------- YOUR CODE HERE ---------------------------------% Instructions: Train the deep network, hidden size here refers to the '% dimension of the input to the classifier, which corresponds % to "hiddenSizeL2".%%[stackedAEOptTheta, cost] = minFunc( @(p) stackedAECost(p, ... inputSize, hiddenSizeL2, ... numClasses,netconfig,lambda, ... trainData, trainLabels), ... stackedAETheta, options);
Step 5: Test the model
Finally, you will need to classify with this model; complete the code in stackedAEPredict.m to classify using the stacked autoencoder with a classification layer.
After completing these steps, running the entire script in stackedAETrain.m will perform layer-wise training of the stacked autoencoder, finetune the model, and measure its performance on the test set. If you've done all the steps correctly, you should get an accuracy of about 87.7% before finetuning and 97.6% after finetuning (for the 10-way classification problem).
代码实现:
n=numel(stack);z=cell(n+1,1);a=cell(n+1,1);a{1}=data;for i=1:n z_temp=stack{i}.w*a{i}; z{i+1}=bsxfun(@plus,z_temp,stack{i}.b); a{i+1}=sigmoid(z{i+1});end%softmax outputH=softmaxTheta*a{i+1};H=bsxfun(@minus, H, max(H, [], 1)); % to prevent overflowExpM=exp(H);P=bsxfun(@rdivide,ExpM,sum(ExpM));[hmax,pred]=max(P);clear hmax;
运行结果:
微调之前 识别精度为:88.080%
微调之后 识别精度为:97.760%
源代码下载:http://download.csdn.net/detail/suan2014/9891006
- UFLDL stackedae_exercise编程答案
- UFLDL softmax回归编程答案
- UFLDL self-taught自我学习编程答案
- UFLDL教程练习答案一(稀疏自编码器和矢量化编程实现)
- UFLDL教程 Exercise:Sparse Autoencoder(答案)
- UFLDL教程 Exercise:Vectorization(答案)
- UFLDL教程答案(1):Exercise:Sparse_Autoencoder
- UFLDL教程答案(2):Exercise:Vectorization
- UFLDL教程答案(3):Exercise:PCA_in_2D&PCA_and_Whitening
- UFLDL教程答案(4):Exercise:Softmax Regression
- UFLDL教程Exercise答案(4):Softmax Regression
- • UFLDL教程练习(exercise)答案(1)
- UFLDL教程答案(5):Exercise:Self-Taught Learning
- UFLDL教程答案(8):Exercise:Convolution and Pooling
- UFLDL教程练习(exercise)答案(1)
- UFLDL教程Exercise答案(3.1):PCA in 2D
- UFLDL教程Exercise答案(8):Convolution and Pooling
- ufldl学习笔记与编程作业:Multi
- 在CentOS安装CMake
- 项目中tomcat 下配置HTTPS
- 四大标签
- Android——Fragment中通过给每个Fragment设置开场动画进入
- Hibernate开启收集缓存统计信息
- UFLDL stackedae_exercise编程答案
- SpringBoot系列(3)---无配置文件SpringMVC
- 五、JAVA语法基础(3)
- 【技能库】--JS时间处理(251)
- Split()用法 isNaN MVC ...基础知识
- springmvc环境搭建中url-pattern /和/*的区别
- Number Sequence F(n)的计算
- Hibernate POJO为什么要实现Serializable接口
- C++控制台输出字体颜色字体背景色