Stanford UFLDL教程 Exercise: Implement deep networks for digit classification
来源:互联网 发布:mac 转移用户文件夹 编辑:程序博客网 时间:2024/05/13 08:52
Exercise: Implement deep networks for digit classification
Contents
[hide]- 1Overview
- 2Dependencies
- 3Step 0: Initialize constants and parameters
- 4Step 1: Train the data on the first stacked autoencoder
- 5Step 2: Train the data on the second stacked autoencoder
- 6Step 3: Train the softmax classifier on the L2 features
- 7Step 4: Implement fine-tuning
- 8Step 5: Test the model
Overview
In this exercise, you will use a stacked autoencoder for digit classification. This exercise is very similar to the self-taught learning exercise, in which we trained a digit classifier using a autoencoder layer followed by a softmax layer. The only difference in this exercise is that we will be using two autoencoder layers instead of one and further finetune the two layers.
The code you have already implemented will allow you to stack various layers and perform layer-wise training. However, to perform fine-tuning, you will need to implement backpropogation through both layers. We will see that fine-tuning significantly improves the model's performance.
In the file stackedae_exercise.zip, we have provided some starter code. You will need to complete the code instackedAECost.m,stackedAEPredict.m andstackedAEExercise.m. We have also providedparams2stack.m andstack2params.m which you might find helpful in constructing deep networks.
Dependencies
The following additional files are required for this exercise:
- MNIST Dataset
- Support functions for loading MNIST in Matlab
- Starter Code (stackedae_exercise.zip)
You will also need your code from the following exercises:
- Exercise:Sparse Autoencoder
- Exercise:Vectorization
- Exercise:Softmax Regression
- Exercise:Self-Taught Learning
If you have not completed the exercises listed above, we strongly suggest you complete them first.
Step 0: Initialize constants and parameters
Open stackedAEExercise.m. In this step, we set meta-parameters to the same values that were used in previous exercise, which should produce reasonable results. You may to modify the meta-parameters if you wish.
Step 1: Train the data on the first stacked autoencoder
Train the first autoencoder on the training images to obtain its parameters. This step is identical to the corresponding step in the sparse autoencoder and STL assignments, complete this part of the code so as to learn a first layer of features using yoursparseAutoencoderCost.m and minFunc.
Step 2: Train the data on the second stacked autoencoder
We first forward propagate the training set through the first autoencoder (usingfeedForwardAutoencoder.m that you completed inExercise:Self-Taught_Learning) to obtain hidden unit activations. These activations are then used to train the second sparse autoencoder. Since this is just an adapted application of a standard autoencoder, it should run similarly with the first. Complete this part of the code so as to learn a first layer of features using your sparseAutoencoderCost.m and minFunc.
This part of the exercise demonstrates the idea of greedy layerwise training with thesame learning algorithm reapplied multiple times.
Step 3: Train the softmax classifier on the L2 features
Next, continue to forward propagate the L1 features through the second autoencoder (usingfeedForwardAutoencoder.m) to obtain the L2 hidden unit activations. These activations are then used to train the softmax classifier. You can either usesoftmaxTrain.m or directly use softmaxCost.m that you completed inExercise:Softmax Regression to complete this part of the assignment.
Step 4: Implement fine-tuning
To implement fine tuning, we need to consider all three layers as a single model. ImplementstackedAECost.m to return the cost and gradient of the model. The cost function should be as defined as the log likelihood and a gradient decay term. The gradient should be computed usingback-propagation as discussed earlier. The predictions should consist of the activations of the output layer of the softmax model.
To help you check that your implementation is correct, you should also check your gradients on a synthetic small dataset. We have implementedcheckStackedAECost.m to help you check your gradients. If this checks passes, you will have implemented fine-tuning correctly.
Note: When adding the weight decay term to the cost, you should regularize only the softmax weights (do not regularize the weights that compute the hidden layer activations).
Implementation Tip: It is always a good idea to implement the code modularly and check (the gradient of) each part of the code before writing the more complicated parts.
Step 5: Test the model
Finally, you will need to classify with this model; complete the code in stackedAEPredict.m to classify using the stacked autoencoder with a classification layer.
After completing these steps, running the entire script in stackedAETrain.m will perform layer-wise training of the stacked autoencoder, finetune the model, and measure its performance on the test set. If you've done all the steps correctly, you should get an accuracy of about 87.7% before finetuning and 97.6% after finetuning (for the 10-way classification problem).
from: http://ufldl.stanford.edu/wiki/index.php/Exercise:_Implement_deep_networks_for_digit_classification
- Stanford UFLDL教程 Exercise: Implement deep networks for digit classification
- UFLDL教程: Exercise: Implement deep networks for digit classification
- UFLDL Exercise: Implement deep networks for digit classification
- UFLDL Exercise:Implement deep networks for digit classification
- UFLDL教程答案(6):Exercise:Implement deep networks for digit classification
- UFLDL教程Exercise答案(6):Implement deep networks for digit classification
- UFLDL Exercise: Implement deep networks for digit classificationz
- Exercise: Implement deep networks for digit classification 代码示例
- Convolutional neural networks(CNN) (九) Implement deep networks for digit classification Exercise
- UFLDL Tutorial_Building Deep Networks for Classification
- UFLDL学习笔记5(Building Deep Networks for Classification)
- Stanford UFLDL教程 Exercise:Vectorization
- Stanford UFLDL教程 Exercise:Sparse Autoencoder
- Stanford UFLDL教程 Exercise:PCA and Whitening
- Stanford UFLDL教程 Exercise:Softmax Regression
- Stanford UFLDL教程 Exercise:Self-Taught Learning
- Stanford UFLDL教程 Exercise:Convolution and Pooling
- Stanford UFLDL教程 Exercise:Sparse Coding
- 第15周实践项目-直接插入排序
- Stanford UFLDL教程 栈式自编码算法
- android 屏幕适配核心概念与单位笔记
- Stanford UFLDL教程 微调多层自编码算法
- swift注意点
- Stanford UFLDL教程 Exercise: Implement deep networks for digit classification
- Spring中jdbcTemplate
- Stanford UFLDL教程 线性解码器
- Android语音识别——谷歌语音识别与百度语音识别
- Stanford UFLDL教程 Exercise:Learning color features with Sparse Autoencoders
- zoc7 mac下中文乱码
- maven 配置和常用命令
- Stanford UFLDL教程 卷积特征提取
- css选择器特异性计算