Self-Taught Learning
来源:互联网 发布:航测无人机软件 编辑:程序博客网 时间:2024/06/10 19:01
自编码器是一个三层的feed-forward神经网络模型,输入层经过隐含层的特征表示后再重构出跟输入层逼近的输出层,中间的隐含层是特征表示层,表示对输入层学习到的特征,这些特征可能更好地表示了数据,如果用学到的特征来训练数据分类或回归可能学习效果更好,于是就有了自我学习和无监督特征学习。
如果我们有很多的未标注数据,那就更好了,我们可以用自编码器学习特征表示,然后用学到的特征表示对已标注数据提取特征,再用机器学习算法比如softmax regression进行训练、预测,即先经过无监督的特征学习,然后再经过有监督的学习。未标注数据与已标注数据来自同一分布时就是半监督学习,来自不同分布就是无监督学习,比如我们的目标是要区分摩托车和汽车,如果未标注数据也是摩托车或汽车,那么这个问题就是半监督学习,如果不是则是自我学习。
自编码的网络结构如下:
通过自编码器得到特征表示的模型参数W1和b1,我们就可以用W1和b1对已标注数据进行特征提取,即算出它们的激活值。
实验数据也是MNIST数据集,这次把5-9类的数据作为无标注数据学习特征表示,然后在0-4类的数据中分为训练集和测试集来运行模型,实验结果的预测准确率为98.32%,而直接用图像像素作为输入得到准确率为96.74%。
%% CS294A/CS294W Self-taught Learning Exercise% Instructions% ------------% % This file contains code that helps you get started on the% self-taught learning. You will need to complete code in feedForwardAutoencoder.m% You will also need to have implemented sparseAutoencoderCost.m and % softmaxCost.m from previous exercises.%%% ======================================================================% STEP 0: Here we provide the relevant parameters values that will% allow your sparse autoencoder to get good filters; you do not need to % change the parameters below.inputSize = 28 * 28;numLabels = 5;hiddenSize = 200;sparsityParam = 0.1; % desired average activation of the hidden units. % (This was denoted by the Greek alphabet rho, which looks like a lower-case "p", % in the lecture notes). lambda = 3e-3; % weight decay parameter beta = 3; % weight of sparsity penalty term maxIter = 400;%% ======================================================================% STEP 1: Load data from the MNIST database%% This loads our training and test data from the MNIST database files.% We have sorted the data for you in this so that you will not have to% change it.% Load MNIST database filesmnistData = loadMNISTImages('mnist/train-images-idx3-ubyte');mnistLabels = loadMNISTLabels('mnist/train-labels-idx1-ubyte');% Set Unlabeled Set (All Images)% Simulate a Labeled and Unlabeled setlabeledSet = find(mnistLabels >= 0 & mnistLabels <= 4);unlabeledSet = find(mnistLabels >= 5); %5-9类作为无标签数据集用来学习特征表示%已标注数据分一半分别用于训练softmax和测试numTrain = round(numel(labeledSet)/2);trainSet = labeledSet(1:numTrain); testSet = labeledSet(numTrain+1:end);unlabeledData = mnistData(:, unlabeledSet);trainData = mnistData(:, trainSet);trainLabels = mnistLabels(trainSet)' + 1; % Shift Labels to the Range 1-5testData = mnistData(:, testSet);testLabels = mnistLabels(testSet)' + 1; % Shift Labels to the Range 1-5% Output Some Statisticsfprintf('# examples in unlabeled set: %d\n', size(unlabeledData, 2));fprintf('# examples in supervised training set: %d\n\n', size(trainData, 2));fprintf('# examples in supervised testing set: %d\n\n', size(testData, 2));%% ======================================================================% STEP 2: Train the sparse autoencoder% This trains the sparse autoencoder on the unlabeled training% images. % Randomly initialize the parameterstheta = initializeParameters(hiddenSize, inputSize);%% ----------------- YOUR CODE HERE ----------------------% Find opttheta by running the sparse autoencoder on% unlabeledTrainingImagesopttheta = theta; %用minFunc里的L-BFGS算法训练sparse autoencoder的模型,要用到sparse autoencoder的计算损失的代码addpath minFunc/options.Method = 'lbfgs';options.maxIter = 400;options.display = 'on';[opttheta, cost] = minFunc( @(p) sparseAutoencoderCost(p, ... inputSize, hiddenSize, ... lambda, sparsityParam, ... beta, unlabeledData), ... theta, options);%% ----------------------------------------------------- % Visualize weightsW1 = reshape(opttheta(1:hiddenSize * inputSize), hiddenSize, inputSize);display_network(W1');%%======================================================================%% STEP 3: Extract Features from the Supervised Dataset% % You need to complete the code in feedForwardAutoencoder.m so that the % following command will extract features from the data.trainFeatures = feedForwardAutoencoder(opttheta, hiddenSize, inputSize, ... trainData);testFeatures = feedForwardAutoencoder(opttheta, hiddenSize, inputSize, ... testData);%%======================================================================%% STEP 4: Train the softmax classifiersoftmaxModel = struct; %% ----------------- YOUR CODE HERE ----------------------% Use softmaxTrain.m from the previous exercise to train a multi-class% classifier. % Use lambda = 1e-4 for the weight regularization for softmax% You need to compute softmaxModel using softmaxTrain on trainFeatures and% trainLabels%softmax训练过程options.maxIter = 100;lambda = 1e-4;inputSize = hiddenSize;softmaxModel = softmaxTrain(inputSize, 5, lambda, ... trainFeatures, trainLabels, options);%% -----------------------------------------------------%%======================================================================%% STEP 5: Testing %% ----------------- YOUR CODE HERE ----------------------% Compute Predictions on the test set (testFeatures) using softmaxPredict% and softmaxModel%用到softmax练习中的预测函数[pred] = softmaxPredict(softmaxModel, testFeatures);acc = mean(pred(:) == testLabels(:));fprintf('Accuracy: %0.3f%%\n', acc*100);%% -----------------------------------------------------% Classification Scorefprintf('Test Accuracy: %f%%\n', 100*mean(pred(:) == testLabels(:)));% (note that we shift the labels by 1, so that digit 0 now corresponds to% label 1)%% Accuracy is the proportion of correctly classified images% The results for our implementation was:%% Accuracy: 98.3%%%
参考:
http://ufldl.stanford.edu/wiki/index.php/Self-Taught_Learning_to_Deep_Networks
0 0
- Self-Taught Learning
- Deep learning----------Self-taught learning
- UFLFL Exercise:Self-Taught Learning
- UFLDL Exercise:Self-Taught Learning
- UFLDL Exercise:Self-Taught Learning
- 【UFLDL-exercise6- Self-Taught Learning】
- UFLDL Exercise: Self-Taught Learning
- Ufldl Exercise:Self-Taught Learning
- 【Deep Learning】4、Self-Taught Learning
- Deep learning:十五(Self-Taught Learning练习)
- Deep learning:十五(Self-Taught Learning练习)
- Self-Taught Learning To Deep Networds
- Self-taught Learning: TransferLearning from Unlabelled Data
- 自我学习(Self-Taught Learning)
- Exercise:Self-Taught Learning 代码示例
- Deep Learning by Andrew Ng --- self-taught
- Stanford UFLDL教程 Exercise:Self-Taught Learning
- UFLDL教程: Exercise:Self-Taught Learning
- 设计模式-单例模式(Singleton)
- bzoj1007
- iOS开发中@property的属性介绍
- 设计模式-简单工厂模式(Simple Factory)
- Hadoop学习之 资料集合
- Self-Taught Learning
- 设计模式-工厂方法模式(factory method)
- 设计模式-抽象工厂模式(Abstract Factory)
- [UG软件安装] 8.5安装后显示英文的解决方法
- 设计模式-原型模式(Prototype)
- MSP430 SPI硬件接口
- 公司终于“又”开始招人了。
- C中如何调用C++函数
- cocos2d-x-2.2.2项目添加到vs