(已翻译)caffe官方例程之hdf5_classification(Brewing Logistic Regression then Going Deeper)
来源:互联网 发布:智能语音翻译软件 编辑:程序博客网 时间:2024/06/06 02:22
Brewing Logistic Regression then Going Deeper
While Caffe is made for deep networks it can likewise represent "shallow" models like logistic regression for classification. We'll do simple logistic regression on synthetic data that we'll generate and save to HDF5 to feed vectors to Caffe. Once that model is done, we'll add layers to improve accuracy. That's what Caffe is about: define a model, experiment, and then deploy.
虽然Caffe是用于深度网络,但它同样可以表示“浅”模型,如分类的逻辑回归。 我们将对我们将生成的合成数据进行简单的逻辑回归,并保存到HDF5格式的数据集中,用以输入到Caffe中。 一旦该模型完成,我们将添加层以提高准确性。 这就是Caffe的意义:定义一个模型,实验,然后部署。
import numpy as npimport matplotlib.pyplot as plt%matplotlib inlineimport osos.chdir('..')import syssys.path.insert(0, './python')import caffeimport osimport h5pyimport shutilimport tempfileimport sklearnimport sklearn.datasetsimport sklearn.linear_modelimport pandas as pd
Synthesize a dataset of 10,000 4-vectors for binary classification with 2 informative features and 2 noise features.
创建一个10000*4的数据,用于二元分类,具有2个信息特征和2个噪声特征。
X, y = sklearn.datasets.make_classification( n_samples=10000, n_features=4, n_redundant=0, n_informative=2, n_clusters_per_class=2, hypercube=False, random_state=0)# Split into train and testX, Xt, y, yt = sklearn.cross_validation.train_test_split(X, y)# Visualize sample of the dataind = np.random.permutation(X.shape[0])[:1000]df = pd.DataFrame(X[ind])_ = pd.scatter_matrix(df, figsize=(9, 9), diagonal='kde', marker='o', s=40, alpha=.4, c=y[ind])
Learn and evaluate scikit-learn's logistic regression with stochastic gradient descent (SGD) training. Time and check the classifier's accuracy.
使用随机梯度下降(SGD)训练来学习和评估scikit-learn逻辑回归,并检查分类器的准确性。
%%timeit# Train and test the scikit-learn SGD logistic regression.clf = sklearn.linear_model.SGDClassifier( loss='log', n_iter=1000, penalty='l2', alpha=5e-4, class_weight='auto')clf.fit(X, y)yt_pred = clf.predict(Xt)print('Accuracy: {:.3f}'.format(sklearn.metrics.accuracy_score(yt, yt_pred)))
Save the dataset to HDF5 for loading in Caffe.
将数据集保存到HDF5文件中,以加载到Caffe中。
# Write out the data to HDF5 files in a temp directory.# This file is assumed to be caffe_root/examples/hdf5_classification.ipynbdirname = os.path.abspath('./examples/hdf5_classification/data')if not os.path.exists(dirname): os.makedirs(dirname)train_filename = os.path.join(dirname, 'train.h5')test_filename = os.path.join(dirname, 'test.h5')# HDF5DataLayer source should be a file containing a list of HDF5 filenames.# To show this off, we'll list the same data file twice.with h5py.File(train_filename, 'w') as f: f['data'] = X f['label'] = y.astype(np.float32)with open(os.path.join(dirname, 'train.txt'), 'w') as f: f.write(train_filename + '\n') f.write(train_filename + '\n') # HDF5 is pretty efficient, but can be further compressed.comp_kwargs = {'compression': 'gzip', 'compression_opts': 1}with h5py.File(test_filename, 'w') as f: f.create_dataset('data', data=Xt, **comp_kwargs) f.create_dataset('label', data=yt.astype(np.float32), **comp_kwargs)with open(os.path.join(dirname, 'test.txt'), 'w') as f: f.write(test_filename + '\n')
Let's define logistic regression in Caffe through Python net specification. This is a quick and natural way to define nets that sidesteps manually editing the protobuf model.
让我们通过Python网络规范在Caffe中定义逻辑回归。 这是一个快速和自然的方式来定义网络,旁边手动编辑protobuf模型。
from caffe import layers as Lfrom caffe import params as Pdef logreg(hdf5, batch_size): # logistic regression: data, matrix multiplication, and 2-class softmax loss n = caffe.NetSpec() n.data, n.label = L.HDF5Data(batch_size=batch_size, source=hdf5, ntop=2) n.ip1 = L.InnerProduct(n.data, num_output=2, weight_filler=dict(type='xavier')) n.accuracy = L.Accuracy(n.ip1, n.label) n.loss = L.SoftmaxWithLoss(n.ip1, n.label) return n.to_proto()train_net_path = 'examples/hdf5_classification/logreg_auto_train.prototxt'with open(train_net_path, 'w') as f: f.write(str(logreg('examples/hdf5_classification/data/train.txt', 10)))test_net_path = 'examples/hdf5_classification/logreg_auto_test.prototxt'with open(test_net_path, 'w') as f: f.write(str(logreg('examples/hdf5_classification/data/test.txt', 10)))
Now, we'll define our "solver" which trains the network by specifying the locations of the train and test nets we defined above, as well as setting values for various parameters used for learning, display, and "snapshotting".
现在,我们将定义我们的“solver”,通过指定上面定义的训练和测试网络的位置用于学习,显示和“快照”的各种参数的值来训练网络。
from caffe.proto import caffe_pb2def solver(train_net_path, test_net_path): s = caffe_pb2.SolverParameter() # Specify locations of the train and test networks. s.train_net = train_net_path s.test_net.append(test_net_path) s.test_interval = 1000 # Test after every 1000 training iterations. s.test_iter.append(250) # Test 250 "batches" each time we test. s.max_iter = 10000 # # of times to update the net (training iterations) # Set the initial learning rate for stochastic gradient descent (SGD). s.base_lr = 0.01 # Set `lr_policy` to define how the learning rate changes during training. # Here, we 'step' the learning rate by multiplying it by a factor `gamma` # every `stepsize` iterations. s.lr_policy = 'step' s.gamma = 0.1 s.stepsize = 5000 # Set other optimization parameters. Setting a non-zero `momentum` takes a # weighted average of the current gradient and previous gradients to make # learning more stable. L2 weight decay regularizes learning, to help prevent # the model from overfitting. s.momentum = 0.9 s.weight_decay = 5e-4 # Display the current training loss and accuracy every 1000 iterations. s.display = 1000 # Snapshots are files used to store networks we've trained. Here, we'll # snapshot every 10K iterations -- just once at the end of training. # For larger networks that take longer to train, you may want to set # snapshot < max_iter to save the network and training state to disk during # optimization, preventing disaster in case of machine crashes, etc. s.snapshot = 10000 s.snapshot_prefix = 'examples/hdf5_classification/data/train' # We'll train on the CPU for fair benchmarking against scikit-learn. # Changing to GPU should result in much faster training! s.solver_mode = caffe_pb2.SolverParameter.CPU return ssolver_path = 'examples/hdf5_classification/logreg_solver.prototxt'with open(solver_path, 'w') as f: f.write(str(solver(train_net_path, test_net_path)))
Time to learn and evaluate our Caffeinated logistic regression in Python.
现在应该是时候去学习和评估我们的Caffeinated逻辑回归,在Python中。
%%timeitcaffe.set_mode_cpu()solver = caffe.get_solver(solver_path)solver.solve()accuracy = 0batch_size = solver.test_nets[0].blobs['data'].numtest_iters = int(len(Xt) / batch_size)for i in range(test_iters): solver.test_nets[0].forward() accuracy += solver.test_nets[0].blobs['accuracy'].dataaccuracy /= test_itersprint("Accuracy: {:.3f}".format(accuracy))
Do the same through the command line interface for detailed output on the model and solving.
通过命令行界面做同样的操作,以获得模型和求解的详细输出。
!./build/tools/caffe train -solver examples/hdf5_classification/logreg_solver.prototxt
If you look at output or the logreg_auto_train.prototxt
, you'll see that the model is simple logistic regression. We can make it a little more advanced by introducing a non-linearity between weights that take the input and weights that give the output -- now we have a two-layer network. That network is given in nonlinear_auto_train.prototxt
, and that's the only change made in nonlinear_logreg_solver.prototxt
which we will now use.
The final accuracy of the new network should be higher than logistic regression!
如果你看输出或logreg_auto_train.prototxt,你会看到模型是简单的逻辑回归。 我们可以通过引入在接受输入的权重和给出输出的权重之间的非线性使它更高级 - 现在我们有一个两层网络。 那个网络是在nonlinear_auto_train.prototxt中给出的,这是我们现在使用的在nonlinear_logreg_solver.prototxt中做的唯一改变。
新网络的最终精度应该高于逻辑回归!
from caffe import layers as Lfrom caffe import params as Pdef nonlinear_net(hdf5, batch_size): # one small nonlinearity, one leap for model kind n = caffe.NetSpec() n.data, n.label = L.HDF5Data(batch_size=batch_size, source=hdf5, ntop=2) # define a hidden layer of dimension 40 n.ip1 = L.InnerProduct(n.data, num_output=40, weight_filler=dict(type='xavier')) # transform the output through the ReLU (rectified linear) non-linearity n.relu1 = L.ReLU(n.ip1, in_place=True) # score the (now non-linear) features n.ip2 = L.InnerProduct(n.ip1, num_output=2, weight_filler=dict(type='xavier')) # same accuracy and loss as before n.accuracy = L.Accuracy(n.ip2, n.label) n.loss = L.SoftmaxWithLoss(n.ip2, n.label) return n.to_proto()train_net_path = 'examples/hdf5_classification/nonlinear_auto_train.prototxt'with open(train_net_path, 'w') as f: f.write(str(nonlinear_net('examples/hdf5_classification/data/train.txt', 10)))test_net_path = 'examples/hdf5_classification/nonlinear_auto_test.prototxt'with open(test_net_path, 'w') as f: f.write(str(nonlinear_net('examples/hdf5_classification/data/test.txt', 10)))solver_path = 'examples/hdf5_classification/nonlinear_logreg_solver.prototxt'with open(solver_path, 'w') as f: f.write(str(solver(train_net_path, test_net_path)))
%%timeitcaffe.set_mode_cpu()solver = caffe.get_solver(solver_path)solver.solve()accuracy = 0batch_size = solver.test_nets[0].blobs['data'].numtest_iters = int(len(Xt) / batch_size)for i in range(test_iters): solver.test_nets[0].forward() accuracy += solver.test_nets[0].blobs['accuracy'].dataaccuracy /= test_itersprint("Accuracy: {:.3f}".format(accuracy))
Do the same through the command line interface for detailed output on the model and solving.
通过命令行界面做同样的操作,以获得模型和求解的详细输出。
!./build/tools/caffe train -solver examples/hdf5_classification/nonlinear_logreg_solver.prototxt
# Clean up (comment this out if you want to examine the hdf5_classification/data directory).shutil.rmtree(dirname)
- (已翻译)caffe官方例程之hdf5_classification(Brewing Logistic Regression then Going Deeper)
- caffe学习笔记9 -- Brewing Logistic Regression then Going Deeper
- Caffe 实例笔记 3 Brewing Logistic Regression then Going Deeper
- Going deeper with convolutions-GoogLeNet(阅读)
- Going deeper with convolutions-GoogLeNet(阅读)
- googlenet笔记(Going Deeper with Convolutions)
- Going deeper with convolutions-GoogLeNet(阅读)
- 机器学习(二)之Logistic Regression
- caffe之python接口实战 :brewing-logreg 官方教程源码解析
- Going Deeper into Regression Analysis with Assumptions, Plots & Solutions
- Caffe官方例程之 特征可视化python代码 (filter visualization notebook)
- caffe 官方例程之R-CNN
- going deeper with convolutions (Googlenet网络解读)
- CNN初探(三)------Going Deeper with Convolutions
- [深度学习] Going Deeper with Convolutions(GooLeNet)阅读笔记
- 论文笔记:Going deeper with convolutions(inception v1)
- Going deeper with convolutions----------(GoogLeNet) 论文解读
- 逻辑回归(logistic regression)
- ScrollView嵌套GridView,GridView显示不全
- a mirror of Android build ENV setup flow
- 去掉字符串中连续出现k个0的子串
- 电子书5
- SQL Server转换成MySQL
- (已翻译)caffe官方例程之hdf5_classification(Brewing Logistic Regression then Going Deeper)
- UVa 202 - Repeating Decimals(AC)
- Linux C高级编程——网络编程之包裹函数
- 俄罗斯方块游戏的控制面板和消息接收框界面的实现
- 跟我学AngularJs:AngularJs入门及第一个实例
- Tcl -- unset
- TCP三次握手与释放链接
- 《软件架构设计》学习笔记--8--6大步骤4:概念架构设计
- 关于集成支付宝SDK的开发