caffe 速览笔记
来源:互联网 发布:java中嵌套if 编辑:程序博客网 时间:2024/06/05 02:45
Caffe Tutorial
Caffe 模型的最基本要素: Blob, Layers, Nets
Blob 传送数据的基本单元
Blob与tensorflow中的tensor类似,都是用来在节点之间传递数据所用,接口也类似,常规的dimension是 数据量N x 通道K x 高度H x 宽度W
Blob内部存有data和diff两大块。前者是传递的数据,后者是计算的梯度。
由于blob同时存在CPU和GPU,所以有两种方式来访问他们:
- const (don`t change the values)
- mutable (changes the values)
const Dtype* cpu_data() const;Dtype* mutable_cpu_data();
要使用合适的方式来访问blob变量,主要是因为blob具有cpu和gpu同步功能,会进行cpu和gpu的通信。所以如果不想更改值的话,最好使用const来调用,以减小通信开销。并且blob的指针不要存在自己的对象中,而使用函数来获取指针,因为syncedmem需要函数的调用来确定什么时候复制数据。
cpu 和 gpu 之间的转换?
Layer 计算与连接的单元
Layer是模型的本质与计算的基本单元。包括卷积、池化、内积、sigmoid等。
每个Layer包括了三个重要的计算:设置、前向传输、后向传输(setup, forward, backward)
- Setup: 初始化layer与连接
- Forward: 根据来自底部的数据进行计算,传到顶部
- Backward:根据顶部传来的梯度进行梯度计算,然后传到底部。具有参数的layer会将梯度进行存储应用
Net 定义和操作
Net是一个layer连接的组合,组成了一个有向无环图。
这样进行定义:
name: "LogReg"layer { name: "mnist" type: "Data" top: "data" top: "label" data_param { source: "input_leveldb" batch_size: 64 }}layer { name: "ip" type: "InnerProduct" bottom: "data" top: "ip" inner_product_param { num_output: 2 }}layer { name: "loss" type: "SoftmaxWithLoss" bottom: "ip" bottom: "label" top: "loss"}
Forward 与 Backward
Forward 与 Backward 的传输是Net的重要计算。
Forward计算:
Backward计算:
Loss
普通:
layer { name: "loss" type: "SoftmaxWithLoss" bottom: "pred" bottom: "label" top: "loss"}
loss weight:
layer { name: "loss" type: "SoftmaxWithLoss" bottom: "pred" bottom: "label" top: "loss" loss_weight: 1}
最后可根据所有loss进行加权相加计算:
loss := 0for layer in layers: for top, loss_weight in layer.tops, layer.loss_weights: loss += loss_weight * sum(top)
Slover
各种用于计算梯度与loss的方法。
//todo
Layers
数据层
可利用设置TransformationParameter来进行输入的预处理(mean subtraction, scaling, random cropping, and mirroring)
当TransformationParameter失效时,可利用 bias, scale, and crop层进行处理。
有以下几种data layers:
- Image Data - read raw images.
- Database - read data from LEVELDB or LMDB.
- HDF5 Input - read HDF5 data, allows data of arbitrary dimensions.
- HDF5 Output - write data as HDF5.
- Input - typically used for networks that are being deployed.
- Window Data - read window data file.
- Memory Data - read data directly from memory.
- Dummy Data - for static data and debugging.
Vison Layers
以images为输入,以images为输出。
有以下几种Vision Layers:
- 卷积层
- 池化层
- Spatial Pyramid Pooling (SPP)
- Crop 裁剪操作
- Deconvolution Layer 逆卷积层
Recurrent Layers
- Recurrent
- RNN
- Long-short Term Memory
Common Layers
- Inner Product
- Dropout
- Embed
Normalization Layers
- Local Response Normalization
- Mean Variance Normalization
- Batch Normalization
Activation / Neuron Layers
- ReLU / Rectified-Linear and Leaky-ReLU - ReLU and Leaky-ReLU rectification.
- PReLU - parametric ReLU.
- ELU - exponential linear rectification.
- Sigmoid
- TanH
- Absolute Value
- Power - f(x) = (shift + scale * x) ^ power.
- Exp - f(x) = base ^ (shift + scale * x).
- Log - f(x) = log(x).
- BNLL - f(x) = log(1 + exp(x)).
- Threshold - performs step function at user defined threshold.
- Bias - adds a bias to a blob that can either be learned or fixed.
- Scale - scales a blob by an amount that can either be learned or fixed.
Utility Layers
- Flatten
- Reshape
Batch Reindex
Split
- Concat
- Slicing
- Eltwise - element-wise operations such as product or sum between two blobs.
- Filter / Mask - mask or select output using last blob.
- Parameter - enable parameters to be shared between layers.
- Reduction - reduce input blob to scalar blob using operations such as sum or mean.
Silence - prevent top-level blobs from being printed during training.
ArgMax
Softmax
Python - allows custom Python layers.
Loss Layers
- Multinomial Logistic Loss
- Infogain Loss - a generalization of MultinomialLogisticLossLayer.
- Softmax with Loss - computes the multinomial logistic loss of the softmax of its inputs. It’s conceptually identical to a softmax layer followed by a multinomial logistic loss layer, but provides a more numerically stable gradient.
- Sum-of-Squares / Euclidean - computes the sum of squares of differences of its two inputs
- Hinge / Margin - The hinge loss layer computes a one-vs-all hinge (L1) or squared hinge loss (L2).
- Sigmoid Cross-Entropy Loss - computes the cross-entropy (logistic) loss, often used for predicting targets interpreted as probabilities.
- Accuracy / Top-k layer - scores the output as an accuracy with respect to target – it is not actually a loss and has no backward step.
- Contrastive Loss
Data inputs and outputs
in:
layer { name: "mnist" # Data layer loads leveldb or lmdb storage DBs for high-throughput. type: "Data" # the 1st top is the data itself: the name is only convention top: "data" # the 2nd top is the ground truth: the name is only convention top: "label" # the Data layer configuration data_param { # path to the DB source: "examples/mnist/mnist_train_lmdb" # type of DB: LEVELDB or LMDB (LMDB supports concurrent reads) backend: LMDB # batch processing improves efficiency. batch_size: 64 } # common data transformations transform_param { # feature scaling coefficient: this maps the [0, 255] MNIST data to [0, 1] scale: 0.00390625 }}
out:
layer { name: "data" type: "Data" [...] transform_param { scale: 0.1 mean_file_size: mean.binaryproto # for images in particular horizontal mirroring and random cropping # can be done as simple data augmentations. mirror: 1 # 1 = on, 0 = off # crop a `crop_size` x `crop_size` patch: # - at random during training # - from the center during testing crop_size: 227 }}
- caffe 速览笔记
- caffe笔记
- Caffe笔记
- 【caffe源码研究】caffe笔记
- caffe 初学笔记
- Caffe学习笔记
- caffe报告笔记
- Caffe+Win使用笔记
- Caffe安装笔记
- caffe 使用笔记
- caffe安装笔记
- Caffe学习笔记
- caffe学习笔记:mnist
- caffe学习笔记
- caffe cifar10 net笔记
- caffe学习笔记
- caffe convolution layer 笔记
- Caffe学习笔记
- 在 2016 年学 JavaScript 是一种什么样的体验?(转)
- BZOJ 1016: [JSOI2008]最小生成树计数 kruskal dfs
- 查询端口号占用
- JS绑定单选复选(模板替换可用)
- word2vec 笔记
- caffe 速览笔记
- Java中 <? super T>,<? extends T>,和<T>的区别
- MFC ODBC 多表查询 使用成员函数GetFieldValue()
- Tensorflow学习之旅(十一)——VGG网络与实践评测
- Linux测试题
- java获取登录的ip地址
- 百度编辑器上传图片至七牛云
- ffmpeg 从mp4上提取H264的nalu
- 互联网协议