Principles of training multi-layer neural network using backpropagation
来源:互联网 发布:java 命令行参数 d 编辑:程序博客网 时间:2024/05/19 03:46
The project describes teaching process of multi-layer neural network employing backpropagation algorithm. To illustrate this process the three layer neural network with two inputs and one output,which is shown in the picture below, is used:
Each neuron is composed of two units. First unit adds products of weights coefficients and input signals. The second unit realise nonlinear function, called neuron activation function. Signal e is adder output signal, and y = f(e) is output signal of nonlinear element. Signal y is also output signal of neuron.
To teach the neural network we need training data set. The training data set consists of input signals (x1 and x2 ) assigned with corresponding target (desired output) z. The network training is an iterative process. In each iteration weights coefficients of nodes are modified using new data from training data set. Modification is calculated using algorithm described below: Each teaching step starts with forcing both input signals from training set. After this stage we can determine output signals values for each neuron in each network layer. Pictures below illustrate how signal is propagating through the network, Symbols w(xm)n represent weights of connections between network input xm and neuron n in input layer. Symbols yn represents output signal of neuron n.
Propagation of signals through the hidden layer. Symbols wmn represent weights of connections between output of neuron m and input of neuron n in the next layer.
Propagation of signals through the output layer.
In the next algorithm step the output signal of the network y is compared with the desired output value (the target), which is found in training data set. The difference is called error signal d of output layer neuron.
It is impossible to compute error signal for internal neurons directly, because output values of these neurons are unknown. For many years the effective method for training multiplayer networks has been unknown. Only in the middle eighties the backpropagation algorithm has been worked out. The idea is to propagate error signal d (computed in single teaching step) back to all neurons, which output signals were input for discussed neuron.
The weights' coefficients wmn used to propagate errors back are equal to this used during computing output value. Only the direction of data flow is changed (signals are propagated from output to inputs one after the other). This technique is used for all network layers. If propagated errors came from few neurons they are added. The illustration is below:
When the error signal for each neuron is computed, the weights coefficients of each neuron input node may be modified. In formulas below df(e)/derepresents derivative of neuron activation function (which weights are modified).
Coefficient h affects network teaching speed. There are a few techniques to select this parameter. The first method is to start teaching process with large value of the parameter. While weights coefficients are being established the parameter is being decreased gradually. The second, more complicated, method starts teaching with small parameter value. During the teaching process the parameter is being increased when the teaching is advanced and then decreased again in the final stage. Starting teaching process with low parameter value enables to determine weights coefficients signs.
0 0
- Principles of training multi-layer neural network using backpropagation
- Principles of training multi-layer neural network using backpropagation
- Principles of training multi-layer neural network using backpropagation
- Multi-Layer Neural Network
- Backpropagation neural network
- UFLDL学习笔记4——Multi-Layer Neural Network
- UFLDL阅读笔记1—Multi-Layer Neural Network
- Neural Network :Backpropagation的实现
- CS231N-Lecture4 Backpropagation&Neural Network
- Fast convolutional neural network training using selective data sampling 笔记
- Principles of Network Applications
- ufldl学习笔记与编程作业:Multi-Layer Neural Network(多层神经网络+识别手写体编程)
- CS231N-Lecture5 Training Neural Network
- 多任务学习“Rotating Your Face Using Multi-task Deep Neural Network”
- Fast convolutional neural network training using selective data sampling: Application to hemorrhage
- Fast convolutional neural network training using selective data sampling: Application to hemorrhage
- <神经科学原理>Principles of Neural Science
- Coursera机器学习-第五周-Neural Network BackPropagation
- java基础——抽象类
- jquery width,innerWidth,outterWidth 及事件操作
- VB…………………………
- 杂七杂八
- lintcode ----等价二叉树
- Principles of training multi-layer neural network using backpropagation
- 【图像处理】线性、位置不变退化图像的频域复原基础(维纳滤波,最小均方滤波,几何滤波)
- Linux系统位数查看方法
- Unity3D之VR开发笔记@Oculus开发UNITY环境搭建
- java 通过jdbc连接oracle集群
- 着色器文件中的 Properties 定义
- leetcode 10 Regular Expression Matching
- 电脑天梯图
- UML