多层神经网络Multi-layer networks
来源:互联网 发布:瞩目视频会议软件 编辑:程序博客网 时间:2024/05/21 14:45
Multi-layer networks
A nonlinear problem
Consider again the best linear fit we found for the car data. Notice that the data points are not evenly distributed around the line: for low weights, we see more miles per gallon than our model predicts. In fact, it looks as if a simple curve might fit these data better than the straight line. We can enable our neural network to do such curve fitting by giving it an additional node which has a suitably curved (nonlinear) activation function. A useful function for this purpose is the S-shaped hyperbolic tangent (tanh) function (Fig. 1). (Fig. 1) (Fig. 2)FIg. 2 shows our new network: an extra node (unit 2) with tanh activation function has been inserted between input and output. Since such a node is "hidden" inside the network, it is commonly called a hidden unit. Note that the hidden unit also has a weight from the bias unit. In general, all non-input neural network units have such a bias weight. For simplicity, the bias unit and weights are usually omitted from neural network diagrams - unless it's explicitly stated otherwise, you should always assume that they are there.
When this network is trained by gradient descent on the car data, it learns to fit the tanh function to the data (Fig. 3). Each of the four weights in the network plays a particular role in this process: the two bias weights shift the tanh function in the x- and y-direction, respectively, while the other two weights scale it along those two directions. Fig. 2 gives the weight values that produced the solution shown in Fig. 3.
Hidden Layers
One can argue that in the example above we have cheated by picking a hidden unit activation function that could fit the data well. What would we do if the data looks like this (Fig. 4)?(Relative concentration of NO and NO2 in exhaust fumes as a function
of the richness of the ethanol/air mixture burned in a car engine.)
Obviously the tanh function can't fit this data at all. We could cook up a special activation function for each data set we encounter, but that would defeat our purpose of learning to model the data. We would like to have a general, non-linear function approximation method which would allow us to fit any given data set, no matter how it looks like.
Fortunately there is a very simple solution: add more hidden units! In fact, a network with just two hidden units using the tanh function (Fig. 5) can fit the dat in Fig. 4 quite well - can you see how? The fit can be further improved by adding yet more units to the hidden layer. Note, however, that having too large a hidden layer - or too many hidden layers - can degrade the network's performance (more on this later). In general, one shouldn't use more hidden units than necessary to solve a given problem. (One way to ensure this is to start training with a very small network. If gradient descent fails to find a satisfactory solution, grow the network by adding a hidden unit, and repeat.)
Theoretical results indicate that given enough hidden units, a network like the one in Fig. 5 can approximate any reasonable function to any required degree of accuracy. In other words, any function can be expressed as a linear combination of tanh functions: tanh is a universal basis function. Many functions form a universal basis; the two classes of activation functions commonly used in neural networks are the sigmoidal (S-shaped) basis functions (to which tanh belongs), and the radialbasis functions.
from: http://www.willamette.edu/~gorr/classes/cs449/multilayer.html
- 多层神经网络Multi-layer networks
- ufldl学习笔记与编程作业:Multi-Layer Neural Network(多层神经网络+识别手写体编程)
- Theano Multi Layer Perceptron 多层感知机
- 多层感知机:Multi-Layer Perceptron
- 多层感知机(Multi-Layer Perception)
- 多层感知机及其BP算法(Multi-Layer Perception)
- 多层感知机 Multi-Layer Perception Tensorflow实现
- Tensorflow实现多层感知机Multi-layer Preceptron
- Multi-linear classification(MLP)多层神经网络基础原理
- 多层神经网络
- 多层神经网络
- 多层神经网络
- 多层神经网络
- 多层感知机(multi-layer perceptron)实现手写体分类(TensorFlow)
- 神经网络简介-多层神经网络
- 多层神经网络和BP反向传播算法Multilayer Networks and Backpropagation
- (三)Multi-class Classification and Neural Networks[多分类问题和神经网络]
- 多柱深度神经网络——Multi-column Deep Neural Networks for Image Classification
- 线性神经网络Linear Neural Networks
- MySQL 5.7 Distrib 5.7.8-rc 主从复制的简单配置,备忘
- 从一个表中查数据,插入另一个表
- [软件人生]关于认知,能力的思考——中国城市里的无知现象片段
- 随机数的获取
- 多层神经网络Multi-layer networks
- C++实现蛇形矩阵
- 通过构造方法为私有属性赋值
- 通过路径分隔符新建文件路径
- WCF基础
- xmemcached简单使用
- SQL Server触发器创建、删除、修改、查看
- 神经网络BP反向传播算法Error Backpropagation
- 坎巴拉太空计划MOD:kRPC0.2.2中文文档