COURSE 1 Neural Networks and Deep Learning
来源:互联网 发布:心理学与生活 知乎 编辑:程序博客网 时间:2024/06/06 17:28
Week1
What is neural network?
It is a powerful learning algorithm inspired by how the brain works.
Example 1 - single neural network
Given data about the size of houses on the real estate market and you want to fit a function that will
predict their price. It is a linear regression problem because the price as a function of size is a continuous
output.
We know the prices can never be negative so we are creating a function called Rectified Linear Unit (ReLU)
which starts at zero.
The input is the size of the house (x)
The output is the price (y)
The “neuron” implements the function ReLU (blue line)
Example 2 – Multiple neural network
The price of a house can be affected by other features such as size, number of bedrooms, zip code and
wealth. The role of the neural network is to predicted the price and it will automatically generate the
hidden units. We only need to give the inputs x and the output y.
Supervised learning for Neural Network
In supervised learning, we are given a data set and already know what our correct output should look like,
having the idea that there is a relationship between the input and the output.
Supervised learning problems are categorized into “regression” and “classification” problems. In a
regression problem, we are trying to predict results within a continuous output, meaning that we are
trying to map input variables to some continuous function. In a classification problem, we are instead
trying to predict results in a discrete output. In other words, we are trying to map input variables into
discrete categories.
There are different types of neural network, for example Convolution Neural Network (CNN) used often
for image application and Recurrent Neural Network (RNN) used for one-dimensional sequence data
such as translating English to Chinses or a temporal component such as text transcript. As for the
autonomous driving, it is a hybrid neural network architecture.
Neural Network examples
Structured vs unstructured data
Structured data refers to things that has a defined meaning such as price, age whereas unstructured
data refers to thing like pixel, raw audio, text.
Why is deep learning taking off?
Deep learning is taking off due to a large amount of data available through the digitization of the society, faster computation and innovation in the development of neural network algorithm.
Two things have to be considered to get to the high level of performance:
- Being able to train a big enough neural network
- Huge amount of labeled data
The process of training a neural network is iterative.
It could take a good amount of time to train a neural network, which affects your productivity. Faster computation helps to iterate and improve new algorithm.
Week2
Binary Classification
In a binary classification problem, the result is a discrete value output
Notation
a training example:
(x,y),x∈ℝnx,y∈{0,1} m training examples:
{(x(1),y(1)),(x(2),y(2)),...,(x(m),y(m))}m=mtrain=# of train examples matrix:
X=[x(1),x(2),...,x(m)]∈ℝnx×mY=[y(1),y(2),...,y(m)]∈ℝ1×m goal:
Given x,ŷ =P(y=1|x),where 0≤ŷ
Logistic Regression
parameters
The input features vector:
x∈ℝnx,where nx is the number of features The training label:
y∈{0,1} The weights:
w∈ℝnX,where nx is the number of features The threshold:
b∈ℝ The output:
ŷ =σ(wTx+b) Sigmoid function:
s=σ(wtx+b)=σ(z)=11+e−z
Loss (error) function:
Cost function:
Gradient Descent
Want to find w and b that minimize J(w, b)
Process
Repeat
Logistic Regression Gradient Descent
Recap
Gradient Descent
Process
Gradient Descent on m examples
Recap
Descent
Pseudocode
Vectorization
Logistic Regression Derivatives
Vectorizing Logistic Regression
Implementing Logistic Regression
Broadcasting in Python
General Principle
Week3
Neural Networks Overview
Neural Network Representation
Computing a Neural Network’s Output
Vectorizing across multiple examples
Activation functions
Why do you need non-linear activation functions
Suppose
Then
It is similar to
If you were to use linear activation functions or we go to call them identity activation functions, then the new network is just outputting a linear function of the input and we’ll talk about deep networks later new networks with many many layers, many many hidden layers and it turns out that if you use a linear activation function or alternatively if you don’t have an activation function. Then no matter how many layers, your neural network has always doing is just computing a linear activation function.
Gradient Descent for Neural Networks
Backpropogation
Random Initialization
If initializing weights to zeros, then all weights will update symmetricly. Then no matter how many nodes in one layer, your neural network has always doing is just using one node in one layer.
Week4
Building Blocks of Deep Neural Networks
Propagation
Forward Propagation for Layer l
Input
Cache
Output
Vectorized
Input
Cache
Output
Backward Propagation for Layer l
Input
Local
Output
Vectorized
Input
Local
Output
Parameters vs Hyperparameters
Parameters
Hyperparameters
Hyperparameters can control W and b
- COURSE 1 Neural Networks and Deep Learning
- Neural Networks And Deep Learning(1)
- Neural Networks and Deep Learning
- Neural Networks and Deep Learning
- Neural networks and Deep Learning
- Neural Networks and Deep Learning
- neural networks and deep learning(Michael Nielsen)笔记(1)
- 《Neural networks and deep learning》概览
- 《Neural networks and deep learning》概览
- 《Neural networks and deep learning》概览
- Neural networks and deep learning materials
- neural-networks-and-deep-learning mnist-loader
- neural-networks-and-deep-learning mnist_pca.py
- neural-networks-and-deep-learning expand_mnist.py
- neural-networks-and-deep-learning mnist_average_darkness.py
- neural-networks-and-deep-learning mnist_svm.py
- neural-networks-and-deep-learning network.py
- neural-networks-and-deep-learning network2.py
- 【Scikit-Learn 中文文档】多类和多标签算法
- Android开发:计算缓存大小并且清空缓存
- 网络工程师必备知识(未写完)
- 分布式事务一些总结与思考
- JDBK连接数据库操作
- COURSE 1 Neural Networks and Deep Learning
- Azkaban-开源任务调度程序(使用篇)
- 欢迎使用CSDN-markdown编辑器
- 开机报 BOOT BCD ,并无法进系统
- IRLS 迭代加权最小二乘
- 12.call和apply的用法
- 解决{dede:field.body/}不解析
- Android实时取景:用SurfaceView实现 -- Camera
- RedHat Enterprise Linux 6 使用 Centos 6 的yum