deeplearning----学习一个简单的分类器



1、零一损失

---

 我们的目的就是让错误次数（零一损失）尽可能的少：

f（x）会得出在当前的theata条件下输入对应的最大概率的输出值。换言之，我们从x预测出f(x),如果这个值就是y，那么预测成功，反之失败。

# zero_one_loss is a Theano variable representing a symbolic# expression of the zero one loss ; to get the actual value this# symbolic expression has to be compiled into a Theano function (see# the Theano tutorial for more details)zero_one_loss = T.sum(T.neq(T.argmax(p_y_given_x), y))#neq是I函数，T.neq(x,y)判断两个值是否不相等，not equal？

2、负对数自然损失

---

 

 由于0-1损失是不可微的,在大型模型中去优化它相当耗费资源,因此我们最大化它的对数似然函数来完成（似然就是可能性）:

也就是最小化负对数似然损失

 

 负对数似然函数：negative log-likelihood (NLL)

# NLL is a symbolic variable ; to get the actual value of NLL, this symbolic# expression has to be compiled into a Theano function (see the Theano# tutorial for more details)NLL = -T.sum(T.log(p_y_given_x)[T.arange(y.shape[0]), y])# note on syntax: T.arange(y.shape[0]) is a vector of integers [0,1,2,...,len(y)].# Indexing a matrix M by the two vectors [0,1,...,K], [a,b,...,k] returns the# elements M[0,a], M[1,b], ..., M[K,k] as a vector.  Here, we use this# syntax to retrieve the log-probability of the correct labels, y.

3、随机梯度下降SGD(Stochastic Gradient Descent)

--------------------------------------------------------------------------------

# GRADIENT DESCENT

while True:
loss = f(params)
d_loss_wrt_params = ... # compute gradient
params -= learning_rate * d_loss_wrt_params
if <stopping condition is met>:
return params



上面是一般梯度下降，基本思路是：损失--》梯度--》参数更新

随机梯度下降是一次选几个样本进行训练。最简单的方式是一次一个：

随机梯度下降SGD(Stochastic Gradient Descent)

---

# STOCHASTIC GRADIENT DESCENT
for (x_i,y_i) in training_set:
                            # imagine an infinite generator
                            # that may repeat examples (if there is only a finite training set)
    loss = f(params, x_i, y_i)
    d_loss_wrt_params = ... # compute gradient
    params -= learning_rate * d_loss_wrt_params
    if <stopping condition is met>:
        return params

4、Minibatch SGD 除了一次使用多个样本，其他和sgd都一样

or (x_batch,y_batch) in train_batches:
                            # imagine an infinite generator
                            # that may repeat examples
    loss = f(params, x_batch, y_batch)
    d_loss_wrt_params = ... # compute gradient using theano
    params -= learning_rate * d_loss_wrt_params
    if <stopping condition is met>:
        return params