Theano 模块 基础知识篇

基础知识：

符号变量：

>>> import numpy as np>>> import matplotlib.pyplot as plt>>> import theano>>> # 根据规定，tensor 子模块重名为T... import theano.tensor as T>>> # The theano.tensor 子模块拥有多种基本变量类型... # Here, we're defining a scalar标量 (0-d) variable.... # The argument参数 gives the variable its name.... foo = T.scalar('foo')>>> # Now, we can define another variable bar which is just foo squared平方.... bar = foo**2>>> # It will also be a theano variable.... print type(bar)<class 'theano.tensor.var.TensorVariable'>>>> print bar.typeTensorType(float64, scalar)>>> # Using theano's pp (pretty print)标准打印 function, we see that... # bar is defined symbolically as the square of foo... print theano.pp(bar)(foo ** TensorConstant{2})

函数：

1.

>>> #想进行与theano有关的计算，需要定义能够被实际的值调用并返回一个实际值得函数。... # 我们不能用theano的对象进行任何计算... # We need to define a theano function first.... # The first argument of theano.function defines the inputs to the function.... # Note that bar relies on foo, so foo is an input to this function.... # theano.function 将接收foo的值，编写代码来计算bar的值... f = theano.function([foo], bar)>>> print f(3)9.0>>>>>> # Alternatively或者, in some cases you can use a symbolic variable's符号变量的 eval求值 method.... # This can be more convenient than defining a function.... # The eval method takes a dictionary where the keys are theano variables and the values are values for those variables.eval方法需要传入一个keys是theano变量，values是变量的值的字典参数。... print bar.eval({foo: 3})9.0

2.

>>> # We can also use Python functions to construct Theano variables.... # It seems pedantic here, but can make syntax cleaner for more complicated examples.... def square(x):...     return x**2...>>> bar = square(foo)>>> print bar.eval({foo: 3})9.0

3.

>>> # We can also use Python functions to construct Theano variables.... # It seems pedantic迂腐 here, 但能够使复杂的实例的语法更加清晰.... def square(x):...     return x**2...>>> bar = square(foo)>>> print bar.eval({foo: 3})9.0

theano.tensor：

>>> #Theano also has variable types for vectors向量, matrices矩阵（复）, and tensors张量. The theano.tensor submodule has various functions for performing operations进行操作 on these variables.... A = T.matrix('A')>>> x = T.vector('x')>>> b = T.vector('b')>>> y = T.dot(A, x) + b #乘法 矩阵每行乘向量每行加向量>>> # Note that squaring a matrix矩阵（单） is element-wise元素方式... z = T.sum(A**2) #元素和>>> # theano.function can compute multiple things at a time第一个参数作为列表可传入多个theano对象... # You can also set default parameter values设置默认参数... linear_mix = theano.function([A, x, theano.Param(b, default=np.array([0, 0]))], [y, z])>>> # We'll cover讨论 theano.config.floatX later... print linear_mix(np.array([[1, 2, 3],...                            [4, 5, 6]], dtype=theano.config.floatX), #A...                  np.array([1, 2, 3], dtype=theano.config.floatX), #x...                  np.array([4, 5], dtype=theano.config.floatX)) #b[array([ 18.,  37.]), array(91.0)]>>> # Using the default value for b... print linear_mix(np.array([[1, 2, 3],...                            [4, 5, 6]]), #A...                  np.array([1, 2, 3])) #x[array([ 14.,  32.]), array(91.0)]

共享变量：

共享变量的特殊性在于，没有显式的值但能够通过set/get方法更新并被function共享。共享变量存在function调用过程中。
1.初始化

>>> shared_var = theano.shared(np.array([[1, 2], [3, 4]], dtype=theano.config.floatX))>>> # The type of the shared variable is deduced推导 from its initialization... print shared_var.type()<TensorType(float64, matrix)>

2.set/get

>>> # We can set the value of a shared variable using set_value... shared_var.set_value(np.array([[3, 4], [2, 1]], dtype=theano.config.floatX))>>> # ..and get it using get_value... print shared_var.get_value()[[ 3.  4.] [ 2.  1.]]

3.在function中隐藏

>>> shared_squared = shared_var**2>>> # The first argument of theano.function (inputs) tells Theano what the arguments to the compiled编译 function should be.... # Note that because shared_var is shared, it already has a value, so it doesn't need to be an input to the function.... # Therefore, Theano implicitly暗中 considers shared_var an input to a function using shared_squared and so we don't need... # to include it in the inputs argument of theano.function.... function_1 = theano.function([], shared_squared)>>> print function_1()[[  9.  16.] [  4.   1.]]

4.利用function来更新共享变量：

>>> #共享变量的值可以通过用function中的更新参数来更新。... subtract = T.matrix('subtract')#更新参数>>> # updates takes a dict where keys are shared variables and values are the new value the shared variable should take... # Here, updates will set shared_var = shared_var - subtract 更新... function_2 = theano.function([subtract], shared_var, updates={shared_var: shared_var - subtract})>>> print "shared_var before subtracting [[1, 1], [1, 1]] by using function_2:"shared_var before subtracting [[1, 1], [1, 1]] by using function_2:>>> print shared_var.get_value()[[ 3.  4.] [ 2.  1.]]>>> # Subtract [[1, 1], [1, 1]] from shared_var... function_2(np.array([[1, 1], [1, 1]]))array([[ 3.,  4.],       [ 2.,  1.]])>>> print "shared_var after calling function_2:"shared_var after calling function_2:>>> print shared_var.get_value()[[ 2.  3.] [ 1.  0.]]>>> # Note that this also changes the output of function_1, because shared_var is shared!... print "New output of function_1() (shared_var**2):"New output of function_1() (shared_var**2):>>> print function_1()[[ 4.  9.] [ 1.  0.]]

渐变：

Theano提供的另一个极大地便利是能够计算 gradients梯度.提供了函数能够快速的计算 derivative导数而不用进行实际的 derive推导。

>>> # Recall回忆起 that bar = foo**2... # We can compute the gradient of bar with respect to foo like so:... bar_grad = T.grad(bar, foo)#arg1 对 arg2 求导>>> # We expect that bar_grad = 2*foo... bar_grad.eval({foo: 10})array(20.0)>>>>>> # Recall that y = Ax + b... # We can also compute a Jacobian雅可比式（行列式/矩阵），导数行列式 like so:... y_J = theano.gradient.jacobian(y, x)>>> linear_mix_J = theano.function([A, x, b], y_J)F:\_python\lib\site-packages\theano\tensor\subtensor.py:110: FutureWarning: comparison to `None` will result in an elementwise object comparison in the future.  start in [None, 0] orF:\_python\lib\site-packages\theano\gof\cmodule.py:284: RuntimeWarning: numpy.ndarray size changed, may indicate binary incompatibility  rval = __import__(module_name, {}, {}, [module_name])>>> # Because it's a linear mix线性组合, we expect the output to always be A... print linear_mix_J(np.array([[9, 8, 7], [4, 5, 6]]), #A...                    np.array([1, 2, 3]), #x...                    np.array([4, 5])) #b[[ 9.  8.  7.] [ 4.  5.  6.]]>>> # We can also compute the Hessian黑塞矩阵 with theano.gradient.hessian (skipping that here)

雅可比矩阵:是一阶偏导数以一定方式排列成的矩阵，其行列式称为雅可比行列式。体现了一个可微方程与给出点的最优线性逼近。
黑塞矩阵：是一个多元函数的二阶偏导数构成的方阵，描述了函数的局部曲率。
楚天舒在豆瓣上对Jacobian 和 Hessian 的解释。
# 首先类比一下一维。Jacobian相当于一阶导数，Hessian相当于二阶导数。 一维函数的导数的motivation是很明显的。二阶导数的零点就是一阶导数的极值点。 对于很多应用，我们不仅关心一阶导数的零点（也就是函数的极值点），也关心一阶导数的极值点，比如信号处理中，信号的一阶导数的极值点反映信号变化的最剧烈程度。极值点寻求在编程时不方便，不如找二阶导数的零点。
# Jacobian对于标量函数f: Rn-> R1，实际是个向量，这个向量实际上就是函数的梯度gradient。gradient根据Cauchy-Swartz公式，指向的是在某处方向导数取极大值的方向。在二维图像处理中，可用gradient来检测灰度值的边缘。
# 对于向量场F: Rn-> Rm, Jacobian的每一行实际都是一个梯度。且有 F（X)=F(P)+J(P)(X-P)+O(||X-P||) 这个式子的每一行都是一个分量的局部线性化。
# 考虑一个二维的数字图像线性变换（Homography， image warping), 以有限差分代替微分，可作类似分析。
# H: 像素（x,y)-->像素(u,v)
# u=u（x,y) v=v(x,y)
# 则其Jacobian为
# [ u'(x) u'(y)]
# [ v'(x) v'(y)]
# 反映了局部图像的变形程度。
# 最理想的情况 u'(x)=1,v'(y)=1,u'(y)=0,v'(x)=0.说明图像维持原状。
# 由于 dudv=|det(Jacobian(x,y))|dxdy （此式的有效性可参考换元法）
# [注：]有的书上称det(Jacobian(x,y)）为Jacobian.
# 说明面积微元改变的程度由|det(Jacobian(x,y))|决定
# 当|det(Jacobian(x,y))|=1时，说明面积不变，
# 当|det(Jacobian(x,y))|<1时，说明面积压缩，出现了像素丢失现象。
# 当|det(Jacobian(x,y))|>1时，说明面积扩张，需要进行像素插值。
# 另外，由Jacobian矩阵的特征值或奇异值，可作类似说明。可参考Wielandt-Hoffman定理
# Hessian矩阵定义在标量函数上，对于矢量函数，则成为一个rank 3的张量。

Debug Mode:

>>> #因为在Theano中实际运行的代码可能和编写的代码大相径庭，debug可能会很困难。默认情况下Theano编译代码达到尽可能快的目的，然而也可以为了方便debug以牺牲速度的方式进行代码的编译。... # A simple division function... num = T.scalar('num')>>> den = T.scalar('den')>>> divide = theano.function([num, den], num/den)>>> print divide(10, 2)5.0>>> # This will cause a NaN... print divide(0, 0)nan>>>>>> # To compile a function in debug mode, just set mode='DebugMode'... divide = theano.function([num, den], num/den, mode='DebugMode')>>> # NaNs now cause errors... print divide(0, 0)Traceback (most recent call last):  File "<stdin>", line 2, in <module>  File "F:\_python\lib\site-packages\theano\compile\function_module.py", line 579, in __call__    outputs = self.fn()  File "F:\_python\lib\site-packages\theano\compile\debugmode.py", line 2030, in deco    return f()  File "F:\_python\lib\site-packages\theano\compile\debugmode.py", line 1804, in f    specific_hint=hint2)theano.compile.debugmode.InvalidValueError: InvalidValueError        type(variable) = TensorType(float64, scalar)        variable       = Elemwise{true_div,no_inplace}.0        type(value)    = <type 'numpy.ndarray'>        dtype(value)   = float64        shape(value)   = ()        value          = nan        min(value)     = nan        max(value)     = nan        isfinite       = False        client_node    = None        hint           = perform output        specific_hint  = non-finite elements not allowed        context        = ...  Elemwise{true_div,no_inplace} [@A] ''   |num [@B]   |den [@C]

使用 CPU 与 GPU：

#Theano can transparently透彻的 compile onto different hardware硬件. What device策略 it uses by default depends on your .theanorc file and any environment variables defined, as described in detail here: http://deeplearning.net/software/theano/library/config.html Currently, you should use float32 when using most GPUs, but most people prefer to use float64 on a CPU. For convenience方便起见, Theano provides the floatX configuration variable配置变量 which designates指定 what float accuracy to use. For example,#you can run a Python script脚本 with certain environment variables set to use the CPU:#THEANO_FLAGS=device=cpu,floatX=float64 python your_script.py# 或 GPU：#THEANO_FLAGS=device=gpu,floatX=float32 python your_script.py# You can get the values being used to configure配置 Theano like so:print theano.config.deviceprint theano.config.floatX# You can also get/set them at runtime:old_floatX = theano.config.floatXtheano.config.floatX = 'float32'# Be careful that you're actually using floatX!# For example, the following will cause var to be a float64 regardless of floatX due to numpy defaults:var = theano.shared(np.array([1.3, 2.4]))print var.type() #!!!# So, whenever you use a numpy array, make sure to set its dtype to theano.config.floatXvar = theano.shared(np.array([1.3, 2.4], dtype=theano.config.floatX))print var.type()# Revert to old valuetheano.config.floatX = old_floatX

cpu
float64
<TensorType(float64, vector)>
<TensorType(float32, vector)>

====================================================================================================================================