Theano（7）：Theano循环语句，Scan

http://deeplearning.net/software/theano/tutorial/loop.html

http://deeplearning.net/software/theano/library/scan.html#lib-scan

Scan

• A general form of recurrence, which can be used for looping.
• Reduction and map (loop over the leading dimensions) are special cases of scan.
• You scan a function along some input sequence, producing an output at each time-step.
• The function can see the previous K time-steps of your function.
• sum() could be computed by scanning the z + x(i) function over a list, given an initial state of z=0.
• Often a for loop can be expressed as a scan() operation, and scan is the closest that Theano comes to looping.
• Advantages of using scan over for loops:
  • Number of iterations to be part of the symbolic graph.
  • Minimizes GPU transfers (if GPU is involved).
  • Computes gradients through sequential steps.
  • Slightly faster than using a for loop in Python with a compiled Theano function.
  • Can lower the overall memory usage by detecting the actual amount of memory needed.

The full documentation can be found in the library: Scan.

计算A^K：

三件事要处理：不变的A，使用non_sequences这个参数；初始值，使用 outputs_info这个参数；prior_result*A，theano框架自动计算。

所以有：

>>> k=T.iscalar('k')>>> a=T.dmatrix('a')>>> # We only care about A**k, but scan has provided us with A**1 through A**k.>>> result, updates = theano.scan(fn=lambda prior_result, a: prior_result*a,...         outputs_info=T.ones_like(a), non_sequences=a, n_steps=k)>>> # Discard the values that we don't care about. Scan is smart enough to notice this and not waste memory saving them.>>> final_result = result[-1]>>> # compiled function that returns a**k>>> power = theano.function(inputs=[a,k], outputs=final_result, updates=updates)>>> print(power([[1,2,3],[4,5,6]],2))[[  1.   4.   9.] [ 16.  25.  36.]]

Scan over一个tensor的第一维（主维度）：

需要loop的tensor(s)通过sequence keyword argument传递给scan函数。

例如构建一个多项式，系数来自指定的vector，底来自指定的scalar：

1.0 * (3 ** 0) + 0.0 * (3 ** 1) + 2.0 * (3 ** 2)

>>> coefs=T.vector('coefs')>>> x=T.scalar('x')>>> maxSupportCoefNum=100>>> #generage the components of the polnomial>>> components, updates=theano.scan(fn=lambda coef, power, base: coef*(base**power),...     outputs_info=None, sequences=[coefs, T.arange(maxSupportCoefNum)], non_sequences=x)>>> poly=components.sum() #sum components up>>> polynomial=theano.function(inputs=[coefs,x], outputs=poly)>>> print(polynomial([1,0,2],3))19.0

 outputs_info 为 None，说明fn不需要初始化。

另外，power从哪里来的呢？？？a handy trick used to simulate python’s enumerate: simply include theano.tensor.arangeto the sequences.
最后说一下scan函数中，fn函数的参数的产生顺序（The general order of function parameters to fn is）：

sequences (if any), prior result(s) (if needed), non-sequences (if any)

elementwise的计算y+x[i]=z[i]：

scalar与vector：

>>> x=T.vector('x')>>> y=T.scalar('y')>>> addEach, updates=theano.scan(lambda xi: y+xi, sequences=x)>>> addFun=theano.function(inputs=[x,y],outputs=[addEach])>>> z=addFun([1,2,3,4],100)>>> print z[array([ 101.,  102.,  103.,  104.])]

scalar与matrix：

>>> import theano>>> import theano.tensor as T>>> x=T.dmatrix('x')>>> y=T.dscalar('y')>>> addEach, updates=theano.scan(lambda xi: y+xi, sequences=x)>>> addFun=theano.function(inputs=[x,y], outputs=[addEach])>>> z=addFun([[1,2],[3,4]],100)>>> print z[array([[ 101.,  102.],       [ 103.,  104.]])]

vector与matrix：

>>> subEach, updates=theano.scan(lambda xi: T.tanh(T.dot(xi,w)+b), sequences=x)>>> mulEach, updates=theano.scan(lambda xi: T.tanh(T.dot(xi,w)+b), sequences=x)>>> mulFun=theano.function(inputs=[x,w,b], outputs=[mulEach])>>> z=mulFun([[1,2],[3,4]],[5,6],[7,8])>>> print z[array([[ 1.,  1.],       [ 1.,  1.]])]>>> mulEach, updates=theano.scan(lambda xi: T.dot(xi,w)+b, sequences=x)>>> mulFun=theano.function(inputs=[x,w,b], outputs=[mulEach])>>> z=mulFun([[1,2],[3,4]],[5,6],[7,8])>>> print z[array([[ 24.,  25.],       [ 46.,  47.]])]

scalar与scalar计算和：

>>> k=theano.shared(0)>>> nStep=T.iscalar('step')>>> addAll, updates=theano.scan(lambda:{k:(k+1)},n_steps=nStep)>>> addFun=theano.function([nStep],[],updates=updates)>>> k.get_value()array(0)>>> addFun(3)[]>>> k.get_value()array(3)>>> k.set_value(5)>>> k.get_value()array(5)>>> addFun(3)[]>>> k.get_value()array(8)

还有很多，参考：
http://deeplearning.net/software/theano/tutorial/loop.html

Computing the sequence x(t) = tanh(x(t - 1).dot(W) + y(t).dot(U) + p(T - t).dot(V))Computing norms of lines of XComputing norms of columns of X

Computing trace of XComputing the sequence x(t) = x(t - 2).dot(U) + x(t - 1).dot(V) + tanh(x(t - 1).dot(W) + b)Computing the Jacobian of y = tanh(v.dot(A)) wrt xComputing tanh(v.dot(W) + b) * d where d is binomial

Computing pow(A, k)Calculating a Polynomial

>>> x=T.vector('x')>>> y=T.scalar('y')>>> addEach, updates=theano.scan(lambda xi: y+xi, sequences=x)>>> addFun=theano.function(inputs=[x,y],outputs=[addEach])>>> z=addFun([1,2,3,4],100)>>> print z[array([ 101.,  102.,  103.,  104.])]