merge

doc/library/index.txt

@@ -19,6 +19,7 @@ Types and Ops that you can use to build and compile expression graphs.
    sparse/index
    scalar/index
    gof/index
+   scan
 
 There are also some top-level imports that you might find more convenient:
 

doc/library/scan.txt

 
 .. _lib_scan:
 
-Scan: Looping in Theano 
-=======================
+================================
+:mod:`scan` -- Looping in Theano 
+================================
 
 
 Guide
------
+=====
+
+The scan functions provides the basic functionality needed to do loops
+in Theano. Scan comes with many whistles and bells, that can be easily 
+introduced through a few examples : 
+
+Computing :math:`A^k`
+---------------------
+
+Assume that, given *k* you want to get ``A**k`` using a loop(note that 
+you do not need to do this in Theano, this example is purely
+didactical). More precisely, if *A* is a tensor you want to compute 
+``A**k`` elemwise. The python/numpy code would loop like
+
+.. code-block:: python
+  
+  result = 1
+  for i in xrange(k):
+    result = result * A
+
+The equivalent Theano code would be 
+
+.. code-block:: python
+
+  # Symbolic description of the result
+  result = theano.scan(fn             = lambda x_tm1,A: x_tm1*A,\
+                       sequences      = [], \
+                       initial_states = T.ones_like(A),\
+                       non_sequences  = A, \
+                       n_steps        = k)
+
+  # compiled function that returns A**k
+  f = theano.function([A,k], result)
+
+Let us go through the example line by line. What we did is first to 
+construct a function (using lambda expression) that given `x_tm1` and 
+`A` returns `x_tm1*A`. Given the order of the parameters, `x_tm1`
+is the value of our output at time step ``t-1``. Therefore 
+``x_t`` (value of output at time `t`) is `A` times value of output 
+at `t-1`. 
+Next we assign an empy list to ``sequences`` (since we do not need to
+iterate over anything) and initialize the output as a tensor with same
+shape as A filled with ones. We give A as a non sequence parameter  and
+tell scan to iterate for k steps. 
+
+
+Recurrent Neural Network with Scan
+----------------------------------
+
+A more practical task would be to implement a RNN using scan. Assume 
+that our RNN is defined as follows :
+
+.. math::
+  x(n) = \tanh( W x(n-1) + W^{in}_1 u(n) + W^{in}_2 u(n-4) +
+  W^{feedback} y(n-1) )
+
+  y(n) = W^{out} x(n- 3)
+
+In this case we have a sequence over which we need to iterate ``u``, 
+and two outputs ``x`` and ``y``. To implement this with scan we first
+construct a function that computes one iteration step : 
+
+.. code-block:: python
+
+  def oneStep(u_tm4, u_t, x_tm3, x_tm1, y_tm1, W, W_in_1, W_in_2,  W_feedback, W_out):
+
+    x_t = T.tanh( theano.dot(x_tm1, W) + \
+                  theano.dot(u_t,   W_in_1) + \
+                  theano.dot(u_tm4, W_in_2) + \
+                  theano.dot(y_tm1, W_feedback))
+    y_t = theano.dot(x_tm3, W_out)
+
+    return [x_t, y_t]
+
+Note the order in which the parameters are given, and in which the
+result is returned. It is crucial to respect cronological order among 
+the taps ( time slices of sequences or outputs) used, and to have same
+order in this function as when applying scan. Given that we have all 
+the Theano variables needed we construct our RNN as follows : 
+
+.. code-block:: python
+
+   u  = T.matrix() # it is a sequence of vectors
+   x0 = T.matrix() # initial state of x has to be a matrix, since 
+                   # it has to cover x[-3] 
+   y0 = T.vector() # y0 is just a vector since scan has only to provide
+                   # y[-1]
+
+
+   x_vals, y_vals = theano.scan(fn             = oneStep, \
+                                sequences      = [u], \
+                                initial_states = [x0,y0], \
+                                non_sequences  = [W,W_in_1,W_in_2,W_feedback, W_out], \
+                                sequences_taps = {0:[-4,0] },\
+                                outputs_taps   = {0:[-3,-1] },)
+        # for second input y, scan adds -1 in output_taps by default
+
+
+
+Now ``x_vals`` and ``y_vals`` are symbolic variables pointing to the
+sequence of x and y values generated by iterating over u.
 
 
 Reference
----------
+=========
 
-.. automodule:: theano.sandbox.scan
+.. automodule:: theano.scan
 
-.. autofunction:: theano.sandbox.scan.scan
+.. autofunction:: theano.scan
 

theano/__init__.py

@@ -60,6 +60,9 @@ FancyModule = Module
 from printing import \
     pprint, pp
 
+from scan import \
+    scan
+
 import tensor
 import scalar
 #import sparse #we don't import by default as we don't want to force having scipy installed.