Some restructuring of the tutorials

doc/tutorial/debug_faq.txt

@@ -97,3 +97,94 @@ Use your imagination :)
 This can be a really powerful debugging tool.
 Note the call to ``fn`` inside the call to ``print_eval``; without it, the graph wouldn't get computed at all!
 
+How to use pdb ?
+----------------
+
+In the majority of cases, you won't be executing from the interactive shell
+but from a set of Python scripts. In such cases, the use of the Python
+debugger can come in handy, especially as your models become more complex.
+Intermediate results don't necessarily have a clear name and you can get
+exceptions which are hard to decipher, due to the "compiled" nature of
+functions.
+
+Consider this example script ("ex.py"):
+
+.. code-block:: python
+
+        import theano
+        import numpy
+        import theano.tensor as T
+
+        a = T.dmatrix('a')
+        b = T.dmatrix('b')
+
+        f = theano.function([a,b], [a*b])
+
+        # matrices chosen so dimensions are unsuitable for multiplication
+        mat1 = numpy.arange(12).reshape((3,4))
+        mat2 = numpy.arange(25).reshape((5,5))
+
+        f(mat1, mat2)
+
+This is actually so simple the debugging could be done easily, but it's for
+illustrative purposes. As the matrices can't be element-wise multiplied
+(unsuitable shapes), we get the following exception:
+
+::
+    File "ex.py", line 14, in <module>
+      f(mat1, mat2)
+    File "/u/username/Theano/theano/compile/function_module.py", line 451, in
+ __call__
+    File "/u/username/Theano/theano/gof/link.py", line 271, in
+ streamline_default_f
+    File "/u/username/Theano/theano/gof/link.py", line 267, in
+ streamline_default_f
+    File "/u/username/Theano/theano/gof/cc.py", line 1049, in execute
+  ValueError: ('Input dimension mis-match. (input[0].shape[0] = 3,
+ input[1].shape[0] = 5)',
+       Elemwise{mul,no_inplace}(a, b), Elemwise{mul,no_inplace}(a, b))
+
+The call stack contains a few useful informations to trace back the source
+of the error. There's the script where the compiled function was called --
+but if you're using (improperly parameterized) prebuilt modules, the error
+might originate from ops in these modules, not this script. The last line
+tells us about the Op that caused the exception. In thise case it's a "mul"
+involving Variables name "a" and "b". But suppose we instead had an
+intermediate result to which we hadn't given a name.
+
+After learning a few things about the graph structure in Theano, we can use
+the Python debugger to explore the graph, and then we can get runtime
+information about the error. Matrix dimensions, especially, are useful to
+pinpoint the source of the error. In the printout, there are also 2 of the 4
+dimensions of the matrices involved, but for the sake of example say we'd
+need the other dimensions to pinpoint the error. First, we re-launch with
+the debugger module and run the program with "c":
+
+::
+        python -m pdb ex.py
+        > /u/username/experiments/doctmp1/ex.py(1)<module>()
+        -> import theano
+        (Pdb) c
+
+Then we get back the above error printout, but the interpreter breaks in
+that state. Useful commands here are
+
+* "up" and "down" (to move up and down the call stack),</li>
+* "l" (to print code around the line in the current stack position),</li>
+* "p variable_name" (to print the string representation of
+'variable_name'),</li>
+* "p dir(object_name)", using the Python dir() function to print the list of
+an object's members
+
+Here, for example, I do "up", and a simple "l" shows me there's a local
+variable "node". This is the "node" from the computation graph, so by
+following the "node.inputs", "node.owner" and "node.outputs" links I can
+explore around the graph.
+
+That graph is purely symbolic (no data, just symbols to manipulate it
+abstractly). To get information about the actual parameters, you explore the
+"thunks" objects, which bind the storage for the inputs (and outputs) with
+the function itself (a "thunk" is a concept related to closures). Here, to
+get the current node's first input's shape, you'd therefore do "p
+thunk.inputs[0][0].shape", which prints out "(3, 4)".
+

doc/tutorial/examples.txt

@@ -197,8 +197,8 @@ array(33.0)
 
 .. _functionstateexample:
 
-Including values in a symbolic graph
-====================================
+Using shared variables
+======================
 
 It is also possible to make a function with an internal state. For
 example, let's say we want to make an accumulator: at the beginning,
@@ -268,8 +268,8 @@ updates).  Also, theano has more control over where and how shared variables are
 allocated, which is one of the important elements of getting good performance
 on the GPU.
 
-It may happen that you have constructed a symbolic graph on top of a
-shared variable, but you do *not* want to use its value. In this case, you can use the
+It may happen that you expressed some formula using a shared variable, but 
+you do *not* want to use its value. In this case, you can use the
 ``givens`` parameter of ``function`` which replaces a particular node in a graph
 for the purpose of one particular function.
 
@@ -290,5 +290,93 @@ substitution to be co-dependent, the order of substitution is not defined, so
 the substitutions have to work in any order.
 
 
+Using Random Numbers
+====================
+
+Because everything has to be expressed symbolically firstly in Theano,
+using pseudo-random numbers is not as straightforward as it is in 
+numpy, though also not to complicated. 
+
+The way to think about putting randomness into Theano's computations is 
+to put random variables in your graph. Theano will allocate a numpy 
+RandomStream object (a random number generator) for each such 
+variable, and draw from it as necessary. I'll call this sort of 
+sequence of random numbers a *random stream*. *Random streams* are at 
+their core shared variables, so the observations on shared variables
+hold here as well.
+
+Brief example
+-------------
+
+Here's a brief example.  The setup code is:
+
+.. code-block:: python
+
+    from theano.tensor.shared_randomstreams import RandomStreams
+    srng = RandomStreams(seed=234)
+    rv_u = srng.uniform((2,2))
+    rv_n = srng.normal((2,2))
+    f = function([], rv_u, updates=[rv_u.update])
+    g = function([], rv_n)                              #omitting rv_n.update
+    nearly_zeros = function([], rv_u + rv_u - 2 * rv_u, updates=[rv_u.update])
+
+Here, 'rv_u' represents a random stream of 2x2 matrices of draws from a uniform
+distribution.  Likewise,  'rv_n' represenents a random stream of 2x2 matrices of
+draws from a normal distribution.  The distributions that are implemented are
+defined in :class:`RandomStreams`.
+
+Now let's use these things.  If we call f(), we get random uniform numbers.
+Since we are updating the internal state of the random number generator (via
+the ``updates`` argument), we get different random numbers every time.
+
+>>> f_val0 = f()
+>>> f_val1 = f()  #different numbers from f_val0
+
+When we omit the updates argument (as in ``g``) to ``function``, then the
+random number generator state is not affected by calling the returned function.  So for example, 
+calling ``g`` multiple times will return the same numbers.
+
+>>> g_val0 = g()  # different numbers from f_val0 and f_val1
+>>> g_val0 = g()  # same numbers as g_val0 !!!
+
+An important remark is that a random variable is drawn at most once during any
+single function execution.  So the ``nearly_zeros`` function is guaranteed to
+return approximately 0 (except for rounding error) even though the ``rv_u``
+random variable appears three times in the output expression.
+
+>>> nearly_zeros = function([], rv_u + rv_u - 2 * rv_u, updates=[rv_u.update])
+
+Seedings Streams
+----------------
+
+Random variables can be seeded individually or collectively.
+
+You can seed just one random variable by seeding or assigning to the
+``.rng.value`` attribute.
+
+>>> rv_u.rng.value.seed(89234)  # seeds the generator for rv_u
+
+You can also seed *all* of the random variables allocated by a :class:`RandomStreams`
+object by that object's ``seed`` method.  This seed will be used to seed a
+temporary random number generator, that will in turn generate seeds for each
+of the random variables.
+
+>>> srng.seed(902340)  # seeds rv_u and rv_n with different seeds each
+
+Sharing Streams between Functions
+---------------------------------
+
+As usual for shared variables, the random number generators used for random
+variables are common between functions.  So our ``nearly_zeros`` function will
+update the state of the generators used in function ``f`` above.
+
+For example:
+
+>>> state_after_v0 = rv_u.rng.value.get_state()
+>>> nearly_zeros()       # this affects rv_u's generator
+>>> v1 = f()             
+>>> rv_u.rng.value.set_state(state_after_v0)
+>>> v2 = f()             # v2 != v1
+
 
 

doc/tutorial/index.txt

@@ -24,6 +24,7 @@ installation (see :ref:`install`).
     adding
     examples
     loading_and_saving
+    symbolic_graphs
     modes
     remarks
     debug_faq