changes in documentation of scan

theano/scan_module/scan.py

@@ -5,7 +5,7 @@ Scanning is a general form of recurrence, which can be used for looping.
 The idea is that you *scan* a function along some input sequence, producing
 an output at each time-step that can be seen (but not modified) by the
 function at the next time-step. (Technically, the function can see the
-previous K  time-steps of your outputs and L time steps (from the past and
+previous K  time-steps of your outputs and L time steps (from past and
 future) of your inputs.
 
 So for example, ``sum()`` could be computed by scanning the ``z+x_i``
@@ -13,15 +13,21 @@ function over a list, given an initial state of ``z=0``.
 
 Special cases:
 
-* A *reduce* operation can be performed by returning only the last
+* A *reduce* operation can be performed by using only the last
   output of a ``scan``.
 * A *map* operation can be performed by applying a function that
   ignores previous steps of the outputs.
 
-Often a for-loop can be expressed as a ``scan()`` operation, and ``scan`` is
-the closest that theano comes to looping. The advantage of using ``scan``
-over for loops is that it allows the number of iterations to be a part of
-the symbolic graph.
+Often a for-loop or while-loop can be expressed as a ``scan()`` operation,
+and ``scan`` is the closest that theano comes to looping. The advantages
+of using ``scan`` over `for` loops in python (amongs other) are:
+
+* it allows the number of iterations to be part of the symbolic graph
+* it allows computing gradients through the for loop
+* there exist a bunch of optimizations that help re-write your loop
+such that less memory is used and that it runs faster
+* it ensures that data is not copied from host to gpu and gpu to
+host at each step
 
 The Scan Op should typically be used by calling any of the following
 functions: ``scan()``, ``map()``, ``reduce()``, ``foldl()``,
@@ -74,27 +80,27 @@ def scan( fn
         ``fn`` is a function that describes the operations involved in one
         step of ``scan``. ``fn`` should construct variables describing the
         output of one iteration step. It should expect as input theano
-        variables representing all the time slices of the input sequences
-        and outputs, and all other arguments given to scan as
-        ``non_sequences``. The order in which scan passes this variables
-        to ``fn``  is the following :
+        variables representing all the slices of the input sequences
+        and previous values of the outputs, as well as all other arguments
+        given to scan as ``non_sequences``. The order in which scan passes
+        these variables to ``fn``  is the following :
 
         * all time slices of the first sequence
         * all time slices of the second sequence
         * ...
         * all time slices of the last sequence
-        * all time slices of the first output
-        * all time slices of the second otuput
+        * all past slices of the first output
+        * all past slices of the second otuput
         * ...
-        * all time slices of the last output
+        * all past slices of the last output
         * all other arguments (the list given as `non_sequences` to
             scan)
 
         The order of the sequences is the same as the one in the list
-        `sequences` given to scan. The order of the outputs is the sane
+        `sequences` given to scan. The order of the outputs is the same
         as the order of ``output_info``. For any sequence or output the
-        order of the time slices is the same as the order of the time
-        taps provided. For example if one writes the following :
+        order of the time slices is the same as the one in which they have
+        been given as taps. For example if one writes the following :
 
         .. code-block:: python
 
@@ -122,25 +128,64 @@ def scan( fn
         The list of ``non_sequences`` can also contain shared variables
         used in the function, though ``scan`` is able to figure those
         out on its own so they can be skipped. For the clarity of the
-        code we recommand though to provide them to scan.
+        code we recommand though to provide them to scan. To some extend
+        ``scan`` can also figure out other ``non sequences`` (not shared)
+        even if not passed to scan (but used by `fn`). A simple example of
+        this would be :
+
+        .. code-block:: python
+
+            import theano.tensor as TT
+            W   = TT.matrix()
+            W_2 = W**2
+            def f(x):
+                return TT.dot(x,W_2)
 
         The function is expected to return two things. One is a list of
         outputs ordered in the same order as ``outputs_info``, with the
         difference that there should be only one output variable per
         output initial state (even if no tap value is used). Secondly
         `fn` should return an update dictionary ( that tells how to
-        update any shared variable after each iteration ste). The
+        update any shared variable after each iteration step). The
         dictionary can optionally be given as a list of tuples. There is
         no constraint on the order of these two list, ``fn`` can return
         either ``(outputs_list, update_dictionary)`` or
         ``(update_dictionary, outputs_list)`` or just one of the two (in
         case the other is empty).
 
+        To use ``scan`` as a while loop, the user needs to change the
+        function ``fn`` such that also a stopping condition is returned.
+        To do so, he/she needs to wrap the condition in an ``until`` class.
+        The condition can be returned as a third element, or all the other
+        outputs and updates can be wrapped in ``until``. A few examples
+        would be :
+
+        .. code-block:: python
+
+            ...
+            return [y1_t, y2_t], {x:x+1}, theano.until(x < 50)
+
+        or
+
+        .. code-block:: python
+
+            ...
+            return theano.until(x<50, [y1_t, y2_t], {x:x+1})
+
+        Note that a number of steps ( considered in here as the maximum
+        number of steps ) is still required even though a condition is
+        passed ( and it is used to allocate memory if needed ). Also when
+        passing multiple argument to ``until`` be aware of its signature:
+
+        .. code-block:: python
+
+            class until(object):
+                def __init__( condition, outputs = [], updates = {}):
 
     :param sequences:
         ``sequences`` is the list of Theano variables or dictionaries
         describing the sequences ``scan`` has to iterate over. If a
-        sequence is given as wrapped in a dictionary a set of optional
+        sequence is given as wrapped in a dictionary, then a set of optional
         information can be provided about the sequence. The dictionary
         should have the following keys:
 
@@ -191,13 +236,6 @@ def scan( fn
           ``fn``. They are provided as a list of *negative* integers,
           where a value ``k`` implies that at iteration step ``t`` scan
           will pass to ``fn`` the slice ``t+k``.
-        * ``return_steps`` -- Integer representing the number of steps
-          to return for the current steps. For example, if ``k`` is
-          provided, ``scan`` will return ``output[-k:]``. This is meant
-          as a hint, based on ``k`` and the past taps of the outputs used,
-          scan can be smart about the amount of memory it requires to
-          store intermidiate results. If not given, or ``0``, ``scan``
-          will return all computed steps.
 
         ``scan`` will follow this logic if partial information is given:
 
@@ -210,12 +248,12 @@ def scan( fn
         * If you wrap an output in a dictionary but you do not provide any
           initial state, it assumes that you are not using any form of
           taps.
-        * If you provide a ``None`` instead of a variable or a dictionary
-          ``scan`` assumes that you will not use any taps for this output
-          (like for example in case of a map)
+        * If you provide a ``None`` instead of a variable or a empty
+          dictionary ``scan`` assumes that you will not use any taps for
+          this output (like for example in case of a map)
 
         If ``outputs_info`` is an empty list or None, ``scan`` assumes
-        that no tap is used for any of the otuputs. If information is
+        that no tap is used for any of the outputs. If information is
         provided just for a subset of the outputs an exception is
         raised (because there is no convention on how scan should map
         the provided information to the outputs of ``fn``)
@@ -223,8 +261,9 @@ def scan( fn
 
     :param non_sequences:
         ``non_sequences`` is the list of arguments that are passed to
-        ``fn`` at each steps. Once can opt to exclude shared variables
-        used in ``fn`` from this list.
+        ``fn`` at each steps. One can opt to exclude variable
+        used in ``fn`` from this list as long as they are part of the
+        computational graph, though for clarity we encourage not to do so.
 
 
     :param n_steps:
@@ -232,10 +271,9 @@ def scan( fn
         or Theano scalar. If any of the input sequences do not have
         enough elements, scan will raise an error. If the *value is 0* the
         outputs will have *0 rows*. If the value is negative, ``scan``
-        run backwards in time. If the ``go_backwards`` flag is already
+        will run backwards in time. If the ``go_backwards`` flag is already
         set and also ``n_steps`` is negative, ``scan`` will run forward
-        in time. If n stpes is not provided, or is a constant that
-        evaluates to ``None``, ``inf`` or ``NaN``, ``scan`` will figure
+        in time. If n stpes is not provided, ``scan`` will figure
         out the amount of steps it should run given its input sequences.
 
 
@@ -257,19 +295,20 @@ def scan( fn
 
 
     :param name:
-        When profiling ``scan`` it is crucial to provide a name for any
+        When profiling ``scan``, it is crucial to provide a name for any
         instance of ``scan``. The profiler will produce an overall
-        profile of your code as well as profiles for doing one iteration
-        step for each instance of ``scan``. The ``name`` of the instance is
-        how you differentiate between all these profiles.
-
+        profile of your code as well as profiles for the computation of
+        one step of each instance of ``scan``. The ``name`` of the instance
+        appears in those profiles and can greatly help to disambiguate
+        information.
 
     :param mode:
         It is recommended to leave this argument to None, especially
         when profiling ``scan`` (otherwise the results are not going to
-        be accurate). If you prefer the computations of one step os
-        ``scan`` to be done differently then the entire function set
-        this parameters (see ``theano.function`` for details about
+        be accurate). If you prefer the computations of one step of
+        ``scan`` to be done differently then the entire function, you
+        can use this parameter to describe how the computations in this
+        loop are done (see ``theano.function`` for details about
         possible values and their meaning).
 
 
@@ -278,9 +317,9 @@ def scan( fn
              Theano variable or a list of Theano variables representing the
              outputs of ``scan`` (in the same order as in
              ``outputs_info``). ``updates`` is a dictionary specifying the
-             update rules for all shared variables used in the scan
-             operation. This dictionary should be passed to
-             ``theano.function`` when you compile your function.
+             update rules for all shared variables used in scan
+             This dictionary should be passed to ``theano.function`` when
+             you compile your function.
     """
     # General observation : this code is executed only once, at creation
     # of the computational graph, so we don't yet need to be smart about