merge

doc/advanced/index.txt

@@ -11,6 +11,7 @@ Structure
 .. toctree::
    :maxdepth: 2
    
+   pipeline
    env
    features
    optimization

doc/advanced/pipeline.txt

+
+.. _pipeline:
+
+====================================
+Overview of the compilation pipeline
+====================================
+
+The purpose of this page is to explain each step of defining and
+compiling a Theano function.
+
+
+Definition of the computation graph
+===================================
+
+By creating Theano :ref:`Variables <variable>` using
+``theano.tensor.lscalar`` or ``theano.tensor.dmatrix`` or by using
+Theano functions such as ``theano.tensor.sin`` or
+``theano.tensor.log``, the user builds a computation graph. The
+structure of that graph and details about its components can be found
+in the :ref:`graphstructures` article.
+
+
+
+Compilation of the computation graph
+====================================
+
+Once the user has built a computation graph, he can use
+``theano.function`` or a ``theano.Method`` in a ``theano.module`` in
+order to make one or more functions that operate on real data. Both
+function and Method take a list of input :ref:`Variables <variable>`
+as well as a list of output Variables that define a precise subgraph
+corresponding to the function(s) we want to define, compile that
+subgraph and produce a callable.
+
+Here is an overview of the various steps that are done with the
+computation graph in the compilation phase:
+
+
+Step 1 - Create an Env
+======================
+
+The subgraph given by the end user is wrapped in a structure called
+:ref:`env`. That structure defines several hooks on adding and
+removing (pruning) nodes as well as on modifying links between nodes
+(for example, modifying an input of an :ref:`apply` node) (see the
+article about :ref:`env` for more information).
+
+Env provides a method to change the input of an Apply node from one
+Variable to another and a more high-level method to replace a Variable
+with another. This is the structure that :ref:`Optimizers
+<optimization>` work on.
+
+Some relevant :ref:`Features <envfeature>` are typically added to the
+Env, namely to prevent any optimization from operating inplace on
+inputs declared as immutable.
+
+
+Step 2 - Execute main Optimizer
+===============================
+
+Once the Env is made, an :ref:`optimizer <optimization>` is produced
+by the :ref:`mode` passed to ``function`` or to the Method/Module's
+``make`` (the Mode basically has two important fields, ``linker`` and
+``optimizer``). That optimizer is applied on the Env using its
+optimize() method.
+
+The optimizer is typically obtained through :ref:`optdb <optdb>`.
+
+
+Step 3 - Execute linker to obtain a thunk
+=========================================
+
+Once the computation graph is optimized, the :ref:`linker` is
+extracted from the Mode. It is then called with the Env as argument to
+produce a ``thunk``, which is a function with no arguments that
+returns nothing. Along with the thunk, one list of input containers (a
+theano.gof.Container is a sort of object that wraps another and does
+type casting) and one list of output containers are produced,
+corresponding to the input and output Variables as well as the updates
+defined for the inputs when applicable. To perform the computations,
+the inputs must be placed in the input containers, the thunk must be
+called, and the outputs must be retrieved from the output containers
+where the thunk put them.
+
+Typically, the linker calls the ``toposort`` method in order to obtain
+a linear sequence of operations to perform. How they are linked
+together depends on the Linker used. The CLinker produces a single
+block of C code for the whole computation, whereas the OpWiseCLinker
+produces one thunk for each individual operation and calls them in
+sequence.
+
+The linker is where some options take effect: the ``strict`` flag of
+an input makes the associated input container do type checking. The
+``borrow`` flag of an output, if False, adds the output to a
+``no_recycling`` list, meaning that when the thunk is called the
+output containers will be cleared (if they stay there, as would be the
+case if ``borrow`` was True, the thunk would be allowed to reuse (or
+"recycle") the storage).
+
+
+Step 4 - Wrap the thunk in a pretty package
+===========================================
+
+The thunk returned by the linker along with input and output
+containers is unwieldy. ``function`` and ``Method`` hide that
+complexity away so that it can be used like a normal function with
+arguments and return values.
+
+
+
+
+
+
+
+
+
+
+
+

theano/gof/opt.py

@@ -498,21 +498,21 @@ class PatternSub(LocalOptimizer):
     arbitrary criterion.
 
     Examples:
-     PatternOptimizer((add, 'x', 'y'), (add, 'y', 'x'))
-     PatternOptimizer((multiply, 'x', 'x'), (square, 'x'))
-     PatternOptimizer((subtract, (add, 'x', 'y'), 'y'), 'x')
-     PatternOptimizer((power, 'x', Constant(double, 2.0)), (square, 'x'))
-     PatternOptimizer((boggle, {'pattern': 'x',
+     PatternSub((add, 'x', 'y'), (add, 'y', 'x'))
+     PatternSub((multiply, 'x', 'x'), (square, 'x'))
+     PatternSub((subtract, (add, 'x', 'y'), 'y'), 'x')
+     PatternSub((power, 'x', Constant(double, 2.0)), (square, 'x'))
+     PatternSub((boggle, {'pattern': 'x',
                                 'constraint': lambda env, expr: expr.type == scrabble}),
                       (scrabble, 'x'))
     """
 
     def __init__(self, in_pattern, out_pattern, allow_multiple_clients = False):
         """
-        Creates a PatternOptimizer that replaces occurrences of
+        Creates a PatternSub that replaces occurrences of
         in_pattern by occurrences of out_pattern.
 
-        If allow_multiple_clients is False, he pattern matching will
+        If allow_multiple_clients is False, the pattern matching will
         fail if one of the subpatterns has more than one client.
         """
         self.in_pattern = in_pattern