merge

.hgignore

@@ -2,8 +2,8 @@ syntax: glob
 *.pyo
 *~
 \#*\#
-doc/oplist.txt
-doc/typelist.txt
+doc/indexes/oplist.txt
+doc/indexes/typelist.txt
 doc/.build/
 compiled/*.cpp
 cutils_ext.cpp

doc/advanced_tutorial/ex1/op.txt

@@ -7,24 +7,6 @@ Now that we have a ``double`` type, we have yet to use it to perform
 computations. We'll start by defining multiplication.
 
 
-What is an Op?
-==============
-
-An :ref:`op` in Theano defines a certain computation on some types of
-inputs, producing some types of outputs. It is equivalent to a
-function definition in most programming languages. From a list of
-input :ref:`Results <result>` and an Op, you can build an :ref:`apply`
-node representing the application of the Op to the inputs.
-
-It is important to understand the distinction between an Op (the
-definition of a function) and an Apply node (the application of a
-function). If you were to interpret the Python language using Theano's
-structures, code going like ``def f(x): ...`` would produce an Op for
-``f`` whereas code like ``a = f(x)`` or ``g(f(4), 5)`` would produce an
-Apply node involving the ``f`` Op.
-
-
-
 Op's contract
 =============
 
@@ -33,10 +15,10 @@ An Op is any object which defines the following methods:
 - **make_node(*inputs)**
     
   - This method is responsible for creating output Results of a suitable Type
-    to serve as the outputs of this Op's application.  It should put these
+    to serve as the outputs of this Op's application.  This method should put these
     outputs into an Apply instance, and return the Apply instance.
   
-  - This important function creates an Apply node representing the
+  - This method creates an Apply node representing the
     application of the Op on the inputs provided. If the Op cannot be
     applied on these inputs, it must raise an appropriate
     exception.
@@ -54,53 +36,75 @@ An Op is any object which defines the following methods:
 
 - **perform(node, inputs, output_storage)**
 
-  - This function computes the function associated to this Op. The
+  - This method computes the function associated to this Op. The
     ``node`` is an Apply node created by the Op's ``make_node``
-    method, the inputs are a list of references to data to operate on
-    and output_storage is a list of storage cells where the results of
-    the computation must be put.
-  
-  - This function must be determined by the inputs.  That is to say, if it is
+    method, ``inputs`` is a list of references to data to operate on,
+    and ``output_storage`` is a list of storage cells where the results of
+    the computation must be put. More specifically:
+
+    - ``node``: This is a reference to an Apply node which was previously
+      obtained via ``mul``'s ``make_node`` method. It is typically not
+      used in simple Ops, but it contains symbolic information that
+      could be required for complex Ops.
+
+    - ``inputs``: This is a list of data.
+
+    - ``output_storage``: This is a list of storage cells.
+      A storage cell is a one-element list. It is forbidden to change
+      the length of the list(s) contained in output_storage.  There is
+      one storage cell for each output of the Op.
+
+      The data you put in ``output_storage`` must match the type of the
+      symbolic output. This is a situation where the ``node`` argument
+      can come in handy.
+
+  - This method must be determined by the inputs. That is to say, if it is
     evaluated once on inputs A and returned B, then if ever inputs C, equal to
     A, are presented again, then outputs equal to B must be returned again.
   
   - You must be careful about aliasing outputs to inputs, and making
-    modifications to any of the inputs.  See `Views and inplace operations`_
+    modifications to any of the inputs. See `Views and inplace operations`_
     before writing a ``perform`` implementation that does either of these
     things.
 
 - **__eq__(self, other)**
 
-  - Returning True here is a promise to the optimization system that the other
+  - ``other`` is also an Op.
+
+  - Returning ``True`` here is a promise to the optimization system that the other
     Op will produce exactly the same graph effects (from perform) as this one,
-    given identical inputs.  This means it will produce the same output values,
+    given identical inputs. This means it will produce the same output values,
     it will destroy the same inputs (same destroy_map), and will alias outputs
     to the same inputs (same view_map).
 
 - **__hash__(self)**
 
-  - If two Op instances compare equal, then they MUST return the same hash
-    value.  
+  - If two Op instances compare equal, then they **must** return the same hash
+    value.
       
   - Equally important, this hash value must not change during the lifetime of
     self.
 
 - **__ne__(self, other)**
 
-  - Recommended, and default: define as (not (self==other))
+  - Recommended
+  
+  - Default: ``(not (self==other))``
+
+- **grad(inputs, output_gradients)**
 
-- **grad(inputs, output_gradients)** *Optional*
+  - Optional.
 
   - If the Op you are defining is differentiable, you can define its
     gradient symbolically in this method. 
     
-  - Both the inputs and output_gradients will be Results. This function must
-    return a list containing one Result (or None) for each input.
-    Each returned Result represents the gradient wrt that input given the
-    symbolic gradients wrt each output.
+  - Both the ``inputs`` and ``output_gradients`` will be Results. This
+    method must return a list containing one Result (or None) for each
+    input. Each returned Result represents the gradient with respect to
+    that input given the symbolic gradients with respect to each output.
 
   - If the output is not differentiable with respect to any inputs, then this
-    function should be defined to return [None for i in inputs].
+    method should be defined to return [None for i in inputs].
 
   - If this method is not defined, then theano assumes it has been forgotten.
     Symbolic differentiation will fail on a graph that includes this Op.
@@ -108,7 +112,9 @@ An Op is any object which defines the following methods:
   - For more information on the use of this method, see ``grad``.
 
 
-For each method, the *default* is what the Op class defines for you.
+For each method, the *default* is what :api:`theano.gof.Op` defines
+for you.  At a bare minimum, a new Op must define ``make_node`` and
+``perform``, which have no defaults.
 
 For more details, including the interface for providing a C implementation of
 perform(), refer to the documentation for :ref:`op`.
@@ -154,9 +160,10 @@ Use this list to make sure that you defined everything you need for your Op:
 Defining mul
 ============
 
-We are going to redefine the two functions that are absolutely
-necessary to redefine: ``make_node`` and ``perform``. First, we'll
-instantiate a ``mul`` Op:
+We'll define multiplication as a *binary* operation, even though a
+multiplication Op could take an arbitrary number of arguments.
+
+First, we'll instantiate a ``mul`` Op:
 
 .. code-block:: python
 
@@ -167,12 +174,13 @@ instantiate a ``mul`` Op:
 **make_node**
 
 This function must take as many arguments as the operation we are
-defining is supposed to take as inputs - in this example that would be
-two (we'll define multiplication as a binary operation here, even
-though a multiplication Op could technically take an arbitrary number
-of arguments). It should ensure that both inputs have the ``double``
-type and it should make an Apply node with an output Result of type
-``double`` (since multiplying two doubles yields a double).
+defining is supposed to take as inputs---in this example that would be
+two.
+This function ensures that both inputs have the ``double``
+type.
+Since multiplying two doubles yields a double,
+this function makes an Apply node with an output Result of type
+``double``.
 
 .. code-block:: python
 
@@ -189,14 +197,8 @@ want to multiply two arbitrary types, it would not make much sense
 (and we'd be screwed when we implement this in C!)
 
 The last line is the meat of the definition. There we create an Apply
-node representing the application of ``mul`` to ``x`` and ``y``. Apply
-takes three arguments: the first one is the Op we are applying. In
-this case, we are applying ``mul``. The second argument is a list of
-input Results - here, ``x`` and ``y``. The third is a list of output
-Results. Since the multiplication of two doubles ought to give us a
-double again, we create a Result of type ``double`` and we place it in
-a list. Since the list only has one element, ``mul`` only has one
-output.
+node representing the application of Op ``mul`` to inputs ``x`` and
+``y``, giving a Result instance of type ``double`` as the output.
 
 .. note::
    Theano relies on the fact that if you call the ``make_node`` method
@@ -208,25 +210,11 @@ output.
 
 **perform**
 
-This code should actually compute the function. It is important to
-understand the role of all three arguments of ``perform``:
-
-- *node*: This is a reference to an Apply node which was previously
-  obtained via ``mul``'s ``make_node`` method. It is not typically
-  useful, but it contains symbolic information that could be required
-  for complex Ops.
-
-- *inputs*: This is a list of data. In this example, the data in
-  ``inputs`` will be instances of Python's built-in type ``float``
-  because this is the type that ``double.filter()`` will always
-  return, per our own definition.
-
-- *output_storage*: This is a list of storage cells. There is one
-  storage cell for each output of the Op. A storage cell is
-  a one-element list (note: it is forbidden to change the
-  length of the list(s) contained in output_storage). In this example,
-  output_storage will contain a single storage cell for the
-  multiplication's result.
+This code actually computes the function.
+In our example, the data in ``inputs`` will be instances of Python's
+built-in type ``float`` because this is the type that ``double.filter()``
+will always return, per our own definition. ``output_storage`` will
+contain a single storage cell for the multiplication's result.
 
 .. code-block:: python
 
@@ -248,12 +236,9 @@ Here, ``z`` is a list of one element. By default, ``z == [None]``.
    :ref:`op` documentation.
 
 .. warning::
-   The data you put in ``output_storage`` must match the type of the
-   symbolic output. This is a situation where the ``node`` argument
-   can come in handy. In this example, we gave ``z`` the Theano type
-   ``double`` in ``make_node``, which means that a Python ``float``
-   must be put there. You should not put, say, an ``int`` in ``z[0]``
-   because Theano assumes Ops handle typing properly.
+   We gave ``z`` the Theano type ``double`` in ``make_node``, which means
+   that a Python ``float`` must be put there. You should not put, say, an
+   ``int`` in ``z[0]`` because Theano assumes Ops handle typing properly.
 
 
 Trying out our new Op
@@ -298,7 +283,7 @@ by modifying ``make_node`` to accept Python ``int`` or ``float`` as
    mul.make_node = make_node
 
 Whenever we pass a Python int or float instead of a Result as ``x`` or
-``y``, make_node will convert it to :ref:`constant` for us. ``gof.Constant``
+``y``, ``make_node`` will convert it to :ref:`constant` for us. ``gof.Constant``
 is a :ref:`result` we statically know the value of.
 
 >>> x = double('x')

doc/advanced_tutorial/ex1/type.txt

@@ -5,43 +5,6 @@ Making the double type
 ======================
 
 
-What is a Type?
-===============
-
-A :ref:`type` in Theano represents a set of constraints on potential
-data objects. These constraints allow Theano to tailor C code to handle
-them and to statically optimize the computation graph. For instance,
-the :ref:`irow <predefinedtypes>` type in the ``theano.tensor`` package
-gives the following constraints on the data the Results of type ``irow``
-may contain:
-
-#. Must be an instance of ``numpy.ndarray``: ``isinstance(x, numpy.ndarray)``
-#. Must be an array of 32-bit integers: ``str(x.dtype) == 'int32'``
-#. Must have a shape of 1xN: ``len(x.shape) == 2 and x.shape[0] == 1``
-
-Knowing these restrictions, Theano may generate C code for addition, etc.
-that declares the right data types and that contains the right number
-of loops over the dimensions.
-
-Note that a Theano :ref:`type` is not equivalent to a Python type or
-class. Indeed, in Theano, :ref:`irow <predefinedtypes>` and :ref:`dmatrix
-<predefinedtypes>` both use ``numpy.ndarray`` as the underlying type
-for doing computations and storing data, yet they are different Theano
-Types. Indeed, the constraints set by ``dmatrix`` are:
-
-#. Must be an instance of ``numpy.ndarray``: ``isinstance(x, numpy.ndarray)``
-#. Must be an array of 64-bit floating point numbers: ``str(x.dtype) == 'float64'``
-#. Must have a shape of MxN, no restriction on M or N: ``len(x.shape) == 2``
-
-These restrictions are different from those of ``irow`` which are listed above.
-
-There are cases in which a Type can fully correspond to a Python type,
-such as the ``double`` Type we will define here which corresponds to
-Python's ``float``. But, it's good to know that this is not necessarily
-the case. Unless specified otherwise, when we say "Type" we mean a
-Theano Type.
-
-
 Type's contract
 ===============
 

doc/advanced_tutorial/graphstructures.txt

@@ -169,6 +169,10 @@ theano's version of a function definition.
 
 An Apply instance has three important fields:
 
+**op**
+  An :ref:`op` that determines the function/transformation being
+  applied here.
+
 **inputs**
   A list of :ref:`Results <result>` that represent the arguments of
   the function.
@@ -177,33 +181,8 @@ An Apply instance has three important fields:
   A list of :ref:`Results <result>` that represent the return values
   of the function.
 
-**op**
-  An :ref:`op` that determines the function/transformation being
-  applied here.
-
-
-.. index::
-   single: Constant
-   single: graph construct; Constant
-
-.. _constant:
-
---------
-Constant
---------
-
-A constant is a :ref:`Result` with one extra field, *data* (only
-settable once). When used in a computation graph as the input of an
-:ref:`Op` :ref:`application <Apply>`, it is assumed that said input
-will *always* take the value contained in the constant's data
-field. Furthermore, it is assumed that the :ref:`Op` will not under
-any circumstances modify the input. This means that a constant is
-eligible to participate in numerous optimizations: constant inlining
-in C code, constant folding, etc.
-
-A constant does not need to be specified in a :ref:`function`'s list
-of inputs.
-
+An Apply instance can be created by calling ``gof.Apply(op, inputs,
+outputs)``.
 
 
 .. index::
@@ -212,6 +191,8 @@ of inputs.
 
 .. _result:
 
+
+
 ------
 Result
 ------
@@ -263,6 +244,28 @@ A Result ``r`` contains four important fields:
 
 Result has one special subclass: :ref:`constant <constant>`.
 
+.. index::
+   single: Constant
+   single: graph construct; Constant
+
+.. _constant:
+
+
+
+Constant
+^^^^^^^^
+
+A constant is a :ref:`Result` with one extra field, *data* (only
+settable once). When used in a computation graph as the input of an
+:ref:`Op` :ref:`application <Apply>`, it is assumed that said input
+will *always* take the value contained in the constant's data
+field. Furthermore, it is assumed that the :ref:`Op` will not under
+any circumstances modify the input. This means that a constant is
+eligible to participate in numerous optimizations: constant inlining
+in C code, constant folding, etc.
+
+A constant does not need to be specified in a :ref:`function`'s list
+of inputs.
 
 
 .. index::
@@ -275,7 +278,20 @@ Result has one special subclass: :ref:`constant <constant>`.
 Op
 --
 
-WRITEME
+An :ref:`op` in Theano defines a certain computation on some types of
+inputs, producing some types of outputs. It is equivalent to a
+function definition in most programming languages. From a list of
+input :ref:`Results <result>` and an Op, you can build an :ref:`apply`
+node representing the application of the Op to the inputs.
+
+It is important to understand the distinction between an Op (the
+definition of a function) and an Apply node (the application of a
+function). If you were to interpret the Python language using Theano's
+structures, code going like ``def f(x): ...`` would produce an Op for
+``f`` whereas code like ``a = f(x)`` or ``g(f(4), 5)`` would produce an
+Apply node involving the ``f`` Op.
+
+
 
 
 
@@ -289,5 +305,37 @@ WRITEME
 Type
 ----
 
-WRITEME
+A :ref:`type` in Theano represents a set of constraints on potential
+data objects. These constraints allow Theano to tailor C code to handle
+them and to statically optimize the computation graph. For instance,
+the :ref:`irow <predefinedtypes>` type in the ``theano.tensor`` package
+gives the following constraints on the data the Results of type ``irow``
+may contain:
+
+#. Must be an instance of ``numpy.ndarray``: ``isinstance(x, numpy.ndarray)``
+#. Must be an array of 32-bit integers: ``str(x.dtype) == 'int32'``
+#. Must have a shape of 1xN: ``len(x.shape) == 2 and x.shape[0] == 1``
+
+Knowing these restrictions, Theano may generate C code for addition, etc.
+that declares the right data types and that contains the right number
+of loops over the dimensions.
+
+Note that a Theano :ref:`type` is not equivalent to a Python type or
+class. Indeed, in Theano, :ref:`irow <predefinedtypes>` and :ref:`dmatrix
+<predefinedtypes>` both use ``numpy.ndarray`` as the underlying type
+for doing computations and storing data, yet they are different Theano
+Types. Indeed, the constraints set by ``dmatrix`` are:
+
+#. Must be an instance of ``numpy.ndarray``: ``isinstance(x, numpy.ndarray)``
+#. Must be an array of 64-bit floating point numbers: ``str(x.dtype) == 'float64'``
+#. Must have a shape of MxN, no restriction on M or N: ``len(x.shape) == 2``
+
+These restrictions are different from those of ``irow`` which are listed above.
+
+There are cases in which a Type can fully correspond to a Python type,
+such as the ``double`` Type we will define here which corresponds to
+Python's ``float``. But, it's good to know that this is not necessarily
+the case. Unless specified otherwise, when we say "Type" we mean a
+Theano Type.
+
 

doc/advanced_tutorial/theano_vs_python.txt

@@ -12,7 +12,7 @@ analogue in Python:
 Theano          Python
 =============== ===========================================================
 Apply           function application / function call
-Result          function data
+Result          function data / variable
 Op              operations carried out in computation / function definition
 Type            data types
 Module          ??? class?

doc/internal/index.txt

@@ -13,5 +13,5 @@ Structure
 
    dev_start_guide
    hg_primer
+   mammouth
    metadocumentation
-   lisa_labo

doc/internal/mammouth.txt

+===========================
+Running Theano on Mammouth
+===========================
+
+To run Theano on the Mammouth cluster, follow these simple steps:
+
+* Make sure to source Fred's .local.bashrc file. It contains all the goodies for using the latest and greatest (optimized) libraries (numpy, scipy, etc.)
+>>> source /home/bastienf/.local.bashrc
+
+* set THEANO_BLAS_LDFLAGS='-lmkl -lguide -fopenmp'
+
+  Note: the -lguide flag works, however the fix should probably be considered temporary.
+  Intel has deprecated libguide.so in favor of the newer library libiomp5.so. However, 
+  both libraries are mutually exclusive and one component (theano, numpy or scipy?) already
+  seems to be using libguide.so (hence -liomp5 causes a linking error when compiling thunks)
+

doc/sandbox/numpy.txt

+
+.. _numpy::
+
+===============
+NumPy refresher
+===============
+
+Give summary of type(x) vs x.type vs x.dtype

scripts/docgen.py

@@ -126,7 +126,7 @@ if __name__ == '__main__':
             os.chdir(workdir)
             os.system('make')
             try:
-                shutil.copy(os.path.join(workdir, 'theano.pdf'), currentdir)
+                shutil.copy(os.path.join(workdir, 'theano.pdf'), outdir)
                 os.chdir(outdir)
                 shutil.rmtree(workdir)
             except OSError, e: