merge

NEWS.txt

+doc/NEWS.txt
\ No newline at end of file

doc/NEWS.txt

+
+.. _NEWS:
+
+=============
+Release Notes
+=============
+
+Theano 0.1
+==========
+
+*Release date: 2009-04-02*
+
+What works
+----------
+
+- building symbolic expression.
+- arranging symbolic expressions into Modules so that multiple functions
+  can work on the same data.
+- symbolic gradient descent.
+- graph optimization.
+- compilation to C for many kinds of expression.
+- a debugging mode that checks that your expression results are correct,
+  using a variety of sanity checks.
+
+What's missing?
+---------------
+
+- An algorithm library. We're missing a library of examples and standard
+  component implementations.  Some examples will find their way into
+  the Theano repo, but standard algorithms will go into the 'pylearn'
+  project (toolbox style). Now that we have a stable foundation, we
+  can reach a consensus on style for algorithms.

doc/advanced/features.txt

@@ -5,6 +5,8 @@
 List of Env Features
 ====================
 
+See :api:`gof.env.Env`.
+
 WRITEME
 
 .. _nodefinder:
@@ -12,4 +14,6 @@ WRITEME
 NodeFinder
 ==========
 
+See :api:`gof.toolbox.NodeFinder`.
+
 WRITEME

doc/advanced/module.txt

@@ -5,15 +5,82 @@
 Module Interface
 ================
 
-WRITEME
+A Theano Module is like Theano's version of a file.
+When you instantiate a ``Module()``, you are creating a blank file.
+Into this file you can put both symbolic and non-symbolic objects.
+Non-symbolic objects are like constants (technically literals) in the file.
+Symbolic objects are like variables and functions.
+The functions in a Module are called Methods.
+The variables in a Module (and submodules) are global.
+Module Methods have access to all these global variables.
+
+To use a Module, you need to compile it.
+This is done by calling `Module.make()`.
+The result of compiling a Module is a ModuleInstance, this is the compiled
+version of your Theano file.
+In the ModuleInstance, your symbolic variables have become containers (containing None),
+and your Methods have become callable functions.
+You should initialize the symbolic variables by calling
+``ModuleInstance.initialize()`` (although make() will call it for you, 
+on the top-level ModuleInstance.)
+
+You can compile a Module several times, to create multiple ModuleInstances.
+Each of these will have its own copy of all program literals.
+
+
+
+Module Graph
+------------
+
+Components can be grouped into a directed graph.
+When we call `make`, this graph is replicated with ComponentInstances instead of
+Components.  Wheras Components are represent symbolic things (ie. Variables), ComponentInstances represent non-symbolic ones (ie. sparse matrices, ndarrays, callable functions).
 
 
+.. index::
+   single: Component
+   single: component; Component
+
+.. _component:
+---------
+Component
+---------
+
+All of the elements of what is called the "module system" or "modules" are
+components.
+
+A component subclass is represents a symbolic theano thing, and implements the
+``build`` function. 
+The ``build`` function is responsible for converting the symbolic thing into a
+non-symbolic thing.
+
+
+Compiling with make
+-------------------
+
+Conversion from a Component graph to a ComponentInstance graph is performed by `Component.make`.
+This method traverses the Component graph in multiple passes. 
+
+In the first pass (the allocate pass), it creates storage for all Variables that are contained in the graph (see
+`Component.allocate`).  These are the module variables.
+
+In the second pass (the build pass), it creates functions that (in general) operate on these module variables.
+This pass also serves to construct all ComponentInstance-derived instances as well, such as
+`ModuleInstance`s.  The objects that are returned from this second pass are the return value of
+`Component.make`.
+
+In the third pass (the initialize pass), is optional and not necessarily recursive through the
+graph.
+The purpose of the third pass is to call the initialize method of the ComponentInstances built
+during the second pass.
+During this pass the ComponentInstance graph is complete. It is a good time to fill storage
+allocated in phase 1 with sensible values.
+
 .. index::
    single: External
    single: component; External
 
 .. _external:
-
 --------
 External
 --------
@@ -27,7 +94,6 @@ WRITEME
    single: component; Member
 
 .. _member:
-
 ------
 Member
 ------
@@ -40,7 +106,6 @@ WRITEME
    single: component; Method
 
 .. _method:
-
 ------
 Method
 ------
@@ -53,13 +118,29 @@ WRITEME
    single: component; Module
 
 .. _module:
-
 ------
 Module
 ------
 
-WRITEME
+A Module instance can contain objects as attributes.
+This makes it something like a class in the way that Method is
+analogous to a function.
+
+A Module is meant to contain Components.
+Attributes which are not Components themselves must at least be transform-able
+into Components by :api:`compile.module.wrap`.  If a Module contains something
+that is not convertible into a Component, then it is not possible to compile
+that Module with ``make``.
 
 
+Old Text
+--------
 
+In the Module system, the analog of the file is the `Module`, the analog of the function is the
+`Method`, and the analog of the variable is the `Member`.  Module, Member, and Method all work
+at the symbolic level.  Once a graph of Modules, Members, and Methods is ready for use, it must
+be compiled with a call to `make` which will return an isomorphic structure in which Modules
+have become `ModuleInstances`, Members have become `Container`s, and Methods have become
+`Function`s.
+This structure contains numbers and functions, and is ready for computation.
 

doc/advanced_tutorial/cop.txt

@@ -35,7 +35,7 @@ There are less methods to define for an Op than for a Type:
   This must return C code that cleans up whatever c_code allocated and
   that we must free.
 
-  *Default* The default behavior is to do nothing.
+  *Default:* The default behavior is to do nothing.
 
 .. function:: c_compile_args()
               c_headers()
@@ -118,14 +118,14 @@ version that it produces in the code I gave above.
 .. code-block:: python
 
    from theano import gof
-   
+
    class BinaryDoubleOp(gof.Op):
-   
+
        def __init__(self, name, fn, ccode):
            self.name = name
            self.fn = fn
            self.ccode = ccode
-   
+
        def make_node(self, x, y):
            if isinstance(x, (int, float)):
                x = gof.Constant(double, x)
@@ -134,29 +134,29 @@ version that it produces in the code I gave above.
            if x.type != double or y.type != double:
                raise TypeError('%s only works on doubles' % self.name)
            return gof.Apply(self, [x, y], [double()])
-   
+
        def perform(self, node, (x, y), (z, )):
            z[0] = self.fn(x, y)
-   
+
        def __str__(self):
            return self.name
-   
+
        def c_code(self, node, name, (x, y), (z, ), sub):
            return self.ccode % locals()
-   
-   
+
+
    add = BinaryDoubleOp(name = 'add',
                         fn = lambda x, y: x + y,
                         ccode = "%(z)s = %(x)s + %(y)s;")
-   
+
    sub = BinaryDoubleOp(name = 'sub',
                         fn = lambda x, y: x - y,
                         ccode = "%(z)s = %(x)s - %(y)s;")
-   
+
    mul = BinaryDoubleOp(name = 'mul',
                         fn = lambda x, y: x * y,
                         ccode = "%(z)s = %(x)s * %(y)s;")
-   
+
    div = BinaryDoubleOp(name = 'div',
                         fn = lambda x, y: x / y,
                         ccode = "%(z)s = %(x)s / %(y)s;")

doc/advanced_tutorial/graphstructures.txt

@@ -146,12 +146,12 @@ Automatic wrapping
 
 All nodes in the graph must be instances of ``Apply`` or ``Result``, but
 ``<Op subclass>.make_node()`` typically wraps constants to satisfy those
-constraints. For example, the :api:`tensor.add <theano.tensor.add>`
+constraints. For example, the :api:`tensor.add <theano.tensor.basic.add>`
 Op instance is written so that:
 
 .. code-block:: python
 
-    e = scalar('x') + 1
+    e = dscalar('x') + 1
 
 builds the following graph:
 

doc/advanced_tutorial/inplace.txt

@@ -7,8 +7,8 @@ Views and inplace operations
 
 Theano allows the definition of Ops which return a :term:`view` on one
 of their inputs or operates :term:`inplace` on one or several
-inputs. This allows more efficient operations on numpy's ndarray data type than
-would be possible otherwise.
+inputs. This allows more efficient operations on numpy's ``ndarray``
+data type than would be possible otherwise.
 However, in order to work correctly, these Ops need to
 implement an additional interface.
 
@@ -29,7 +29,7 @@ Views
 
 A "view" on an object ``x`` is an object ``y`` which shares memory
 with ``x`` in some way. In other words, changing ``x`` might also
-change ``y`` and vice versa. For example, imagine a "vector" structure
+change ``y`` and vice versa. For example, imagine a ``vector`` structure
 which contains two fields: an integer length and a pointer to a memory
 buffer. Suppose we have:
 
@@ -51,7 +51,7 @@ range ``0xDEADBEFF - 0xDEADBFDF`` and z the range ``0xCAFEBABE -
 considered to be a view of ``x`` and vice versa.
 
 Suppose you had an Op which took ``x`` as input and returned
-``y``. You would need to tell Theano that y is a view of x. For this
+``y``. You would need to tell Theano that ``y`` is a view of ``x``. For this
 purpose, you would set the ``view_map`` field as follows:
 
 
@@ -103,7 +103,7 @@ operation on ``x``.
 
    .. code-block:: python
 
-      x, y = dscalars('xy')
+      x, y = dscalars('x', 'y')
       r1 = log(x)
 
       # r2 is x AFTER the add_inplace - x still represents the value before adding y
@@ -119,7 +119,7 @@ operation on ``x``.
 
    Needless to say, this goes for user-defined inplace operations as
    well: the modified input must figure in the list of outputs you
-   give to Apply in the definition of make_node.
+   give to ``Apply`` in the definition of ``make_node``.
 
    Also, for technical reasons but also because they are slightly
    confusing to use as evidenced by the previous code, Theano does not
@@ -132,7 +132,7 @@ operation on ``x``.
    introduces inconsistencies.
 
 
-Take the previous definitions of x, y and z and suppose an Op which
+Take the previous definitions of ``x``, ``y`` and ``z`` and suppose an Op which
 adds one to every byte of its input. If we give ``x`` as an input to
 that Op, it can either allocate a new buffer of the same size as ``x``
 (that could be ``z``) and set that new buffer's bytes to the variable of
@@ -141,7 +141,7 @@ it could add one to each byte *in* the buffer ``x``, therefore
 changing it. That would be an inplace Op.
 
 Theano needs to be notified of this fact. The syntax is similar to
-that of view_map:
+that of ``view_map``:
 
 
 .. code-block:: python
@@ -160,10 +160,10 @@ first input (position 0).
    myop.destroy_map = {1: [0]} # second output operates inplace on first input
 
    myop.destroy_map = {0: [0], # first output operates inplace on first input
-                    1: [1]} # *AND* second output operates inplace on second input
+                       1: [1]} # *AND* second output operates inplace on second input
 
    myop.destroy_map = {0: [0], # first output operates inplace on first input
-                    1: [0]} # *AND* second output *ALSO* operates inplace on first input
+                       1: [0]} # *AND* second output *ALSO* operates inplace on first input
 
    myop.destroy_map = {0: [0, 1]} # first output operates inplace on both the first and second input
    # unlike for views, the previous line is legal and supported
@@ -194,7 +194,7 @@ input(s)'s memory). From there, go to the previous section.
    the value of ``x`` it might invert the order and that will
    certainly lead to erroneous computations.
 
-   You can often identify an incorrect view_map or destroy_map by using
-   :ref:`DebugMode`.  *Be sure to use DebugMode when developing a new Op that
-   uses view_map and/or destroy_map.*
+   You can often identify an incorrect ``view_map`` or ``destroy_map``
+   by using :ref:`DebugMode`.  *Be sure to use DebugMode when developing
+   a new Op that uses ``view_map`` and/or ``destroy_map``.*
 

doc/advanced_tutorial/op.txt

@@ -12,16 +12,17 @@ computations. We'll start by defining multiplication.
 Op's contract
 =============
 
-An Op (:api:`gof.op.Op`) is any object which defines the following methods:
+An Op (:api:`gof.op.Op`) is any object which defines the
+following methods:
 
 
 .. function:: make_node(*inputs)
-    
+
   This method is responsible for creating output Variables of a
   suitable Type to serve as the outputs of this Op's application.
   This method should put these outputs into an Apply instance, and
   return the Apply instance.
-  
+
   This method creates an Apply node representing the application of
   the Op on the inputs provided. If the Op cannot be applied on
   these inputs, it must raise an appropriate exception.
@@ -30,13 +31,13 @@ An Op (:api:`gof.op.Op`) is any object which defines the following methods:
   ordered correctly: a subsequent ``self.make_node(*apply.inputs)``
   must produce something equivalent to the first ``apply``.
 
-``default_output``
+.. attribute:: default_output
 
-  *Default*: None
+  *Default:* None
 
   If this member variable is an integer, then the default
   implementation of ``__call__`` will return
-  `node.outputs[self.default_output]``, where `node` was returned
+  ``node.outputs[self.default_output]``, where ``node`` was returned
   by ``make_node``.  Otherwise, the entire list of outputs will be
   returned.
 
@@ -45,7 +46,7 @@ An Op (:api:`gof.op.Op`) is any object which defines the following methods:
   Syntactic shortcut to make_node which returns the output
   Variables of the Op.
 
-  *Default*: this is done for you by Op.
+  *Default:* this is done for you by Op.
 
 .. function:: perform(node, inputs, output_storage)
 
@@ -64,26 +65,26 @@ An Op (:api:`gof.op.Op`) is any object which defines the following methods:
 
     - ``output_storage``: This is a list of storage cells.
       A storage cell is a one-element list. It is forbidden to change
-      the length of the list(s) contained in output_storage.  There is
+      the length of the list(s) contained in ``output_storage``.  There is
       one storage cell for each output of the Op.
 
       The data you put in ``output_storage`` must match the type of the
       symbolic output. This is a situation where the ``node`` argument
       can come in handy.
 
-      A function Mode may allow output_storage elements to persist between
-      evaluations, or it may reset output_storage cells to hold a value of
-      None.  This feature can allow perform to reuse memory between calls, for
-      example.
+      A function Mode may allow ``output_storage`` elements to persist between
+      evaluations, or it may reset ``output_storage`` cells to hold a value of
+      ``None``.  This feature can allow ``perform`` to reuse memory between
+      calls, for example.
 
   This method must be determined by the inputs. That is to say, if
   it is evaluated once on inputs A and returned B, then if ever
   inputs C, equal to A, are presented again, then outputs equal to
   B must be returned again.
-  
+
   You must be careful about aliasing outputs to inputs, and making
-  modifications to any of the inputs. See `Views and inplace
-  operations <views_and_inplace>`_ before writing a ``perform``
+  modifications to any of the inputs. See :ref:`Views and inplace
+  operations <views_and_inplace>` before writing a ``perform``
   implementation that does either of these things.
 
 .. function:: __eq__(other)
@@ -95,20 +96,21 @@ An Op (:api:`gof.op.Op`) is any object which defines the following methods:
   (from perform) as this one, given identical inputs. This means it
   will produce the same output values, it will destroy the same
   inputs (same destroy_map), and will alias outputs to the same
-  inputs (same view_map).
+  inputs (same view_map). For more details, see
+  :ref:`views_and_inplace`.
 
 .. function:: __hash__()
 
   If two Op instances compare equal, then they **must** return the
   same hash value.
-      
+
   Equally important, this hash value must not change during the
   lifetime of self.  Op instances should be immutable in this
   sense.
 
 .. function:: __ne__(other)
-  
-  Default: ``(not (self==other))``
+
+  *Default:* ``(not (self==other))``
 
 .. function:: grad(inputs, output_gradients)
 
@@ -116,30 +118,28 @@ An Op (:api:`gof.op.Op`) is any object which defines the following methods:
 
   If the Op you are defining is differentiable, you can define its
   gradient symbolically in this method.
-    
+
   Both the ``inputs`` and ``output_gradients`` will be
   Variables. This method must return a list containing one Variable
-  (or None) for each input. Each returned Variable represents the
+  (or ``None``) for each input. Each returned Variable represents the
   gradient with respect to that input given the symbolic gradients
   with respect to each output.
 
   If the output is not differentiable with respect to any inputs,
-  then this method should be defined to return [None for i in
-  inputs].
+  then this method should be defined to return ``[None for i in
+  inputs]``.
 
-  If this method is not defined, then theano assumes it has been
+  If this method is not defined, then Theano assumes it has been
   forgotten.  Symbolic differentiation will fail on a graph that
   includes this Op.
 
-  For more information on the use of this method, see ``grad``.
-
 
 For each method, the *default* is what :api:`theano.gof.op.Op` defines
 for you.  At a bare minimum, a new Op must define ``make_node`` and
 ``perform``, which have no defaults.
 
 For more details, including the interface for providing a C
-implementation of perform(), refer to the documentation for :ref:`op`.
+implementation of ``perform()``, refer to the documentation for :ref:`op`.
 
 
 Defining an Op: ``mul``
@@ -252,7 +252,7 @@ AttributeError: 'int' object has no attribute 'type'
 Automatic Constant Wrapping
 ---------------------------
 
-Well, ok. We'd like our Op to be a bit more flexible. This can be done
+Well, OK. We'd like our Op to be a bit more flexible. This can be done
 by modifying ``make_node`` to accept Python ``int`` or ``float`` as
 ``x`` and/or ``y``:
 

doc/advanced_tutorial/optimization.txt

@@ -18,7 +18,7 @@ Env is a wrapper around a whole computation graph, you can see its
 :ref:`documentation <env>` for more details) and navigates through it
 in a suitable way, replacing some Variables by others in the process. A
 local optimization, on the other hand, is defined as a function on a
-*single* :ref:`apply` node and must return either False (to mean that
+*single* :ref:`apply` node and must return either ``False`` (to mean that
 nothing is to be done) or a list of new Variables that we would like to
 replace the node's outputs with. A :ref:`navigator` is a special kind
 of global optimization which navigates the computation graph in some
@@ -49,7 +49,7 @@ methods:
 
   This method takes an Env object and adds :ref:`features
   <envfeature>` to it. These features are "plugins" that are needed
-  for the apply method to do its job properly.
+  for the ``apply`` method to do its job properly.
 
 .. function:: optimize(env)
 
@@ -69,7 +69,7 @@ A local optimization is an object which defines the following methods:
 
 .. function:: transform(node)
 
-  This method takes an :ref:`apply` node and returns either False to
+  This method takes an :ref:`apply` node and returns either ``False`` to
   signify that no changes are to be done or a list of Variables which
   matches the length of the node's ``outputs`` list. When the
   LocalOptimizer is applied by a Navigator, the outputs of the node
@@ -99,9 +99,9 @@ Here is the code for a global optimization implementing the
 simplification described above:
 
 .. code-block:: python
-   
+
    from theano.gof import toolbox
-   
+
    class Simplify(gof.Optimizer):
        def add_requirements(self, env):
            env.extend(toolbox.ReplaceValidate())
@@ -116,38 +116,39 @@ simplification described above:
                            env.replace_validate(z, b)
                        elif y == b:
                            env.replace_validate(z, a)
-   
+
    simplify = Simplify()
 
 Here's how it works: first, in ``add_requirements``, we add the
 ``ReplaceValidate`` :ref:`envfeature` located in
-``theano.gof.toolbox``. This feature adds the ``replace_validate``
-method to the env, which is an enhanced version of ``replace`` that
+:api:`theano.gof.toolbox`. This feature adds the ``replace_validate``
+method to ``env``, which is an enhanced version of ``replace`` that
 does additional checks to ensure that we are not messing up the
-computation graph (note: if ReplaceValidate was already added by
+computation graph (note: if ``ReplaceValidate`` was already added by
 another optimizer, ``extend`` will do nothing). In a nutshell,
-``toolbox.ReplaceValidate`` grants access to ``env.replace_validate``
+``toolbox.ReplaceValidate`` grants access to ``env.replace_validate``,
 and ``env.replace_validate`` allows us to replace a Variable with
 another while respecting certain validation constraints. You can
 browse the list of :ref:`features <envfeaturelist>` and see if some of
 them might be useful to write optimizations with. For example, as an
-exercise, try to rewrite Simplify using :ref:`nodefinder` (hint: you
-want to use the method it publishes in place of the call to toposort!)
+exercise, try to rewrite Simplify using :ref:`nodefinder`. (Hint: you
+want to use the method it publishes instead of the call to toposort!)
 
 Then, in ``apply`` we do the actual job of simplification. We start by
 iterating through the graph in topological order. For each node
 encountered, we check if it's a ``div`` node. If not, we have nothing
-to do here. If so, we put in x, y and z the numerator, denominator and
-quotient (output) of the division. The simplification only occurs when
-the numerator is a multiplication, so we check for that. If the
-numerator is a multiplication we put the two operands in a and b, so
+to do here. If so, we put in ``x``, ``y`` and ``z`` the numerator,
+denominator and quotient (output) of the division.
+The simplification only occurs when the numerator is a multiplication,
+so we check for that. If the numerator is a multiplication we put the
+two operands in ``a`` and ``b``, so
 we can now say that ``z == (a*b)/y``. If ``y==a`` then ``z==b`` and if
-``y==b`` then ``z==a``. When either case happens then we can replace z
-by either a or b using ``env.replace_validate`` - else we do
+``y==b`` then ``z==a``. When either case happens then we can replace
+``z`` by either ``a`` or ``b`` using ``env.replace_validate`` - else we do
 nothing. You might want to check the documentation about :ref:`variable`
 and :ref:`apply` to get a better understanding of the
 pointer-following game you need to get ahold of the nodes of interest
-for the simplification (x, y, z, a, b, etc.)
+for the simplification (``x``, ``y``, ``z``, ``a``, ``b``, etc.).
 
 Test time:
 
@@ -198,7 +199,7 @@ places. Note that ``add(x, y)`` and ``add(y, x)`` are still considered
 to be different because Theano has no clue that ``add`` is
 commutative. You may write your own global optimizer to identify
 computations that are identical with full knowledge of the rules of
-arithmetics that your ops implement. Theano might provide facilities
+arithmetics that your Ops implement. Theano might provide facilities
 for this somewhere in the future.
 
 .. note::
@@ -217,7 +218,7 @@ The local version of the above code would be the following:
 
 
 .. code-block:: python
-   
+
    class LocalSimplify(gof.LocalOptimizer):
        def transform(self, node):
            if node.op == div:
@@ -234,7 +235,7 @@ The local version of the above code would be the following:
            # but it isn't now
            # TODO: do this and explain it
            return [] # that's not what you should do
-   
+
    local_simplify = LocalSimplify()
 
 The definition of transform is the inner loop of the global optimizer,

doc/advanced_tutorial/type.txt

@@ -39,30 +39,30 @@ default values.
   ``filter(value, strict = True)`` does not raise an exception, the
   value is compatible with the Type.
 
-  *Default*: True iff ``filter(value, strict = True)`` does not raise
+  *Default:* True iff ``filter(value, strict = True)`` does not raise
   an exception.
 
 .. function:: values_eq(a, b)
 
   Returns True iff ``a`` and ``b`` are equal.
 
-  *Default*: ``a == b``
+  *Default:* ``a == b``
 
 .. function:: values_eq_approx(a, b)
 
   Returns True iff ``a`` and ``b`` are approximately equal, for a
   definition of "approximately" which varies from Type to Type.
 
-  *Default*: ``values_eq(a, b)``
+  *Default:* ``values_eq(a, b)``
 
 .. function:: make_variable(name=None)
 
   Makes a :term:`Variable` of this Type with the specified name, if
-  ``name is not None``. If ``name is ``None``, then the Variable does
+  ``name`` is not ``None``. If ``name`` is ``None``, then the Variable does
   not have a name. The Variable will have its ``type`` field set to
   the Type object.
 
-  *Default*: there is a generic definition of this in Type. The
+  *Default:* there is a generic definition of this in Type. The
   Variable's ``type`` will be the object that defines this method (in
   other words, ``self``).
 
@@ -70,21 +70,21 @@ default values.
 
   Syntactic shortcut to ``make_variable``.
 
-  *Default*: ``make_variable``
+  *Default:* ``make_variable``
 
 .. function:: __eq__(other)
 
   Used to compare Type instances themselves
 
-  *Default*: ``object.__eq__``
+  *Default:* ``object.__eq__``
 
 .. function:: __hash__()
 
-  Types should not be mutable, so it should be Ok to define a hash
+  Types should not be mutable, so it should be OK to define a hash
   function.  Typically this function should hash all of the terms
   involved in ``__eq__``.
 
-  *Default*: ``id(self)``
+  *Default:* ``id(self)``
 
 For each method, the *default* is what ``Type`` defines
 for you. So, if you create an instance of ``Type`` or an
@@ -99,7 +99,7 @@ For more details you can go see the documentation for :ref:`type`.
 Defining double
 ===============
 
-We are going to base Type ``double`` on Python's ``float``. We are
+We are going to base Type ``double`` on Python's ``float``. We
 must define ``filter`` and shall override ``values_eq_approx``.
 
 
@@ -139,17 +139,17 @@ graph in such a way that it produces slightly different variables, for
 example because of numerical instability like rounding errors at the
 end of the mantissa. For instance, ``a + a + a + a + a + a`` might not
 actually produce the exact same output as ``6 * a`` (try with a=0.1),
-but with ``values_eq_approx`` we with don't necessarily mind.
+but with ``values_eq_approx`` we don't necessarily mind.
 
 We added an extra ``tolerance`` argument here. Since this argument is
-not part of the API, it must have a default value which we
+not part of the API, it must have a default value, which we
 chose to be 1e-4.
 
 .. note::
 
     ``values_eq`` is never actually used by Theano, but it might be used
-    internally in the future. Equality testing in DebugMode is done
-    using ``values_eq_approx``.
+    internally in the future. Equality testing in
+    :ref:`DebugMode <debugmode>` is done using ``values_eq_approx``.
 
 **Putting them together**
 
@@ -160,7 +160,7 @@ the Type is to instantiate a plain Type and set the needed fields:
 
 .. code-block:: python
 
-   from Theano import gof
+   from theano import gof
 
    double = gof.Type()
    double.filter = filter
@@ -175,19 +175,19 @@ and define ``filter`` and ``values_eq_approx`` in the subclass:
    from theano import gof
 
    class Double(gof.Type):
-   
+
        def filter(self, x, strict=False):
            if strict and not isinstance(x, float):
                raise TypeError('Expected a float!')
            return float(x)
-   
+
        def values_eq_approx(self, x, y, tolerance=1e-4):
            return abs(x - y) / (abs(x) + abs(y)) < tolerance
-   
+
    double = Double()
 
 ``double`` is then an instance of Type ``Double``, which in turn is a
-sublcass of ``Type``.
+subclass of ``Type``.
 
 There is a small issue with defining ``double`` this way. All
 instances of ``Double`` are technically the same Type. However, different
@@ -199,7 +199,7 @@ instances of ``Double`` are technically the same Type. However, different
 False
 
 Theano compares Types using ``==`` to see if they are the same.
-This happens in DebugMode.  Also, ops can (and should) ensure that their inputs
+This happens in DebugMode.  Also, Ops can (and should) ensure that their inputs
 have the expected Type by checking something like ``if x.type == lvector``.
 
 There are several ways to make sure that equality testing works properly:
@@ -243,7 +243,7 @@ attempt to clear up the confusion:
   that Type instance. If you were to parse the C expression ``c = a +
   b;``, ``a``, ``b`` and ``c`` would all be Variable instances.
 
-* A **subclass of Type** is a way of implementing 
+* A **subclass of Type** is a way of implementing
   a set of Type instances that share
   structural similarities. In the ``double`` example that we are doing,
   there is actually only one Type in that set, therefore the subclass
@@ -265,18 +265,18 @@ Final version
    from theano import gof
 
    class Double(gof.Type):
-   
+
        def filter(self, x, strict=False):
            if strict and not isinstance(x, float):
                raise TypeError('Expected a float!')
            return float(x)
-   
+
        def values_eq_approx(self, x, y, tolerance=1e-4):
            return abs(x - y) / (abs(x) + abs(y)) < tolerance
 
        def __str__(self):
            return "double"
-   
+
    double = Double()
 
 

doc/basic_tutorial/module.txt

@@ -5,9 +5,11 @@ Using Module
 
 Now that we're familiar with the basics, we introduce Theano's more
 advanced interface, Module. This interface allows you to define Theano
-"objects" which can have many state variables and many methods sharing
-these states. The Module system simplifies the way to define complex
+"files" which can have variables and methods sharing
+those variables. The Module system simplifies the way to define complex
 systems such as a neural network.
+It also lets you load and save these complex systems using Python's pickle
+mechanism.
 
 
 Remake of the "state" example
@@ -44,12 +46,15 @@ This deserves to be broken up a bit...
 >>> m = Module()
 
 Here we instantiate an empty Module.
+If you can imagine that Theano is a way of generating code (expression
+graphs),
+then a ``Module()`` is like a fresh blank file.
 
 
 >>> m.state = T.dscalar()
 >>> m.inc = T.dscalar('inc')
 
-Then we declare Variables for use with our Module.
+Then we declare Variables for use in our Module.
 Since we assign these input Variables as attributes of the Module,
 they will be *member Variables* of the Module.
 Member Variables are special in a few ways, which we will see shortly.
@@ -57,7 +62,8 @@ Member Variables are special in a few ways, which we will see shortly.
 .. note::
 
    There is no need to name the Variable explicitly here. ``m.state`` will
-   be given the name ``'state'`` automatically.
+   be given the name ``'state'`` automatically, because it is being assigned
+   to the attribute named ``'state'``.
 
 
 .. note::

doc/conf.py

@@ -64,7 +64,7 @@ today_fmt = '%B %d, %Y'
 
 # List of directories, relative to source directories, that shouldn't be searched
 # for source files.
-exclude_dirs = ['images', 'trac']
+exclude_dirs = ['images', 'scripts', 'trac']
 
 # The reST default role (used for this markup: `text`) to use for all documents.
 #default_role = None

doc/contents.txt

@@ -8,7 +8,6 @@ Contents
 .. toctree::
    :maxdepth: 2
 
-   index
    introduction
    LICENSE
    install
@@ -19,6 +18,7 @@ Contents
    glossary
    links
    internal/index
+   NEWS
 
 
 ..   examples/index

doc/index.txt

@@ -5,8 +5,7 @@ Quick Start
 
 Theano is a Python library that allows you to define, optimize, and
 efficiently evaluate mathematical expressions involving multi-dimensional
-arrays. Using Theano, it is not uncommon to see speed improvements of
-ten-fold over using pure NumPy.
+arrays.
 
 The latest release is version `0.1
 <http://pylearn.org/theano/downloads/Theano-0.1.tar.gz>`_.
@@ -14,7 +13,13 @@ You can download the latest `PDF documentation
 <http://pylearn.org/theano/theano.pdf>`_, rather than reading it online.
 You can go to the :ref:`Table of Contents <contents>`.
 
-Or, you can choose your own adventure...
+News
+----
+
+* 2009-04-01: Theano 0.1 released. See the :ref:`release notes <NEWS>`.
+
+Choose your own adventure...
+----------------------------
 
 * You have no idea what Theano is and you read the :ref:`introduction
   <introduction>`.

doc/install.txt

@@ -125,7 +125,7 @@ Mac
 
 - Install some kind of BLAS library (TODO: how?)
 
-- Set ``THEANO_BLAS_LDFLAGS to something which will link against said BLAS
+- Set ``THEANO_BLAS_LDFLAGS`` to something which will link against said BLAS
   library.  E.g., ``THEANO_BLAS_LDFLAGS='-lcblas -latlas -lgfortran'``.
 
 This advice has not been tested recently, so please inform us of your results.

doc/internal/how_to_release.txt

@@ -11,6 +11,7 @@ Clone the code::
 
 Edit ``setup.py`` to contain the newest version number::
 
+    cd Theano-0.X
     vi setup.py     # Edit the version "field"
 
 The homepage must link to the download URL, for PyPi to correctly get the
@@ -21,8 +22,8 @@ Edit ``doc/index.txt`` to contain a link to what will be the download URL::
 
 Tag the release. The syntax is something like the following::
 
-    cd Theano-0.X
-    hg tag
+    hg tag Theano-0.X
+    hg push
 
 Now, package the release and move it to the static theano directory::
 
@@ -30,7 +31,6 @@ Now, package the release and move it to the static theano directory::
     cd ..
     tar cvf Theano-0.X.tar Theano-0.X
     gzip -9 Theano-0.X.tar
-    rm -Rf Theano-0.X
     mv Theano-0.X.tar.gz www/theano_static/downloads/
     ~/repos/theano/.hg/refresh-epydoc.sh
 
@@ -42,7 +42,13 @@ directory::
 
 Finally, use setuptools to register and upload the release::
 
+    cd Theano-0.X
     python setup.py register sdist bdist_egg upload
+    # If you get an error message about needing to be identified, then store
+    # your pypi information in ~/.pypirc
+    # You can remove this file after upload.
+    cd ..
+    rm -Rf Theano-0.X
 
 I wrote the above without actually running it. This needs to be
 scrutinized when you are actually do a release.

doc/topics/unittest.txt

@@ -393,6 +393,12 @@ Here is an example showing how to use verify_grad:
 >>>    # ...
 >>>    tensor.verify_grad(Flatten(), [a_val])
 
+.. note::
+
+    Although ``verify_grad`` is defined in ``theano.tensor.basic``, unittests
+    should use the version of ``verify_grad`` defined in ``theano.tests.unittest_tools``.
+    This is simply a wrapper function which takes care of seeding the random 
+    number generator appropriately before calling ``theano.tensor.basic.verify_grad``
 
 makeTester and makeBroadcastTester
 ==================================

theano/compile/debugmode.py

@@ -130,6 +130,12 @@ class BadDestroyMap(DebugModeError):
         print >> sio, "  changed input type:", self.node.inputs[self.idx].type
         print >> sio, "  repr (old val):", repr(self.old_val)
         print >> sio, "  repr (new val):", repr(self.new_val)
+        print >> sio, "  value dtype (new <space> old):", self.new_val.dtype, self.old_val.dtype
+        print >> sio, "  value shape (new <space> old):", self.new_val.shape, self.old_val.shape
+        print >> sio, "  value min (new <space> old):", self.new_val.min(), self.old_val.min()
+        print >> sio, "  value max (new <space> old):", self.new_val.max(), self.old_val.max()
+        print >> sio, "  value min (new-old):", (self.new_val-self.old_val).min()
+        print >> sio, "  value max (new-old):", (self.new_val-self.old_val).max()
         print >> sio, ""
         print >> sio, "  Hint: this can also be caused by a deficient values_eq_approx() or __eq__() implementation that compares node input values"
         return sio.getvalue()

theano/compile/module.py

 """Classes implementing Theano's Module system.
 
-Rationale
-=========
-
-Functions in theano can share containers, when the `value` argument to `In` is a Container
-instance.  This feature makes it possible for multiple functions to use (and update) the same
-inputs.
-
-Modules provide a more intuitive syntax that makes this feature easier to use.  
-They draw on the metaphor of a python import--a module has functions and variables, and
-can contain other modules.  All functions have access to all variables, and whenever any
-function modifies a file-level variable, then that change is visible to all other functions.
-
-In the Module system, the analog of the file is the `Module`, the analog of the function is the
-`Method`, and the analog of the variable is the `Member`.  Module, Member, and Method all work
-at the symbolic level.  Once a graph of Modules, Members, and Methods is ready for use, it must
-be compiled with a call to `make` which will return an isomorphic structure in which Modules
-have become `ModuleInstances`, Members have become `Container`s, and Methods have become
-`Function`s.
-This structure contains numbers and functions, and is ready for computation.
-
-
-Design Documentation
-====================
-
-Module Graph
-------------
-
-Components form a tree structure.  Each component may have a _parent_ to which it is _bound_.
-When we call `make`, this tree structure is replicated with ComponentInstances instead of
-Components.  Wheras Components are primarily symbolic, ComponentInstances are sparse matrices,
-ndarrays, callable functions, etc.
-
-Compilation via make
---------------------
-
-Conversion from a Component graph to a ComponentInstance graph is performed by `Component.make`.
-This method traverses the Component graph in two passes. 
-
-In the first pass (the allocate pass), it creates storage for all Variables that are contained in the graph (see
-`Component.allocate`).  These are the module variables.
-
-In the second pass (the build pass), it creates functions that (in general) operate on these module variables.
-This pass also serves to construct all ComponentInstance-derived instances as well, such as
-`ModuleInstance`s.  The objects that are returned from this second pass are the return value of
-`Component.make`.
-
-In the third pass (the initialize pass), is optional and not necessarily recursive through the
-graph.
-The purpose of the third pass is to call the initialize method of the ComponentInstances built
-during the second pass.
-During this pass the ComponentInstance graph is complete. It is a good time to fill storage
-allocated in phase 1 with sensible values.
-
-Class Structure
----------------
-
-The most important classes for the user API here are `Module`, `ModuleInstance`, and `Method`.  
-Several other classes are defined to factorize functionality.
-
-- `Component`: WRITEME: what properties make something a Component?
-
-- `_RComponent`: WRITEME: what properties make something a Component?
-
-- `External`: WRITEME: what properties hold? What 
-
-- `Member`: WRITEME: what properties hold? What do they do?
+For design notes, see doc/advanced/module.txt
 
 """
 
@@ -249,7 +184,6 @@ class External(_RComponent):
             rval += '\n= %s' % (pprint(self.r, dict(target = self.r)))
         return rval
 
-
 class Member(_RComponent):
     """
     Member represents a Variable which is a state of a Composite. That
@@ -836,7 +770,10 @@ class ComponentDict(Composite):
 
     def set(self, item, value):
         if not isinstance(value, Component):
-            raise TypeError('ComponentDict may only contain Components.', value, type(value))
+            msg = """
+            ComponentDict may only contain Components. 
+            (Hint: maybe value here needs to be wrapped, see theano.compile.module.register_wrapper.)"""
+            raise TypeError(msg, value, type(value))
         #value = value.bind(self, item)
         value.name = name_join(self.name, str(item))
         self._components[item] = value
@@ -868,6 +805,16 @@ class ComponentDict(Composite):
 __autowrappers = []
 
 def register_wrapper(condition, wrapper):
+    """
+    :type condition: function x -> bool
+
+    :param condition: this function should return True iff `wrapper` can sensibly turn x into a
+    Component.
+
+    :type wrapper: function x -> Component
+
+    :param wrapper: this function should convert `x` into an instance of a Component subclass.
+    """
     __autowrappers.append((condition, wrapper))
 
 def wrapper(x):
@@ -881,8 +828,13 @@ def wrapper(x):
 
 def wrap(x):
     """
-    Wraps x in a Component. Wrappers can be registered using
-    register_wrapper to allow wrapping more types.
+    Wraps `x` in a `Component`. Wrappers can be registered using
+    `register_wrapper` to allow wrapping more types.
+
+    It is necessary for Module attributes to be wrappable.
+    A Module with an attribute that is not wrappable as a Component, will cause
+    `Component.make` to fail.
+
     """
     w = wrapper(x)
     if w is not None:

theano/compile/tests/test_module.py

@@ -502,6 +502,51 @@ class T_module(unittest.TestCase):
         self.assertRaises(NotImplementedError, c.set,"n",1)
 
 
+    def test_wrappable_as_tensor(self):
+        M = Module()
+        M.a = [1,2,3]
+        M.make()
+        m = M.make()
+        print m.a
+        print m.a[0], type(m.a[0]), m.a[0] == 1
+        print list(m.a)
+        assert list(m.a) == [1,2,3]
+        assert m.a is not M.a
+        try:
+            m.a = [4, 5, 6]
+            assert False
+        except Exception, e:
+            if e[0].startswith("Cannot set readonly"):
+                pass
+            else:
+                raise
+
+        try:
+            m.a[0] = 4
+            assert False
+        except Exception, e:
+            if e[0].startswith("Cannot set readonly"):
+                pass
+            else:
+                raise
+
+    def test_mixed_list(self):
+        M = Module()
+        M.a = [1,2,T.lscalar()]
+        m = M.make()
+        assert list(m.a) == [1,2,None]
+
+        assert m.a is not M.a
+        try:
+            m.a[0] = 4
+            assert False
+        except Exception, e:
+            if e[0].startswith("Cannot set readonly"):
+                pass
+            else:
+                raise
+        m.a[2] = 3
+        assert list(m.a) == [1,2,3]
 
 def test_multiple_references():
 

theano/gof/env.py

@@ -499,6 +499,9 @@ class Env(utils.object2):
     def __str__(self):
         return "[%s]" % ", ".join(graph.as_string(self.inputs, self.outputs))
 
+    def __repr__(self):
+        return self.__str__()
+
 
     ### clone ###
 

theano/sandbox/test_theano_object.py

+from theano_object import *
+
+
+RUN_TESTS = False
+def run(TF):
+    def deco(f):
+        if TF and RUN_TESTS:
+            print 'running test', f.__name__
+            f()
+        return f if RUN_TESTS else None
+    return deco
+
+
+class MyModule(TheanoObject):
+
+    def __init__(self, a=3, b=9):
+        super(MyModule, self).__init__()
+        self.a = self.symbolic_member(2)
+        self.b = self.symbolic_member(3)
+        self.c = 100 #a constant
+        self.d = [self.symbolic_member(5), self.symbolic_member(6)]
+        self.e = ['a', self.symbolic_member(6)]
+
+    @symbolic_fn
+    def add(self, x):
+        return RVal(self.a + self.b + x)
+
+    @symbolic_fn_opts(mode='FAST_COMPILE')
+    def sub(self, x):
+        outputs = (self.a - x, self.b - x)
+        updates = {self.b: self.b-x}
+        return RVal(outputs, updates)
+
+    def normal_function(self, x):
+        return self.add(x) + self.sub(x)  #use numpy addition
+
+    @symbolic_fn
+    def use_submodule(self, x):
+        return RVal(self.a + x + self.submodule.b)
+
+@run(True)
+def test_outputs():
+    MM = MyModule(3, 4)
+    assert MM.add(5) == 12
+    assert MM.b.get() == 4
+    MM.sub(3)
+    assert MM.b.get() == 1 #test get()
+    assert MM.add(5) == 9 #test that b's container is shared between add and sub
+    MM.b.set(2) #test set
+    assert MM.b.get() == 2 #test get()
+    assert MM.add(5) == 10 #test that b's container is shared between add and sub
+
+@run(True)
+def test_submodule():
+    MM = MyModule(1,2)
+    MM.submodule = MyModule(3,4)
+    assert MM.add(5) == 8
+    MM.submodule.sub(7)
+    assert MM.submodule.b.get() == -3
+    assert MM.use_submodule(0) == -2 #self.a is 1 + self.submodule.b is -3
+
+
+@run(False)
+def test_misc_prints():
+    MM = MyModule()
+    print MM
+    print 'add', MM.add(4)
+    print 'b', MM.value(MM.b)
+    print 'sub', MM.sub(45)
+    print 'b', MM.value(MM.b)
+    print MM.sub(23)
+    print MM.add(9)
+    print MM.add(19)
+    print 'b', MM.value(MM.b)
+    print 'a', MM.value(MM.a)
+    MM.value_set(MM.a,6)
+    MM.value_set(MM.b,6)
+    print MM.add(6)
+
+    try:
+        MM.b = 5
+    except Exception, e:
+        print e
+    MM.del_member(MM.b)
+    try:
+        print 'b', MM.value(MM.b)
+    except Exception, e:
+        print e
+    MM.b = 'asdffd'
+    try:
+        print 'b', MM.value(MM.b)
+    except Exception, e:
+        print e
+    try:
+        print 'b', MM.value(MM.b)
+    except Exception, e:
+        print 'E', e
+    print MM.b
+    print 'a', MM.value(MM.a)
+
+

theano/sandbox/theano_object.py

+"""DRAFT: TheanoObject
+
+N.B. the gotcha with this design is listed in the documentation of `TheanoObject`
+
+"""
+import theano
+from theano import tensor
+import numpy
+
+def theano_type(x):
+    """Return a theano Type instance suitable for containing value `x`."""
+    if type(x) is int:
+        return tensor.lscalar
+    else:
+        raise NotImplementedError()
+
+class symbolic_fn_callable(object):
+    """This is the class whose instance you get when you access a symbolic function in a
+    `TheanoObject`.
+
+    When you call a symbolic function (`symbolic_fn`) of a TheanoObject the `__call__` of this
+    class handles your request.
+
+    You can also access the symbolic outputs and updates of a symbolic function though this
+    class.
+
+    .. code-block:: python
+       
+       class T(TheanoObject):
+          @symbolic_fn
+          def add(self, x):
+             ...
+             add_outputs = ...
+             add_updates = ...
+             return RVal(add_outputs, add_updates)
+       t = T() 
+       t.add.outputs(5)         # returns `add_outputs` from when `x=theano_type(5)`
+       t.add.updates(5)         # returns `add_updates` from when `x=theano_type(5)`
+       t.add.theano_function(5) # returns the `Function` compiled when `x=theano_type(5)`
+       t.add(5)                 # runs the `Function` compiled when `x=theano_type(5)`
+                                #     with arguments `(5,)`
+    """
+    def __init__(self, fn, mode):
+        self.fn = fn
+        self.mode = mode
+
+    def on(self, o_self):
+        """Silly method to work with symbolic_fn.__get__"""
+        self.o_self = o_self
+        return self
+    
+    def run_symbolic(self, *args, **kwargs):
+        return self.o_self._get_method_impl(self.fn, self.o_self, args, kwargs, mode=self.mode)
+
+    def __call__(self, *args, **kwargs):
+        return self.run_symbolic(*args, **kwargs)['theano_function'](*args, **kwargs)
+
+    def theano_function(self, *args, **kwargs):
+        return self.run_symbolic(*args, **kwargs)['theano_function']
+
+    def outputs(self, *args, **kwargs):
+        return self.run_symbolic(*args, **kwargs)['outputs']
+
+    def updates(self, *args, **kwargs):
+        return self.run_symbolic(*args, **kwargs)['updates']
+
+class symbolic_fn(object):
+    """A property-like class for decorating symbolic functions in `TheanoObject`
+    """
+    def __init__(self, fn, mode=None):
+        self.fn = fn
+        self.callable = symbolic_fn_callable(fn, mode)
+    
+    def __get__(self, o_self, o_cls):
+        return self.callable.on(o_self)
+
+    def __set__(self, o_self, new_val):
+        pass
+        #return NotImplemented
+
+def symbolic_fn_opts(**kwargs):
+    """Return a decorator for symbolic_functions in a `TheanoObject`
+
+    `kwargs` passed here are passed to `theano.function` via `symbolic_fn`
+    """
+    def deco(f):
+        return symbolic_fn(f, **kwargs)
+    return deco
+
+class RVal(object):
+    """A Return-Value object for a `symbolic_fn` """
+    outputs = []
+    """The method will compute values for the variables in this list"""
+    
+    updates = {}
+    """The method will update module variables in this dictionary
+
+    For items ``(k,v)`` in this dictionary, ``k`` must be a `symbolic_member` of some module.
+    On each call to this compiled function, the value of ``k`` will be replaced with the
+    computed value of the Variable ``v``.
+    
+    """
+    def __init__(self, outputs, updates={}):
+        self.outputs = outputs
+        assert type(updates) is dict
+        self.updates = updates
+
+class TheanoObject(object):
+    """Base for Theano-supported classes
+
+    This class provides support for symbolic_fn class attributes.
+    These will be compiled on demand so that they can be used just like normal (non-symbolic)
+    methods.
+    
+    The symbolic functions in a TheanoObject can share member variables that have been created
+    using the `symbolic_member` method.
+
+    :note: Other variables (ones not created using ``self.symbolic_member``) referred to in the
+    body of a symbolic function will *not* be shared between symbolic functions, or between
+    symbolic functions and this class.  These other variables will be locked away in the
+    closure of a symbolic function when that function is compiled.  
+    
+
+    :warning: It is not recommended for code to interleave
+    (a) changes to non-symbolic instance variables with
+    (b) calls to symbolic functions that use those instance variables. 
+    A symbolic function may be
+    compiled multiple times because it must be compiled for each set of argument types.
+    Each time the function is compiled, the values of non-symbolic variables will be locked
+    into the compiled function.  Subsequent changes to those non-symbolic instance variables
+    will not have any effect on the behaviour of the already-compiled symbolic function.
+
+    :todo: Is there an efficient way of recognizing when a compiled symbolic function is stale,
+    wrt the current values of the class's instance variables?
+
+    - One option is to re-evaluate symbolic functions symbolically and see if the graph can be
+      completely merged with the original graph.  This is not fast enough to do all the time by
+      default though.
+
+    """
+    def __init__(self):
+        self.module_method_cache = {}
+
+    def _get_method_impl(self, fn, o_self, args, kwargs, mode):
+        """Retrieve information about the symbolic function (`fn`) in TheanoObject instance
+        `o_self`, being evaluated on arguments `args` and `kwargs`.
+
+        :rtype: dict with entries 'theano_function', 'outputs', 'updates'
+
+        :return: the theano function compiled for these arguments, the symbolic outputs of that
+        function, and the symbolic updates performed by that function.
+
+        :note: This function caches return values in self.`module_method_cache`.
+
+        :todo: This may at some point become a class-level cache rather than an instance-level
+        cache.
+
+        """
+        if kwargs:
+            raise NotImplementedError()
+
+        cache = self.module_method_cache
+
+        args_types = tuple(theano_type(arg) for arg in args)
+        key = (fn, args_types)
+
+        if key not in cache:
+            inputs = [a() for a in args_types]
+            print 'compiling', fn, 'for inputs', inputs
+            rval = fn(o_self, *inputs)
+
+            print 'compiling to compute outputs', rval.outputs
+
+            if isinstance(rval.outputs, (tuple, list)):
+                all_required_inputs = theano.gof.graph.inputs(rval.outputs)
+            else:
+                all_required_inputs = theano.gof.graph.inputs([rval.outputs])
+
+            # construct In instances for the symbolic_member instances that can automatically be
+            # included here.
+            module_inputs = [theano.compile.io.In(
+                    variable=v, 
+                    value=v._theanoclass_container,
+                    mutable=(v in rval.updates),
+                    update=rval.updates.get(v, None))
+                for v in all_required_inputs \
+                        if hasattr(v, '_theanoclass_container') and not (v in inputs)]
+
+            cache[key] = dict(theano_function=theano.function(inputs+module_inputs, rval.outputs),
+                    updates=rval.updates,
+                    outputs=rval.outputs,
+                    mode=mode)
+
+        return cache[key]
+
+    def symbolic_member(self, ival, name=None):
+        """Create a Variable instance to hold value `ival`.
+
+        This function also immediately creates a Container object for ival.
+        When the returned Variable is used as input to a `TheanoObject` `symbolic_fn`, (but
+        does not appear as an argument to that symbolic_fn), then this Container will be used to
+        retrieve (and store) values for the Variable.
+
+        This Variable's Container's contents can be retrieved by its `get()` method.
+        This Variable's Container's contents can be written using its `set(newval)` method.
+
+        """
+        if type(ival) is not int:
+            raise NotImplementedError()
+
+        v = tensor.lscalar(name)
+        v._theanoclass_container = \
+                theano.gof.Container(v, 
+                        storage = [numpy.asarray(ival, dtype='int64')],
+                        readonly=False)
+        assert not hasattr(v, 'set')
+        assert not hasattr(v, 'get')
+        v.get = lambda : v._theanoclass_container.data
+        def setval_in_v(newval):
+            v._theanoclass_container.data = newval
+        v.set = setval_in_v
+        return v
+
+
+        
+

theano/scalar/basic.py

@@ -766,7 +766,7 @@ tan = Tan(upgrade_to_float, name = 'tan')
 
 class Cosh(UnaryScalarOp):
     """
-    sinh(x) = (exp(x) + exp(-x)) / 2
+    cosh(x) = (exp(x) + exp(-x)) / 2
     """
     def impl(self, x):
         return math.cosh(x)

theano/sparse/tests/test_basic.py

@@ -11,7 +11,7 @@ from theano import gof
 
 from theano.sparse.basic import _is_dense, _is_sparse, _is_dense_variable, _is_sparse_variable
 from theano.sparse.basic import _mtypes, _mtype_to_str
-from theano.tests import unittest_tools
+from theano.tests import unittest_tools as utt
 
 
 def eval_outputs(outputs):
@@ -19,7 +19,7 @@ def eval_outputs(outputs):
 
 class T_transpose(unittest.TestCase):
     def setUp(self):
-        unittest_tools.seed_rng()
+        utt.seed_rng()
 
     def test_transpose_csc(self):
         sp = sparse.csc_matrix(sparse.eye(5,3))
@@ -126,7 +126,7 @@ class T_Add(unittest.TestCase):
 
 class T_conversion(unittest.TestCase):
     def setUp(self):
-        unittest_tools.seed_rng()
+        utt.seed_rng()
 
     def test0(self):
         a = tensor.as_tensor_variable(numpy.random.rand(5))
@@ -157,7 +157,7 @@ class T_conversion(unittest.TestCase):
 import scipy.sparse as sp
 class test_structureddot(unittest.TestCase):
     def setUp(self):
-        unittest_tools.seed_rng()
+        utt.seed_rng()
 
     def test_structuredot(self):
         bsize = 2
@@ -193,7 +193,7 @@ class test_structureddot(unittest.TestCase):
             assert _is_dense(c)
             assert numpy.all(outvals == c)
 
-            tensor.verify_grad(buildgraphCSC, [kernvals,imvals])
+            utt.verify_grad(buildgraphCSC, [kernvals,imvals])
 
             ##
             # Test compressed-sparse row matrices ###
@@ -215,7 +215,7 @@ class test_structureddot(unittest.TestCase):
             assert _is_dense(c)
             assert numpy.all(outvals == c)
 
-            tensor.verify_grad( buildgraphCSR, [kernvals,imvals])
+            utt.verify_grad( buildgraphCSR, [kernvals,imvals])
 
 
 if __name__ == '__main__':

theano/tensor/basic.py

@@ -20,7 +20,6 @@ from ..gof.python25 import partial
 
 from .. import compile, printing
 from ..printing import pprint, Print
-from ..tests import unittest_tools
 
 ### set up the external interface
 from elemwise import Elemwise, DimShuffle, CAReduce, Sum
@@ -159,6 +158,15 @@ def constant(x, name=None, ndim=None):
 def value(x, name=None, ndim=None):
     return constant_or_value(x, rtype=TensorValue, name=name, ndim=ndim)
 
+def _obj_is_wrappable_as_tensor(x):
+    try:
+        constant(x)
+        return True
+    except TypeError:
+        return False
+def _wrap_tensor_into_member(x):
+    return compile.module.Member(constant(x))
+compile.module.register_wrapper(_obj_is_wrappable_as_tensor, _wrap_tensor_into_member)
 
 
 class TensorType(Type):
@@ -250,11 +258,16 @@ class TensorType(Type):
         if type(a) is numpy.ndarray and type(b) is numpy.ndarray:
             if a.shape != b.shape:
                 return False
-            if a.shape == ():
-                ones = numpy.ones(2)
-                return numpy.allclose(ones * a, ones*b)
+            if a.dtype != b.dtype:
+                return False
+            if 'int' in str(a.dtype):
+                return numpy.all(a==b)
             else:
-                return numpy.allclose(a,b)
+                if a.shape == (): #for comparing scalars, use broadcasting.
+                    ones = numpy.ones(2)
+                    return numpy.allclose(ones * a, ones*b)
+                else:
+                    return numpy.allclose(a,b)
         return False
 
     def __hash__(self):
@@ -924,7 +937,8 @@ def argmax(x, axis=None):
 
 @constructor
 def min(x, axis=None):
-    if 'float'in str(x.dtype):
+    str_x_type = str(x.dtype)
+    if str_x_type.startswith('float') or str_x_type.startswith('int'):
         return -max(-x, axis=axis)
     else:
         #Be careful about unsigned integers, complex
@@ -932,7 +946,8 @@ def min(x, axis=None):
 
 @constructor
 def argmin(x, axis=None):
-    if 'float'in str(x.dtype):
+    str_x_type = str(x.dtype)
+    if str_x_type.startswith('float') or str_x_type.startswith('int'):
         return argmax(-x, axis=axis)
     else:
         #Be careful about unsigned integers, complex

theano/tensor/tests/test_nnet.py

@@ -5,86 +5,86 @@ from theano import tensor as T
 from theano import gof
 import test_basic as TT
 import numpy
-from theano.tests import unittest_tools
+from theano.tests import unittest_tools as utt
 
 from theano.tensor.nnet import *
 
 
 class T_sigmoid(unittest.TestCase):
     def setUp(self):
-        unittest_tools.seed_rng()
+        utt.seed_rng()
     def test_elemwise(self):
-        TT.verify_grad(sigmoid, [numpy.random.rand(3,4)])
+        utt.verify_grad(sigmoid, [numpy.random.rand(3,4)])
 
 class T_softplus(unittest.TestCase):
     def setUp(self):
-        unittest_tools.seed_rng()
+        utt.seed_rng()
     def test_elemwise(self):
-        TT.verify_grad(softplus, [numpy.random.rand(3,4)])
+        utt.verify_grad(softplus, [numpy.random.rand(3,4)])
 
 class T_Softmax(unittest.TestCase):
     def setUp(self):
-        unittest_tools.seed_rng()
+        utt.seed_rng()
     def test0(self):
         def f(a):
             return softmax(a)[:,0]
-        TT.verify_grad(f, [numpy.random.rand(3,4)])
+        utt.verify_grad(f, [numpy.random.rand(3,4)])
     def test1(self):
         def f(a):
             return softmax(a)[:,1]
-        TT.verify_grad(f, [numpy.random.rand(3,4)])
+        utt.verify_grad(f, [numpy.random.rand(3,4)])
     def test2(self):
         def f(a):
             return softmax(a)[:,2]
-        TT.verify_grad(f, [numpy.random.rand(3,4)])
+        utt.verify_grad(f, [numpy.random.rand(3,4)])
     def test3(self):
         def f(a):
             return softmax(a)[:,3]
-        TT.verify_grad(f, [numpy.random.rand(3,4)])
+        utt.verify_grad(f, [numpy.random.rand(3,4)])
 
 
 class T_SoftmaxWithBias(unittest.TestCase):
     def setUp(self):
-        unittest_tools.seed_rng()
+        utt.seed_rng()
     def test0(self):
         def f(a, b):
             return softmax_with_bias(a, b)[:,0]
-        TT.verify_grad(f, [numpy.random.rand(3,4),
+        utt.verify_grad(f, [numpy.random.rand(3,4),
             numpy.random.rand(4)])
     def test1(self):
         def f(a, b):
             return softmax_with_bias(a, b)[:,1]
-        TT.verify_grad(f, [numpy.random.rand(3,4),
+        utt.verify_grad(f, [numpy.random.rand(3,4),
             numpy.random.rand(4)])
     def test2(self):
         def f(a, b):
             return softmax_with_bias(a, b)[:,2]
-        TT.verify_grad(f, [numpy.random.rand(3,4),
+        utt.verify_grad(f, [numpy.random.rand(3,4),
             numpy.random.rand(4)])
     def test3(self):
         def f(a, b):
             return softmax_with_bias(a, b)[:,3]
-        TT.verify_grad(f, [numpy.random.rand(3,4),
+        utt.verify_grad(f, [numpy.random.rand(3,4),
             numpy.random.rand(4)])
 
 class T_CrossentropySoftmax1Hot(unittest.TestCase):
     def setUp(self):
-        unittest_tools.seed_rng()
+        utt.seed_rng()
     def test0(self):
         y_idx = [0,1,3]
         def f(a, b):
             return crossentropy_softmax_1hot_with_bias(a, b, y_idx)[0]
-        TT.verify_grad(f, [numpy.random.rand(3,4),
+        utt.verify_grad(f, [numpy.random.rand(3,4),
             numpy.random.rand(4)])
     def test1(self):
         y_idx = [0,1,3]
         def f(a):
             return crossentropy_softmax_1hot(a, y_idx)[0]
-        TT.verify_grad(f, [numpy.random.rand(3,4)])
+        utt.verify_grad(f, [numpy.random.rand(3,4)])
 
 class T_prepend(unittest.TestCase):
     def setUp(self):
-        unittest_tools.seed_rng()
+        utt.seed_rng()
     def test0(self):
         """basic functionality"""
         x=tensor.matrix('x')
@@ -110,7 +110,7 @@ class T_prepend(unittest.TestCase):
 
 class T_solve(unittest.TestCase):
     def setUp(self):
-        self.rng = numpy.random.RandomState(unittest_tools.fetch_seed(666))
+        self.rng = numpy.random.RandomState(utt.fetch_seed(666))
 
     def test0(self):
         A=self.rng.randn(5,5)

theano/tensor/tests/test_xlogx.py

@@ -8,11 +8,11 @@ import test_basic as TT
 
 import random
 import numpy.random
-from theano.tests import unittest_tools
+from theano.tests import unittest_tools as utt
 
 class T_XlogX(unittest.TestCase):
     def setUp(self):
-        unittest_tools.seed_rng()
+        utt.seed_rng()
 
     def test0(self):
         x = as_tensor_variable([1, 0])
@@ -23,7 +23,7 @@ class T_XlogX(unittest.TestCase):
 #        class Dummy(object):
 #            def make_node(self, a):
 #                return [xlogx(a)[:,2]]
-        TT.verify_grad(xlogx, [numpy.random.rand(3,4)])
+        utt.verify_grad(xlogx, [numpy.random.rand(3,4)])
 
 
 if __name__ == '__main__':

theano/tests/unittest_tools.py

 import unittest
 import numpy
+import theano.tensor as T
 
 import os, sys
 
@@ -40,3 +41,14 @@ def seed_rng(pseed=None):
                 'instead of seed %i given as parameter' % (seed, pseed)
     numpy.random.seed(seed)
     return seed
+
+def verify_grad(op, pt, n_tests=2, rng=None, eps=1.0e-7, tol=0.0001):
+    """
+    Wrapper for tensor/basic.py:verify_grad
+    Takes care of seeding the random number generator if None is given
+    """
+    if rng is None:
+        seed_rng()
+        rng = numpy.random
+    T.verify_grad(op, pt, n_tests, rng, eps, tol)
+