merge

doc/advanced_tutorial/ex1/op.txt

@@ -7,24 +7,6 @@ Now that we have a ``double`` type, we have yet to use it to perform
 computations. We'll start by defining multiplication.
 
 
-What is an Op?
-==============
-
-An :ref:`op` in Theano defines a certain computation on some types of
-inputs, producing some types of outputs. It is equivalent to a
-function definition in most programming languages. From a list of
-input :ref:`Results <result>` and an Op, you can build an :ref:`apply`
-node representing the application of the Op to the inputs.
-
-It is important to understand the distinction between an Op (the
-definition of a function) and an Apply node (the application of a
-function). If you were to interpret the Python language using Theano's
-structures, code going like ``def f(x): ...`` would produce an Op for
-``f`` whereas code like ``a = f(x)`` or ``g(f(4), 5)`` would produce an
-Apply node involving the ``f`` Op.
-
-
-
 Op's contract
 =============
 
@@ -33,10 +15,10 @@ An Op is any object which defines the following methods:
 - **make_node(*inputs)**
     
   - This method is responsible for creating output Results of a suitable Type
-    to serve as the outputs of this Op's application.  It should put these
+    to serve as the outputs of this Op's application.  This method should put these
     outputs into an Apply instance, and return the Apply instance.
   
-  - This important function creates an Apply node representing the
+  - This method creates an Apply node representing the
     application of the Op on the inputs provided. If the Op cannot be
     applied on these inputs, it must raise an appropriate
     exception.
@@ -45,6 +27,15 @@ An Op is any object which defines the following methods:
     correctly: a subsequent ``self.make_node(*apply.inputs)`` must produce
     something equivalent to the first ``apply``.
 
+- default_output
+
+  - *Default*: None
+
+  - If this member variable is an integer, then the default implementation of
+    ``__call__`` will return `node.outputs[self.default_output]``, where
+    `node` was returned by ``make_node``.  Otherwise, the entire list of
+    outputs will be returned.
+
 - **__call__(*inputs)**
 
   - Syntactic shortcut to make_node which returns the output Results
@@ -54,53 +45,80 @@ An Op is any object which defines the following methods:
 
 - **perform(node, inputs, output_storage)**
 
-  - This function computes the function associated to this Op. The
+  - This method computes the function associated to this Op. The
     ``node`` is an Apply node created by the Op's ``make_node``
-    method, the inputs are a list of references to data to operate on
-    and output_storage is a list of storage cells where the results of
-    the computation must be put.
-  
-  - This function must be determined by the inputs.  That is to say, if it is
+    method, ``inputs`` is a list of references to data to operate on,
+    and ``output_storage`` is a list of storage cells where the results of
+    the computation must be put. More specifically:
+
+    - ``node``: This is a reference to an Apply node which was previously
+      obtained via ``mul``'s ``make_node`` method. It is typically not
+      used in simple Ops, but it contains symbolic information that
+      could be required for complex Ops.
+
+    - ``inputs``: This is a list of data.
+
+    - ``output_storage``: This is a list of storage cells.
+      A storage cell is a one-element list. It is forbidden to change
+      the length of the list(s) contained in output_storage.  There is
+      one storage cell for each output of the Op.
+
+      The data you put in ``output_storage`` must match the type of the
+      symbolic output. This is a situation where the ``node`` argument
+      can come in handy.
+
+      A function Mode may allow output_storage elements to persist between
+      evaluations, or it may reset output_storage cells to hold a value of
+      None.  This feature can allow perform to reuse memory between calls, for
+      example.
+
+  - This method must be determined by the inputs. That is to say, if it is
     evaluated once on inputs A and returned B, then if ever inputs C, equal to
     A, are presented again, then outputs equal to B must be returned again.
   
   - You must be careful about aliasing outputs to inputs, and making
-    modifications to any of the inputs.  See `Views and inplace operations`_
+    modifications to any of the inputs. See `Views and inplace operations`_
     before writing a ``perform`` implementation that does either of these
     things.
 
 - **__eq__(self, other)**
 
-  - Returning True here is a promise to the optimization system that the other
+  - ``other`` is also an Op.
+
+  - Returning ``True`` here is a promise to the optimization system that the other
     Op will produce exactly the same graph effects (from perform) as this one,
-    given identical inputs.  This means it will produce the same output values,
+    given identical inputs. This means it will produce the same output values,
     it will destroy the same inputs (same destroy_map), and will alias outputs
     to the same inputs (same view_map).
 
 - **__hash__(self)**
 
-  - If two Op instances compare equal, then they MUST return the same hash
-    value.  
+  - If two Op instances compare equal, then they **must** return the same hash
+    value.
       
   - Equally important, this hash value must not change during the lifetime of
     self.
 
 - **__ne__(self, other)**
 
-  - Recommended, and default: define as (not (self==other))
+  - Recommended
+  
+  - Default: ``(not (self==other))``
 
-- **grad(inputs, output_gradients)** *Optional*
+- **grad(inputs, output_gradients)**
+
+  - Optional.
 
   - If the Op you are defining is differentiable, you can define its
     gradient symbolically in this method. 
     
-  - Both the inputs and output_gradients will be Results. This function must
-    return a list containing one Result (or None) for each input.
-    Each returned Result represents the gradient wrt that input given the
-    symbolic gradients wrt each output.
+  - Both the ``inputs`` and ``output_gradients`` will be Results. This
+    method must return a list containing one Result (or None) for each
+    input. Each returned Result represents the gradient with respect to
+    that input given the symbolic gradients with respect to each output.
 
   - If the output is not differentiable with respect to any inputs, then this
-    function should be defined to return [None for i in inputs].
+    method should be defined to return [None for i in inputs].
 
   - If this method is not defined, then theano assumes it has been forgotten.
     Symbolic differentiation will fail on a graph that includes this Op.
@@ -108,7 +126,9 @@ An Op is any object which defines the following methods:
   - For more information on the use of this method, see ``grad``.
 
 
-For each method, the *default* is what the Op class defines for you.
+For each method, the *default* is what :api:`theano.gof.Op` defines
+for you.  At a bare minimum, a new Op must define ``make_node`` and
+``perform``, which have no defaults.
 
 For more details, including the interface for providing a C implementation of
 perform(), refer to the documentation for :ref:`op`.
@@ -154,9 +174,10 @@ Use this list to make sure that you defined everything you need for your Op:
 Defining mul
 ============
 
-We are going to redefine the two functions that are absolutely
-necessary to redefine: ``make_node`` and ``perform``. First, we'll
-instantiate a ``mul`` Op:
+We'll define multiplication as a *binary* operation, even though a
+multiplication Op could take an arbitrary number of arguments.
+
+First, we'll instantiate a ``mul`` Op:
 
 .. code-block:: python
 
@@ -167,12 +188,13 @@ instantiate a ``mul`` Op:
 **make_node**
 
 This function must take as many arguments as the operation we are
-defining is supposed to take as inputs - in this example that would be
-two (we'll define multiplication as a binary operation here, even
-though a multiplication Op could technically take an arbitrary number
-of arguments). It should ensure that both inputs have the ``double``
-type and it should make an Apply node with an output Result of type
-``double`` (since multiplying two doubles yields a double).
+defining is supposed to take as inputs---in this example that would be
+two.
+This function ensures that both inputs have the ``double``
+type.
+Since multiplying two doubles yields a double,
+this function makes an Apply node with an output Result of type
+``double``.
 
 .. code-block:: python
 
@@ -189,14 +211,8 @@ want to multiply two arbitrary types, it would not make much sense
 (and we'd be screwed when we implement this in C!)
 
 The last line is the meat of the definition. There we create an Apply
-node representing the application of ``mul`` to ``x`` and ``y``. Apply
-takes three arguments: the first one is the Op we are applying. In
-this case, we are applying ``mul``. The second argument is a list of
-input Results - here, ``x`` and ``y``. The third is a list of output
-Results. Since the multiplication of two doubles ought to give us a
-double again, we create a Result of type ``double`` and we place it in
-a list. Since the list only has one element, ``mul`` only has one
-output.
+node representing the application of Op ``mul`` to inputs ``x`` and
+``y``, giving a Result instance of type ``double`` as the output.
 
 .. note::
    Theano relies on the fact that if you call the ``make_node`` method
@@ -208,25 +224,11 @@ output.
 
 **perform**
 
-This code should actually compute the function. It is important to
-understand the role of all three arguments of ``perform``:
-
-- *node*: This is a reference to an Apply node which was previously
-  obtained via ``mul``'s ``make_node`` method. It is not typically
-  useful, but it contains symbolic information that could be required
-  for complex Ops.
-
-- *inputs*: This is a list of data. In this example, the data in
-  ``inputs`` will be instances of Python's built-in type ``float``
-  because this is the type that ``double.filter()`` will always
-  return, per our own definition.
-
-- *output_storage*: This is a list of storage cells. There is one
-  storage cell for each output of the Op. A storage cell is
-  a one-element list (note: it is forbidden to change the
-  length of the list(s) contained in output_storage). In this example,
-  output_storage will contain a single storage cell for the
-  multiplication's result.
+This code actually computes the function.
+In our example, the data in ``inputs`` will be instances of Python's
+built-in type ``float`` because this is the type that ``double.filter()``
+will always return, per our own definition. ``output_storage`` will
+contain a single storage cell for the multiplication's result.
 
 .. code-block:: python
 
@@ -248,12 +250,9 @@ Here, ``z`` is a list of one element. By default, ``z == [None]``.
    :ref:`op` documentation.
 
 .. warning::
-   The data you put in ``output_storage`` must match the type of the
-   symbolic output. This is a situation where the ``node`` argument
-   can come in handy. In this example, we gave ``z`` the Theano type
-   ``double`` in ``make_node``, which means that a Python ``float``
-   must be put there. You should not put, say, an ``int`` in ``z[0]``
-   because Theano assumes Ops handle typing properly.
+   We gave ``z`` the Theano type ``double`` in ``make_node``, which means
+   that a Python ``float`` must be put there. You should not put, say, an
+   ``int`` in ``z[0]`` because Theano assumes Ops handle typing properly.
 
 
 Trying out our new Op
@@ -298,7 +297,7 @@ by modifying ``make_node`` to accept Python ``int`` or ``float`` as
    mul.make_node = make_node
 
 Whenever we pass a Python int or float instead of a Result as ``x`` or
-``y``, make_node will convert it to :ref:`constant` for us. ``gof.Constant``
+``y``, ``make_node`` will convert it to :ref:`constant` for us. ``gof.Constant``
 is a :ref:`result` we statically know the value of.
 
 >>> x = double('x')

doc/advanced_tutorial/ex1/type.txt

@@ -5,43 +5,6 @@ Making the double type
 ======================
 
 
-What is a Type?
-===============
-
-A :ref:`type` in Theano represents a set of constraints on potential
-data objects. These constraints allow Theano to tailor C code to handle
-them and to statically optimize the computation graph. For instance,
-the :ref:`irow <predefinedtypes>` type in the ``theano.tensor`` package
-gives the following constraints on the data the Results of type ``irow``
-may contain:
-
-#. Must be an instance of ``numpy.ndarray``: ``isinstance(x, numpy.ndarray)``
-#. Must be an array of 32-bit integers: ``str(x.dtype) == 'int32'``
-#. Must have a shape of 1xN: ``len(x.shape) == 2 and x.shape[0] == 1``
-
-Knowing these restrictions, Theano may generate C code for addition, etc.
-that declares the right data types and that contains the right number
-of loops over the dimensions.
-
-Note that a Theano :ref:`type` is not equivalent to a Python type or
-class. Indeed, in Theano, :ref:`irow <predefinedtypes>` and :ref:`dmatrix
-<predefinedtypes>` both use ``numpy.ndarray`` as the underlying type
-for doing computations and storing data, yet they are different Theano
-Types. Indeed, the constraints set by ``dmatrix`` are:
-
-#. Must be an instance of ``numpy.ndarray``: ``isinstance(x, numpy.ndarray)``
-#. Must be an array of 64-bit floating point numbers: ``str(x.dtype) == 'float64'``
-#. Must have a shape of MxN, no restriction on M or N: ``len(x.shape) == 2``
-
-These restrictions are different from those of ``irow`` which are listed above.
-
-There are cases in which a Type can fully correspond to a Python type,
-such as the ``double`` Type we will define here which corresponds to
-Python's ``float``. But, it's good to know that this is not necessarily
-the case. Unless specified otherwise, when we say "Type" we mean a
-Theano Type.
-
-
 Type's contract
 ===============
 

doc/advanced_tutorial/graphstructures.txt

@@ -169,6 +169,10 @@ theano's version of a function definition.
 
 An Apply instance has three important fields:
 
+**op**
+  An :ref:`op` that determines the function/transformation being
+  applied here.
+
 **inputs**
   A list of :ref:`Results <result>` that represent the arguments of
   the function.
@@ -177,33 +181,8 @@ An Apply instance has three important fields:
   A list of :ref:`Results <result>` that represent the return values
   of the function.
 
-**op**
-  An :ref:`op` that determines the function/transformation being
-  applied here.
-
-
-.. index::
-   single: Constant
-   single: graph construct; Constant
-
-.. _constant:
-
---------
-Constant
---------
-
-A constant is a :ref:`Result` with one extra field, *data* (only
-settable once). When used in a computation graph as the input of an
-:ref:`Op` :ref:`application <Apply>`, it is assumed that said input
-will *always* take the value contained in the constant's data
-field. Furthermore, it is assumed that the :ref:`Op` will not under
-any circumstances modify the input. This means that a constant is
-eligible to participate in numerous optimizations: constant inlining
-in C code, constant folding, etc.
-
-A constant does not need to be specified in a :ref:`function`'s list
-of inputs.
-
+An Apply instance can be created by calling ``gof.Apply(op, inputs,
+outputs)``.
 
 
 .. index::
@@ -212,6 +191,8 @@ of inputs.
 
 .. _result:
 
+
+
 ------
 Result
 ------
@@ -263,6 +244,28 @@ A Result ``r`` contains four important fields:
 
 Result has one special subclass: :ref:`constant <constant>`.
 
+.. index::
+   single: Constant
+   single: graph construct; Constant
+
+.. _constant:
+
+
+
+Constant
+^^^^^^^^
+
+A constant is a :ref:`Result` with one extra field, *data* (only
+settable once). When used in a computation graph as the input of an
+:ref:`Op` :ref:`application <Apply>`, it is assumed that said input
+will *always* take the value contained in the constant's data
+field. Furthermore, it is assumed that the :ref:`Op` will not under
+any circumstances modify the input. This means that a constant is
+eligible to participate in numerous optimizations: constant inlining
+in C code, constant folding, etc.
+
+A constant does not need to be specified in a :ref:`function`'s list
+of inputs.
 
 
 .. index::
@@ -275,7 +278,20 @@ Result has one special subclass: :ref:`constant <constant>`.
 Op
 --
 
-WRITEME
+An :ref:`op` in Theano defines a certain computation on some types of
+inputs, producing some types of outputs. It is equivalent to a
+function definition in most programming languages. From a list of
+input :ref:`Results <result>` and an Op, you can build an :ref:`apply`
+node representing the application of the Op to the inputs.
+
+It is important to understand the distinction between an Op (the
+definition of a function) and an Apply node (the application of a
+function). If you were to interpret the Python language using Theano's
+structures, code going like ``def f(x): ...`` would produce an Op for
+``f`` whereas code like ``a = f(x)`` or ``g(f(4), 5)`` would produce an
+Apply node involving the ``f`` Op.
+
+
 
 
 
@@ -289,5 +305,37 @@ WRITEME
 Type
 ----
 
-WRITEME
+A :ref:`type` in Theano represents a set of constraints on potential
+data objects. These constraints allow Theano to tailor C code to handle
+them and to statically optimize the computation graph. For instance,
+the :ref:`irow <predefinedtypes>` type in the ``theano.tensor`` package
+gives the following constraints on the data the Results of type ``irow``
+may contain:
+
+#. Must be an instance of ``numpy.ndarray``: ``isinstance(x, numpy.ndarray)``
+#. Must be an array of 32-bit integers: ``str(x.dtype) == 'int32'``
+#. Must have a shape of 1xN: ``len(x.shape) == 2 and x.shape[0] == 1``
+
+Knowing these restrictions, Theano may generate C code for addition, etc.
+that declares the right data types and that contains the right number
+of loops over the dimensions.
+
+Note that a Theano :ref:`type` is not equivalent to a Python type or
+class. Indeed, in Theano, :ref:`irow <predefinedtypes>` and :ref:`dmatrix
+<predefinedtypes>` both use ``numpy.ndarray`` as the underlying type
+for doing computations and storing data, yet they are different Theano
+Types. Indeed, the constraints set by ``dmatrix`` are:
+
+#. Must be an instance of ``numpy.ndarray``: ``isinstance(x, numpy.ndarray)``
+#. Must be an array of 64-bit floating point numbers: ``str(x.dtype) == 'float64'``
+#. Must have a shape of MxN, no restriction on M or N: ``len(x.shape) == 2``
+
+These restrictions are different from those of ``irow`` which are listed above.
+
+There are cases in which a Type can fully correspond to a Python type,
+such as the ``double`` Type we will define here which corresponds to
+Python's ``float``. But, it's good to know that this is not necessarily
+the case. Unless specified otherwise, when we say "Type" we mean a
+Theano Type.
+
 

doc/advanced_tutorial/theano_vs_python.txt

@@ -12,7 +12,7 @@ analogue in Python:
 Theano          Python
 =============== ===========================================================
 Apply           function application / function call
-Result          function data
+Result          function data / variable
 Op              operations carried out in computation / function definition
 Type            data types
 Module          ??? class?

doc/basic_tutorial/function.txt

+
+.. _function:
+
+===============
+theano.function
+===============
+
+
+This page is about ``theano.function``, the interface for compiling graphs into callable objects.
+
+The signature for this function is:
+
+.. code-block:: python
+
+    def function(inputs, outputs, mode='FAST_RUN'):
+        ...
+
+You've already seen example usage in the basic tutorial... something like this:
+
+>>> x = theano.tensor.dscalar()
+>>> f = theano.function([x], 2*x)
+>>> print f(4)                    # prints 8.0
+
+The idea here is that we've compiled the symbolic graph (``2*x``) into a function that can be called on a number and will do some computations.
+
+Inputs
+======
+
+The ``inputs`` argument to ``theano.function`` is a list, containing the ``Result`` instances for which values will be specified at the time of the function call.  But inputs can be more than just Results.  
+``In`` instances let us attach properties to ``Results`` to tell function more about how to use them.
+
+**In(result, name=None, value=None, update=None, mutable=False)** returns an ``In`` instance:
+
+  - ``result``: a Result instance. 
+
+    This will be assigned a value before running the function,
+    not computed from its owner.
+  
+  - ``name``: Any type. (If autoname_input=True, defaults to result.name). 
+
+    If name is a valid Python identifier, this input can be set by
+    ``kwarg``, and its value can be accessed by ``self.<name>``.
+
+    Default: ``None``
+     
+  - ``value``: literal or Container
+
+    This is the default value of the Input.
+
+    Default: ``None``
+
+  - ``update``: Result instance
+
+    This expression Result will replace ``value`` after each function call.
+
+    Default: ``None``
+
+  - ``mutable``: Bool (requires value)
+
+    If ``True``, permit the compiled function to modify the python object being used as the default value.
+
+    Default: ``False``
+
+  - ``autoname``: Bool
+
+    ``True``: if ``name`` is None and the Result has a name, it will be taken
+    as the input's name.
+
+    ``False``: the name is the exact value passed as the name parameter
+    (possibly ``None``).
+
+    Default: ???
+
+Value: initial and default values
+---------------------------------
+
+A non-None `value` argument makes an In() instance an optional parameter
+of the compiled function.  For example, in the following code we are
+defining an arity-2 function ``inc``.
+
+>>> u, x, s = T.scalars('uxs')
+>>> inc = function([u, In(x, value=3), In(s, update=(s+x*u), value=10.0)], [])
+
+Since we provided a ``value`` for ``s`` and ``x``, we can call it with just a value for ``u`` like this:
+
+>>> inc(5)         # update s with 10+3*5
+[]
+>>> print inc[s]
+25.0
+
+The effect of this call is to increment the storage associated to ``s`` in ``inc`` by 15.
+
+If we pass two arguments to ``inc``, then we override the value associated to
+``x``, but only for this one function call.
+
+>>> inc(3, 4)      # update s with 25 + 3*4
+[]
+>>> print inc[s]
+37.0
+>>> print inc[x]   # the override value of 4 was only temporary
+3.0
+
+If we pass three arguments to ``inc``, then we override the value associated 
+with ``x`` and ``u`` and ``s``.
+Since ``s``'s value is updated on every call, the old value of ``s`` will be ignored and then replaced.
+
+>>> inc(3, 4, 7)      # update s with 7 + 3*4
+[]
+>>> print inc[s]
+19.0
+
+We can also assign to ``inc[s]`` directly:
+
+>>> inc[s] = 10
+>>> inc[s]
+array(10.0)
+
+
+
+Advanced: Sharing Storage Between Functions
+-------------------------------------------
+
+``value`` can be a :api:`theano.gof.Container` as well as a literal.
+This permits linking a value of a Result in one function to the value of a Result in another function.
+By using a ``Container`` as a value we can implement shared variables between functions.
+
+For example, consider the following program.
+
+>>> x, s = T.scalars('xs')
+>>> inc = function([x, In(s, update=(s+x), value=10.0)], [])
+>>> dec = function([x, In(s, update=(s-x), value=inc.container[s])], [])
+>>> dec(3)
+[]
+>>> print inc[s]
+7.0
+>>> inc(2)
+[]
+>>> print dec[s]
+9.0
+
+The functions ``inc`` and ``dec`` operate on a shared internal value for ``s``.
+Theano's Module system uses this mechanism to share storage between Methods.
+
+The container being shared doesn't have to correspond to the same Result in both functions, 
+but that's usually how this mechanism is used.
+
+Input Argument Restrictions
+---------------------------
+
+The following restrictions apply to the inputs to ``theano.function``:
+
+- Every input list element must be a valid ``In`` instance, or must be
+  upgradable to a valid ``In`` instance. See the shortcut rules below.
+
+- The same restrictions apply as in Python function definitions:
+  default arguments and keyword arguments must come at the end of
+  the list. Un-named mandatory arguments must come at the beginning of
+  the list.
+
+- Names have to be unique within an input list.  If multiple inputs
+  have the same name, then the function will raise an exception. [***Which
+  exception?**]
+
+- Two ``In`` instances may not name the same Result. I.e. you cannot
+give the same parameter multiple times.
+
+If no name is specified explicitly for an In instance, then its name
+will be taken from the Result's name. Note that this feature can cause
+harmless-looking input lists to not satisfy the two conditions above.
+In such cases, Inputs should be named explicitly to avoid problems
+such as duplicate names, and named arguments preceding unnamed ones.
+This automatic naming feature can be disabled by instantiating an In
+instance explicitly with the ``autoname`` flag set to False.
+
+
+Access to function values and containers
+----------------------------------------
+
+For each input, ``theano.function`` will create a ``Container`` if the
+value was not already a ``Container``. At the time of a function call,
+each of these containers must be filled with a value. Each input (but
+especially ones with a default value or an update expression) may have a
+value between calls. The function interface defines a way to get at
+both the current value associated with an input, as well as the container
+which will contain all future values:
+
+  - The ``value`` property accesses the current values. It is both readable
+    and writable, but assignments (writes) may be implemented by an internal
+    copy and/or casts.
+
+  - The ``container`` property accesses the corresponding container.
+    This property accesses is a read-only dictionary-like interface. It is
+    useful for fetching the container associated with a particular input to
+    share containers between functions, or to have a sort of pointer to an
+    always up-to-date value.
+
+Both ``value`` and ``container`` properties provide dictionary-like access based on three types of keys:
+
+- integer keys: you can look up a value/container by its position in the input list;
+- name keys: you can look up a value/container by its name;
+- Result keys: you can look up a value/container by the Result it corresponds to.
+
+In addition to these access mechanisms, there is an even more convenient
+method to access values by indexing a Function directly by typing
+``fn[<name>]``, as in the examples above.
+
+To show some examples of these access methods...
+
+.. code-block:: python
+
+    a, b, c = T.scalars('xys') # set the internal names of graph nodes
+    # Note that the name of c is 's', not 'c'!
+    fn = function([a, b, ((c, c+a+b), 10.0)], [])
+
+    #the value associated with c is accessible in 3 ways
+    assert fn['s'] is fn.value[c]
+    assert fn['s'] is fn.container[c].value
+
+    assert fn['s'] == 10.0
+    fn(1, 2)
+    assert fn['s'] == 13.0
+    fn.s = 99.0
+    fn(1, 0)
+    assert fn['s'] == 100.0
+    fn.value[c] = 99.0
+    fn(1,0)
+    assert fn['s'] == 100.0
+    assert fn['s'] == fn.value[c]
+    assert fn['s'] == fn.container[c].value
+
+
+Input Shortcuts
+---------------
+
+Every element of the inputs list will be upgraded to an In instance if necessary.
+
+- a Result instance ``r`` will be upgraded like ``In(r)``
+
+- a tuple ``(name, r)`` will be ``In(r, name=name)``
+
+- a tuple ``(r, val)`` will be ``In(r, value=value, autoname=True)``
+
+- a tuple ``((r,up), val)`` will be ``In(r, value=value, update=up, autoname=True)``
+
+- a tuple ``(name, r, val)`` will be ``In(r, name=name, value=value)``
+
+- a tuple ``(name, (r,up), val)`` will be ``In(r, name=name, value=val, update=up, autoname=True)``
+
+Example:
+
+.. code-block:: python
+
+    import theano
+    from theano import tensor as T
+    from theano.compile.io import In
+    x = T.scalar()
+    y = T.scalar('y')
+    z = T.scalar('z')
+    w = T.scalar('w')
+
+    fn = theano.function(inputs = [x, y, In(z, value=42), ((w, w+x), 0)],
+                         outputs = x + y + z)
+    # the first two arguments are required and the last two are
+    # optional and initialized to 42 and 0, respectively.
+    # The last argument, w, is updated with w + x each time the
+    # function is called.
+
+    fn(1)               # illegal because there are two required arguments
+    fn(1, 2)            # legal, z is 42, w goes 0 -> 1 (because w <- w + x), returns array(45.0)
+    fn(1, y = 2)        # legal, z is 42, w goes 1 -> 2, returns array(45.0)
+    fn(x = 1, y = 2)    # illegal because x was not named
+    fn(1, 2, 3)         # legal, z is 3, w goes 2 -> 3, returns array(6.0)
+    fn(1, z = 3, y = 2) # legal, z is 3, w goes 3 -> 4, returns array(6.0)
+    fn(1, 2, w = 400)   # legal, z is 42 again, w goes 400 -> 401, returns array(45.0)
+    fn(1, 2)            # legal, z is 42, w goes 401 -> 402, returns array(45.0)
+
+In the example above, ``z`` has value 42 when no value is explicitly given.
+This default value is potentially used at every function invocation, because
+``z`` has no ``update`` or storage associated with it.
+
+
+Outputs
+=======
+
+The ``outputs`` argument to function can be one of
+
+- ``None``, or
+- a Result or ``Out`` instance, or
+- a list of Results or ``Out`` instances.
+
+An ``Out`` instance is a structure that lets us attach options to individual output ``Result`` instances, 
+similarly to how ``In`` lets us attach options to individual input ``Result`` instances.
+
+**Out(result, borrow=False)** returns an ``Out`` instance:
+
+  * ``borrow``
+  
+    If ``True``, a reference to function's internal storage
+    is OK.  A value returned for this output might be clobbered by running
+    the function again, but the function might be faster.
+
+    Default: ``False``
+
+
+
+
+If a single ``Result`` or ``Out`` instance is given as argument, then the compiled function will return a single value.
+
+If a list of ``Result`` or ``Out`` instances is given as argument, then the compiled function will return a list of their values.
+
+.. code-block:: python
+
+    x, y, s = T.matrices('xys')
+
+    # print a list of 2 ndarrays
+    fn1 = theano.function([x], [x+x, Out((x+x).T, borrow=True])
+    print fn1(ndarray([[1,0],[0,1]]))
+
+
+    # print a list of 1 ndarray
+    fn2 = theano.function([x], [x+x])
+    print fn2(ndarray([[1,0],[0,1]]))
+
+    # print an ndarray
+    fn3 = theano.function([x], outputs=x+x)
+    print fn3(ndarray([[1,0],[0,1]]))
+
+
+
+Modes
+=====
+
+The ``mode`` parameter to ``theano.function`` controls how the
+inputs-to-outputs graph is transformed into a callable object.
+
+Theano defines the following modes by name:
+
+- ``FAST_COMPILE``: Apply just a few optimizations, but use C op implementations where possible.
+- ``FAST_RUN``: Apply all optimizations, and use C op implementations where possible.
+- ``DEBUG_MODE``: Verify the correctness of all optimizations, and compare C and python 
+    implementations. This mode can take much longer than the other modes, 
+    but can identify many kinds of problems.
+
+For a finer level of control over which optimizations are applied, and whether
+C or python implementations are used, read :api:`theano.compile.Mode`.
+

doc/basic_tutorial/index.txt

@@ -30,6 +30,7 @@ Now we're ready for the tour:
 
    adding
    examples
+   function
    module
    module_vs_op
    randomstreams

doc/sandbox/numpy.txt

+
+.. _numpy::
+
+===============
+NumPy refresher
+===============
+
+Give summary of type(x) vs x.type vs x.dtype

theano/compile/function_module.py

@@ -110,8 +110,20 @@ class Function(object):
     outputs, performs the packing and unpacking of inputs and return values.  It implements the
     square-bracket indexing so that you can look up the value of a symbolic node.
 
+    Functions are copyable via {{{fn.copy()}}} and {{{copy.copy(fn)}}}.
     When a function is copied, this instance is duplicated.  Contrast with self.maker
     (instance of `FunctionMaker`) that is shared between copies.
+    The meaning of copying a function is that the containers and their current values will all be duplicated.
+    This requires that mutable inputs be copied, whereas immutable inputs may be shared between copies.
+
+
+
+    A Function instance is hashable, on the basis of its memory address (its id).
+
+    A Function instance is only equal to itself.
+    
+    A Function instance may be serialized using the `pickle` or `cPickle` modules.
+    This will save all default inputs, the graph, and *** to the pickle file (WRITEME).
 
     """
 

theano/compile/io.py

-
+"""Define `SymbolicInput`, `SymbolicOutput`, `In`, `Out` """
+__docformat__ = 'restructuredtext en'
 
 class SymbolicInput(object):
     """