
=======================================
:mod:`module` -- a theano object system
=======================================

.. note::

    Module addresses similar needs to `shared`.  New code is encouraged to
    use `shared` variables.

Now that we're familiar with the basics, we introduce Theano's more
advanced interface, Module. This interface allows you to define Theano
"files" which can have variables and methods sharing
those variables. The Module system simplifies the way to define complex
systems such as a neural network.
It also lets you load and save these complex systems using Python's pickle
mechanism.


Remake of the "state" example
=============================

Let's use Module to re-implement :ref:`the example using state
<functionstateexample>`.

>>> m = Module()
>>> m.state = T.dscalar()
>>> m.inc = T.dscalar('inc')
>>> m.new_state = m.state + m.inc
>>> m.add = Method(m.inc, m.new_state, {m.state: m.new_state})
>>> m.sub = Method(m.inc, None, {m.state: m.state - m.inc})
>>> acc = m.make(state = 0)
>>> acc.state, acc.inc
(array(0.0), None)
>>> acc.add(2)
array(2.0)
>>> acc.state, acc.inc
(array(2.0), None)
>>> acc.state = 39.99
>>> acc.add(0.01)
array(40.0)
>>> acc.state
array(40.0)
>>> acc.sub(20)
>>> acc.state
array(20.0)

This deserves to be broken up a bit...


>>> m = Module()

Here we instantiate an empty Module.
If you can imagine that Theano is a way of generating code (expression
graphs),
then a ``Module()`` is like a fresh blank file.


>>> m.state = T.dscalar()
>>> m.inc = T.dscalar('inc')

Then we declare Variables for use in our Module.
Since we assign these input Variables as attributes of the Module,
they will be *member Variables* of the Module.
Member Variables are special in a few ways, which we will see shortly.

.. note::

   There is no need to name the Variable explicitly here. ``m.state`` will
   be given the name ``'state'`` automatically, because it is being assigned
   to the attribute named ``'state'``.


.. note::

  Since we made it a member of ``m``, the ``acc`` object will have an
  attribute called ``inc``.  This attribute will keep its default value of
  None throughout the example.


>>> m.new_state = m.state + m.inc

This line creates a Variable corresponding to some symbolic computation.
Although this line also assigns a Variable to a Module attribute, it
does not become a member Variable like ``state`` and ``inc`` because it
represents an expression *result*.


>>> m.add = Method(m.inc, m.new_state, {m.state: m.new_state})

Here we declare a Method. The three arguments are as follow:

* **inputs**: a list of input Variables
* **outputs**: a list of output Variables, or ``None``. ``None`` is equivalent
  to returning an empty list of outputs.
* **updates**: a dictionary mapping member Variables to Variables.  When we
  call the function that this Method compiles to, it will replace (update) the
  values associated with the member Variables.

>>> m.sub = Method(m.inc, None, {m.state: m.state - m.inc})

We declare another Method, that has no outputs.

>>> acc = m.make(state = 0)

This line is what does the magic (well, compilation).
The ``m`` object contains symbolic things such as Variables and Methods.
Calling ``make`` on ``m`` creates an object that can do real
computation and whose attributes contain values such as numbers and numpy
ndarrays.

At this point something special happens for our member Variables too.
In the ``acc`` object, make allocates room to store numbers for ``m``'s
member Variables.  By using the string ``'state'`` as a keyword
argument, we tell Theano to store the number ``0`` for the member
Variable called ``state``.  By not mentioning the ``inc`` variable, we
associate ``None`` to the ``inc`` Variable.


>>> acc.state, acc.inc
(array(0.0), None)

Since ``state`` was declared as a member Variable of ``m``, we can
access it's value in the ``acc`` object by the same attribute.
Ditto for ``inc``.

.. note::

  Members can also be accessed using a dictionary-like notation.
  The syntax ``acc['state']`` is equivalent to ``acc.state``.


>>> acc.add(2)
array(2.0)
>>> acc.state, acc.inc
(array(2.0), None)

When we call the ``acc.add`` method, the value ``2`` is used for the
symbolic ``m.inc``.  The first line evaluates the output and all the
updates given in the ``updates`` argument of the call to Method that
declared ``acc``. We only had one update which mapped ``state`` to
``new_state`` and you can see that it works as intended, adding the
argument to the internal state.

Note also that ``acc.inc`` is still ``None`` after our call.  Since
``m.inc`` was listed as an input in the call to Method that created
``m.add``, when ``acc.add`` was created by the call to ``m.make``, a
private storage container was allocated to hold the first parameter.
If we had left ``m.inc`` out of the Method input list, then ``acc.add``
would have used ``acc.inc`` instead.


>>> acc.state = 39.99

The state can also be set.  When we manually set the value of a member
attribute like this, then subsequent calls to the methods of our module will
use the new value.

>>> acc.add(0.01)
array(40.0)
>>> acc.state
array(40.0)
>>> acc.sub(20)
>>> acc.state
array(20.0)

Here, note that ``acc.add`` and ``acc.sub`` share access to the same ``state``
value but update it in different ways.



Using Inheritance
=================

A friendlier way to use Module is to implement your functionality as a
subclass of Module:

.. literalinclude:: ../../examples/module/accumulator.py

This is just like the previous example except slightly fancier.

.. warning::

   Do not forget to call the constructor of the parent class!
   (That's the call to ``super().__init__`` in the previous code block.)

   If you forget it, you'll get strange behavior :(


Extending your Module with Python methods
=========================================

Let's say we want to add a method to our accumulator to print out the
state and we want to call it ``print_state``.  There are two mechanisms to do
this: let's call them _instance_method and InstanceType.


Mechanism 1: _instance_method
-----------------------------

This is the preferred way of adding a few instance methods with a minimum of
boilerplate code.

All we need to do to use this mechanism is to give a method called
``_instance_print_state`` to our Module class.


Any method called like ``_instance_XXX`` will cause the object
obtained through a call to ``make`` to have a method called ``XXX``.
Note that when we define ``_instance_print_state`` there are two "self"
arguments: ``self`` which is *symbolic* and ``obj`` which is the compiled
object (the one that contains values).

Hint: ``self.state`` is the symbolic state variable and
prints out as "state", whereas ``obj.state`` is the state's actual
value in the accumulator and prints out as "0.0".


Mechanism 2: InstanceType
-------------------------

If a number of instance methods are going to be defined, and especially if you
will want to inherit from the kind of class that gets instantiated by ``make``,
you might prefer to consider using the InstanceType mechanism.


Adding custom initialization
============================

As was said in the previous section, you can add functionality with
``_instance_XXX`` methods. One of these methods is actually special:
``_instance_initialize`` will be called with whatever arguments you
give to ``make``. There is a default behavior which we have used,
where we give the states' initial values with keyword arguments
(``acc.make(state = 0)``). If you want more personalized behavior, you
can override the default with your own method, which has to be called
``_instance_initialize``.

.. TODO
    Here is an example where we take width and height arguments to
    initialize a state with a matrix of zeros:



Nesting Modules
===============

Probably the most powerful feature of Theano's Modules is that one can be
included as an attribute to another so that the storage of each is available
to both.


As you read through examples of Theano code, you will probably see many
instances of Modules being nested in this way.
