merge

doc/library/tensor/shared_randomstreams.txt

@@ -23,79 +23,8 @@ put random variables in your graph.  Theano will allocate a numpy RandomState
 object for each such variable, and draw from it as necessary.  I'll call this sort of sequence of
 random numbers a *random stream*.
 
-Brief example
--------------
-
-Here's a brief example.  The setup code is:
-
-.. code-block:: python
-
-    from theano.tensor.shared_randomstreams import RandomStreams
-    srng = RandomStreams(seed=234)
-    rv_u = srng.uniform((2,2))
-    rv_n = srng.normal((2,2))
-    f = function([], rv_u, updates=[rv_u.update])
-    g = function([], rv_n)                              #omitting rv_n.update
-    nearly_zeros = function([], rv_u + rv_u - 2 * rv_u, updates=[rv_u.update])
-
-Here, 'rv_u' represents a random stream of 2x2 matrices of draws from a uniform
-distribution.  Likewise,  'rv_n' represenents a random stream of 2x2 matrices of
-draws from a normal distribution.  The distributions that are implemented are
-defined in :class:`RandomStreams`.
-
-Now let's use these things.  If we call f(), we get random uniform numbers.
-Since we are updating the internal state of the random number generator (via
-the ``updates`` argument, we get different random numbers every time.
-
->>> f_val0 = f()
->>> f_val1 = f()  #different numbers from f_val0
-
-When we omit the updates argument (as in ``g``) to ``function``, then the
-random number generator state is not affected by calling the returned function.  So for example, 
-calling ``g`` multiple times will return the same numbers.
-
->>> g_val0 = g()  # different numbers from f_val0 and f_val1
->>> g_val0 = g()  # same numbers as g_val0 !!!
-
-An important remark is that a random variable is drawn at most once during any
-single function execution.  So the ``nearly_zeros`` function is guaranteed to
-return approximately 0 (except for rounding error) even though the ``rv_u``
-random variable appears three times in the output expression.
-
->>> nearly_zeros = function([], rv_u + rv_u - 2 * rv_u, updates=[rv_u.update])
-
-Seedings Streams
-----------------
-
-Random variables can be seeded individually or collectively.
-
-You can seed just one random variable by seeding or assigning to the
-``.rng.value`` attribute.
-
->>> rv_u.rng.value.seed(89234)  # seeds the generator for rv_u
-
-You can also seed *all* of the random variables allocated by a :class:`RandomStreams`
-object by that object's ``seed`` method.  This seed will be used to seed a
-temporary random number generator, that will in turn generate seeds for each
-of the random variables.
-
->>> srng.seed(902340)  # seeds rv_u and rv_n with different seeds each
-
-Sharing Streams between Functions
----------------------------------
-
-As usual for shared variables, the random number generators used for random
-variables are common between functions.  So our ``nearly_zeros`` function will
-update the state of the generators used in function ``f`` above.
-
-For example:
-
->>> state_after_v0 = rv_u.rng.value.get_state()
->>> nearly_zeros()       # this affects rv_u's generator
->>> v1 = f()             
->>> rv_u.rng.value.set_state(state_after_v0)
->>> v2 = f()             # v2 != v1
-
+For an example of how to use random numbers, see
+:ref:`using_random_numbers`.
 
 
 Reference

doc/tutorial/examples.txt

@@ -289,6 +289,7 @@ careful though, not to allow the expressions introduced by a givens
 substitution to be co-dependent, the order of substitution is not defined, so
 the substitutions have to work in any order.
 
+.. _using_random_numbers:
 
 Using Random Numbers
 ====================

doc/tutorial/loading_and_saving.txt

@@ -5,29 +5,148 @@
 Loading and Saving
 ==================
 
-Many Theano objects can be serialized.  However, you will want to consider different mechanisms
-depending on the amount of time you anticipate between saving and reloading.  For short-term
-(such as temp files and network transfers) pickling is possible.  For longer-term (such as
-saving models from an experiment) you should not rely on pickled theano objects; we recommend
-loading and saving the underlying shared objects as you would in the course of any other python
-program.
+Python's standard way of saving class instances and reloading them
+is the pickle_ mechanism. Many Theano objects can be serialized (and
+deserialized) by ``pickle``, however, a limitation of ``pickle`` is that
+it does not save the code or data of a class along with the instance of
+the class being serialized. As a result, reloading objects created by a
+previous version of a class can be really problematic.
 
-pickling -- Short-term serialization
-=====================================
+Thus, you will want to consider different mechanisms depending on
+the amount of time you anticipate between saving and reloading.  For
+short-term (such as temp files and network transfers), pickling of
+the Theano objects or classes is possible.  For longer-term (such as
+saving models from an experiment) you should not rely on pickled Theano
+objects; we recommend loading and saving the underlying shared objects
+as you would in the course of any other Python program.
 
-Pickling and unpickling of functions. Caveats... basically don't do this for long-term storage.
 
-***TODO***
+.. _pickle: http://docs.python.org/library/pickle.html
 
-not-pickling -- Long-term serialization
-=======================================
 
-***TODO***
+The basics of pickling
+======================
 
-Give a short example of how to add a __getstate__ and __setstate__ to a class.  Point out to
-use protocol=-1 for numpy ndarrays.
+The two modules ``pickle`` and ``cPickle`` have the same functionalities, but
+``cPickle``, coded in C, is much faster.
 
-Point to the python docs for further reading.
+>>> import cPickle
 
+You can serialize (or *save*, or *pickle*) objects to a file with
+``cPickle.dump``:
 
+>>> f = file('obj.save', 'wb')
+>>> cPickle.dump(my_obj, f, protocol=cPickle.HIGHEST_PROTOCOL)
+>>> f.close()
+
+.. note::
+
+    If you want your saved object to be stored efficiently, don't forget
+    to use ``cPickle.HIGHEST_PROTOCOL``, the resulting file can be
+    dozens of times smaller than with the default protocol.
+
+.. note::
+
+    Opening your file in binary mode (``'b'``) is required for portability
+    (especially between Unix and Windows).
+
+To de-serialize (or *load*, or *unpickle*) a pickled file, use
+``cPickle.load``:
+
+>>> f = file('obj.save', 'rb')
+>>> loaded_obj = cPickle.load(f)
+>>> f.close()
+
+
+You can pickle several objects into the same file, and load them all (in the
+same order):
+
+>>> f = file('objects.save', 'wb')
+>>> for obj in [obj1, obj2, obj3]:
+>>>     cPickle.dump(obj, f, protocol=cPickle.HIGHEST_PROTOCOL)
+>>> f.close()
+
+Then:
+
+>>> f = file('objects.save', 'rb')
+>>> loaded_objects = []
+>>> for i in range(3):
+>>>     loaded_objects.append(cPickle.load(f))
+>>> f.close()
+
+For more details about pickle's usage, see
+`Python documentation <http://docs.python.org/library/pickle.html#usage>`_.
+
+
+Short-term serialization
+========================
+
+If you are confident that the class instance you are serializing will be
+deserialized by a compatible version of the code, pickling the whole model is
+an adequate solution. It would be the cas, for instance, if you are saving
+models and reloading them during the same execution of your program, or if the
+class you're saving has been really stable for a while.
+
+You can control what pickle will save from your object, by defining a
+`__getstate__
+<http://docs.python.org/library/pickle.html#object.__getstate__>`_ method,
+and similarly `__setstate__
+<http://docs.python.org/library/pickle.html#object.__getstate__>`_.
+
+This will be especially useful if, for instance, your model class contains a
+link to the data set currently in use, that you probably don't want to pickle
+along every instance of your model.
+
+For instance, you can define functions along the lines of:
+
+.. code-block:: python
+
+    def __getstate__(self):
+        state = dict(self.__dict__)
+        del state['training_set']
+        return state
+
+    def __setstate__(self, d):
+        self.__dict__.update(d)
+        self.training_set = cPickle.load(file(self.training_set_file, 'rb'))
+
+
+Long-term serialization
+=======================
+
+If the implementation of the class you want to save is quite unstable, for
+instance if functions are created or removed, class members are renamed, you
+should save and load only the immutable (and necessary) part of your class.
+
+You can do that by defining __getstate__ and __setstate__ functions as above,
+maybe defining the attributes you want to save, rather than the ones you
+don't.
+
+For instance, if the only parameters you want to save are a weight
+matrix ``W`` and a bias ``b``, you can define:
+
+.. code-block:: python
+
+    def __getstate__(self):
+        return (W, b)
+
+    def __setstate__(self, (W,b)):
+        self.W = W
+        self.b = b
+
+If, at some point in time, ``W`` is renamed to ``weights`` and ``b`` to
+``bias``, the older pickled files will still be usable, if you update these
+functions to reflect the change in name:
+
+.. code-block:: python
+
+    def __getstate__(self):
+        return (weights, bias)
+
+    def __setstate__(self, (W,b)):
+        self.weights = W
+        self.bias = b
+
+For more information on advanced use of pickle and its internals, see Python's
+pickle_ documentation.