more aliasing tutorial text

doc/tutorial/aliasing.txt

@@ -24,19 +24,21 @@ changes to values in that pool.
  1. Theano manages its own memory space, which typically does not overlap with
     the memory of normal python variables that non-theano code creates.
 
- 2. Theano's memory space includes the buffers allocated to store shared
+ 1. Theano functions only modify buffers that are in its memory space.
+
+ 1. Theano's memory space includes the buffers allocated to store shared
     variables and the temporaries used to evaluate Functions.
 
- 3. Physically, Theano's memory space may be spread across the host, a GPU
+ 1. Physically, Theano's memory space may be spread across the host, a GPU
     device(s), and in the future may even include objects on a remote machine.
 
- 4. The memory allocated for a shared variable buffer is unique: it is never
+ 1. The memory allocated for a shared variable buffer is unique: it is never
     aliased to another shared variable.
 
- 5. Theano's managed memory is constant while Theano Functions are not running
+ 1. Theano's managed memory is constant while Theano Functions are not running
     and theano library code is not running.
 
- 6. The default behaviour of Function is to return user-space values for
+ 1. The default behaviour of Function is to return user-space values for
     outputs, and to expect user-space values for inputs.
     
 The distinction between Theano-managed memory and user-managed memory can be
@@ -51,20 +53,159 @@ to use the ``borrow=True`` argument and reap the benefit of faster code.
 Borrowing when creating shared variables
 ========================================
 
-TODO
+A ``borrow`` argument can be provided to the shared-variable constructor.
+
+.. code-block:: python
+
+    import numpy, theano
+    np_array = numpy.ones(2, dtype='float32')
+
+    s_default = shared(np_array)
+    s_false   = shared(np_array, borrow=False)
+    s_true    = shared(np_array, borrow=True)
+
+By default (``s_default``) and when explicitly setting ``borrow=False``, the
+shared variable we construct gets a [deep] copy of ``np_array``.  So changes we
+subsequently make to ``np_array`` have no effect on our shared variable.
+
+.. code-block:: python
+
+    np_array += 1 # now it is an array of 2.0 s
+
+    s_default.value  # -> array([1.0, 1.0])
+    s_false.value    # -> array([2.0, 2.0])
+
+If we are running this with the CPU as the device, 
+then changes we make to np_array *right away* will show up in ``s_false.value``
+because numpy arrays are mutable, and ``s_false`` is using the ``np_array``
+object as it's internal buffer.
+
+However, this aliasing of ``np_array`` and ``s_false`` is *inconsistent and fragile*!
+It is inconsistent because if Theano is using a GPU device, then the borrow flag
+has no effect.
+It is fragile because
+if we call a theano function that updates the value of ``s_false`` the aliasing
+relationship *may* or *may not* be broken (it depends on what the Theano
+function does).
+
+*Take home message:*
+
+It is safe practice (and a good idea) to use ``borrow=True`` in a shared
+variable constructor when the shared variable stands for a large object (in
+terms of memory footprint) and you do not want to create copies of it in memory
+.
+
+It is not a reliable technique to use ``borrow=True`` to modify shared variables
+by side-effect, because with some devices (e.g. GPU devices) this technique will
+not work.
+
+Borrowing when accessing value of shared variables
+==================================================
+
+Retrieving
+----------
+
+A ``borrow`` argument can also be used to control how a shared variable's value is retrieved.
+
+.. code-block:: python
+
+    s = shared(np_array)
+
+    v_false = s.get_value(borrow=False) # N.B. borrow default is False
+    v_true = s.get_value(borrow=True)
 
-Borrowing when construction Function objects
+
+When ``borrow=False`` is passed to ``get_value``, it means that the return value
+may not be aliased to any part of Theano's internal memory.
+When ``borrow=True`` is passed to ``get_value``, it means that the return value
+*might* be aliased to some of Theano's internal memory.
+But both of these calls might create copies of the internal memory.
+
+The reason that ``borrow=True`` might still make a copy is that the internal
+representation of a shared variable might not be what you expect.  When you
+create a variable by passing a numpy array for example, then ``get_value()``
+must return a numpy array too.  That's how Theano can make the GPU use
+transparent.  But when you are using a GPU (or in future perhaps a remote machine), then the numpy.ndarray
+is not the internal representation of your data.
+If you really want Theano to return its internal representation *and never copy it*
+then you should use the ``return_internal_type=True`` argument to
+``get_value``.  It will never copy the internal object (always return in
+constant time), but might return various datatypes depending on contextual
+factors (e.g. the compute device, the dtype of the numpy array).
+
+.. code-block:: python
+
+    v_internal = s.get_internal_value(borrow=True, return_internal_type=True)
+
+It is possible to use ``borrow=False`` in conjunction with
+``return_internal_type=True``, which will return a deep copy of the internal object.
+This is primarily for internal debugging, not for typical use.
+
+*Take home message:*
+
+It is safe (and sometimes much faster) to use ``get_value(borrow=True)`` when
+your code does not modify the return value.  *Do not use this to modify a shared
+variable by side-effect* because it will make your code device-dependent.
+Modification of GPU variables by this sort of side-effect is impossible.
+
+Assigning
+---------
+
+Shared variables also have a ``set_value`` method that can accept an optional ``borrow=True`` argument.
+The semantics are similar to those of creating a new shared variable -
+``borrow=False`` is the default and ``borrow=True`` means that Theano *may*
+reuse the buffer you provide as the internal storage for the variable.
+
+A standard pattern for manually updating the value of a shared variable is as
+follows.
+
+.. code-block:: python
+
+    s.set_value(
+        some_inplace_fn(s.get_value(borrow=True)),
+        borrow=True)
+
+This pattern works regardless of the compute device, and when the compute device
+makes it possible to expose Theano's internal variables without a copy, then it
+goes as fast as an in-place update.
+
+
+Borrowing when constructing Function objects
 ============================================
 
-TODO
+A ``borrow`` argument can also be provided to the ``In`` and ``Out`` objects
+that control how ``theano.function`` handles its arguments and return value[s]. 
+
+.. code-block:: python
+
+    import theano, theano.tensor
+
+    x = theano.tensor.matrix()
+    y = 2*x
+    f = theano.function([theano.In(x, borrow=True)], theano.Out(y, borrow=True))
 
 
-Borrowing when accessing value of shared variable
-=================================================
+Borrowing an input means that Theano will treat the argument you provide as if
+it were part of Theano's pool of temporaries.  Consequently, your input
+may be reused as a buffer (and overwritten!) during the computation of other variables in the
+course of evaluating that function (e.g. ``f``).  
 
-TODO
+Borrowing an output means that Theano will not insist on allocating a fresh
+output buffer every time you call the function.  It will possibly reuse the same one as
+a previous call, and overwrite the old contents.  Consequently, it may overwrite
+old return values by side effect.
 
-Describe interaction with 'raw' flag as well.
+It is also possible to pass an ``return_internal_type=True`` flag to the ``Out``
+variable which has the same interpretation as the ``return_internal_type`` flag
+to the shared variable's ``get_value`` function.
 
+*Take home message:*
+When an input ``x`` to a function is not needed after the function returns and you
+would like to make it available to Theano as additional workspace, then consider
+marking it with ``In(x, borrow=True)``.  It may make the function faster and
+reduce its memory requirement.
+When a return value ``y`` is large (in terms of memory footprint), and you only need to read from it once, right
+away when it's returned, then consider marking it with an ``Out(y,
+borrow=True)``.