Tutorial updated

doc/internal/dev_start_guide.txt

@@ -41,13 +41,13 @@ As a developer, you should clone this repository like this:
 
 .. code-block:: bash
 
-    hg clone 'http://username:password@pylearn.org/hg/Theano'
+    hg clone 'http://username:password@hg.assembla.com/theano Theano'
 
 You can also clone the code anonymously:
 
 .. code-block:: bash
 
-    hg clone http://pylearn.org/hg/Theano
+    hg clone http://hg.assembla.com/theano Theano
 
 Setting up your environment
 ===========================

doc/library/compile/index.txt

@@ -15,11 +15,13 @@
 
     TODO
 
+
+.. _libdoc_compile_function:
+
 compile.function
 ================
 
-This page is about :api:`theano.function
-<theano.compile.function_module.function>`, the interface for compiling
+This page is about `theano.function`, the interface for compiling
 graphs into callable objects.
 
 The signature for this function is:
@@ -402,6 +404,11 @@ For a finer level of control over which optimizations are applied, and whether
 C or python implementations are used, read :api:`compile.mode.Mode`.
 
 
+.. _compile_debugMode:
+
+DebugMode ??
+
+
 .. toctree::
 
     function

doc/library/tensor/index.txt

@@ -12,6 +12,8 @@
 
 TODO: What does (or compatible) mean?  Talk about casting rules, refer .
 
+.. _libdoc_tensor_type:
+
 TensorType
 ==========
 
@@ -20,6 +22,8 @@ TensorType
    .. method:: quux()
 
 
+.. _libdoc_tensor_creation:
+
 Creation
 ========
 
@@ -80,6 +84,8 @@ Basic indexing.
 
 Advanced indexing.
 
+.. _libdoc_tensor_elementwise:
+
 Elementwise
 ===========
 
@@ -92,9 +98,48 @@ Logic Functions
 Mathematical
 ------------
 
+.. _libdoc_tensor_broadcastable:
+
 Broadcasting in Theano vs. Numpy
 --------------------------------
 
+Broadcasting is a mechanism which allows tensors with
+different numbers of dimensions to be added or multiplied
+together by (virtually) replicating the smaller tensor along
+the dimensions that it is lacking.
+
+In a nutshell, broadcasting is the mechanism by which a scalar
+may be added to a matrix, a vector to a matrix or a scalar to
+a vector.
+
+.. figure:: bcast.png
+
+Broadcasting a row matrix. T and F respectively stand for
+True and False and indicate along which dimensions we allow
+broadcasting.
+
+If the second argument were a vector, its shape would be
+``(2,)`` and its broadcastable pattern ``(F,)``. They would
+be automatically expanded to the **left** to match the
+dimensions of the matrix (adding ``1`` to the shape and ``T``
+to the pattern), resulting in ``(1, 2)`` and ``(T, F)``.
+It would then behave just like the example above.
+
+
+Unlike numpy which does broadcasting dynamically, Theano needs
+to know, for any operation which supports broadcasting, which
+dimensions will need to be broadcasted. When applicable, this
+information is given in the :ref:`type` of a *Variable*.
+
+See also:
+
+* :ref:`How broadcasting is used in Theano's tensor types <tensortypes>`
+
+* `SciPy documentation about numpy's broadcasting <http://www.scipy.org/EricsBroadcastingDoc>`_
+
+* `OnLamp article about numpy's broadcasting <http://www.onlamp.com/pub/a/python/2000/09/27/numerically.html>`_
+
+
 Random Sampling
 ===============
 

doc/tutorial/adding.txt

@@ -26,17 +26,32 @@ array(28.4)
 
 
 Let's break this down into several steps. The first step is to define
-two symbols  representing the quantities that you want
-to add. Note that from now on, we will use the term :term:`Variable`
-to mean "symbol" (in other words, ``x``, ``y``, ``z`` are all Variable
-objects). The output of the function ``f`` is a ``numpy.ndarray``
-with zero dimensions.
+two symbols (*Variables*) representing the quantities that you want
+to add. Note that from now on, we will use the term 
+*Variable* to mean "symbol" (in other words, 
+``x``, ``y``, ``z`` are all *Variable* objects). The output of the function 
+``f`` is a ``numpy.ndarray`` with zero dimensions.
 
 If you are following along and typing into an interpreter, you may have
 noticed that there was a slight delay in executing the ``function``
 instruction. Behind the scenes, ``f`` was being compiled into C code.
 
 
+.. note:
+
+  A *Variable* is the main data structure you work with when
+  using Theano. The symbolic inputs that you operate on are
+  *Variables* and what you get from applying various operations to
+  these inputs are also *Variables*. For example, when I type
+  
+  >>> x = theano.tensor.ivector()
+  >>> y = -x
+  
+  ``x`` and ``y`` are both Variables, i.e. instances of the
+  ``theano.gof.graph.Variable`` class. The
+  type of both ``x`` and ``y`` is ``theano.tensor.ivector``.
+
+
 -------------------------------------------
 
 **Step 1**
@@ -46,11 +61,11 @@ instruction. Behind the scenes, ``f`` was being compiled into C code.
 
 In Theano, all symbols must be typed. In particular, ``T.dscalar``
 is the type we assign to "0-dimensional arrays (`scalar`) of doubles
-(`d`)". It is a Theano :term:`Type`.
+(`d`)". It is a Theano :ref:`type`.
 
 ``dscalar`` is not a class. Therefore, neither ``x`` nor ``y``
 are actually instances of ``dscalar``. They are instances of
-:api:`TensorVariable <theano.tensor.basic.TensorVariable>`. ``x`` and ``y``
+:ref:`TensorVariable <libdoc_tensor_type>`. ``x`` and ``y``
 are, however, assigned the theano Type ``dscalar`` in their ``type``
 field, as you can see here:
 
@@ -66,7 +81,7 @@ True
 You can learn more about the structures in Theano in :ref:`graphstructures`.
 
 By calling ``T.dscalar`` with a string argument, you create a
-:term:`Variable` representing a floating-point scalar quantity with the
+*Variable* representing a floating-point scalar quantity with the
 given name. If you provide no argument, the symbol will be unnamed. Names
 are not required, but they can help debugging.
 
@@ -78,8 +93,8 @@ The second step is to combine ``x`` and ``y`` into their sum ``z``:
 
 >>> z = x + y
 
-``z`` is yet another :term:`Variable` which represents the addition of
-``x`` and ``y``. You can use the :api:`pp <theano.printing.pp>`
+``z`` is yet another *Variable* which represents the addition of
+``x`` and ``y``. You can use the :ref:`pp <libdoc_printing>`
 function to pretty-print out the computation associated to ``z``.
 
 >>> print pp(z)
@@ -94,7 +109,7 @@ and giving ``z`` as output:
 
 >>> f = function([x, y], z)
 
-The first argument to ``function`` is a list of :term:`Variables <Variable>`
+The first argument to :ref:`function <libdoc_compile_function>` is a list of Variables
 that will be provided as inputs to the function. The second argument
 is a single Variable *or* a list of Variables. For either case, the second
 argument is what we want to see as output when we apply the function.
@@ -131,7 +146,7 @@ array([[ 11.,  22.],
 
 It is possible to add scalars to matrices, vectors to matrices,
 scalars to vectors, etc. The behavior of these operations is defined
-by :term:`broadcasting`.
+by :ref:`broadcasting <libdoc_tensor_broadcastable>`.
 
 The following types are available:
 

doc/tutorial/debug_faq.txt

@@ -110,9 +110,7 @@ put logic inside of the print_eval function  that would, for example, only
 print something out if a certain kind of Op was used, at a certain program
 position, or if a particular value shows up in one of the inputs or outputs.
 
-This can be a really powerful debugging tool.  Read about more things you can
-do with :api:`link.WrapLinkerMany`.
+.. TODO: documentation for link.WrapLinkerMany
 
-Note well the call to ``fn`` inside the call to ``print_eval``; without it,
-the graph wouldn't get computed at all!
+This can be a really powerful debugging tool.  Note the call to ``fn`` inside the call to ``print_eval``; without it, the graph wouldn't get computed at all!
 

doc/tutorial/debugmode.txt

@@ -7,7 +7,7 @@ Using DebugMode
 
 
 The DebugMode evaluation mode (available via ``mode='DEBUG_MODE'``,
-:api:`DebugMode`) includes a number of self-checks and assertions that
+see :ref:`this <function_mode>`) includes a number of self-checks and assertions that
 can help to diagnose several kinds of programmer errors that can lead
 to incorrect output.
 
@@ -41,7 +41,7 @@ In the example above, there is no way to guarantee that a future call to say,
 
 If you instantiate DebugMode using the constructor ``compile.DebugMode``
 rather than the keyword ``DEBUG_MODE`` you can configure its behaviour via
-constructor arguments.  See :api:`DebugMode` for details.
+constructor arguments.  See :ref:`DebugMode <compile_debugMode>` for details.
 
 The keyword version of DebugMode (which you get by using ``mode='DEBUG_MODE``)
 is quite strict, and can raise several different Exception types.
@@ -56,7 +56,6 @@ This error is typically not raised directly.
 However, you can use ``except DebugModeError: ...`` to catch any of the more
 specific types of Exception.
 
-For detailed documentation see :api:`DebugModeError`.
 
 
 BadCLinkerOutput
@@ -66,7 +65,6 @@ This exception means that python (``perform``) and c (``c_code``) for an Op
 didn't compute the same thing like they were supposed to.
 The problem might be a bug in either ``perform`` or ``c_code`` (or both).
 
-For detailed documentation see :api:`BadCLinkerOutput`.
 
 
 BadOptimization
@@ -82,7 +80,6 @@ exception object will indicate which optimization was at fault.
 The exception object also contains information such as a snapshot of the
 before/after graph where the optimization introduced the error.
 
-For detailed documentation see :api:`BadOptimization`.
 
 
 BadDestroyMap
@@ -93,7 +90,6 @@ supposed to.  If either the ``perform`` or ``c_code`` implementation of an Op
 might modify any input, it has to advertise that fact via the ``destroy_map``
 attribute.
 
-For detailed documentation on the Exception, see :api:`BadDestroyMap`.
 
 For detailed documentation on the ``destroy_map`` attribute, see :ref:`inplace`.
 
@@ -105,7 +101,6 @@ This happens when an Op's perform() or c_code() creates an alias or alias-like
 dependency between an input and an output... and it didn't warn the
 optimization system via the ``view_map`` attribute.
 
-For detailed documentation on the Exception, see :api:`BadViewMap`.
 
 For detailed documentation on the ``view_map`` attribute, see :ref:`views`.
 
@@ -119,7 +114,6 @@ steps are ordered by ``id(object)`` somehow, such as via the default object
 hash function.  A Stochastic optimization invalidates the pattern of work
 whereby we debug in DEBUG_MODE and then run the full-size jobs in FAST_RUN.
 
-For detailed documentation see :api:`StochasticOrder`.
 
 
 
@@ -136,6 +130,5 @@ introduced into the computations.  It indicates which Op created the first
 NaN.  These floating-point values can be allowed by passing the
 ``check_isfinite=False`` argument to DebugMode. 
 
-For detailed documentation see :api:`InvalidValueError`.
 
 

doc/tutorial/examples.txt

@@ -22,7 +22,7 @@ the logistic curve, which is given by:
     A plot of the logistic function, with x on the x-axis and s(x) on the
     y-axis.
 
-You want to compute the function :term:`elementwise` on matrices of
+You want to compute the function :ref:`elementwise <libdoc_tensor_elementwise>` on matrices of
 doubles, which means that you want to apply this function to each
 individual element of the matrix.
 
@@ -58,7 +58,7 @@ Computing more than one thing at the same time
 ==============================================
 
 Theano supports functions with multiple outputs. For example, we can
-compute the :term:`elementwise` difference, absolute difference, and
+compute the :ref:`elementwise <libdoc_tensor_elementwise>` difference, absolute difference, and
 squared difference between two matrices ``a`` and ``b`` at the same time:
 
 >>> a, b = T.dmatrices('a', 'b')
@@ -134,19 +134,17 @@ array([[ 0.25      ,  0.19661193],
 
 The resulting function computes the gradient of its first argument
 with respect to the second. In this way, Theano can be used for
-`automatic differentiation`_.
+`automatic differentiation <http://en.wikipedia.org/wiki/Automatic_differentiation>`_.
 
 .. note::
    
-   The second argument of ``T.grad`` can be a list, case in which it 
-   will 
-
-   The variable of ``T.grad`` has the same dimensions as the
-   second argument. This is exactly like the first derivative if the
-   first argument is a scalar or a tensor of size 1 but not if it is
-   larger. For more information on the semantics when the first
-   argument has a larger size and details about the implementation,
-   see :api:`tensor.grad`.
+   The second argument of ``T.grad`` can be a list, in which case the
+   output is also a list. The order in both list is important, element
+   *i* of the output list is the gradient of the first argument of
+   ``T.grad`` with respect to the *i*-th element of the list given as second argument.
+   The first arguement of ``T.grad`` has to be a scalar (a tensor
+   of size 1). For more information on the semantics of the arguments of 
+   ``T.grad`` and details about the implementation, see :ref:`this <libdoc_gradient>`.
 
 
 Setting a default value for an argument
@@ -294,8 +292,9 @@ the substitutions have to work in any order.
 Mode
 ====
 
-The ``mode`` parameter to :api:`theano.function` controls how the
-inputs-to-outputs graph is transformed into a callable object.
+The ``mode`` parameter to :ref:`theano.function <libdoc_compile_function>` 
+controls how the inputs-to-outputs graph is transformed into a callable 
+object.
 
 Theano defines the following modes by name:
 
@@ -307,15 +306,11 @@ Theano defines the following modes by name:
 
 The default mode is typically ``FAST_RUN``, but it can be controlled via
 the environment variable ``THEANO_DEFAULT_MODE``, which can in turn be
-overridden by setting :api:`theano.compile.mode.default_mode` directly,
+overridden by setting `theano.compile.mode.default_mode` directly,
 which can in turn be overridden by passing the keyword argument to
-:api:`theano.function`.
+:ref:`theano.function <libdoc_compile_function>`.
 
-For a finer level of control over which optimizations are applied, and
-whether C or python implementations are used, read
-:api:`compile.mode.Mode`.
 
 
 
-.. _automatic differentiation: http://en.wikipedia.org/wiki/Automatic_differentiation
 

doc/tutorial/numpy.txt

@@ -48,7 +48,7 @@ and 2 columns.
 Broadcasting
 ============
 
-Numpy does :term:`broadcasting` of numpy arrays of different shapes during
+Numpy does *broadcasting* of arrays of different shapes during
 arithmetic operations. What this means in general is that the smaller 
 array is *broadcasted* across the larger array so that they have
 compatible shapes. The example below shows an instance of
@@ -60,6 +60,6 @@ compatible shapes. The example below shows an instance of
 array([2., 4., 6.])
 
 The smaller array ``b`` in this case is *broadcasted* to the same size
-as a during the multiplication. This trick is often useful in
+as ``a`` during the multiplication. This trick is often useful in
 simplifying how expression are written. More details about *broadcasting*
-can be found at `numpy user guide <http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html>`__ .
+can be found at `numpy user guide <http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html>`__.

doc/tutorial/tools.txt

@@ -28,7 +28,7 @@ Predefined types
 ----------------
 
 Predefined types are
-located in the :api:`theano.tensor` package. The name of the types follow
+located in the :ref:`theano.tensor <libdoc_tensor>` package. The name of the types follow
 a recipe:
 
 ``<dtype><dimensionality>``
@@ -48,26 +48,26 @@ d    double   floating point 64
 
 Dimensionality is one of:
 
-====== ====== ========================================== =============================================
-code   shape  Rows :term:`broadcastable <broadcasting>`? Columns :term:`broadcastable <broadcasting>`?
-====== ====== ========================================== =============================================
-scalar []     Yes                                        Yes
-vector [n]    Yes                                        N/A (vectors are used like row vectors)
-row    [1, n] Yes                                        No
-col    [m, 1] No                                         Yes
-matrix [m, n] No                                         No
-====== ====== ========================================== =============================================
+====== ====== ======================================================== ===========================================================
+code   shape  Rows :ref:`broadcastable <libdoc_tensor_broadcastable>`? Columns :ref:`broadcastable <libdoc_tensor_broadcastable>`?
+====== ====== ======================================================== ===========================================================
+scalar []     Yes                                                      Yes
+vector [n]    Yes                                                      N/A (vectors are used like row vectors)
+row    [1, n] Yes                                                      No
+col    [m, 1] No                                                       Yes
+matrix [m, n] No                                                       No
+====== ====== ======================================================== ============================================================
 
 So, if you want a row of 32-bit floats, it is available
-as :api:`theano.tensor.frow <theano.tensor.basic.frow>`.
+as :ref:`theano.tensor.frow <libdoc_tensor_type>`.
 If you want a matrix of unsigned 32-bit integers it is available as
-:api:`theano.tensor.imatrix <theano.tensor.basic.imatrix>`.
+:ref:`theano.tensor.imatrix <libdoc_tensor_type>`.
 
 Each of the types described above can be constructed by two methods:
-a singular version (e.g., :api:`dmatrix <theano.tensor.basic.dmatrix>`)
-and a plural version (:api:`dmatrices <theano.tensor.dmatrices>`).
+a singular version (e.g., :ref:`dmatrix <libdoc_tensor_creation>`)
+and a plural version (:ref:`dmatrices <libdoc_tensor_creation>`).
 When called, the singular version takes a single
-argument which is the name of the :term:`Variable` we want to make and it
+argument which is the name of the *Variable* we want to make and it
 makes a single Variable of that type. The plural version can either take
 an integer or several strings. If an integer is provided, the method
 will return that many Variables and if strings are provided, it will
@@ -91,7 +91,7 @@ Custom tensor types
 
 If you wish to use a type of tensor which is not already available here
 (for example, a 3D tensor) you can build an appropriate type using
-:api:`theano.tensor.TensorType <theano.tensor.basic.TensorType>`.
+:ref:`theano.tensor.TensorType <libdoc_tensor_type>`.
 The first argument you pass is the `dtype` and the second is the
 `broadcastable pattern`.
 
@@ -116,10 +116,10 @@ complex128  complex          128 (two float64)
 
 .. note::
 
-   Even though :api:`theano.tensor` does not define any type
+   Even though :ref:`theano.tensor <libdoc_tensor>` does not define any type
    using ``complex`` dtypes (``complex64`` or ``complex128``),
    you can define them explicitly with
-   :api:`TensorType <theano.tensor.basic.TensorType>` (see example
+   :ref:`TensorType <libdoc_tensor_type>` (see example
    below). However, few operations are fully supported for complex
    types: as of version 0.1, only elementary operations (``+-*/``)
    have C implementations. Additionally, complex types have received
@@ -128,8 +128,7 @@ complex128  complex          128 (two float64)
 
 The broadcastable pattern indicates both the number of dimensions and
 whether a particular dimension must have length 1.
-Here is a table mapping the :term:`broadcastable
-<broadcasting>` pattern to what kind of tensor it encodes:
+Here is a table mapping the :ref:`broadcastable <libdoc_tensor_broadcastable>` pattern to what kind of tensor it encodes:
 
 ===================== =================================
 pattern               interpretation