Major refresh of the installation documentation and developer guide.

doc/install.txt

@@ -20,7 +20,7 @@ to be installed:
         We develop mainly on 64-bit Linux machines. 32-bit architectures are
         not well-tested.
 
-    Python >= 2.4
+    Python_ >= 2.4
         Including the development package (``python-dev`` or ``python-devel``
 	on most Linux distributions).
 
@@ -33,7 +33,8 @@ to be installed:
         Earlier versions have memory leaks.
 
     `SciPy <http://scipy.org>`_
-	Specifically ``scipy.sparse``, and ``scipy.weave``. We recommend SciPy
+	Only currently required for sparse matrix and special functions
+	support, but highly recommended. We recommend SciPy
 	>=0.7 if you are using sparse matrices, because ``scipy.sparse``
 	is buggy in 0.6 (the ``scipy.csc_matrix`` version of ``dot()`` has a
 	bug with singleton dimensions, there may be more bugs).
@@ -45,6 +46,7 @@ to be installed:
 
 .. _BLAS: http://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms
 .. _Accelerate framework: http://developer.apple.com/performance/accelerateframework.html
+.. _Python: http://www.python.org/
 
 The following libraries and software are optional:
 
@@ -68,47 +70,100 @@ Basic user install instructions
 -------------------------------
 
 The easiest way to obtain the released version of Theano is from PyPI using
-the ``easy_install`` command provided by distribute_ (and it's predecessor,
-`setuptools`_). If you have one of these installed, installing Theano should
-be simply a matter of::
+pip_ by typing
 
-    easy_install Theano
+.. code-block:: bash
+
+    pip install Theano
+
+You may need to add ``sudo``  before this command to install into your
+system's ``site-packages`` directory. If you do not have administrator access
+to your machine, you can install to an alternate prefix using
+
+.. code-block:: bash
+
+    pip install Theano --install-option='--prefix=YOURPREFIX'
+
+e.g. using ``--install-option='--prefix=~/.local'`` on Python 2.4 would
+install Theano into ``.local/lib/python2.4/site-packages`` inside your home
+directory on Mac OS X or Unix/Linux (this ``site-packages`` directory must be
+listed in your ``$PYTHONPATH``; for Python 2.6 and later, ``~/.local`` is
+automatically searched and does *not* need to be explicitly included in
+``$PYTHONPATH``, see :ref:`config_pythonpath` for instructions).
+
+Alternatively you can use virtualenv_ to create an isolated ``site-packages``
+directory; see the `virtualenv documentation`_ for details.
+
+.. note::
+
+    Theano *can* be installed with easy_install_, however we recommend pip_ as
+    a bug in ``easy_install`` prevents ``theano.test()`` from running the Theano
+    test suite. If you do not have pip_ installed but do have ``easy_install``,
+    you can get pip_ by typing ``easy_install pip``.
 
 .. _distribute: http://packages.python.org/distribute/
 .. _setuptools: http://pypi.python.org/pypi/setuptools
-
+.. _easy_install: http://packages.python.org/distribute/easy_install.html
+.. _virtualenv: http://pypi.python.org/pypi/virtualenv
+.. _virtualenv documentation: http://virtualenv.openplans.org/
+.. _pip: http://pypi.python.org/pypi/pip
 
 .. _install_bleeding_edge:
 
 Bleeding-edge (developer) install instructions
 ----------------------------------------------
 
-If you are a developer of Theano, then check out the :ref:`dev_start_guide`
-guide.
+If you are a developer of Theano, then check out the :ref:`dev_start_guide`.
 
 The following are general instructions that will set you up with the
-bleeding-edge version of Theano. First, get the code using `mercurial
-<http://www.selenic.com/mercurial/wiki/>`__:
+bleeding-edge version of Theano. First, get the code using `Mercurial
+<http://mercurial.selenic.com/>`__:
 
 .. code-block:: bash
 
     hg clone http://hg.assembla.com/theano Theano
 
-Configuring ``PYTHONPATH``
---------------------------
+From here, the easiest way to get started is
+
+.. code-block:: bash
 
-The subdirectory ``Theano/theano`` has to be located in a path mentioned in
-your ``PYTHONPATH``. In order to do that, you can either create a symbolic
-link to Theano/theano in a directory already mentioned in your ``PYTHONPATH``
-environment variable, or modify the ``PYTHONPATH`` so that it mentions Theano.
+    cd Theano
+    python setup.py develop
+
+This will install a ``.pth`` file in your ``site-packages`` directory that
+tells Python where to look for your Theano installation (i.e. in the
+directory your just checked out of Mercurial). Using ``develop`` mode is
+preferable to ``install`` as any modifications you make in the checkout
+directory (or changes you pull with Mercurial) will be automatically reflected
+in the "installed" version without re-running ``python setup.py install``.
 
-To create a symbolic link:
+If you do not have permission to modify your ``site-packages`` directory you
+can specify an alternative installation prefix using
 
 .. code-block:: bash
 
-    ln -s Theano/theano <someplace on your PYTHONPATH>/theano
+    python setup.py develop --prefix=YOURPREFIX
+
+A common choice is ``~/.local`` which is automatically searched for Python >=
+2.6; for earlier Python versions and other installation prefixes, the prefix
+specified must contain ``lib/pythonA.B/site-packages``, where ``A.B`` is e.g.
+2.5, and this ``site-packages`` directory must be listed in ``$PYTHONPATH``.
+
+An alternative, perhaps simpler way of creating and using an isolated
+``site-packages`` is to use virtualenv_; see the `virtualenv documentation`_
+for details. If you find yourself using virtualenv frequently you may find the
+virtualenvwrapper_ package useful for switching between them.
+
+.. _virtualenv: http://pypi.python.org/pypi/virtualenv
+.. _virtualenv documentation: http://virtualenv.openplans.org/
+.. _virtualenvwrapper: http://www.doughellmann.com/projects/virtualenvwrapper/
+
+.. _config_pythonpath:
+
+Configuring ``PYTHONPATH``
+##########################
 
-To modify the environment variable PYTHONPATH in bash, you may do this:
+To modify the environment variable ``PYTHONPATH`` in bash, you may do this:
 
 .. code-block:: bash
 
@@ -120,16 +175,26 @@ In csh:
 
     setenv PYTHONPATH <path to Theano's parent dir>/Theano:$PYTHONPATH
 
+To make this change stick you'll usually need to add the command to one or
+more of your shell's startup scripts, i.e. ``~/.bashrc`` or ``~/.cshrc``.
+Consult your shell's documentation for details.
+
 Testing your installation
 ---------------------------
 
-Once you have completed these steps, you should run the theano test suite like this:
+Once you have completed these steps, you should run the Theano test suite. At
+a Python (or IPython) interpreter,
 
-.. code-block:: bash
+.. code-block:: python
 
-    cd Theano
-    THEANO_FLAGS=warn.ignore_bug_before=all nosetests #execute all the tests
+    >>> import theano
+    >>> theano.test()
 
+You can also run them in-place from the Mercurial checkout directory by typing
+
+.. code-block:: bash
+
+    nosetests
 
 ``THEANO_FLAGS`` is an environment variable that define Theano flags
 (:ref:`libdoc_config`).  For Windows users, you can remove it or see the
@@ -143,7 +208,7 @@ the ``theano-users@googlegroups.com`` mailing list.
 
     `warn.ignore_bug_before=all` removes warnings that you don't need to see
     here.  It is also recommended for a new user to set this flag to a
-    different value into their .theanorc file.  See
+    different value in their ``.theanorc`` file.  See
     :attr:`config.warn.ignore_bug_before` for more details.
 
 Troubleshooting: Make sure you have a BLAS library
@@ -168,7 +233,6 @@ entry like this:
 .. code-block:: cfg
 
     # other stuff can go here
-    
     [blas]
     ldflags = -lf77blas -latlas -lgfortran #put your flags here
 

doc/internal/dev_start_guide.txt

-
 .. _dev_start_guide:
 
 =====================
 Developer Start Guide
 =====================
 
+To get up to speed, you'll need to
 
-- Learn some non-basic python to understand what's going on in some of the
-  tricker files (like tensor.py).
-
-- Roughly go through the numpy documentation.
-
-- Learn to write reStructuredText_ for epydoc_.
-
-- ExternalTools - packages that play well with Numpy
-
-- EssentialUnitTest - essential usage of python.unittest
+- Learn some non-basic Python to understand what's going on in some of the
+  trickier files (like tensor.py).
+- Go through the `NumPy documentation`_.
+- Learn to write reStructuredText_ for epydoc_ and Sphinx_.
+- Learn about how unittest_ and nose_ work
 
+.. _Sphinx: http://sphinx.pocoo.org/
+.. _reStructuredText: http://docutils.sourceforge.net/rst.html
+.. _epydoc: http://epydoc.sourceforge.net/
+.. _NumPy documentation: http://docs.scipy.org/numpy/
+.. _unittest: http://docs.python.org/library/unittest.html
+.. _nose: http://somethingaboutorange.com/mrl/projects/nose/
 
 Accounts
-========
+--------
 
 To obtain developer access: register with `Assembla
-<http://www.assembla.com/>`_ and add yourself as a watcher on the `Theano space 
-<http://www.assembla.com/spaces/theano>`_. Then send an email to an admin asking 
-to be promoted to a member of the project.
+<http://www.assembla.com/>`_ and add yourself as a watcher on the `Theano
+space <http://www.assembla.com/spaces/theano>`_. Then send an email to an
+admin asking to be promoted to a member of the project.
 
 
 Theano code
-===========
+-----------
 
-*To get the source via mercurial,* you must have `mercurial
-<http://www.selenic.com/mercurial/wiki/>`__ installed.
+*To get the source via Mercurial,* you must have `Mercurial
+<http://mercurial.selenic.com/>`__ installed.
 
 The code that makes up Theano is in a `single repository
 <http://www.assembla.com/spaces/theano/trac_mercurial_tool>`__. As a developer,
@@ -47,71 +48,47 @@ You can also clone the code anonymously:
 
     hg clone http://hg.assembla.com/theano Theano
 
-Setting up your environment
-===========================
-
-Some notes on the environment variable $PYTHONPATH.
-
-The subdirectory Theano/theano has to be located in a location
-mentionned in your PYTHONPATH. In order to do that, you can either
-create a symbolic link to Theano/theano in a directory already
-mentionned in your PYTHONPATH environment variable, or modify the
-PYTHONPATH so that it mentions Theano.
-
-To create a symbolic link:
-
-.. code-block:: bash
-
-    ln -s Theano/theano <someplace on your PYTHONPATH>/theano
-
-To modify the environment variable PYTHONPATH in bash, you may do this:
-
-.. code-block:: bash
+Then follow the :ref:`install_bleeding_edge`.
 
-    export PYTHONPATH=<path to Theano's parent dir>/Theano:$PYTHONPATH
+Details about ``PYTHONPATH``
+----------------------------
 
-In csh:
+``$PYTHONPATH`` should contain a ":"-separated list of paths, each of which
+contains one or several Python packages, in the order in which you would like
+Python to search for them. If a package has sub-packages of interest to you,
+do **not** add them to ``$PYTHONPATH``: it is not portable, might shadow other
+packages or short-circuit important things in its ``__init__``.
 
-.. code-block:: csh
+It is advisable to never import Theano's files from outside Theano itself
+(this is good advice for Python packages in general). Use ``from theano import
+tensor`` instead of ``import tensor``. ``$PYTHONPATH`` should only contain
+paths to complete packages.
 
-    setenv PYTHONPATH <path to Theano's parent dir>/Theano:$PYTHONPATH
-
-Olivier Breuleux explains:
-
-$PYTHONPATH should contain a ":"-separated list of paths, each of which
-contains one or several Python packages, in the order in which you would
-like Python to search for them. If a package has sub-packages of interest
-to you, do _not_ add them in the path: it is not portable, might shadow
-other packages or short-circuit important things in its __init__.
-
-I advise to never import theano's files from outside theano itself
-(and I think that is good advice for Python packages in general). Use
-"from theano import tensor" instead of "import tensor". ... $PYTHONPATH
-... should only contain paths to complete packages, so you don't get
-surprised if I add files that enter in conflict with other packages.
-
-When you install a package, only the package name can be imported
-directly. If you want a sub-package, you must import it from the main
-package. That's how it will work in 99.9% of installs because it is the
-default. Therefore, if you stray from this practice, your code will not
-be portable. Also, some ways to circumvent circular dependencies might
-make it so you have to import files in a certain order, which is best
-handled by the package's own __init__.py.
+When you install a package, only the package name can be imported directly. If
+you want a sub-package, you must import it from the main package. That's how
+it will work in 99.9% of installs because it is the default. Therefore, if you
+stray from this practice, your code will not be portable. Also, some ways to
+circumvent circular dependencies might make it so you have to import files in
+a certain order, which is best handled by the package's own ``__init__.py``.
 
 More instructions
 =================
 
 Once you have completed these steps, you should run the tests like this:
 
-.. code-block:: bash
+.. code-block:: python
+
+    >>> import theano
+    >>> theano.test()
 
-    cd Theano
-    nosetests #execute all the tests
+Or by running ``nosetests`` from your checkout directory.
 
 All tests should pass. If some test fails on your machine, you are
-encouraged to tell us what went wrong on the ``theano-users@googlegroups.com``
+encouraged to tell us what went wrong on the `theano-users`_
 mailing list.
 
+.. _theano-users: https://groups.google.com/group/theano-users
+
 To update your library to the latest revision, change directory (``cd``)
 to your ``Theano`` folder and execute the following command:
 
@@ -122,26 +99,9 @@ to your ``Theano`` folder and execute the following command:
 Nightly test
 ============
 
-Each night we execute all the unit tests automatically.
-The result is sent by email to one of the developers, who (at his discretion) forwards it to the mailing list.
-Said developer may not read his email obsessively (especially on the weekend)
-so there might be some latency in the process.
-For more detail :ref:`see <metadocumentation_nightly_build>`.
-
-
-.. TODO: fix this links
-
-.. _non-basic python: http://pylearn.org/theano/wiki/NonbasicPython
-.. _reStructuredText: http://docutils.sourceforge.net/rst.html
-.. _epydoc: http://epydoc.sourceforge.net/
-.. _basicnumpy: http://pylearn.org/theano/wiki/BasicNumpy
-
-
-.. _README: ../README.html
-.. _Download: ../README.html#downloading-theano
-.. _Documentation: index.html
-.. _Wiki: http://pylearn.org/theano
-.. _TRAC: http://trac.edgewall.org/
-.. _task list: http://pylearn.org/theano/query?status=accepted&status=assigned&status=new&status=reopened&group=milestone&max=200&col=id&col=summary&col=status&col=owner&col=type&col=priority&col=component&col=time&report=9&order=priority
-
+Each night we execute all the unit tests automatically.  The result is sent by
+email to one of the developers, who (at his discretion) forwards it to the
+mailing list.  Said developer may not read his email obsessively (especially
+on the weekend) so there might be some latency in the process.  For more
+detail :ref:`see <metadocumentation_nightly_build>`.