Reorganize Developer documentation

doc/dev_start_guide.txt

@@ -4,6 +4,18 @@
 Developer Start Guide
 =====================
 
+Resources
+=========
+
+See :ref:`theano_community` for a list of Theano resources. The
+following groups/mailing-lists are especially useful to Theano
+contributors: `theano-dev`_, `theano-buildbot`_, and `theano-github`_.
+
+.. _theano-dev: https://groups.google.com/group/theano-dev
+.. _theano-github: https://groups.google.com/group/theano-github
+.. _theano-buildbot: https://groups.google.com/group/theano-buildbot
+
+
 To get up to speed, you'll need to
 
 - Learn some non-basic Python to understand what's going on in some of the
@@ -19,104 +31,144 @@ To get up to speed, you'll need to
 .. _unittest: http://docs.python.org/library/unittest.html
 .. _nose: http://somethingaboutorange.com/mrl/projects/nose/
 
-Accounts
---------
+
+Installation and configuration
+==============================
 
 To obtain developer access: register with `GitHub
 <http://www.github.com/>`_ and create a fork of `Theano
 <http://www.github.com/Theano/Theano>`_.
 
-Coding style
-------------
-
-We didn't had any coding style when we started Theano. We now use
-`this one
-<http://deeplearning.net/software/pylearn/v2_planning/API_coding_style.html>`_
-except that we don't use the numpy docstring standard.
-
-We do not plan to change all existing code to follow this coding
-style, but as we modify the code, we update it accordingly.
+This will create your own Theano project on GitHub, referred later
+as "YourProfile/Theano", or "origin", from which you will be able to
+contribute to the original Theano/Theano, also called "central".
 
-Mailing list
-------------
 
-See the Theano main page for the theano-dev, theano-buildbot and
-theano-github mailing list. They are useful to Theano contributors.
+Create a local copy
+-------------------
 
-Git config
-----------
+Clone your fork locally with
 
 .. code-block:: bash
 
-   git config --global user.email you@yourdomain.example.com
-   git config --global user.name "Your Name Comes Here"
+    git clone git@github.com:your_github_login/Theano.git
 
-Typical development workflow
-----------------------------
+From your local repository, your own fork on GitHub will be called "origin".
 
-Clone your fork locally with
+Then, add a reference to the original ("central") Theano repository with
 
 .. code-block:: bash
 
-    git clone git@github.com:your_github_login/Theano.git
+    git remote add central git://github.com/Theano/Theano.git
 
-and add a reference to the 'central' Theano repository with
+You can choose another name than "central" to reference Theano/Theano
+(for instance, NumPy uses "upstream"), but this documentation will stick
+to "central."
 
-.. code-block:: bash
+You can then test your installation of Theano by following the steps of
+:ref:`testing_installation`.
 
-    git remote add central git://github.com/Theano/Theano.git
 
-When working on a new feature in your own fork, start from an up-to-date copy
-of the trunk:
+Using your local copy
+---------------------
+
+To update your library to the latest revision, you should have a local branch
+that tracks central/master. You can add one (named "trunk" here) with:
 
 .. code-block:: bash
 
     git fetch central
-    git checkout -b my_shiny_feature central/master
+    git branch trunk central/master
 
-Once your code is ready for others to review, push your branch to your github fork:
+Once you have such a branch, in order to update it, do:
 
 .. code-block:: bash
 
-    git push -u origin my_shiny_feature
+    git checkout trunk
+    git pull
 
-then go to your fork's github page on the github website, select your feature
-branch and hit the "Pull Request" button in the top right corner.
-If you don't get any feedback, bug us on the theano-dev mailing list.
+Keep in mind that this branch should be "read-only": if you want to
+patch Theano, you should work in another branch, like described in the
+:ref:`dev_workflow` section below.
 
-When your pull request has been merged, you can delete the branch
-from the github list of branches. This is useful to avoid having too many
-branches staying there. Deleting this remote branch is achieved with:
+
+Configure Git
+-------------
+
+On your local machine, you need to configure git with basic informations:
 
 .. code-block:: bash
 
-   git push origin :my_shiny_feature
+   git config --global user.email you@yourdomain.example.com
+   git config --global user.name "Your Name Comes Here"
+
+
+You can also instruct git to use color in diff. For this, you need to
+add those lines in the file ~/.gitconfig
 
-You can keep your local repository up to date with central/master with those
-commands:
+.. code-block:: cfg
+
+    [color]
+       branch = auto
+       diff = auto
+       interactive = auto
+       status = auto
+
+
+.. _dev_workflow:
+
+Development Workflow
+====================
+
+Start a new local branch
+------------------------
+
+When working on a new feature in your own fork, start from an up-to-date copy
+of the `master` branch (the principal one) of the central repository
+(Theano/Theano on GitHub):
 
 .. code-block:: bash
 
-   git checkout master
-   git fetch central
-   git merge central/master
+    git fetch central
+    git checkout -b my_shiny_feature central/master
 
-If you want to fix a commit already submitted within a pull request (e.g. to
-fix a small typo), you can do it like this to keep history clean:
+.. note::
+
+    This last line is a shortcut for:
+
+    .. code-block:: bash
+
+        git branch my_shiny_feature central/master
+        git checkout my_shiny_feature
+
+
+Submit your changes to the central repository
+---------------------------------------------
+
+Once your code is ready for others to review, you need to push your
+branch to your github fork first:
 
 .. code-block:: bash
 
-   git checkout my_shiny_feature
-   git commit --amend
-   git push -u origin my_shiny_feature:my_shiny_feature
+    git push -u origin my_shiny_feature
+
+Then, go to your fork's github page on the github website, select your
+feature branch and hit the "Pull Request" button in the top right
+corner.  This will signal the maintainers that you wish to submit your
+changes for inclusion in central/master.
+If you don't get any feedback, bug us on the theano-dev mailing list.
 
 
+Tips for Quality Contributions
+==============================
+
 Coding Style Auto Check
 -----------------------
 
 In Theano, we use the same coding style as the `Pylearn
 <http://deeplearning.net/software/pylearn/v2_planning/API_coding_style.html>`_
-project. The principal thing to know is that we follow the
+project, except that we don't use the numpy docstring standard.
+The principal thing to know is that we follow the
 `pep8 <http://www.python.org/dev/peps/pep-0008/>`_ coding style.
 
 We use git hooks provided in the project `pygithooks
@@ -132,120 +184,96 @@ now. So we strongly suggest that you use the "increment" pygithooks
 config option to have a good workflow. See the pygithooks main page
 for how to set it up for Theano and how to enable this option.
 
-Cleaning up history
--------------------
 
-Sometimes you may have commits in your feature branch that
-are not needed in the final pull request. There is a `page
-<http://sandofsky.com/blog/git-workflow.html>`_ that talks about
-this. In summary:
+Unit tests
+----------
 
-* Commits to the trunk should be a lot cleaner than commits to your
-  feature branch; not just for ease of reviewing but also
-  because intermediate commits can break blame (the bisecting tool).
-* `git merge --squash` will put all of the commits from your feature branch into one commit.
-* There are other tools that are useful if your branch is too big for one squash.
+Before submitting a pull request, you should run the unit test suite,
+and make sure that your changes did not create any new Error or Failure.
+You can consult `theano-buildbot`_ for the result of a recent run
+of the test suite with various options.
 
+Each night we execute all the unit tests automatically.  The result is sent by
+email to the `theano-buildbot`_ mailing list.
 
-To checkout another user branch in his repo:
+For more detail, see :ref:`metadocumentation_nightly_build`.
+
+To run all the tests with the same configuration as the buildbot, run this script:
 
 .. code-block:: bash
 
-   git remote add REPO_NAME HIS_REPO_PATH
-   git checkout -b LOCAL_BRANCH_NAME REPO_NAME/REMOVE_BRANCH_NAME
+   theano/misc/do_nightly_build
 
-You can find move information and tips in the `numpy development
-<http://docs.scipy.org/doc/numpy/dev/gitwash/development_workflow.html>`_
-page.
+This function accepts arguments that it forward to nosetests. You can
+run only some tests or enable pdb by giving the equivalent nosetests
+parameters.
 
 
-Details about ``PYTHONPATH``
-----------------------------
 
-``$PYTHONPATH`` should contain a ":"-separated list of paths, each of which
-contains one or several Python packages, in the order in which you would like
-Python to search for them. If a package has sub-packages of interest to you,
-do **not** add them to ``$PYTHONPATH``: it is not portable, might shadow other
-packages or short-circuit important things in its ``__init__``.
+More Advanced Git Usage
+=======================
 
-It is advisable to never import Theano's files from outside Theano itself
-(this is good advice for Python packages in general). Use ``from theano import
-tensor`` instead of ``import tensor``. ``$PYTHONPATH`` should only contain
-paths to complete packages.
+You can find information and tips in the `numpy development
+<http://docs.scipy.org/doc/numpy/dev/gitwash/development_workflow.html>`_
+page. Here are a few.
 
-When you install a package, only the package name can be imported directly. If
-you want a sub-package, you must import it from the main package. That's how
-it will work in 99.9% of installs because it is the default. Therefore, if you
-stray from this practice, your code will not be portable. Also, some ways to
-circumvent circular dependencies might make it so you have to import files in
-a certain order, which is best handled by the package's own ``__init__.py``.
 
-More instructions
+Clean up branches
 -----------------
 
-Once you have completed these steps, you should run the tests like this:
-
-.. code-block:: python
-
-   import theano
-   theano.test()
-
-Or by running ``nosetests`` from your checkout directory.
-
-All tests should pass. If some test fails on your machine, you are
-encouraged to tell us what went wrong on the `theano-users`_
-mailing list.
-
-.. _theano-users: https://groups.google.com/group/theano-users
-
-To update your library to the latest revision, you should have a branch
-that tracks the main trunk. You can add one with:
+When your pull request has been merged, you can delete the branch from
+your GitHub fork's list of branches. This is useful to avoid having too
+many branches staying there. Deleting this remote branch is achieved
+with:
 
 .. code-block:: bash
 
-    git fetch central
-    git branch trunk central/master
+   git push origin :my_shiny_feature
 
-Once you have such a branch, in order to update it, do:
+This lines pushes to the "origin" repository (your fork of Theano on
+GitHub), into the branch "my_shiny_feature", an empty content (that's
+why there is nothing before the colon), effectively removing it.
+
+The branch will still be present in your local clone of the repository.
+If you want to delete it from there, too, you can run:
 
 .. code-block:: bash
 
-    git checkout trunk
-    git pull
+   git branch -d my_shiny_feature
 
-Keep in mind that this branch should be "read-only": if you want to patch
-Theano, do it in another branch like described above.
 
-Optional
---------
+Edit a submitted pull request
+-----------------------------
 
-You can instruct git to do color diff. For this, you need to add those lines in the file ~/.gitconfig
+If you want to fix a commit already submitted within a pull request
+(e.g. to fix a small typo), before the pull request is accepted, you can
+do it like this to keep history clean:
 
 .. code-block:: bash
 
+   git checkout my_shiny_feature
+   git commit --amend
+   git push -u origin my_shiny_feature:my_shiny_feature
 
-    [color]
-       branch = auto
-       diff = auto
-       interactive = auto
-       status = auto
 
-Nightly test
-------------
+Cleaning up history
+-------------------
 
-Each night we execute all the unit tests automatically.  The result is sent by
-email to the `theano-buildbot`_ mailing list.
+Sometimes you may have commits in your feature branch that
+are not needed in the final pull request. There is a `page
+<http://sandofsky.com/blog/git-workflow.html>`_ that talks about
+this. In summary:
 
-.. _theano-buildbot: https://groups.google.com/group/theano-buildbot
+* Commits to the trunk should be a lot cleaner than commits to your
+  feature branch; not just for ease of reviewing but also
+  because intermediate commits can break blame (the bisecting tool).
+* `git merge --squash` will put all of the commits from your feature branch into one commit.
+* There are other tools that are useful if your branch is too big for one squash.
 
-For more detail, see :ref:`see <metadocumentation_nightly_build>`.
 
-To run all the tests with the same configuration as the buildbot, run this script:
+To checkout another user branch in his repo:
 
 .. code-block:: bash
 
-   theano/misc/do_nightly_build
-
-This function accepts arguments that it forward to nosetests. You can
-run only some tests or enable pdb by giving the equivalent nosetests
-parameters.
+   git remote add REPO_NAME HIS_REPO_PATH
+   git checkout -b LOCAL_BRANCH_NAME REPO_NAME/REMOVE_BRANCH_NAME

doc/index.txt

@@ -91,6 +91,9 @@ Check out how Theano can be used for Machine Learning: `Deep Learning Tutorials
 
 Theano was featured at `SciPy 2010 <http://www.iro.umontreal.ca/~lisa/publications2/index.php/publications/show/461>`__.
 
+
+.. _theano_community:
+
 Community
 =========
 

doc/install.txt

@@ -240,6 +240,8 @@ to your ``Theano`` folder and execute the following command:
 You should update frequently, bugs are fixed on a very regular basis.
 
 
+.. _testing_installation:
+
 Testing your installation
 ~~~~~~~~~~~~~~~~~~~~~~~~~