There are many ways to configure BLAS for Theano. This is done with the
There are many ways to configure BLAS for Theano. This is done with the Theano
Theano flags blas.ldflags (:ref:`libdoc_config`). The default is to use
flags ``blas.ldflags`` (:ref:`libdoc_config`). The default is to use the BLAS
the blas installation information in numpy, accessible via ``numpy.distutils.__config__.show()``.
installation information in NumPy, accessible via
You can tell theano to use a different
``numpy.distutils.__config__.show()``. You can tell theano to use a different
version of BLAS, in case you didn't compile numpy with a fast BLAS or if numpy was compiled with a
version of BLAS, in case you didn't compile numpy with a fast BLAS or if numpy
static library of blas (the latter is not supported in theano).
was compiled with a static library of BLAS (the latter is not supported in
Theano).
The short way to configure the theano flags blas.ldflags is by setting the
environmental variable ``THEANO_FLAGS`` to ``blas.ldflags=XXX``
The short way to configure the Theano flags ``blas.ldflags`` is by setting the
(in bash ``export THEANO_FLAGS=blas.ldflags=XXX``)
environmental variable ``THEANO_FLAGS`` to ``blas.ldflags=XXX`` (in bash
``export THEANO_FLAGS=blas.ldflags=XXX``)
The ``${HOME}/.theanorc`` file is the simplest way to set a relatively permanent
option like this one. Add a ``[blas]`` section with an ``ldflags``
The ``${HOME}/.theanorc`` file is the simplest way to set a relatively
permanent option like this one. Add a ``[blas]`` section with an ``ldflags``
entry like this:
entry like this:
.. code-block:: text
.. code-block:: text
...
@@ -154,20 +159,20 @@ entry like this:
...
@@ -154,20 +159,20 @@ entry like this:
# other stuff can go here
# other stuff can go here
For more information on the formatting of ``~/.theanorc`` and the configuration options that you can put there,
For more information on the formatting of ``~/.theanorc`` and the
see :ref:`libdoc_config`.
configuration options that you can put there, see :ref:`libdoc_config`.
Here are some different way to configure BLAS:
Here are some different way to configure BLAS:
0) Do nothing and use the default config, which is to link against the same BLAS that numpy was built with. This doesn't work in the case numpy was compiled with a static library (e.g. ATLAS is compiled by default only as a static lib).
0) Do nothing and use the default config, which is to link against the same BLAS against which NumPy was built. This doesn't work in the case NumPy was compiled with a static library (e.g. ATLAS is compiled by default only as a static library).
1) Disable the usage of BLAS and fall back on numpy for dot products. To do this, set the value of blas.ldflags as the empty string (ex: ``export THEANO_FLAGS=blas.ldflags=``). Depending on the kind of matrix operations your theano code performs, this might slow some things down (vs. linking with BLAS directly).
1) Disable the usage of BLAS and fall back on NumPy for dot products. To do this, set the value of ``blas.ldflags`` as the empty string (ex: ``export THEANO_FLAGS=blas.ldflags=``). Depending on the kind of matrix operations your Theano code performs, this might slow some things down (vs. linking with BLAS directly).
2) You can install the default (reference) version of BLAS if the numpy version (which theano links against) doesn't work. If you have root or sudo access in fedora you can do ``sudo yum install blas blas-devel``. Under Ubuntu/Debian ``sudo apt-get install libblas-dev``. Then put the theano flags ``blas.ldflags=-lblas``. Not that the default version of blas is not optimized. Using an optimized version can give up to 10x speedups in the BLAS functions that we use.
2) You can install the default (reference) version of BLAS if the NumPy version (against which Theano links) doesn't work. If you have root or sudo access in fedora you can do ``sudo yum install blas blas-devel``. Under Ubuntu/Debian ``sudo apt-get install libblas-dev``. Then use the Theano flags ``blas.ldflags=-lblas``. Not that the default version of blas is not optimized. Using an optimized version can give up to 10x speedups in the BLAS functions that we use.
3) Install the ATLAS library. ATLAS is an open source optimized version of BLAS. You can install a precompiled version on most OSes, but if you're willing to invest the time, you can compile it to have a faster version (we have seen speed-ups of up to 3x, especialy on more recent computers, against the precompiled one). On Fedora, ``sudo yum install atlas-devel``. Under Ubuntu, ``sudo apt-get install libatlas-base-dev libatlas-base`` or ``libatlas3gf-sse2`` if your CPU supports SSE2 instructions. Then set the theano flags ``blas.ldflags`` to ``-lf77blas -latlas -lgfortran``. Note that these flags are sometimes OS dependent.
3) Install the ATLAS library. ATLAS is an open source optimized version of BLAS. You can install a precompiled version on most OSes, but if you're willing to invest the time, you can compile it to have a faster version (we have seen speed-ups of up to 3x, especialy on more recent computers, against the precompiled one). On Fedora, ``sudo yum install atlas-devel``. Under Ubuntu, ``sudo apt-get install libatlas-base-dev libatlas-base`` or ``libatlas3gf-sse2`` if your CPU supports SSE2 instructions. Then set the Theano flags ``blas.ldflags`` to ``-lf77blas -latlas -lgfortran``. Note that these flags are sometimes OS dependent.
4) Use a faster version like MKL, GOTO, ... You are on your own to install it. See the doc of that software and set the theano flags ``blas.ldflags`` correctly (for example, for MKL this might be ``-lmkl -lguide -lpthread`` or ``-lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lguide -liomp5 -lmkl_mc -lpthread``).
4) Use a faster version like MKL, GOTO, ... You are on your own to install it. See the doc of that software and set the Theano flags ``blas.ldflags`` correctly (for example, for MKL this might be ``-lmkl -lguide -lpthread`` or ``-lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lguide -liomp5 -lmkl_mc -lpthread``).
.. note::
.. note::
...
@@ -208,21 +213,21 @@ Mac
...
@@ -208,21 +213,21 @@ Mac
$ sudo port install gcc44 py25-scipy mercurial python_select
$ sudo port install gcc44 py25-scipy mercurial python_select
This will install all the required theano dependencies. Note that
This will install all the required Theano dependencies. Note that
compiling gcc takes significant time (hours)! SciPy depends on ATLAS (a
compiling gcc takes significant time (hours)! SciPy depends on ATLAS (a
good BLAS implementation) and numpy, so these will be installed for you automatically.
good BLAS implementation) and NumPy, so these will be installed for you automatically.
- You might have some old versions of gcc, SciPy, numpy, Python installed on
- You might have some old versions of gcc, SciPy, NumPy, Python installed on
your system, perhaps via XCode. It is a good idea to use **either** the
your system, perhaps via Xcode. It is a good idea to use **either** the
MacPorts version of everything **or** some other set of compatible versions
MacPorts version of everything **or** some other set of compatible versions
(provided by fink, or by XCode). The advantages of MacPorts are the
(provided by Fink, or by Xcode). The advantages of MacPorts are the
transparency with which everything can be installed and the fact that
transparency with which everything can be installed and the fact that
packages are updated quite frequently.
packages are updated quite frequently.
- In order to use the MacPorts version of python, you might
- In order to use the MacPorts version of python, you might
need to explicitly select it with ``sudo python_select python25``. The
need to explicitly select it with ``sudo python_select python25``. The
reason this is necessary is because you might have an Apple-provided python
reason this is necessary is because you might have an Apple-provided python
(via, for example, an XCode installation). After performing this step, you
(via, for example, an Xcode installation). After performing this step, you
should check that the symbolic link provided by ``which python`` points to
should check that the symbolic link provided by ``which python`` points to
the MacPorts python. For instance, on Snow Leopard with the latest MacPorts,
the MacPorts python. For instance, on Snow Leopard with the latest MacPorts,
the output of ``which python`` is ``/opt/local/bin/python`` and this symbolic
the output of ``which python`` is ``/opt/local/bin/python`` and this symbolic
...
@@ -230,7 +235,7 @@ Mac
...
@@ -230,7 +235,7 @@ Mac
python_select python26-apple`` (which you should **not** do), the link
python_select python26-apple`` (which you should **not** do), the link
points to ``/usr/bin/python2.6``.
points to ``/usr/bin/python2.6``.
- Once this is fixed, please check that the scipy module that is imported in
- Once this is fixed, please check that the ``scipy`` module that is imported in
Python is the right one (and is a recent one). For instance, ``import
Python is the right one (and is a recent one). For instance, ``import
scipy`` followed by ``print scipy.version`` and ``print scipy.__path__``
scipy`` followed by ``print scipy.version`` and ``print scipy.__path__``
should result in a version number of at least 0.7.0 and a path that starts
should result in a version number of at least 0.7.0 and a path that starts
...
@@ -242,11 +247,11 @@ Mac
...
@@ -242,11 +247,11 @@ Mac
- Put ``export PYTHONPATH=/opt/local/lib/python2.5/site-packages:$PYTHONPATH``
- Put ``export PYTHONPATH=/opt/local/lib/python2.5/site-packages:$PYTHONPATH``
in your ``.bashrc`` in order to include your MacPorts Python packages
in your ``.bashrc`` in order to include your MacPorts Python packages
(numpy, scipy) in Python's path.
(NumPy, SciPy) in Python's path.
- Make sure that the gcc version that you have installed on your system is
- Make sure that the gcc version that you have installed on your system is
up-to-date (at the very least 3.4, but 4.x is better). If you have an old
up-to-date (at the very least 3.4, but 4.x is better). If you have an old
version of XCode lying around, chances are that your gcc install is old. You
version of Xcode lying around, chances are that your gcc install is old. You
should also check ``which gcc``: if it says ``/usr/bin/gcc`` then you
should also check ``which gcc``: if it says ``/usr/bin/gcc`` then you
should install gcc_select from MacPorts (``sudo port install gcc_select``)
should install gcc_select from MacPorts (``sudo port install gcc_select``)
and use the MacPorts-provided gcc. Use ``gcc_select -l`` to see which gcc
and use the MacPorts-provided gcc. Use ``gcc_select -l`` to see which gcc
...
@@ -275,7 +280,7 @@ Mac
...
@@ -275,7 +280,7 @@ Mac
Theano-generated object files against the ``framework`` library in Leopard.
Theano-generated object files against the ``framework`` library in Leopard.
For this reason, we've disabled linking with ``-framework Python``, since on
For this reason, we've disabled linking with ``-framework Python``, since on
most configurations this solves the ``Bus error`` problem. If this default
most configurations this solves the ``Bus error`` problem. If this default
configuration causes problems with your Python/theano installation and you think
configuration causes problems with your Python/Theano installation and you think
that linking with ``-framework Python`` might help, then either set
that linking with ``-framework Python`` might help, then either set
``THEANO_FLAGS=cmodule.mac_framework_link`` or edit your ``~/.theanorc`` to
``THEANO_FLAGS=cmodule.mac_framework_link`` or edit your ``~/.theanorc`` to
contain
contain
...
@@ -297,7 +302,7 @@ Windows V1 (bigger install, but simpler instructions + tentative GPU instruction
...
@@ -297,7 +302,7 @@ Windows V1 (bigger install, but simpler instructions + tentative GPU instruction
- Install `Python(x,y) <http://www.pythonxy.com>`_ in a directory without blank
- Install `Python(x,y) <http://www.pythonxy.com>`_ in a directory without blank
spaces in the name (in particular not into ``C:\Program Files``).
spaces in the name (in particular not into ``C:\Program Files``).
It is a single installation
It is a single installation
file that contains additional packages like Numpy, Scipy, IPython, Matplotlib,
file that contains additional packages like NumPy, SciPy, IPython, Matplotlib,
MinGW, Nose, etc. Note that this implies you do not already have a Python
MinGW, Nose, etc. Note that this implies you do not already have a Python
installation (if you do have one, then you will need to either remove it first,
installation (if you do have one, then you will need to either remove it first,
or install those additional packages manually as described in the V2 instructions).
or install those additional packages manually as described in the V2 instructions).
...
@@ -339,7 +344,7 @@ Windows V1 (bigger install, but simpler instructions + tentative GPU instruction
...
@@ -339,7 +344,7 @@ Windows V1 (bigger install, but simpler instructions + tentative GPU instruction
- You are now ready to run Theano.
- You are now ready to run Theano.
It will use NumPy for dot products, which is still pretty fast (see below for
It will use NumPy for dot products, which is still pretty fast (see below for
optional instructions on how to compile your own BLAS library).
optional instructions on how to compile your own BLAS library).
To test that theano correctly reads your configuration file, run Python
To test that Theano correctly reads your configuration file, run Python
(e.g. by just typing ``python`` in a shell) and run the following code:
(e.g. by just typing ``python`` in a shell) and run the following code:
.. code-block:: python
.. code-block:: python
...
@@ -373,7 +378,7 @@ Windows V1.5 (optional follow-up to V1 instructions)
...
@@ -373,7 +378,7 @@ Windows V1.5 (optional follow-up to V1 instructions)
"View all files", and make sure you do not mix it up with
"View all files", and make sure you do not mix it up with
mingw-get-inst).
mingw-get-inst).
b) Unpack it into your pythonxy\mingw directory.
b) Unpack it into your ``pythonxy\mingw`` directory.
c) On the command-line, install MSYS with
c) On the command-line, install MSYS with
...
@@ -382,7 +387,7 @@ Windows V1.5 (optional follow-up to V1 instructions)
...
@@ -382,7 +387,7 @@ Windows V1.5 (optional follow-up to V1 instructions)
mingw-get install msys-base
mingw-get install msys-base
d) Create an easily accessible shortcut (e.g. on your desktop) to
d) Create an easily accessible shortcut (e.g. on your desktop) to
pythonxy\mingw\msys\1.0\msys.bat. Run it and within the MSYS
``pythonxy\mingw\msys\1.0\msys.bat``. Run it and within the MSYS
console, run the MSYS post-install script:
console, run the MSYS post-install script:
.. code-block:: bash
.. code-block:: bash
...
@@ -390,7 +395,7 @@ Windows V1.5 (optional follow-up to V1 instructions)
...
@@ -390,7 +395,7 @@ Windows V1.5 (optional follow-up to V1 instructions)
/postinstall/pi.sh
/postinstall/pi.sh
It will ask for your MinGW installation directory (e.g.
It will ask for your MinGW installation directory (e.g.
c:/pythonxy/mingw).
``c:\pythonxy\mingw``).
e) Download `ActivePerl <http://www.activestate.com/activeperl>`_ and
e) Download `ActivePerl <http://www.activestate.com/activeperl>`_ and
install it.
install it.
...
@@ -414,8 +419,8 @@ Windows V1.5 (optional follow-up to V1 instructions)
...
@@ -414,8 +419,8 @@ Windows V1.5 (optional follow-up to V1 instructions)
find many error messages in err.txt, but also a libgoto2.dll
find many error messages in err.txt, but also a libgoto2.dll
file in the exports folder. [NOTE: INSTRUCTIONS TO BE CONTINUED]
file in the exports folder. [NOTE: INSTRUCTIONS TO BE CONTINUED]
i) Copy libgoto2.dll from the exports folder to pythonxy\mingw\bin
i) Copy libgoto2.dll from the exports folder to ``pythonxy\mingw\bin``
and pythonxy\mingw\lib.
and ``pythonxy\mingw\lib``.
j) Modify your .theanorc (or .theanorc.txt) with "ldflags = -lgoto2".
j) Modify your .theanorc (or .theanorc.txt) with "ldflags = -lgoto2".
This setting can also be changed in Python for testing purposes:
This setting can also be changed in Python for testing purposes:
...
@@ -424,7 +429,7 @@ Windows V1.5 (optional follow-up to V1 instructions)
...
@@ -424,7 +429,7 @@ Windows V1.5 (optional follow-up to V1 instructions)
theano.config.blas.ldflags = "-lgoto2"
theano.config.blas.ldflags = "-lgoto2"
- (Optional). To test the BLAS performance, you can run the script check_blas.py.
- (Optional). To test the BLAS performance, you can run the script ``check_blas.py``.
For comparison I also downloaded and compiled the unoptimized standard
For comparison I also downloaded and compiled the unoptimized standard
BLAS. The results were the following (Intel Core2 Duo 1.86 GHz):
BLAS. The results were the following (Intel Core2 Duo 1.86 GHz):
...
@@ -439,7 +444,7 @@ Windows V1.5 (optional follow-up to V1 instructions)
...
@@ -439,7 +444,7 @@ Windows V1.5 (optional follow-up to V1 instructions)
- (Optional) Gpu on Windows. Not sur it work! Can you report success/error on the `theano-users <http://groups.google.com/group/theano-users>`_ mailing list?
- (Optional) Gpu on Windows. Not sur it work! Can you report success/error on the `theano-users <http://groups.google.com/group/theano-users>`_ mailing list?
Those are indication for 32 bits version of python, the one that come with pythonxy is 32 bits.
Those are indication for 32-bit version of Python, the one that come with Python(x,y) is 32-bit.
Space or non ascii caracter are not always supported in path. Python support
Space or non ascii caracter are not always supported in path. Python support
them, so your configuration file path can contain them.
them, so your configuration file path can contain them.
...
@@ -460,34 +465,34 @@ Windows V1.5 (optional follow-up to V1 instructions)
...
@@ -460,34 +465,34 @@ Windows V1.5 (optional follow-up to V1 instructions)
Then
Then
1) Install cuda driver(32 bits on 32 bits Windows, idem for 64 bits).
1) Install CUDA driver (32-bit on 32-bit Windows, idem for 64-bit).
2) Install cuda toolkit 32 bits(even if you computer is 64 bits,
2) Install CUDA toolkit 32-bit (even if you computer is 64-bit,
must match the python installation version)
must match the Python installation version)
3) Install cuda sdk 32 bits
3) Install CUDA SDK 32-bit
4) Test some pre-compiled example of the sdk
4) Test some pre-compiled example of the sdk
5) Download Visual Studio 2008 Express(free, VS2010 not supported by nvcc 3.1,
5) Download Visual Studio 2008 Express(free, VS2010 not supported by nvcc 3.1,
VS2005, not available for download, but supported by nvcc, the non free version should work too)
VS2005, not available for download, but supported by nvcc, the non free version should work too)
6) Follow the instruction in the GettingStartedWindows.pdf file from cuda web
6) Follow the instruction in the GettingStartedWindows.pdf file from CUDA web
site to compile cuda code with VS2008. If that don't work, you won't be
site to compile CUDA code with VS2008. If that don't work, you won't be
able to compile gpu code with theano.
able to compile GPU code with Theano.
7) Put into you PATH environment variable the directory where cl.exe is.
7) Put into you PATH environment variable the directory where cl.exe is.
In my case it is: C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin
In my case it is: ``C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin``
8) Make sure the theano folder is in your PYTHONPATH environment variable.
8) Make sure the Theano folder is in your ``PYTHONPATH`` environment variable.
9) Then in theano do: import theano.sandbox.cuda
9) Then in Python do: ``import theano.sandbox.cuda``
That will print some error if their is an error to compile the first cuda file.
That will print some error if their is an error to compile the first CUDA file.
10) Then run the theano cuda test file. In Windows command line (cmd.exe),
10) Then run the Theano CUDA test file. In Windows command line (cmd.exe),
run the program nosetests inside the theano repository.
run the program nosetests inside the Theano repository.
nosetests is installed by pythonxy.
nosetests is installed by Python(x,y).
Windows V2(smaller install, but longer instruction)
Windows V2(smaller install, but longer instruction)
