=============================
Multi cores support in Theano
=============================

BLAS operation
==============

BLAS is an interface for some mathematics operations between vectors,
vector and matrix and matrices (e.g. the dot product between vector/matrix
and matrix/matrix). Many different implementations exist of that
interface and some of them are parallel.

Theano try to use that interface as frequently as possible for
performance reason. So if Theano link to one parallel implementation,
those operation will run in parallel in Theano.

The most frequent way to control the number of threads used is via the
``OMP_NUM_THREADS`` environment variable. Set it to the number of threads
you want to use before starting the python process.


Parallel element wise op with OpenMP
====================================

Because element wise ops work on every tensor entry independently they can be
easily parallelized using OpenMP.

To use OpenMP you must set the OpenMP flag in Theano configuration.

You can use the flag ``openmp_elemwise_minsize`` to set the minimum tensor size
for which the operation is parallelized because for short tensor using OpenMP
can slow down the operation.

If it is no specified the default value ``200000`` is used.

For simple(fast) operation you can obtain a speed up for very long tensor
while for more complex operation you ca obtain a good speed up also for not
too long tensor. 

There is a script ``elemwise_openmp_speedup.py`` in ``theano/misc/`` which you
can use to choose that value for your machine.
The script run two elemwise operation (a fast and a slow one) for a vector of
size ``openmp_elemwise_minsize`` with and without OpenMP and show the time
difference between the two cases.





