提交 9f37dc88 authored 作者: Pascal Lamblin's avatar Pascal Lamblin

Fix typos, some rephrasing.

上级 b1737f4f
...@@ -4,10 +4,11 @@ ...@@ -4,10 +4,11 @@
Implementing some specific Op Implementing some specific Op
============================= =============================
This page guide on the implementation of some specify type of Ops. This page is a guide on the implementation of some specific types of Ops,
and point to some examples of such implementations.
For the random number, it explain the different implementation For the random number generating Ops, it explains different possible
strategy. implementation strategies.
.. _scalar_ops: .. _scalar_ops:
...@@ -15,19 +16,25 @@ strategy. ...@@ -15,19 +16,25 @@ strategy.
Scalar/Elemwise/Reduction Ops Scalar/Elemwise/Reduction Ops
============================= =============================
Implementing a Theano scalar allow that scalar operation to be reused Implementing a Theano scalar Op allows that scalar operation to be reused
by our elemwise operation. If the scalar operation it have c code, the by our elemwise operations on tensors. If the scalar operation has C code, the
elemwise implementation it will automaticaly have c code too. This elemwise implementation it will automaticaly have C code too. This
will enable the fusion of elemwise operation with your new scalar will enable the fusion of elemwise operations using your new scalar
operation. It can also reuse the GPU elemwise code. It is similar for operation. It can also reuse the GPU elemwise code. It is similar for
reduction operation. reduction operation.
There is those 2 PR that add `GammaLn and Psi For examples of how to add new scalar operations, you can have a look at
those 2 pull requests, that add `GammaLn and Psi
<https://github.com/Theano/Theano/pull/686/>`_ and `Gamma <https://github.com/Theano/Theano/pull/686/>`_ and `Gamma
<https://github.com/Theano/Theano/pull/826/>`_ scalar op. <https://github.com/Theano/Theano/pull/826/>`_ scalar Ops.
Take care Be careful about some possible problems in the definition of the
`Fix to grad() methods <https://github.com/Theano/Theano/commit/002872ad97919b97eaf58e095044e3c3067668e4>`_ and `impl() methods related to SciPy <https://github.com/Theano/Theano/commit/08d16c0aa6681fc53d8d0f40342551eb47ff536e>`_ ``grad`` method, and about dependencies that may not be available. In
particular, see the following fixes:
`Fix to grad() methods
<https://github.com/Theano/Theano/commit/002872ad97919b97eaf58e095044e3c3067668e4>`_
and `impl() methods related to SciPy
<https://github.com/Theano/Theano/commit/08d16c0aa6681fc53d8d0f40342551eb47ff536e>`_.
.. _scipy_ops: .. _scipy_ops:
...@@ -67,20 +74,21 @@ Here is some code that allows the Op to be optional: ...@@ -67,20 +74,21 @@ Here is some code that allows the Op to be optional:
Sparse Ops Sparse Ops
========== ==========
There is few differences if you want to make an op that use There are a few differences to keep in mind if you want to make an op
:ref:`sparse <tutsparse>` inputs or outputs. In particular, in the that uses :ref:`sparse <tutsparse>` inputs or outputs, rather than the
``make_node()`` function, you call usual dense tensors. In particular, in the
``theano.sparse.as_sparse_variable(x)`` on sparse input variable ``make_node()`` function, you have to call
``theano.sparse.as_sparse_variable(x)`` on sparse input variables,
instead of ``as_tensor_variable(x)``. instead of ``as_tensor_variable(x)``.
Another difference is that you need to use SparseVariable and Another difference is that you need to use ``SparseVariable`` and
SparseType instead of TensorVariable and TensorType. ``SparseType`` instead of ``TensorVariable`` and ``TensorType``.
Don't forget that we support only sparse matrix (so only 2 dimensions) Don't forget that we support only sparse matrices (so only 2 dimensions)
and they don't support broadcast operation by default as scipy sparse and they don't support broadcasting operation by default, as SciPy sparse
matrix (but a few op do it when called manually). Also, we support 2 matrix class does (but a few Ops do it when called manually). Also, we support 2
formats for sparse type: ``csr`` and ``csr``. So in ``make_mode()``, formats for sparse type: ``csr`` and ``csc``. So in ``make_mode()``,
you create outputs variables like this: you can create outputs variables like this:
.. code-block:: python .. code-block:: python
...@@ -89,14 +97,14 @@ you create outputs variables like this: ...@@ -89,14 +97,14 @@ you create outputs variables like this:
See the sparse :class:`theano.sparse.basic.Cast` op `code See the sparse :class:`theano.sparse.basic.Cast` op `code
<https://github.com/Theano/Theano/blob/master/theano/sparse/basic.py#L753>`_ <https://github.com/Theano/Theano/blob/master/theano/sparse/basic.py#L753>`_
for a good example for a sparse op with python code. for a good example for a sparse op with Python code.
.. note:: .. note::
From the definition of CSR and CSC format, CSR column indices are From the definition of CSR and CSC format, CSR column indices are
not necessarily sorted. Likewise for CSC row indices. Use not necessarily sorted. Likewise for CSC row indices. Use
:class:`EnsureSortedIndices :class:`EnsureSortedIndices
<theano.sparse.basic.EnsureSortedIndices>` if your code don't <theano.sparse.basic.EnsureSortedIndices>` if your code does not
support it. support it.
Also, there can be explicit zeros in your inputs. Use Also, there can be explicit zeros in your inputs. Use
...@@ -110,21 +118,23 @@ for a good example for a sparse op with python code. ...@@ -110,21 +118,23 @@ for a good example for a sparse op with python code.
Sparse Gradient Sparse Gradient
--------------- ---------------
There is 2 types of :ref:`gradients <tutsparse_gradient>` : ``normal`` There are 2 types of :ref:`gradients <tutsparse_gradient>` for sparse
operations: ``normal``
gradient and ``structured`` gradient. Please document what your op gradient and ``structured`` gradient. Please document what your op
implement in its docstring. It is important that the user know it and implements in its docstring. It is important that the user knows it, and
it is not always easy to infer from the code. Also make clear witch it is not always easy to infer from the code. Also make clear which
inputs/outputs are sparse and witch ones are dense. inputs/outputs are sparse and which ones are dense.
Sparse c code Sparse C code
------------- -------------
Theano don't have a native c code interface for sparse matrix. The Theano does not have a native C code interface for sparse matrices. The
reason is simple, we use the scipy sparse matrix object and they don't reason is simple, we use the SciPy sparse matrix object and they don't
have a c object. So we use a simple trick: a sparse matrix is made of have a C object. So we use a simple trick: a sparse matrix is made of
4 fields that are vector: data, indices, indptr and shape. So to make 4 fields that are NumPy vector arrays: ``data``, ``indices``, ``indptr``
an op with c code that have sparse variables as inputs, we make an op and ``shape``. So to make
that take as input the needed fields of those sparse variables. an op with C code that has sparse variables as inputs, we actually make an op
that takes as input the needed fields of those sparse variables.
You can extract the 4 fields with You can extract the 4 fields with
:func:`theano.sparse.basic.csm_properties`. You can use :func:`theano.sparse.basic.csm_properties`. You can use
...@@ -136,48 +146,49 @@ fields. ...@@ -136,48 +146,49 @@ fields.
You can look at the `AddSD You can look at the `AddSD
<https://github.com/Theano/Theano/blob/master/theano/sparse/basic.py#L1704>`_ <https://github.com/Theano/Theano/blob/master/theano/sparse/basic.py#L1704>`_
sparse op for an example with c code. It implement the addition of a sparse op for an example with C code. It implements the addition of a
sparse matrix with a dense matrix. sparse matrix with a dense matrix.
Sparse Tests Sparse Tests
------------ ------------
You can reuse the test system for tensor variable. To generate the You can reuse the test system for tensor variables. To generate the
needed sparse variable and data, you can use needed sparse variable and data, you can use
:func:`theano.sparse.tests.test_basic.sparse_random_inputs`. It take :func:`theano.sparse.tests.test_basic.sparse_random_inputs`. It takes
take many paramters including parameters for the format (csr or csc), the shape, the many parameters, including parameters for the format (csr or csc), the shape, the
dtype, to have explicit 0 and to have unsorted indices. dtype, whether to have explicit 0 and whether to have unsorted indices.
.. _random_ops: .. _random_ops:
Random distribution Random distribution
=================== ===================
We have 3 base random number generators. One that wrap NumPy random We have 3 base random number generators. One that wraps NumPy's random
generator, one that implement MRG31k3p and one that wrap CURAND. generator, one that implements MRG31k3p and one that wraps CURAND.
The fastest, but less developed is CURAND. It work only on CUDA enable The fastest, but less developed, is CURAND. It works only on CUDA-enabled
GPUs. It don't work on the CPU and it have less random distribution. GPUs. It does not work on the CPU and it has fewer random distributions
implemented.
The recommended and 2nd faster is MRG. It work on the GPU and CPU and The recommended and 2nd faster is MRG. It works on the GPU and CPU and
have more distribution. has more implemented distributions.
The slowest is our wrapper on NumPy random generator. The slowest is our wrapper on NumPy's random generator.
We explain and guide on 3 possibles implementations of new We explain and provide advice on 3 possibles implementations of new
distribution here:: distributions here::
1) Extend our wrapper to NumPy random function. 1) Extend our wrapper around NumPy random functions.
See this `PR <https://github.com/Theano/Theano/pull/1607>`_ as an example. See this `PR <https://github.com/Theano/Theano/pull/1607>`_ as an example.
2) Extend MRG implementation by reusing existing Theano Op. Look into 2) Extend MRG implementation by reusing existing Theano Op. Look into
the ``theano/sandbox/rng_mrg.py`` file and grep for all code about the ``theano/sandbox/rng_mrg.py`` file and grep for all code about
binomal(). This distribution use the output of the uniform binomal(). This distribution uses the output of the uniform
distribution and convert it to a binomial distribution with distribution and converts it to a binomial distribution with
existing Theano op. The test go in existing Theano operations. The tests go in
``theano/sandbox/test_rng_mrg.py`` ``theano/sandbox/test_rng_mrg.py``
3) Extend MRG implementation with a new Op that take an uniform as 3) Extend MRG implementation with a new Op that takes an uniform as
input. Look in the ``theano/sandbox/{rng_mrg,multinomial}.py`` file input. Look in the ``theano/sandbox/{rng_mrg,multinomial}.py`` file
and its test in ``theano/sandbox/test_multinomal.py``. This is and its test in ``theano/sandbox/test_multinomal.py``. This is
recommended when current Theano ops aren't well suited to modify recommended when current Theano ops aren't well suited to modify
...@@ -195,24 +206,24 @@ OpenMP Ops ...@@ -195,24 +206,24 @@ OpenMP Ops
========== ==========
To allow consistent interface of Ops that support OpenMP, we have some To allow consistent interface of Ops that support OpenMP, we have some
helper code. Doing this also allow to enable/disable OpenMP globally helper code. Doing this also allows to enable/disable OpenMP globally
or per op for fine grin control. or per op for fine-grained control.
Your Op need to inherit from ``theano.gof.OpenMPOp``. If it override Your Op needs to inherit from ``theano.gof.OpenMPOp``. If it overrides
the ``__init__()`` method, it must have an ``openmp=None`` parameter the ``__init__()`` method, it must have an ``openmp=None`` parameter
and must call ``super(MyOpClass, self).__init__(openmp=openmp)``. and must call ``super(MyOpClass, self).__init__(openmp=openmp)``.
The ``OpenMPOp`` class also implement ``c_compile_args`` and The ``OpenMPOp`` class also implements ``c_compile_args`` and
``make_thunk``. This make it add the correct g++ flag to compile with ``make_thunk``. This makes it add the correct g++ flag to compile with
OpenMP. It also disable OpenMP and print a warning if the version of OpenMP. It also disables OpenMP and prints a warning if the version of
g++ don't support it. g++ don't support it.
The Theano flag ``openmp`` is currently False by default as we don't The Theano flag ``openmp`` is currently False by default as we don't
have code that get speed up with it. The only current implementation have code that gets speed up with it. The only current implementation
is ConvOp. It speed up some cases, but slow down others. That is why is ConvOp. It speeds up some cases, but slows down others. That is why
we disable it by default. But we have all the code to have it enabled we disable it by default. But we have all the code to have it enabled
by default if there is more then 1 cores and that the environment by default if there is more then 1 core and that the environment
variable OMP_NUM_THREADS isn't 1. This allow Theano to respect the variable OMP_NUM_THREADS isn't 1. This allows Theano to respect the
current convention. current convention.
.. note: .. note:
...@@ -228,6 +239,6 @@ current convention. ...@@ -228,6 +239,6 @@ current convention.
Numba Ops Numba Ops
========= =========
Want C speed without doing C code for your new Op? You can use Numba Want C speed without writing C code for your new Op? You can use Numba
to generate the C code for you! Here is an `example to generate the C code for you! Here is an `example
Op <https://gist.github.com/nouiz/5492778#file-theano_op-py>`_ doing that. Op <https://gist.github.com/nouiz/5492778#file-theano_op-py>`_ doing that.
Markdown 格式
0%
您添加了 0 到此讨论。请谨慎行事。
请先完成此评论的编辑!
注册 或者 后发表评论