Skip to content

P
pytensor
提交 fe3ff975 authored 作者: Nicolas Bouchard's avatar Nicolas Bouchard
浏览文件
操作

Move format explanation to doc, added new doc and updated list of ops.

上级 2d18b424
隐藏空白字符变更
内嵌 并排
正在显示 包含 221 行增加 和 118 行删除
doc/library/sparse/index.txt
浏览文件 @ fe3ff975
.. _libdoc_sparse:
===========================================================
:mod:`sparse` -- Symbolic Sparse Matrices [doc TODO]
===========================================================
=========================================
:mod:`sparse` -- Symbolic Sparse Matrices
=========================================
The sparse module has been used in:
In the tutorial section, you can find a :ref:`sparse tutorial
<tutsparse>`.
- NLP: Dense linear transformations of sparse vectors.
- Audio: Filterbank in Fourier domain.
The sparse module is less mature than the tensor module.
This documentation is also not mature.
The sparse submodule is not loaded when we import theano. You must import theano.sparse to enable it.
The sparse module provides two kinds of sparse tensors are supported: CSC matrices and CSR matrices.
Operations that are implemented:
grad?
- conversion from sparse <-> dense
- theano.sparse.{dense_from_sparse,dense_from_sparse}
-
The sparse submodule is not loaded when we import Theano. You must
import ``theano.sparse`` to enable it.
- [un]packing of sparse matrices from indexlists and nonzero elements.
- packing: theano.sparse.{CRC,CSR}
- unpacking: theano.sparse.csm_properties
The sparse module provide the same functionalities as the tensor
module. The difference lies under the cover because sparse matrices
does not store data in a contiguous array. Note that there are no GPU
implementations for sparse matrices implemented in Theano. The sparse
module has been used in:
- transpose
- theano.sparse.transpose
- negation
- neg
- addition/multiplication (elemwise)
- theano.sparse.{add,mul}
- sparse + sparse, sparse + dense, dense + sparse
- sparse * sparse, sparse * dense, dense * sparse
- StructuredDot
- with gradient defined such that sparsity pattern is
constant. This function is called "structured_dot"
- theano.sparse.structured_dot and its grad (structured_dot_grad)
- theano.dot call it.
- dot(sparse, dense) and dot(dense, sparse), dot(sparse, sparse)
- NLP: Dense linear transformations of sparse vectors.
- Audio: Filterbank in Fourier domain.
- Dot
- performs the true dot without special semantics.
- dot(sparse, dense), dot(dense, sparse), dot(sparse, sparse)
- When the operation has the form dot(csr_matrix, dense) the gradient of
this operation can be performed inplace by UsmmCscDense. This leads to
significant speed-ups.
- Subtensor
Compressed Sparse Format
========================
This section tries to explain how information is store for the two
sparse formats of SciPy supported by Theano. There is more formats
that can be used with SciPy and some documentation about them may be
found `here
<http://deeplearning.net/software/theano/sandbox/sparse.html>`_.
.. Changes to this section should also result in changes to tutorial/sparse.txt.
Theano supports two *compressed sparse formats* ``csc`` and ``csr``,
respectively based on columns and rows. They have both the same
attributes: ``data``, ``indices``, ``indptr`` and ``shape``.
* The ``data`` attribute is a one-dimentionnal ``ndarray`` which
contains all the non-zero elements of the sparse matrix.
* The ``indices`` and ``indptr`` attributes are used to store the
position of the data in the sparse matrix.
* The ``shape`` attribute is exactly the same as the ``shape``
attribute of a dense (i.e. generic) matrix. It can be explicitly
specified at the creation of a sparse matrix if it cannot be
infered from the first three attributes.
CSC Matrix
----------
In the *Compressed Sparse Column* format, ``indices`` stands for index
inside the column vectors of the matrix and ``indptr`` tells where the
column starts in the ``data`` and in the ``indices``
attributes. ``indptr`` can be tought as giving the slice which must be
applied to the other attribute in order to get each column of the
matrix. In other words, ``slice(indptr[i], indptr[i+1])`` correspond
to the slice needed to find the i-th column of the matrix in the
``data`` and in the ``indices`` fields.
The following example builds a matrix and returns its columns. It
prints the i-th column, i.e. a list of indices in the column and their
corresponding value in the second list.
>>> data = np.asarray([7, 8, 9])
>>> indices = np.asarray([0, 1, 2])
>>> indptr = np.asarray([0, 2, 3, 3])
>>> m = sp.csc_matrix((data, indices, indptr), shape=(3, 3))
>>> print m.toarray()
[[7 0 0]
[8 0 0]
[0 9 0]]
>>> i = 0
>>> print m.indices[m.indptr[i]:m.indptr[i+1]], m.data[m.indptr[i]:m.indptr[i+1]]
[0, 1] [7, 8]
>>> i = 1
>>> print m.indices[m.indptr[i]:m.indptr[i+1]], m.data[m.indptr[i]:m.indptr[i+1]]
[2] [9]
>>> i = 2
>>> print m.indices[m.indptr[i]:m.indptr[i+1]], m.data[m.indptr[i]:m.indptr[i+1]]
[] []
CSR Matrix
----------
In the *Compressed Sparse Row* format, ``indices`` stands for index
inside the row vectors of the matrix and ``indptr`` tells where the
row starts in the ``data`` and in the ``indices``
attributes. ``indptr`` can be tought as giving the slice which must be
applied to the other attribute in order to get each row of the
matrix. In other words, ``slice(indptr[i], indptr[i+1])`` correspond
to the slice needed to find the i-th row of the matrix in the ``data``
and in the ``indices`` fields.
The following example builds a matrix and returns its rows. It prints
the i-th row, i.e. a list of indices in the row and their corresponding value
in the second list.
>>> data = np.asarray([7, 8, 9])
>>> indices = np.asarray([0, 1, 2])
>>> indptr = np.asarray([0, 2, 3, 3])
>>> m = sp.csr_matrix((data, indices, indptr), shape=(3, 3))
>>> print m.toarray()
[[7 8 0]
[0 0 9]
[0 0 0]]
>>> i = 0
>>> print m.indices[m.indptr[i]:m.indptr[i+1]], m.data[m.indptr[i]:m.indptr[i+1]]
[0, 1] [7, 8]
>>> i = 1
>>> print m.indices[m.indptr[i]:m.indptr[i+1]], m.data[m.indptr[i]:m.indptr[i+1]]
[2] [9]
>>> i = 2
>>> print m.indices[m.indptr[i]:m.indptr[i+1]], m.data[m.indptr[i]:m.indptr[i+1]]
[] []
List of Implemented Operations
==============================
- Moving from and to sparse
- :class:`DenseFromSparse` and ``dense_from_sparse``
- :class:`SparseFromDense` and ``csr_from_dense``, ``csc_from_dense``
- Construction of Sparses and their Properties
- :class:`CSC` to construct a ``csc`` matrix
- :class:`CSR` to construct a ``csr`` matrix
- :class:`CSMProperties` to get the properties of a sparse matrix
- ``sp_ones_like``
- ``sp_zeros_like``
- :class:`SquareDiagonal` and ``square_diagonal``
- Cast
- :class:`Cast` with ``bcast``, ``wcast``, ``icast``, ``lcast``,
``fcast``, ``dcast``, ``ccast``, and ``zcast``
- Transpose
- :class:`Transpose` and ``transpose``
- Basic Arithmetic
- :class:`Neg` for negation
- ``add`` for addition
- ``sub`` for substraction
- ``mul`` for multiplication
- ``col_scale`` to multiply by a vector along the columns
- ``row_slace`` to multiply by a vector along the rows
- Monoid (Is applied only with one sparse as input)
- ``structured_sigmoid``
- ``structured_exp``
- ``structured_log``
- ``structured_pow``
- ``structured_minimum``
- ``structured_maximum``
- ``structured_add``
- ``sin``
- ``arcsin``
- ``tan``
- ``arctan``
- ``sinh``
- ``arcsinh``
- ``tanh``
- ``arctanh``
- ``rint``
- ``ceil``
- ``floor``
- ``sgn``
- ``log1p``
- ``sqr``
- ``sqrt``
- Dot Product
- :class:`Dot` and ``dot``
- :class:`StructuredDot` and ``structured_dot``
- :class:`SamplingDot` and ``sampling_dot``
- :class:`Usmm` and ``usmm``
- Slice Operations
- sparse_variable[N, N], return a tensor scalar
- sparse_variable[M:N, O:P], return a sparse matrix
- Don't support [M, N:O] and [M:N, O] as we don't support sparse vector
- Sparse variable don't support [M, N:O] and [M:N, O] as we don't support sparse vector
and returning a sparse matrix would break the numpy interface.
Use [M:M+1, N:O] and [M:N, O:O+1] instead.
There are no GPU implementations for sparse matrices implemented in Theano.
Some documentation for sparse has been written
`here <http://deeplearning.net/software/theano/sandbox/sparse.html>`_.
- :class:`Diag` and ``diag``
- Concatenation
- :class:`HStack` and ``hstack``
- :class:`VStack` and ``vstack``
- Probability
- :class:`Poisson` and ``poisson``
- :class:`Binomial` and ``csc_fbinomial``, ``csc_dbinomial``
``csr_fbinomial``, ``csr_dbinomial``
- :class:`Multinomial` and ``multinomial``
- Internal Representation
- :class:`EnsureSortedIndices` and ``ensure_sorted_indices``
- :class:`Remove0` and ``remove0``
- ``clean`` to resort indices and remove zeros
===================================================================
:mod:`sparse` -- Sparse Op
......
doc/tutorial/sparse.txt
浏览文件 @ fe3ff975
......@@ -46,86 +46,54 @@ perhaps a tensor variable could be a better choice.
More documentation may be found in the :ref:`Sparse Library Reference <libdoc_sparse>`.
Before going further, here are the ``import`` statements that are assumed for the rest of the
tutorial:
>>> import theano
>>> import numpy as np
>>> import scipy.sparse as sp
>>> from theano import sparse
Compressed Sparse Format
========================
.. Changes to this section should also result in changes to library/sparse/index.txt.
Theano supports two *compressed sparse formats* ``csc`` and ``csr``, respectively based on columns
and rows. They have both the same attributes: ``data``, ``indices``, ``indptr`` and ``shape``.
* The ``shape`` attribute is exactly the same as the ``shape`` attribute of a dense (i.e. generic)
matrix. It can be explicitly specified at the creation of a sparse matrix if it cannot be infered
from the first three attributes.
* The ``data`` attribute is a one-dimentionnal ``ndarray`` which contains all the non-zero
elements of the sparse matrix.
* The ``indices`` and ``indptr`` attributes are used to store the position of the data in the
sparse matrix.
Before going further, here are the ``import`` statements that are assumed for the rest of the
tutorial:
* The ``shape`` attribute is exactly the same as the ``shape`` attribute of a dense (i.e. generic)
matrix. It can be explicitly specified at the creation of a sparse matrix if it cannot be infered
from the first three attributes.
>>> import theano
>>> import numpy as np
>>> import scipy.sparse as sp
>>> from theano import sparse
Which format should I use?
--------------------------
CSC Matrix
----------
At the end, the format does not affect the length of the ``data`` and ``indices`` attributes. They are both
completly fixed by the number of elements you want to store. The only thing that changes with the format
is ``indptr``. In ``csc`` format, the matrix is compressed along columns so a lower number of columns will
result in less memory use. On the other hand, with the ``csr`` format, the matrix is compressed along
the rows and with a matrix that have a lower number of rows, ``csr`` format is a better choice. So here is the rule:
In the *Compressed Sparse Column* format, ``indices`` stands for the column index
of the data and ``indptr`` tells where the column starts in the ``data`` and in the
``indices`` attributes. ``indptr`` can be tought as giving the slice which must be
applied to the other attribute in order to get each column of the matrix. The following
example builds a matrix and returns its columns. It prints the i-th column indexes in the
first list and their corresponding value in the second list.
>>> data = np.asarray([7, 8, 9])
>>> indices = np.asarray([0, 1, 2])
>>> indptr = np.asarray([0, 2, 3, 3])
>>> m = sp.csc_matrix((data, indices, indptr), shape=(3, 3))
>>> print m.toarray()
[[7 0 0]
[8 0 0]
[0 9 0]]
>>> i = 0
>>> print m.indices[m.indptr[i]:m.indptr[i+1]], m.data[m.indptr[i]:m.indptr[i+1]]
[0, 1] [7, 8]
>>> i = 1
>>> print m.indices[m.indptr[i]:m.indptr[i+1]], m.data[m.indptr[i]:m.indptr[i+1]]
[2] [9]
>>> i = 2
>>> print m.indices[m.indptr[i]:m.indptr[i+1]], m.data[m.indptr[i]:m.indptr[i+1]]
[] []
CSR Matrix
----------
.. note::
If shape[0] > shape[1], use ``csr`` format. Otherwise, use ``csc``.
Sometimes, since the sparse module is young, ops does not exist for both format. So here is
what may be the most relevent rule:
.. note::
In the *Compressed Sparse Row* format, ``indices`` stands for the row index
of the data and ``indptr`` tells where the row starts in the ``data`` and in the
``indices`` attributes. ``indptr`` can be tought as giving the slice which must be
applied to the other attribute in order to get each row of the matrix. The following
example builds a matrix and returns its rows. It prints the i-th row indexes in the
first list and their corresponding values in the second list.
>>> data = np.asarray([7, 8, 9])
>>> indices = np.asarray([0, 1, 2])
>>> indptr = np.asarray([0, 2, 3, 3])
>>> m = sp.csr_matrix((data, indices, indptr), shape=(3, 3))
>>> print m.toarray()
[[7 8 0]
[0 0 9]
[0 0 0]]
>>> i = 0
>>> print m.indices[m.indptr[i]:m.indptr[i+1]], m.data[m.indptr[i]:m.indptr[i+1]]
[0, 1] [7, 8]
>>> i = 1
>>> print m.indices[m.indptr[i]:m.indptr[i+1]], m.data[m.indptr[i]:m.indptr[i+1]]
[2] [9]
>>> i = 2
>>> print m.indices[m.indptr[i]:m.indptr[i+1]], m.data[m.indptr[i]:m.indptr[i+1]]
[] []
Use the format compatible with the ops in your computation graph.
The documentation about the ops and their supported format may be found in
the :ref:`Sparse Library Reference <libdoc_sparse>`.
Handling Sparse in Theano
=========================
......
theano/sparse/basic.py
浏览文件 @ fe3ff975
......@@ -4270,6 +4270,9 @@ class Dot(gof.op.Op):
:note: The grad implemented is regular, i.e. not structured.
:note: At least one of `x` or `y` must be a sparse matrix.
:note: When the operation has the form dot(csr_matrix, dense)
the gradient of this operation can be performed inplace
by UsmmCscDense. This leads to significant speed-ups.
"""
def __eq__(self, other):
......
Markdown 格式
0% 或
您添加了 0 人 到此讨论。请谨慎行事。
请先完成此评论的编辑！
请 注册 或者 后发表评论