提交 c01948b6 authored 作者: Arnaud Bergeron's avatar Arnaud Bergeron

Fix most of the errors/warnings in the current docs.

Also sprinkle a bit of pep8.
上级 3810c977
.. _NEWS:
============= =============
Release Notes Release Notes
============= =============
......
...@@ -15,7 +15,7 @@ Profiling ...@@ -15,7 +15,7 @@ Profiling
Theano output: Theano output:
.. literalinclude:: logreg_profile.txt .. literalinclude:: logreg_profile.prof
Compilation pipeline Compilation pipeline
-------------------- --------------------
......
...@@ -216,5 +216,3 @@ optimization can pre-check whether it will get rejected by using the ...@@ -216,5 +216,3 @@ optimization can pre-check whether it will get rejected by using the
which Ops can be performed inplace. You may then skip the optimization if it is which Ops can be performed inplace. You may then skip the optimization if it is
incompatible with this check. Note however that this check does not cover all incompatible with this check. Note however that this check does not cover all
cases where an optimization may be rejected (it will not detect cycles). cases where an optimization may be rejected (it will not detect cycles).
.. _optdb:
...@@ -176,19 +176,19 @@ has more implemented distributions. ...@@ -176,19 +176,19 @@ has more implemented distributions.
The slowest is our wrapper on NumPy's random generator. The slowest is our wrapper on NumPy's random generator.
We explain and provide advice on 3 possibles implementations of new We explain and provide advice on 3 possibles implementations of new
distributions here:: distributions here:
1) Extend our wrapper around NumPy random functions. 1. Extend our wrapper around NumPy random functions.
See this `PR <https://github.com/Theano/Theano/pull/1607>`_ as an example. See this `PR <https://github.com/Theano/Theano/pull/1607>`_ as an example.
2) Extend MRG implementation by reusing existing Theano Op. Look into 2. Extend MRG implementation by reusing existing Theano Op. Look into
the ``theano/sandbox/rng_mrg.py`` file and grep for all code about the ``theano/sandbox/rng_mrg.py`` file and grep for all code about
binomial(). This distribution uses the output of the uniform binomial(). This distribution uses the output of the uniform
distribution and converts it to a binomial distribution with distribution and converts it to a binomial distribution with
existing Theano operations. The tests go in existing Theano operations. The tests go in
``theano/sandbox/test_rng_mrg.py`` ``theano/sandbox/test_rng_mrg.py``
3) Extend MRG implementation with a new Op that takes a uniform sample as 3. Extend MRG implementation with a new Op that takes a uniform sample as
input. Look in the ``theano/sandbox/{rng_mrg,multinomial}.py`` file input. Look in the ``theano/sandbox/{rng_mrg,multinomial}.py`` file
and its test in ``theano/sandbox/test_multinomal.py``. This is and its test in ``theano/sandbox/test_multinomal.py``. This is
recommended when current Theano ops aren't well suited to modify recommended when current Theano ops aren't well suited to modify
......
...@@ -13,9 +13,10 @@ arrays efficiently. Theano features: ...@@ -13,9 +13,10 @@ arrays efficiently. Theano features:
* **dynamic C code generation** -- Evaluate expressions faster. * **dynamic C code generation** -- Evaluate expressions faster.
* **extensive unit-testing and self-verification** -- Detect and diagnose many types of mistake. * **extensive unit-testing and self-verification** -- Detect and diagnose many types of mistake.
Theano has been powering large-scale computationally intensive scientific investigations Theano has been powering large-scale computationally intensive
since 2007. But it is also approachable enough to be used in the classroom scientific investigations since 2007. But it is also approachable
(IFT6266 at the University of Montreal). enough to be used in the classroom (IFT6266 at the University of
Montreal).
News News
==== ====
...@@ -59,22 +60,24 @@ directory, so that when you pull updates via Git, they will be ...@@ -59,22 +60,24 @@ directory, so that when you pull updates via Git, they will be
automatically reflected the "installed" version. For more information about automatically reflected the "installed" version. For more information about
installation and configuration, see :ref:`installing Theano <install>`. installation and configuration, see :ref:`installing Theano <install>`.
Status .. only:: html
======
.. image:: https://secure.travis-ci.org/Theano/Theano.png?branch=master Status
:target: http://travis-ci.org/Theano/Theano/builds ======
.. image:: https://pypip.in/v/Theano/badge.png .. image:: https://secure.travis-ci.org/Theano/Theano.png?branch=master
:target: https://crate.io/packages/Theano/ :target: http://travis-ci.org/Theano/Theano/builds
:alt: Latest PyPI version
.. image:: https://pypip.in/d/Theano/badge.png .. image:: https://pypip.in/v/Theano/badge.png
:target: https://crate.io/packages/Theano/ :target: https://crate.io/packages/Theano/
:alt: Number of PyPI downloads :alt: Latest PyPI version
.. _available on PyPI: http://pypi.python.org/pypi/Theano .. image:: https://pypip.in/d/Theano/badge.png
.. _Related Projects: https://github.com/Theano/Theano/wiki/Related-projects :target: https://crate.io/packages/Theano/
:alt: Number of PyPI downloads
.. _available on PyPI: http://pypi.python.org/pypi/Theano
.. _Related Projects: https://github.com/Theano/Theano/wiki/Related-projects
Citing Theano Citing Theano
============== ==============
......
...@@ -967,7 +967,7 @@ Reductions ...@@ -967,7 +967,7 @@ Reductions
* a *list of ints* - computed along these axes * a *list of ints* - computed along these axes
.. function:: ptp(x, axis = None) .. function:: ptp(x, axis = None)
Range of values (maximum - minimum) along an axis. Range of values (maximum - minimum) along an axis.
The name of the function comes from the acronym for peak to peak. The name of the function comes from the acronym for peak to peak.
...@@ -977,7 +977,7 @@ Reductions ...@@ -977,7 +977,7 @@ Reductions
flatten the array. flatten the array.
:Returns: A new array holding the result. :Returns: A new array holding the result.
Indexing Indexing
======== ========
...@@ -1544,8 +1544,9 @@ Linear Algebra ...@@ -1544,8 +1544,9 @@ Linear Algebra
If an integer i, it is converted to an array containing If an integer i, it is converted to an array containing
the last i dimensions of the first tensor and the first the last i dimensions of the first tensor and the first
i dimensions of the second tensor (excluding the first i dimensions of the second tensor (excluding the first
(batch) dimension): (batch) dimension)::
axes = [range(a.ndim - i, b.ndim), range(1,i+1)] axes = [range(a.ndim - i, b.ndim), range(1,i+1)]
If an array, its two elements must contain compatible axes If an array, its two elements must contain compatible axes
...@@ -1555,17 +1556,17 @@ Linear Algebra ...@@ -1555,17 +1556,17 @@ Linear Algebra
3rd axis of b must have the same shape; the same is true for 3rd axis of b must have the same shape; the same is true for
the 3rd axis of a and the 5th axis of b. the 3rd axis of a and the 5th axis of b.
:type axes: int or array-like of length 2 :type axes: int or array-like of length 2
:returns: a tensor with shape equal to the concatenation of a's shape :returns: a tensor with shape equal to the concatenation of a's shape
(less any dimensions that were summed over) and b's shape (less any dimensions that were summed over) and b's shape
(less first dimension and any dimensions that were summed over). (less first dimension and any dimensions that were summed over).
:rtype: tensor of tensordots :rtype: tensor of tensordots
A hybrid of batch_dot and tensordot, this function computes the A hybrid of batch_dot and tensordot, this function computes the
tensordot product between the two tensors, by iterating over the tensordot product between the two tensors, by iterating over the
first dimension using scan to perform a sequence of tensordots. first dimension using scan to perform a sequence of tensordots.
:note: See :func:`tensordot` and :func:`batched_dot` for :note: See :func:`tensordot` and :func:`batched_dot` for
supplementary documentation. supplementary documentation.
...@@ -1598,85 +1599,92 @@ Gradient / Differentiation ...@@ -1598,85 +1599,92 @@ Gradient / Differentiation
:rtype: variable or list of variables (matching `wrt`) :rtype: variable or list of variables (matching `wrt`)
:returns: gradients of the cost with respect to each of the `wrt` terms :returns: gradients of the cost with respect to each of the `wrt` terms
.. function:: subgraph_grad(wrt, end, start=None, cost=None, details=False) .. function:: subgraph_grad(wrt, end, start=None, cost=None, details=False)
With respect to `wrt`, computes gradients of cost and/or from existing With respect to `wrt`, computes gradients of cost and/or from existing
`start` gradients, up to the `end` variables of a symbolic digraph. `start` gradients, up to the `end` variables of a symbolic digraph.
In other words, computes gradients for a subgraph of the In other words, computes gradients for a subgraph of the
symbolic theano function. Ignores all disconnected inputs. symbolic theano function. Ignores all disconnected inputs.
This can be useful when one needs to perform the gradient descent This can be useful when one needs to perform the gradient descent
iteratively (e.g. one layer at a time in an MLP), or when a particular iteratively (e.g. one layer at a time in an MLP), or when a particular
operation is not differentiable in theano (e.g. stochastic sampling operation is not differentiable in theano (e.g. stochastic sampling
from a multinomial). In the latter case, the gradient of the from a multinomial). In the latter case, the gradient of the
non-differentiable process could be approximated by user-defined non-differentiable process could be approximated by user-defined
formula, which could be calculated using the gradients of a cost formula, which could be calculated using the gradients of a cost
with respect to samples (0s and 1s). These gradients are obtained with respect to samples (0s and 1s). These gradients are obtained
by performing a subgraph_grad from the `cost` or previously known gradients by performing a subgraph_grad from the `cost` or previously known gradients
(`start`) up to the outputs of the stochastic process (`end`). (`start`) up to the outputs of the stochastic process (`end`).
A dictionary mapping gradients obtained from the user-defined A dictionary mapping gradients obtained from the user-defined
differentiation of the process, to variables, could then be fed into differentiation of the process, to variables, could then be fed into
another subgraph_grad as `start` with any other `cost` (e.g. weight decay). another subgraph_grad as `start` with any other `cost` (e.g. weight decay).
In an MLP, we could use subgraph_grad to iteratively backpropagate: In an MLP, we could use subgraph_grad to iteratively backpropagate:
>>> x, t = theano.tensor.fvector('x'), theano.tensor.fvector('t') >>> x, t = theano.tensor.fvector('x'), theano.tensor.fvector('t')
>>> w1 = theano.shared(np.random.randn(3,4)) >>> w1 = theano.shared(np.random.randn(3,4))
>>> w2 = theano.shared(np.random.randn(4,2)) >>> w2 = theano.shared(np.random.randn(4,2))
>>> a1 = theano.tensor.tanh(theano.tensor.dot(x,w1)) >>> a1 = theano.tensor.tanh(theano.tensor.dot(x,w1))
>>> a2 = theano.tensor.tanh(theano.tensor.dot(a1,w2)) >>> a2 = theano.tensor.tanh(theano.tensor.dot(a1,w2))
>>> cost2 = theano.tensor.sqr(a2 - t).sum() >>> cost2 = theano.tensor.sqr(a2 - t).sum()
>>> cost2 += theano.tensor.sqr(w2.sum()) >>> cost2 += theano.tensor.sqr(w2.sum())
>>> cost1 = theano.tensor.sqr(w1.sum()) >>> cost1 = theano.tensor.sqr(w1.sum())
>>> params = [[w2],[w1]] >>> params = [[w2],[w1]]
>>> costs = [cost2,cost1] >>> costs = [cost2,cost1]
>>> grad_ends = [[a1], [x]] >>> grad_ends = [[a1], [x]]
>>> next_grad = None >>> next_grad = None
>>> param_grads = [] >>> param_grads = []
>>> for i in xrange(2): >>> for i in xrange(2):
>>> param_grad, next_grad = theano.subgraph_grad( >>> param_grad, next_grad = theano.subgraph_grad(
>>> wrt=params[i], end=grad_ends[i], >>> wrt=params[i], end=grad_ends[i],
>>> start=next_grad, cost=costs[i] >>> start=next_grad, cost=costs[i]
>>> ) >>> )
>>> next_grad = dict(zip(grad_ends[i], next_grad)) >>> next_grad = dict(zip(grad_ends[i], next_grad))
>>> param_grads.extend(param_grad) >>> param_grads.extend(param_grad)
:type wrt : List of Variables. :type wrt: list of variables
Gradients are computed with respect to `wrt`. :param wrt:
Gradients are computed with respect to `wrt`.
:type end : List of Variables.
Theano variables at which to end gradient descent :type end: list of variables
(they are considered constant in theano.grad). :param end:
For convenience, the gradients with respect to these variables Theano variables at which to end gradient descent (they are
are also returned. considered constant in theano.grad). For convenience, the
gradients with respect to these variables are also returned.
:type start : Dictionary of Variables
:param start: If not None, a dictionary mapping variables to :type start: dictionary of variables
their gradients. This is useful when the gradient on some :param start:
variables are known. These are used to compute the gradients If not None, a dictionary mapping variables to their
backwards up to the variables in `end` gradients. This is useful when the gradient on some variables
(they are used as known_grad in theano.grad). are known. These are used to compute the gradients backwards up
to the variables in `end` (they are used as known_grad in
:type cost: Scalar (0-dimensional) Variable. theano.grad).
:param cost:
Additional costs for which to compute the gradients. :type cost: scalar (0-dimensional) variable
For example, these could be weight decay, an l1 constraint, :param cost:
MSE, NLL, etc. May optionally be None if start is provided.
Warning : If the gradients of `cost` with respect to any Additional costs for which to compute the gradients. For
of the `start` variables is already part of the `start` example, these could be weight decay, an l1 constraint, MSE,
dictionary, then it may be counted twice with respect to `wrt` NLL, etc. May optionally be None if start is provided.
and `end`.
.. warning::
:type details: bool.
:param details: When True, additionally returns the If the gradients of `cost` with respect to any of the `start`
list of gradients from `start` and of `cost`, respectively, variables is already part of the `start` dictionary, then it
with respect to `wrt` (not `end`). may be counted twice with respect to `wrt` and `end`.
:type details: bool
:param details:
When True, additionally returns the list of gradients from
`start` and of `cost`, respectively, with respect to `wrt` (not
`end`).
:rtype: Tuple of 2 or 4 Lists of Variables :rtype: Tuple of 2 or 4 Lists of Variables
:return: Returns lists of gradients with respect to `wrt` and `end`, :return: Returns lists of gradients with respect to `wrt` and `end`,
respectively. respectively.
......
=======
OpenCL
=======
Migrate the GPU code-generators to the PyCUDA style, and eventually to OpenCL.
This means mainly to use a different kind of code-generation strategy. The
kernel itself is compiled, but the calling code remains in python or cython. We
would no longer generate entire C files this way, and no longer use the CLinker
for GPU code.
...@@ -137,7 +137,7 @@ Theano defines the following modes by name: ...@@ -137,7 +137,7 @@ Theano defines the following modes by name:
- ``'DebugMode``: Verify the correctness of all optimizations, and compare C and Python - ``'DebugMode``: Verify the correctness of all optimizations, and compare C and Python
implementations. This mode can take much longer than the other modes, but can identify implementations. This mode can take much longer than the other modes, but can identify
several kinds of problems. several kinds of problems.
- ``'ProfileMode'``(deprecated): Same optimization as FAST_RUN, but print some profiling information. - ``'ProfileMode'`` (deprecated): Same optimization as FAST_RUN, but print some profiling information.
The default mode is typically ``FAST_RUN``, but it can be controlled via The default mode is typically ``FAST_RUN``, but it can be controlled via
the configuration variable :attr:`config.mode`, the configuration variable :attr:`config.mode`,
......
...@@ -72,4 +72,4 @@ to run the example: ...@@ -72,4 +72,4 @@ to run the example:
The output: The output:
.. literalinclude:: profiling_example_out.txt .. literalinclude:: profiling_example_out.prof
...@@ -5,14 +5,16 @@ ...@@ -5,14 +5,16 @@
Using the GPU Using the GPU
============= =============
For an introductory discussion of *Graphical Processing Units* (GPU) and their use for For an introductory discussion of *Graphical Processing Units* (GPU)
intensive parallel computation purposes, see `GPGPU <http://en.wikipedia.org/wiki/GPGPU>`_. and their use for intensive parallel computation purposes, see `GPGPU
<http://en.wikipedia.org/wiki/GPGPU>`_.
One of Theano's design goals is to specify computations at an
abstract level, so that the internal function compiler has a lot of flexibility One of Theano's design goals is to specify computations at an abstract
about how to carry out those computations. One of the ways we take advantage of level, so that the internal function compiler has a lot of flexibility
this flexibility is in carrying out calculations on an Nvidia graphics card when about how to carry out those computations. One of the ways we take
the device present in the computer is CUDA-enabled. advantage of this flexibility is in carrying out calculations on an
Nvidia graphics card when the device present in the computer is
CUDA-enabled.
Setting Up CUDA Setting Up CUDA
---------------- ----------------
......
...@@ -24,6 +24,7 @@ class OpFromGraph(gof.Op): ...@@ -24,6 +24,7 @@ class OpFromGraph(gof.Op):
- Add support for the GPU? Probably just need an opt to remove transfer - Add support for the GPU? Probably just need an opt to remove transfer
- Add support to pickle this Op. - Add support to pickle this Op.
- Add support/test with random generator - Add support/test with random generator
:note: :note:
- We support shared variables in the inner graph. This is automatic and - We support shared variables in the inner graph. This is automatic and
invisible to the user. They can be as input to the node or in the invisible to the user. They can be as input to the node or in the
......
...@@ -490,6 +490,7 @@ def register_rebroadcast_c_code(typ, code, version=()): ...@@ -490,6 +490,7 @@ def register_rebroadcast_c_code(typ, code, version=()):
%(oname)s for the input and output C variable names %(oname)s for the input and output C variable names
respectively. %(axis)s for the axis that we need to respectively. %(axis)s for the axis that we need to
check. This code is put in a loop for all axis check. This code is put in a loop for all axis
:param version: A number indicating the version of the code, for cache. :param version: A number indicating the version of the code, for cache.
""" """
Rebroadcast.c_code_and_version[typ] = (code, version) Rebroadcast.c_code_and_version[typ] = (code, version)
...@@ -497,14 +498,18 @@ def register_rebroadcast_c_code(typ, code, version=()): ...@@ -497,14 +498,18 @@ def register_rebroadcast_c_code(typ, code, version=()):
class Rebroadcast(gof.Op): class Rebroadcast(gof.Op):
""" """
Change the input's broadcastable fields in Change the input's broadcastable fields in some predetermined way.
some predetermined way.
e.g.: Rebroadcast((0, True), (1, False))(x) :code:`Rebroadcast((0, True), (1, False))(x)` would make :code:`x`
would make x broadcastable in axis 0 broadcastable in axis 0 and not broadcastable in axis 1
and not broadcastable in axis 1
See also the unbroadcast, addbroadcast and patternbroadcast functions. .. seealso::
..note: work inplace and work for CudaNdarrayType :func:`unbroadcast <theano.tensor.unbroadcast>`
:func:`addbroadcast <theano.tensor.addbroadcast>`
:func:`patternbroadcast <theano.tensor.patternbroadcast>`
..note: works inplace and works for CudaNdarrayType
""" """
view_map = {0: [0]} view_map = {0: [0]}
# Mapping from Type to C code (and version) to use. # Mapping from Type to C code (and version) to use.
......
...@@ -95,15 +95,15 @@ def memoize(f): ...@@ -95,15 +95,15 @@ def memoize(f):
def deprecated(filename, msg=''): def deprecated(filename, msg=''):
"""Decorator which will print a warning message on the first call. """Decorator which will print a warning message on the first call.
Use it like this: Use it like this::
@deprecated('myfile', 'do something different...') @deprecated('myfile', 'do something different...')
def fn_name(...) def fn_name(...)
... ...
And it will print And it will print::
WARNING myfile.fn_name deprecated. do something different... WARNING myfile.fn_name deprecated. do something different...
""" """
def _deprecated(f): def _deprecated(f):
......
...@@ -546,59 +546,65 @@ def grad(cost, wrt, consider_constant=None, ...@@ -546,59 +546,65 @@ def grad(cost, wrt, consider_constant=None,
def subgraph_grad(wrt, end, start=None, cost=None, details=False): def subgraph_grad(wrt, end, start=None, cost=None, details=False):
''' '''
With respect to `wrt`, computes gradients of cost and/or from existing With respect to `wrt`, computes gradients of cost and/or from
`start` gradients, up to the `end` variables of a symbolic digraph. existing `start` gradients, up to the `end` variables of a
In other words, computes gradients for a subgraph of the symbolic digraph. In other words, computes gradients for a
symbolic theano function. Ignores all disconnected inputs. subgraph of the symbolic theano function. Ignores all disconnected
inputs.
This can be useful when one needs to perform the gradient descent This can be useful when one needs to perform the gradient descent
iteratively (e.g. one layer at a time in an MLP), or when a particular iteratively (e.g. one layer at a time in an MLP), or when a
operation is not differentiable in theano (e.g. stochastic sampling particular operation is not differentiable in theano
from a multinomial). In the latter case, the gradient of the (e.g. stochastic sampling from a multinomial). In the latter case,
non-differentiable process could be approximated by user-defined the gradient of the non-differentiable process could be
formula, which could be calculated using the gradients of a cost approximated by user-defined formula, which could be calculated
with respect to samples (0s and 1s). These gradients are obtained using the gradients of a cost with respect to samples (0s and
by performing a subgraph_grad from the `cost` or previously known gradients 1s). These gradients are obtained by performing a subgraph_grad
(`start`) up to the outputs of the stochastic process (`end`). from the `cost` or previously known gradients (`start`) up to the
A dictionary mapping gradients obtained from the user-defined outputs of the stochastic process (`end`). A dictionary mapping
differentiation of the process, to variables, could then be fed into gradients obtained from the user-defined differentiation of the
another subgraph_grad as `start` with any other `cost` (e.g. weight decay). process, to variables, could then be fed into another
subgraph_grad as `start` with any other `cost` (e.g. weight
decay).
:type wrt : List of Variables. :type wrt: list of variables
Gradients are computed with respect to `wrt`. :param wrt:
Gradients are computed with respect to `wrt`.
:type end : List of Variables. :type end: list of variables
Theano variables at which to end gradient descent :param end:
(they are considered constant in theano.grad). Theano variables at which to end gradient descent (they are
For convenience, the gradients with respect to these variables considered constant in theano.grad). For convenience, the
are also returned. gradients with respect to these variables are also returned.
:type start : Dictionary of Variables :type start: dictionary of variables
:param start: If not None, a dictionary mapping variables to :param start:
their gradients. This is useful when the gradient on some If not None, a dictionary mapping variables to their
variables are known. These are used to compute the gradients gradients. This is useful when the gradient on some variables
backwards up to the variables in `end` are known. These are used to compute the gradients backwards up
(they are used as known_grad in theano.grad). to the variables in `end` (they are used as known_grad in
theano.grad).
:type cost: Scalar (0-dimensional) Variable. :type cost: scalar (0-dimensional) variable
:param cost: :param cost:
Additional costs for which to compute the gradients. Additional costs for which to compute the gradients. For
For example, these could be weight decay, an l1 constraint, example, these could be weight decay, an l1 constraint, MSE,
MSE, NLL, etc. May optionally be None if start is provided. NLL, etc. May optionally be None if start is provided. Warning
Warning : If the gradients of `cost` with respect to any : If the gradients of `cost` with respect to any of the `start`
of the `start` variables is already part of the `start` variables is already part of the `start` dictionary, then it may
dictionary, then it may be counted twice with respect to `wrt` be counted twice with respect to `wrt` and `end`.
and `end`.
:type details: bool. :type details: bool
:param details: When True, additionally returns the :param details:
list of gradients from `start` and of `cost`, respectively, When True, additionally returns the list of gradients from
with respect to `wrt` (not `end`). `start` and of `cost`, respectively, with respect to `wrt` (not
`end`).
:rtype: Tuple of 2 or 4 Lists of Variables :rtype: Tuple of 2 or 4 Lists of Variables
:return: Returns lists of gradients with respect to `wrt` and `end`, :return: Returns lists of gradients with respect to `wrt` and `end`,
respectively. respectively.
''' '''
assert ((cost is not None) or (start is not None)) assert ((cost is not None) or (start is not None))
assert isinstance(end, list) assert isinstance(end, list)
......
...@@ -999,8 +999,9 @@ def guess_n_streams(size, warn=True): ...@@ -999,8 +999,9 @@ def guess_n_streams(size, warn=True):
""" """
Return a guess at a good number of streams. Return a guess at a good number of streams.
:param warn: If True, warn when a guess cannot be made (in which case :param warn:
we return 60 * 256). If True, warn when a guess cannot be made (in which case we
return 60 * 256).
""" """
# TODO: a smart way of choosing the number of streams, see #612. # TODO: a smart way of choosing the number of streams, see #612.
# Note that this code was moved out of `MRG_RandomStreams` so that it can # Note that this code was moved out of `MRG_RandomStreams` so that it can
...@@ -1134,20 +1135,25 @@ class MRG_RandomStreams(object): ...@@ -1134,20 +1135,25 @@ class MRG_RandomStreams(object):
ndim may be a plain integer to supplement the missing ndim may be a plain integer to supplement the missing
information. information.
:param low: Lower bound of the interval on which values are sampled. :param low:
If the ``dtype`` arg is provided, ``low`` will be cast into dtype. Lower bound of the interval on which values are sampled. If
This bound is excluded. the ``dtype`` arg is provided, ``low`` will be cast into
dtype. This bound is excluded.
:param high: Higher bound of the interval on which values are sampled. :param high:
If the ``dtype`` arg is provided, ``high`` will be cast into dtype. Higher bound of the interval on which values are sampled.
This bound is excluded. If the ``dtype`` arg is provided, ``high`` will be cast into
dtype. This bound is excluded.
:param size: Can be a list of integer or Theano variable :param size:
(ex: the shape of other Theano Variable) Can be a list of integer or Theano variable (ex: the shape
of other Theano Variable)
:param dtype:
The output data type. If dtype is not specified, it will be
inferred from the dtype of low and high, but will be at
least as precise as floatX.
:param dtype: The output data type. If dtype is not specified, it will
be inferred from the dtype of low and high, but will be at least as
precise as floatX.
""" """
low = as_tensor_variable(low) low = as_tensor_variable(low)
high = as_tensor_variable(high) high = as_tensor_variable(high)
...@@ -1274,14 +1280,18 @@ class MRG_RandomStreams(object): ...@@ -1274,14 +1280,18 @@ class MRG_RandomStreams(object):
def normal(self, size, avg=0.0, std=1.0, ndim=None, def normal(self, size, avg=0.0, std=1.0, ndim=None,
dtype=None, nstreams=None): dtype=None, nstreams=None):
""" """
:param size: Can be a list of integers or Theano variables (ex: the :param size:
shape of another Theano Variable) Can be a list of integers or Theano variables (ex: the shape
of another Theano Variable)
:param dtype:
The output data type. If dtype is not specified, it will be
inferred from the dtype of low and high, but will be at
least as precise as floatX.
:param dtype: The output data type. If dtype is not specified, it will :param nstreams:
be inferred from the dtype of low and high, but will be at least as Number of streams.
precise as floatX.
:param nstreams: Number of streams.
""" """
# We need an even number of ]0,1[ samples. Then we split them # We need an even number of ]0,1[ samples. Then we split them
# in two halves. First half becomes our U1's for Box-Muller, # in two halves. First half becomes our U1's for Box-Muller,
......
...@@ -66,8 +66,10 @@ class Conv3D(theano.Op): ...@@ -66,8 +66,10 @@ class Conv3D(theano.Op):
b_ = T.as_tensor_variable(b) b_ = T.as_tensor_variable(b)
d_ = T.as_tensor_variable(d) d_ = T.as_tensor_variable(d)
node = theano.Apply(self, inputs=[V_, W_,b_,d_], outputs = [ T.TensorType(V_.dtype, (V_.broadcastable[0],False,False,False, W_.broadcastable[0]))() ] ) bcast = (V_.broadcastable[0], False, False, False, W_.broadcastable[0])
node = theano.Apply(self, inputs=[V_, W_, b_, d_],
outputs=[T.TensorType(V_.dtype, bcast)()])
return node return node
...@@ -118,8 +120,6 @@ class Conv3D(theano.Op): ...@@ -118,8 +120,6 @@ class Conv3D(theano.Op):
dCdW.name = 'Conv3D_dCdW(dCdH='+dCdH_name+',V='+V_name+',W='+W_name+')' dCdW.name = 'Conv3D_dCdW(dCdH='+dCdH_name+',V='+V_name+',W='+W_name+')'
dCdb.name = 'Conv3D_dCdb(dCdH='+dCdH_name+',V='+V_name+',W='+W_name+',b='+b_name+')' dCdb.name = 'Conv3D_dCdb(dCdH='+dCdH_name+',V='+V_name+',W='+W_name+',b='+b_name+')'
return [ dCdV, dCdW, dCdb, dCdd ] return [ dCdV, dCdW, dCdb, dCdd ]
def perform(self, node, inputs, output_storage): def perform(self, node, inputs, output_storage):
...@@ -149,8 +149,7 @@ class Conv3D(theano.Op): ...@@ -149,8 +149,7 @@ class Conv3D(theano.Op):
rval = (batch_size, output_height, output_width, output_dur, output_channels ) rval = (batch_size, output_height, output_width, output_dur, output_channels )
return [rval]
return [ rval ]
def c_support_code(self): def c_support_code(self):
return blas_header_text() return blas_header_text()
...@@ -174,7 +173,6 @@ class Conv3D(theano.Op): ...@@ -174,7 +173,6 @@ class Conv3D(theano.Op):
H = outputs[0] H = outputs[0]
codeSource = """ codeSource = """
///////////// < code generated by Conv3D > ///////////// < code generated by Conv3D >
...@@ -279,7 +277,6 @@ class Conv3D(theano.Op): ...@@ -279,7 +277,6 @@ class Conv3D(theano.Op):
const long long outputWidth = int( (vidWidth - filterWidth) / dc )+1; const long long outputWidth = int( (vidWidth - filterWidth) / dc )+1;
const long long outputDur = int( (vidDur - filterDur) / dt ) +1; const long long outputDur = int( (vidDur - filterDur) / dt ) +1;
npy_intp dims[5]; npy_intp dims[5];
dims[0] = batchSize; dims[0] = batchSize;
dims[4] = outputChannels; dims[4] = outputChannels;
...@@ -287,8 +284,6 @@ class Conv3D(theano.Op): ...@@ -287,8 +284,6 @@ class Conv3D(theano.Op):
dims[2] = outputWidth; dims[2] = outputWidth;
dims[3] = outputDur; dims[3] = outputDur;
if(!(%(H)s) || PyArray_DIMS(%(H)s)[0]!=dims[0] || if(!(%(H)s) || PyArray_DIMS(%(H)s)[0]!=dims[0] ||
PyArray_DIMS(%(H)s)[1]!=dims[1] || PyArray_DIMS(%(H)s)[1]!=dims[1] ||
PyArray_DIMS(%(H)s)[2]!=dims[2] || PyArray_DIMS(%(H)s)[2]!=dims[2] ||
...@@ -303,10 +298,8 @@ class Conv3D(theano.Op): ...@@ -303,10 +298,8 @@ class Conv3D(theano.Op):
} }
{ // extra scope so fail works { // extra scope so fail works
#define ELEM_AT(x, i) * ( dtype_ ## x *) ( PyArray_BYTES(x) + (i) ) #define ELEM_AT(x, i) * ( dtype_ ## x *) ( PyArray_BYTES(x) + (i) )
const int ws0 = PyArray_STRIDES(%(W)s)[0]; const int ws0 = PyArray_STRIDES(%(W)s)[0];
const int ws1 = PyArray_STRIDES(%(W)s)[1]; const int ws1 = PyArray_STRIDES(%(W)s)[1];
const int ws2 = PyArray_STRIDES(%(W)s)[2]; const int ws2 = PyArray_STRIDES(%(W)s)[2];
...@@ -319,22 +312,14 @@ class Conv3D(theano.Op): ...@@ -319,22 +312,14 @@ class Conv3D(theano.Op):
const int bs = PyArray_STRIDES(%(b)s)[0]; const int bs = PyArray_STRIDES(%(b)s)[0];
const int hs4 = PyArray_STRIDES(%(H)s)[4]; const int hs4 = PyArray_STRIDES(%(H)s)[4];
// Compute H // Compute H
//H[i,j,x,y,t] = b_j + sum_k sum_l sum_m sum_z W[j,z,k,l,m] V[i,z, dr*r+k,dc*c+l,dt*t+m] //H[i,j,x,y,t] = b_j + sum_k sum_l sum_m sum_z W[j,z,k,l,m] V[i,z, dr*r+k,dc*c+l,dt*t+m]
//TODO: add special cases //TODO: add special cases
// ex: filterDur == 1 && batchSize == 1 && dt = 1 (for SFA) // ex: filterDur == 1 && batchSize == 1 && dt = 1 (for SFA)
// ex: inputChannels == 1 """ // ex: inputChannels == 1 """
# if the data types are not mixed, we can insert special case
# optimizations based on BLAS
#if the data types are not mixed, we can insert special case optimizations based on BLAS
VV, WV, bv, dv = node.inputs VV, WV, bv, dv = node.inputs
HV = node.outputs[0] HV = node.outputs[0]
if (theano.config.blas.ldflags and if (theano.config.blas.ldflags and
...@@ -546,7 +531,7 @@ class Conv3D(theano.Op): ...@@ -546,7 +531,7 @@ class Conv3D(theano.Op):
return strutil.render_string(codeSource,locals()) return strutil.render_string(codeSource,locals())
global conv3D
conv3D = Conv3D() conv3D = Conv3D()
""" """
3D "convolution" of multiple filters on a minibatch 3D "convolution" of multiple filters on a minibatch
......
Markdown 格式
0%
您添加了 0 到此讨论。请谨慎行事。
请先完成此评论的编辑!
注册 或者 后发表评论