This method computes the function associated to this Op. The
``node`` is an Apply node created by the Op's ``make_node``
``node`` is an Apply node created by the Op's ``make_node``
method, ``inputs`` is a list of references to data to operate on,
method, ``inputs`` is a list of references to data to operate on,
and ``output_storage`` is a list of storage cells where the variables of
and ``output_storage`` is a list of storage cells where the
the computation must be put. More specifically:
variables of the computation must be put. More specifically:
- ``node``: This is a reference to an Apply node which was previously
- ``node``: This is a reference to an Apply node which was previously
obtained via ``mul``'s ``make_node`` method. It is typically not
obtained via ``mul``'s ``make_node`` method. It is typically not
...
@@ -74,108 +76,70 @@ An Op (:api:`gof.op.Op`) is any object which defines the following methods:
...
@@ -74,108 +76,70 @@ An Op (:api:`gof.op.Op`) is any object which defines the following methods:
None. This feature can allow perform to reuse memory between calls, for
None. This feature can allow perform to reuse memory between calls, for
example.
example.
- This method must be determined by the inputs. That is to say, if it is
This method must be determined by the inputs. That is to say, if
evaluated once on inputs A and returned B, then if ever inputs C, equal to
it is evaluated once on inputs A and returned B, then if ever
A, are presented again, then outputs equal to B must be returned again.
inputs C, equal to A, are presented again, then outputs equal to
B must be returned again.
- You must be careful about aliasing outputs to inputs, and making
modifications to any of the inputs. See `Views and inplace operations
<views_and_inplace>`_
before writing a ``perform`` implementation that does either of these
things.
- **__eq__(self, other)**
You must be careful about aliasing outputs to inputs, and making
modifications to any of the inputs. See `Views and inplace
operations <views_and_inplace>`_ before writing a ``perform``
implementation that does either of these things.
- ``other`` is also an Op.
.. function:: __eq__(other)
- Returning ``True`` here is a promise to the optimization system that the other
``other`` is also an Op.
Op will produce exactly the same graph effects (from perform) as this one,
given identical inputs. This means it will produce the same output values,
it will destroy the same inputs (same destroy_map), and will alias outputs
to the same inputs (same view_map).
- **__hash__(self)**
Returning ``True`` here is a promise to the optimization system
that the other Op will produce exactly the same graph effects
(from perform) as this one, given identical inputs. This means it
will produce the same output values, it will destroy the same
inputs (same destroy_map), and will alias outputs to the same
inputs (same view_map).
- If two Op instances compare equal, then they **must** return the same hash
.. function:: __hash__()
value.
- Equally important, this hash value must not change during the lifetime of
If two Op instances compare equal, then they **must** return the
self. Op instances should be immutable in this sense.
same hash value.
- **__ne__(self, other)**
Equally important, this hash value must not change during the
lifetime of self. Op instances should be immutable in this
sense.
- Recommended
.. function:: __ne__(other)
- Default: ``(not (self==other))``
Default: ``(not (self==other))``
- **grad(inputs, output_gradients)**
.. function:: grad(inputs, output_gradients)
- Optional.
Optional.
- If the Op you are defining is differentiable, you can define its
If the Op you are defining is differentiable, you can define its
gradient symbolically in this method.
gradient symbolically in this method.
- Both the ``inputs`` and ``output_gradients`` will be Variables. This
Both the ``inputs`` and ``output_gradients`` will be
method must return a list containing one Variable (or None) for each
Variables. This method must return a list containing one Variable
input. Each returned Variable represents the gradient with respect to
(or None) for each input. Each returned Variable represents the
that input given the symbolic gradients with respect to each output.
gradient with respect to that input given the symbolic gradients
with respect to each output.
- If the output is not differentiable with respect to any inputs, then this
If the output is not differentiable with respect to any inputs,
method should be defined to return [None for i in inputs].
then this method should be defined to return [None for i in
inputs].
- If this method is not defined, then theano assumes it has been forgotten.
If this method is not defined, then theano assumes it has been
Symbolic differentiation will fail on a graph that includes this Op.
forgotten. Symbolic differentiation will fail on a graph that
includes this Op.
- For more information on the use of this method, see ``grad``.
For more information on the use of this method, see ``grad``.
For each method, the *default* is what :api:`theano.gof.op.Op` defines
For each method, the *default* is what :api:`theano.gof.op.Op` defines
for you. At a bare minimum, a new Op must define ``make_node`` and
for you. At a bare minimum, a new Op must define ``make_node`` and
``perform``, which have no defaults.
``perform``, which have no defaults.
For more details, including the interface for providing a C implementation of
For more details, including the interface for providing a C
perform(), refer to the documentation for :ref:`op`.
implementation of perform(), refer to the documentation for :ref:`op`.
Checklist
---------
Use this list to make sure that you defined everything you need for your Op:
* Are there parameters that are not inputs but parametrize the behavior of your Op? (see parametrization section below)
* Yes?
* Define ``__init__`` with those parameters. They will be instance variables.
* Override ``__eq__``, ``__ne__`` and ``__hash__`` (optional)
* Consider making pre-made instances for common parameters. This will simplify usage.
* No? (usual case for simple Ops)
* Consider making a singleton of your Op (this can be as simple as
``my_op = MyOp()``). This will save you from having to implement __eq__
and company. The singleton approach does not work when an Op instance
has parameters (Did you pass anything to __init__?)
* Always define *make_node* (see make_node section below).
* Always define *perform* (see perform section below).
* Do you need performance only C can offer?
* Define *c_code* and *c_code_cleanup* (see HowtoMakeCeeOps)
* Remember to use the 'c' or 'c|py' linker on graphs using your Op! [*This is described where?*]
* Is your Op differentiable? Do you want to use it in differentiable
expressions?
* Define *grad* (see grad section below)
* Does your Op modify any of its inputs?
* *IMPORTANT:* read the destroyers and viewers section.
* Does any output from the Op share any sort of state with an input?
* *IMPORTANT:* read the destroyers and viewers section.
* Does your Op have more than one output?
* Consider setting the default_output attribute to the index of that output. (It will make your Op usable in ``PatternOptimizers``, and make user code look like the Op has only that output.)
[*Consider changing the order of the checklist above and the sections below such that the stuff you ALWAYS have to do, which is the most basic stuff anyhow, goes towards the top.*]
Defining an Op: ``mul``
Defining an Op: ``mul``
...
@@ -259,28 +223,6 @@ Here, ``z`` is a list of one element. By default, ``z == [None]``.
...
@@ -259,28 +223,6 @@ Here, ``z`` is a list of one element. By default, ``z == [None]``.
that a Python ``float`` must be put there. You should not put, say, an
that a Python ``float`` must be put there. You should not put, say, an
``int`` in ``z[0]`` because Theano assumes Ops handle typing properly.
``int`` in ``z[0]`` because Theano assumes Ops handle typing properly.
**eq** and **hash**
Correct implementations of eq and hash permit Theano to recognize one
of the most obvious opportunities
for optimization: not repeatedly computing the same thing.
.. code-block:: python
def __eq__(self, other):
return type(self) == type(other) and (self.name == other.name) and (self.fn == other.fn)
This page is about ``theano.function``, the interface for compiling graphs into callable objects.
This page is about ``theano.function``, the interface for compiling graphs into callable objects.
...
@@ -31,47 +31,35 @@ Inputs
...
@@ -31,47 +31,35 @@ Inputs
The ``inputs`` argument to ``theano.function`` is a list, containing the ``Variable`` instances for which values will be specified at the time of the function call. But inputs can be more than just Variables.
The ``inputs`` argument to ``theano.function`` is a list, containing the ``Variable`` instances for which values will be specified at the time of the function call. But inputs can be more than just Variables.
``In`` instances let us attach properties to ``Variables`` to tell function more about how to use them.
``In`` instances let us attach properties to ``Variables`` to tell function more about how to use them.
**In(variable, name=None, value=None, update=None, mutable=False)** returns an ``In`` instance:
- ``variable``: a Variable instance.
.. class:: In
This will be assigned a value before running the function,
Theano relies heavily on unit testing. Its importance cannot be stressed enough !
Theano relies heavily on unit testing. Its importance cannot be
stressed enough!
Unit Testing revolves around the following principles:
Unit Testing revolves around the following principles:
* ensuring correctness: making sure that your Op, Type or Optimization works in the way you intended it to work. It is important for this testing to be as thorough as possible: test not only the obvious cases, but more importantly the corner cases which are more likely to trigger bugs down the line.
* ensuring correctness: making sure that your Op, Type or Optimization
* test all possible failure paths. This means testing that your code fails in the appropriate manner, by raising the correct errors when in certain situations.
works in the way you intended it to work. It is important for this
* sanity check: making sure that everything still runs after you've done your modification. If your changes cause unit tests to start failing, it could be that you've changed an API on which other users rely on. It is therefore your responsibility to either a) provide the fix or b) inform the author of your changes and coordinate with that person to produce a fix. If this sounds like too much of a burden... then good ! APIs aren't meant to be changed on a whim !
testing to be as thorough as possible: test not only the obvious
cases, but more importantly the corner cases which are more likely
This page is in no way meant to replace tutorials on Python's unittest module, for this we refer the reader to the `official documentation <http://docs.python.org/library/unittest.html>`_. We will however adress certain specificities about how unittests relate to theano.
to trigger bugs down the line.
* test all possible failure paths. This means testing that your code
fails in the appropriate manner, by raising the correct errors when
in certain situations.
* sanity check: making sure that everything still runs after you've
done your modification. If your changes cause unit tests to start
failing, it could be that you've changed an API on which other users
rely on. It is therefore your responsibility to either a) provide
the fix or b) inform the author of your changes and coordinate with
that person to produce a fix. If this sounds like too much of a
burden... then good! APIs aren't meant to be changed on a whim!
This page is in no way meant to replace tutorials on Python's unittest
module, for this we refer the reader to the `official documentation
<http://docs.python.org/library/unittest.html>`_. We will however
adress certain specificities about how unittests relate to theano.
Unittest Primer
Unittest Primer
===============
===============
A unittest is a subclass of ``unittest.TestCase``, with member functions with
A unittest is a subclass of ``unittest.TestCase``, with member
names that start with the string ``test``. For example:
functions with names that start with the string ``test``. For
example:
>>> class MyTestCase(unittest.TestCase):
>>> class MyTestCase(unittest.TestCase):
>>> def test0(self):
>>> def test0(self):
...
@@ -34,15 +53,16 @@ names that start with the string ``test``. For example:
...
@@ -34,15 +53,16 @@ names that start with the string ``test``. For example:
How to Run Unit Tests ?
How to Run Unit Tests ?
-----------------------
-----------------------
Two options are avaiable.
Two options are available:
Nosetests
Nosetests
~~~~~~~~~
~~~~~~~~~
The easiest by far is to use ``nosetests`` which
The easiest by far is to use ``nosetests`` which is a command line
is a command line utility that recurses through a given directory, finds all
utility that recurses through a given directory, finds all unittests
unittests matching a specific criteria and executes them. By default, it will
matching a specific criteria and executes them. By default, it will
find & execute tests case in test*.py files whose method name starts with 'test'.
find & execute tests case in test*.py files whose method name starts
with 'test'.
Running all unit tests
Running all unit tests
...
@@ -64,11 +84,12 @@ Running a specific unit test
...
@@ -64,11 +84,12 @@ Running a specific unit test
Using unittest module
Using unittest module
~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~
To launch tests cases from within python, you can also use the functionality
To launch tests cases from within python, you can also use the
offered by the ``unittest`` module. The simplest thing is to run all the tests in a file
functionality offered by the ``unittest`` module. The simplest thing
using ``unittest.main()``. Python's built-in unittest module uses metaclasses
is to run all the tests in a file using ``unittest.main()``. Python's
to know about all the ``unittest.TestCase`` classes you have created. This
built-in unittest module uses metaclasses to know about all the
call will run them all, printing '.' for passed tests, and a stack trace for
``unittest.TestCase`` classes you have created. This call will run
them all, printing '.' for passed tests, and a stack trace for
exceptions. The standard footer code in theano's test files is:
exceptions. The standard footer code in theano's test files is:
>>> if __name__ == '__main__':
>>> if __name__ == '__main__':
...
@@ -92,13 +113,15 @@ To run just a single ``MyTestCase`` member test function called ``test0``:
...
@@ -92,13 +113,15 @@ To run just a single ``MyTestCase`` member test function called ``test0``:
Folder Layout
Folder Layout
-------------
-------------
"tests" directories are scattered throughout theano. Each tests subfolder is
"tests" directories are scattered throughout theano. Each tests
meant to contain the unittests which validate the .py files in the parent folder.
subfolder is meant to contain the unittests which validate the .py
files in the parent folder.
Files containing unittests should be prefixed with the word "test".
Files containing unittests should be prefixed with the word "test".
Optimally every python module should have a unittest file associated with it,
Optimally every python module should have a unittest file associated
as shown below. Unittests testing functionality of module <module>.py should therefore be stored in tests/test_<module>.py
with it, as shown below. Unittests testing functionality of module
<module>.py should therefore be stored in tests/test_<module>.py
>>> Theano/theano/tensor/basic.py
>>> Theano/theano/tensor/basic.py
>>> Theano/theano/tensor/elemwise.py
>>> Theano/theano/tensor/elemwise.py
...
@@ -112,20 +135,22 @@ How to Write a Unittest
...
@@ -112,20 +135,22 @@ How to Write a Unittest
Test Cases and Methods
Test Cases and Methods
----------------------
----------------------
Unittests should be grouped "logically" into test cases, which are meant to
Unittests should be grouped "logically" into test cases, which are
group all unittests operating on the same element and/or concept. Test cases
meant to group all unittests operating on the same element and/or
are implemented as Python classes which inherit from unittest.TestCase
concept. Test cases are implemented as Python classes which inherit
from unittest.TestCase
Test cases contain multiple test methods. These should be prefixed with the
Test cases contain multiple test methods. These should be prefixed
word "test".
with the word "test".
Test methods should be as specific as possible and cover a particular aspect
Test methods should be as specific as possible and cover a particular
of the problem. For example, when testing the TensorDot Op, one test method
aspect of the problem. For example, when testing the TensorDot Op, one
could check for validity, while another could verify that the proper errors
test method could check for validity, while another could verify that
are raised when inputs have invalid dimensions.
the proper errors are raised when inputs have invalid dimensions.
Test method names should be as explicit as possible, so that users can see at
Test method names should be as explicit as possible, so that users can
first glance, what functionality is being tested and what tests need to be added.
see at first glance, what functionality is being tested and what tests
need to be added.
Example:
Example:
...
@@ -136,10 +161,11 @@ Example:
...
@@ -136,10 +161,11 @@ Example:
>>> def test_invalid_dims(self):
>>> def test_invalid_dims(self):
>>> # do more stuff
>>> # do more stuff
Test cases can define a special setUp method, which will get called before
Test cases can define a special setUp method, which will get called
each test method is executed. This is a good place to put functionality which
before each test method is executed. This is a good place to put
is shared amongst all test methods in the test case (i.e initializing data,
functionality which is shared amongst all test methods in the test
parameters, seeding random number generators -- more on this later)
case (i.e initializing data, parameters, seeding random number
generators -- more on this later)
>>> class TestTensorDot(unittest.TestCase):
>>> class TestTensorDot(unittest.TestCase):
>>> def setUp(self):
>>> def setUp(self):
...
@@ -147,15 +173,16 @@ parameters, seeding random number generators -- more on this later)
...
@@ -147,15 +173,16 @@ parameters, seeding random number generators -- more on this later)
This makes the test case less manageable and forces the user to update the
This makes the test case less manageable and forces the user to update
variables each time the input is changed or possibly when the module being
the variables each time the input is changed or possibly when the
tested changes (after a bug fix for example). It also constrains the test case
module being tested changes (after a bug fix for example). It also
to specific input/output data pairs. The section on random values covers why this
constrains the test case to specific input/output data pairs. The
might not be such a good idea.
section on random values covers why this might not be such a good
idea.
Here is a list of useful functions, as defined by TestCase:
Here is a list of useful functions, as defined by TestCase:
* checking the state of boolean variables: assert, failUnless, assertTrue, failIf, assertFalse
* checking the state of boolean variables: assert, failUnless,
* checking for (in)equality constraints: assertEqual, failUnlessEqual, assertNotEqual, failIfEqual
assertTrue, failIf, assertFalse
* checking for (in)equality constraints up to a given precision (very useful in theano): assertAlmostEqual, failUnlessAlmostEqual, assertNotAlmostEqual, failIfAlmostEqual
* checking for (in)equality constraints: assertEqual, failUnlessEqual,
assertNotEqual, failIfEqual
* checking for (in)equality constraints up to a given precision (very
useful in theano): assertAlmostEqual, failUnlessAlmostEqual,
assertNotAlmostEqual, failIfAlmostEqual
Checking for errors
Checking for errors
-------------------
-------------------
On top of verifying that your code provides the correct output, it is equally
On top of verifying that your code provides the correct output, it is
important to test that it fails in the appropriate manner, raising the
equally important to test that it fails in the appropriate manner,
appropriate exceptions, etc. Silent failures are deadly, as they can go unnoticed
raising the appropriate exceptions, etc. Silent failures are deadly,
for a long time and a hard to detect "after-the-fact".
as they can go unnoticed for a long time and a hard to detect
"after-the-fact".
Example:
Example:
...
@@ -216,7 +251,8 @@ Useful functions, as defined by TestCase:
...
@@ -216,7 +251,8 @@ Useful functions, as defined by TestCase:
Test Cases and Theano Modes
Test Cases and Theano Modes
---------------------------
---------------------------
When compiling theano functions or modules, a mode parameter can be given to specify which linker and optimizer to use.
When compiling theano functions or modules, a mode parameter can be
given to specify which linker and optimizer to use.
Example:
Example:
...
@@ -224,9 +260,15 @@ Example:
...
@@ -224,9 +260,15 @@ Example:
>>> m = theano.Module()
>>> m = theano.Module()
>>> minstance = m.make(mode='DEBUG_MODE')
>>> minstance = m.make(mode='DEBUG_MODE')
Whenever possible, unit tests should omit this parameter. Leaving-out the mode will ensure that unit tests use the default mode (defined in compile.mode.default_mode). This default_mode is set to the THEANO_DEFAULT_MODE environment variable, if it is present. If not, it defaults to 'FAST_RUN'.
Whenever possible, unit tests should omit this parameter. Leaving-out
the mode will ensure that unit tests use the default mode (defined in
compile.mode.default_mode). This default_mode is set to the
THEANO_DEFAULT_MODE environment variable, if it is present. If not, it
defaults to 'FAST_RUN'.
This allows the user to easily switch the mode in which unittests are run. For example to run all tests in all modes from a BASH script, type this:
This allows the user to easily switch the mode in which unittests are
run. For example to run all tests in all modes from a BASH script,
type this:
.. code-block:: bash
.. code-block:: bash
...
@@ -237,14 +279,15 @@ This allows the user to easily switch the mode in which unittests are run. For e
...
@@ -237,14 +279,15 @@ This allows the user to easily switch the mode in which unittests are run. For e
Using Random Values in Test Cases
Using Random Values in Test Cases
---------------------------------
---------------------------------
numpy.random is often used in unit tests to initialize large data structures,
numpy.random is often used in unit tests to initialize large data
for use as inputs to the function or module being tested. When
structures, for use as inputs to the function or module being
doing this, it is imperative that the random number generator be seeded at the
tested. When doing this, it is imperative that the random number
be beginning of each unit test. This will ensure that unittest behaviour is
generator be seeded at the be beginning of each unit test. This will
consistent from one execution to another (i.e always pass or always fail).
ensure that unittest behaviour is consistent from one execution to
another (i.e always pass or always fail).
Instead of using numpy.random.seed to do this, we encourage users to do the
Instead of using numpy.random.seed to do this, we encourage users to
following:
do the following:
>>> from theano.tests import unittest_tools
>>> from theano.tests import unittest_tools
>>>
>>>
...
@@ -257,19 +300,24 @@ following:
...
@@ -257,19 +300,24 @@ following:
The behaviour of seed_rng is as follows:
The behaviour of seed_rng is as follows:
* If an explicit seed is given, it will be used for seending numpy's rng.
* If an explicit seed is given, it will be used for seending numpy's rng.
* If not, it will try to get a seed from the THEANO_UNITTEST_SEED variable.
* If not, it will try to get a seed from the THEANO_UNITTEST_SEED variable.
* If THEANO_UNITTEST_SEED is set to "random", it will seed the rng. with None, which is equivalent to seeding with a random seed.
* If THEANO_UNITTEST_SEED is set to "random", it will seed the
rng. with None, which is equivalent to seeding with a random seed.
* If THEANO_UNITTEST_SEED is not defined, it will use a default seed of 666.
* If THEANO_UNITTEST_SEED is not defined, it will use a default seed of 666.
The main advantage of using unittest_tools.seed_rng is that it allows us to
The main advantage of using unittest_tools.seed_rng is that it allows
change the seed used in the unitests, without having to manually edit all the
us to change the seed used in the unitests, without having to manually
files. For example, this allows the nightly build to run nosetests repeatedly,
edit all the files. For example, this allows the nightly build to run
changing the seed on every run (hence achieving a higher confidence that the
nosetests repeatedly, changing the seed on every run (hence achieving
variables are correct), while still making sure unittests are deterministic.
a higher confidence that the variables are correct), while still
making sure unittests are deterministic.
Users who prefer their unittests to be random (when run on their local machine)
Users who prefer their unittests to be random (when run on their local
can simply set THEANO_UNITTEST_SEED to 'random'.
machine) can simply set THEANO_UNITTEST_SEED to 'random'.
Similarly, to provide a seed to numpy.random.RandomState, simply use:
Similarly, to provide a seed to numpy.random.RandomState, simply use:
...
@@ -277,23 +325,27 @@ Similarly, to provide a seed to numpy.random.RandomState, simply use:
...
@@ -277,23 +325,27 @@ Similarly, to provide a seed to numpy.random.RandomState, simply use:
>>> # OR providing an explicit seed
>>> # OR providing an explicit seed
>>> rng = numpy.random.RandomState(unittest_tools.fetch_seed(1231)) #again not recommended
>>> rng = numpy.random.RandomState(unittest_tools.fetch_seed(1231)) #again not recommended
Note that the ability to change the seed from one nosetest to another, is incompatible with the method of hard-coding the baseline variables (against which we compare the theano outputs). These must then be determined "algorithmically". Although this represents more work, the test suite will be better because of it.
Note that the ability to change the seed from one nosetest to another,
is incompatible with the method of hard-coding the baseline variables
(against which we compare the theano outputs). These must then be
determined "algorithmically". Although this represents more work, the
test suite will be better because of it.
Creating an Op UnitTest
Creating an Op UnitTest
=======================
=======================
A few tools have been developed to help automate the development of unitests
A few tools have been developed to help automate the development of
for Theano Ops.
unitests for Theano Ops.
Validating the Gradient
Validating the Gradient
-----------------------
-----------------------
The ``verify_grad`` function can be used to validate that the ``grad``
The ``verify_grad`` function can be used to validate that the ``grad``
function of your Op is properly implemented. ``verify_grad`` is based on the
function of your Op is properly implemented. ``verify_grad`` is based
Finite Difference Method where the derivative of function ``f`` at point ``x``
on the Finite Difference Method where the derivative of function ``f``
is approximated as:
at point ``x`` is approximated as:
.. math::
.. math::
...
@@ -302,8 +354,12 @@ is approximated as:
...
@@ -302,8 +354,12 @@ is approximated as:
``verify_grad`` performs the following steps:
``verify_grad`` performs the following steps:
* approximates the gradient numerically using the Finite Difference Method
* approximates the gradient numerically using the Finite Difference Method
* calculate the gradient using the symbolic expression provided in the ``grad`` function
* compares the two values. The tests passes if they are equal to within a certain tolerance.
* calculate the gradient using the symbolic expression provided in the
``grad`` function
* compares the two values. The tests passes if they are equal to
within a certain tolerance.
Here is the prototype for the verify_grad function.
Here is the prototype for the verify_grad function.
...
@@ -315,12 +371,17 @@ the given tolerance.
...
@@ -315,12 +371,17 @@ the given tolerance.
The parameters are as follows:
The parameters are as follows:
* op: something that behaves like an Op instance with a single output (can be a python
* op: something that behaves like an Op instance with a single output
function combining multiple ops)
(can be a python function combining multiple ops)
* pt: the list of numpy.ndarrays to use as inputs to the op
* pt: the list of numpy.ndarrays to use as inputs to the op
* n_tests: number of times to run the test
* n_tests: number of times to run the test
* rng: random number generator from which to draw random samples
* rng: random number generator from which to draw random samples
* eps: stepsize used in the Finite Difference Method
* eps: stepsize used in the Finite Difference Method
* tol: relative tolerance used as threshold for gradient comparison
* tol: relative tolerance used as threshold for gradient comparison
Here is an example showing how to use verify_grad:
Here is an example showing how to use verify_grad:
...
@@ -336,14 +397,15 @@ Here is an example showing how to use verify_grad:
...
@@ -336,14 +397,15 @@ Here is an example showing how to use verify_grad:
makeTester and makeBroadcastTester
makeTester and makeBroadcastTester
==================================
==================================
Most Op unittests perform the same function. All such tests must verify that
Most Op unittests perform the same function. All such tests must
the op generates the proper output, that the gradient is valid, that the Op
verify that the op generates the proper output, that the gradient is
fails in known/expected ways. Because so much of this is common, two helper
valid, that the Op fails in known/expected ways. Because so much of
functions exists to make your lives easier: ``makeTester`` and
this is common, two helper functions exists to make your lives easier:
``makeBroadcastTester`` (defined in module ``theano.tensor.tests.test_basic``).
``makeTester`` and ``makeBroadcastTester`` (defined in module
``theano.tensor.tests.test_basic``).
Here is an example of ``makeTester`` generating testcases for the Dot product
Here is an example of ``makeTester`` generating testcases for the Dot
op:
product op:
>>> DotTester = makeTester(name = 'DotTester',
>>> DotTester = makeTester(name = 'DotTester',
>>> op = dot,
>>> op = dot,
...
@@ -357,29 +419,34 @@ op:
...
@@ -357,29 +419,34 @@ op:
>>> bad2 = (rand(5, 7), rand(8,3))),
>>> bad2 = (rand(5, 7), rand(8,3))),
>>> grad = dict())
>>> grad = dict())
In the above example, we provide a name and a reference to the op we want to
In the above example, we provide a name and a reference to the op we
test. We then provide in the ``expected`` field, a function which
want to test. We then provide in the ``expected`` field, a function
``makeTester`` can use to compute the correct values. The following five
which ``makeTester`` can use to compute the correct values. The
parameters are dictionaries which contain:
following five parameters are dictionaries which contain:
* checks: dictionary of validation functions (dictionary key is a description
* checks: dictionary of validation functions (dictionary key is a
of what each function does). Each function accepts two parameters and
description of what each function does). Each function accepts two
performs some sort of validation check on each op-input/op-output value pairs.
parameters and performs some sort of validation check on each
If the function returns False, an Exception is raised containing the
op-input/op-output value pairs. If the function returns False, an
check's description.
Exception is raised containing the check's description.
* good: contains valid input values, for which the output should match the
expected output. Unittest will fail if this is not the case.
* good: contains valid input values, for which the output should match
* bad_build: invalid parameters which should generate an Exception when
the expected output. Unittest will fail if this is not the case.
attempting to build the graph (call to ``make_node`` should fail).
Fails unless an Exception is raised.
* bad_build: invalid parameters which should generate an Exception
* bad_runtime: invalid parameters which should generate an Exception at
when attempting to build the graph (call to ``make_node`` should
runtime, when trying to compute the actual output values (call to
fail). Fails unless an Exception is raised.
* bad_runtime: invalid parameters which should generate an Exception
at runtime, when trying to compute the actual output values (call to
``perform`` should fail). Fails unless an Exception is raised.
``perform`` should fail). Fails unless an Exception is raised.
* grad: dictionary containing input values which will be used in the call to
``verify_grad``
* grad: dictionary containing input values which will be used in the
call to ``verify_grad``
``makeBroadcastTester`` is a wrapper function for makeTester.
``makeBroadcastTester`` is a wrapper function for makeTester. If an
If an ``inplace=True`` parameter is passed to it, it will take care of adding
``inplace=True`` parameter is passed to it, it will take care of
an entry to the ``checks`` dictionary. This check will ensure that inputs and
adding an entry to the ``checks`` dictionary. This check will ensure
outputs are equal, after the Op's perform function has been applied.
that inputs and outputs are equal, after the Op's perform function has
