提交 58374668 authored 作者: James Bergstra's avatar James Bergstra

merge

.. _gradient:
===========================
Computation of the Gradient
===========================
WRITEME
...@@ -5,6 +5,9 @@ ...@@ -5,6 +5,9 @@
Advanced Topics Advanced Topics
=============== ===============
Structure
=========
.. toctree:: .. toctree::
:maxdepth: 2 :maxdepth: 2
...@@ -16,4 +19,11 @@ Advanced Topics ...@@ -16,4 +19,11 @@ Advanced Topics
function function
module module
Concepts
========
.. toctree::
:maxdepth: 2
gradient
...@@ -8,6 +8,19 @@ Module Interface ...@@ -8,6 +8,19 @@ Module Interface
WRITEME WRITEME
.. index::
single: External
single: component; External
.. _external:
--------
External
--------
WRITEME
.. index:: .. index::
single: Member single: Member
......
...@@ -25,6 +25,13 @@ import sys, os ...@@ -25,6 +25,13 @@ import sys, os
# coming with Sphinx (named 'sphinx.ext.*') or your custom ones. # coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
extensions = ['sphinx.ext.autodoc', 'ext'] extensions = ['sphinx.ext.autodoc', 'ext']
try:
from sphinx.ext import pngmath
extensions.append('sphinx.ext.pngmath')
except ImportError:
pass
# Add any paths that contain templates here, relative to this directory. # Add any paths that contain templates here, relative to this directory.
templates_path = ['.templates'] templates_path = ['.templates']
......
...@@ -4,94 +4,7 @@ ...@@ -4,94 +4,7 @@
Graph: interconnected Apply and Result instances Graph: interconnected Apply and Result instances
================================================ ================================================
*TODO: There is similar documentation in the* `wiki <http://lgcm.iro.umontreal.ca/theano/wiki/GraphStructures>`__. *However, the <MOVED TO advanced/graphstructures.txt>
wiki has more information about certain topics. Merge these two pieces of
documentation.*
In theano, a graph is an implicit concept, not a class or an instance.
When we create `Results` and then `apply` `operations` to them to make more `Results`, we build a bi-partite, directed, acyclic graph.
Results point to `Apply` instances (via their `owner` attribute) and `Apply` instances point to `Results` (via their `inputs` and `outputs` fields).
To see how `Result`, `Type`, `Apply`, and `Op` all work together, compare the following code fragment and illustration.
.. code-block:: python
x = matrix('x')
y = matrix('y')
z = x + y
.. image:: http://lgcm.iro.umontreal.ca/theano/attachment/wiki/GraphStructures/apply.png?format=raw
Arrows represent references (python's pointers), the blue box is an Apply instance, red boxes are `Result` nodes, green circles are `Op` instances, purple boxes are `Type` instances.
Two examples
============
Here's how to build a graph the convenient way...
.. code-block:: python
from theano.tensor import *
# create 3 Results with owner = None
x = matrix('x')
y = matrix('y')
z = matrix('z')
# create 2 Results (one for 'e', one intermediate for y*z)
# create 2 Apply instances (one for '+', one for '*')
e = x + y * z
Long example
============
The example above uses several syntactic shortcuts.
If we had wanted a more brute-force approach to graph construction, we could have typed this.
.. code-block:: python
from theano.tensor import *
# We instantiate a type that represents a matrix of doubles
float64_matrix = Tensor(dtype = 'float64', # double
broadcastable = (False, False)) # matrix
# We make the Result instances we need.
x = Result(type = float64_matrix, name = 'x')
y = Result(type = float64_matrix, name = 'y')
z = Result(type = float64_matrix, name = 'z')
# This is the Result that we want to symbolically represents y*z
mul_result = Result(type = float64_matrix)
assert mul_result.owner is None
# We instantiate a symbolic multiplication
node_mul = Apply(op = mul,
inputs = [y, z],
outputs = [mul_result])
assert mul_result.owner is node_mul and mul_result.index == 0 # these fields are set by Apply
# This is the Result that we want to symbolically represents x+(y*z)
add_result = Result(type = float64_matrix)
assert add_result.owner is None
# We instantiate a symbolic addition
node_add = Apply(op = add,
inputs = [x, mul_result],
outputs = [add_result])
assert add_result.owner is node_add and add_result.index == 0 # these fields are set by Apply
e = add_result
# We have access to x, y and z through pointers
assert e.owner.inputs[0] is x
assert e.owner.inputs[1] is mul_result
assert e.owner.inputs[1].owner.inputs[0] is y
assert e.owner.inputs[1].owner.inputs[1] is z
Note how the call to `Apply` modifies the `owner` and `index` fields of the `Result` s passed as outputs to point to itself and the rank they occupy in the output list. This whole machinery builds a DAG (Directed Acyclic Graph) representing the computation, a graph that theano can compile and optimize.
.. _README: ../README.html .. _README: ../README.html
.. _Download: ../README.html#downloading-theano .. _Download: ../README.html#downloading-theano
......
.. _examples:
========
Examples
========
WRITEME
Should this be auto-generated?
...@@ -35,27 +35,29 @@ efficient machine learning algorithms while minimizing human ...@@ -35,27 +35,29 @@ efficient machine learning algorithms while minimizing human
time. Theano was named after the `Greek mathematician`_ who may have time. Theano was named after the `Greek mathematician`_ who may have
been Pythagoras' wife. been Pythagoras' wife.
Theano is released under a BSD license (:ref:`link <license>`)
You can keep reading from :ref:`here <usingtheano>`. You can keep reading from :ref:`here <usingtheano>`.
Getting started Getting started
=============== ===============
TODO: I want to bold the links below. How the fuck do I bold links? ``**`link name`_**`` doesn't work! :(
:ref:`install` :ref:`install`
Instructions to download and install Theano on your system. Instructions to download and install Theano on your system.
:ref:`basictutorial` :ref:`basictutorial`
Getting started with Theano's basic features. Go there if you are new! Getting started with Theano's basic features. Go there if you are
new!
:ref:`advtutorial` :ref:`advtutorial`
This tutorial is for more advanced users who want to define their own This tutorial is for more advanced users who want to define their
operations and optimizations. It is recommended to go through the own operations and optimizations. It is recommended to go through
:ref:`basictutorial` first. the :ref:`basictutorial` first.
...@@ -71,6 +73,7 @@ Contents ...@@ -71,6 +73,7 @@ Contents
advtutorial/index advtutorial/index
advanced/index advanced/index
indexes/index indexes/index
examples/index
glossary glossary
links links
LICENSE LICENSE
......
...@@ -18,7 +18,7 @@ to be installed: ...@@ -18,7 +18,7 @@ to be installed:
- numpy >=1.2 (earlier versions have memory leaks) - numpy >=1.2 (earlier versions have memory leaks)
- SciPy (specifically numpy, sparse, weave). We recommend scipy >=0.7 if you are using sparse matrices, because scipy.sparse is buggy in 0.6. (scipy.csc_matrix dot has a bug with singleton dimensions. There may be more bugs.) - SciPy (specifically numpy, sparse, weave). We recommend scipy >=0.7 if you are using sparse matrices, because scipy.sparse is buggy in 0.6. (scipy.csc_matrix dot has a bug with singleton dimensions. There may be more bugs.)
- g++, python-dev (optional but highly recommended, to compile generated C code) - g++, python-dev (optional but highly recommended, to compile generated C code)
- docutils, pygments (optional, to build documentation) - sphinx >=0.5.1, pygments (optional, to build documentation) (also latex and dvipng if you want math to show up as images...)
- mercurial (optional, to download the source) - mercurial (optional, to download the source)
- nose (nosetests) (optional, for testing) - nose (nosetests) (optional, for testing)
...@@ -134,3 +134,19 @@ As of now, the Windows platform is not supported. In fact, it has ...@@ -134,3 +134,19 @@ As of now, the Windows platform is not supported. In fact, it has
never even been tested, so feel free to explore this uncharted never even been tested, so feel free to explore this uncharted
territory and inform us of your progress! territory and inform us of your progress!
----------------------------
Generating the documentation
----------------------------
This should give you the gist of it:
.. code-block:: bash
$ python scripts/docgen.py --help
Usage: scripts/docgen.py [OPTIONS]
-o <dir>: output the html files in the specified dir
--rst: only compile the doc (requires sphinx)
--epydoc: only compile the api documentation (requires epydoc)
--help: this help
...@@ -39,6 +39,8 @@ efficient machine learning algorithms while minimizing human ...@@ -39,6 +39,8 @@ efficient machine learning algorithms while minimizing human
time. Theano was named after the `Greek mathematician`_ who may have time. Theano was named after the `Greek mathematician`_ who may have
been Pythagoras' wife. been Pythagoras' wife.
Theano is released under a BSD license (:ref:`link <license>`)
.. _usingtheano: .. _usingtheano:
......
.. _Basic Tutorial - More Examples: .. _basictutexamples:
============= =============
More examples More examples
...@@ -12,7 +12,9 @@ Logistic function ...@@ -12,7 +12,9 @@ Logistic function
Let's say that you want to compute the logistic curve, which is given Let's say that you want to compute the logistic curve, which is given
by: by:
``s(x) = 1 / (1 + e**(-x))`` .. math::
s(x) = \frac{1}{1 + e^{-x}}
You want to compute the function :term:`elementwise` on matrices of You want to compute the function :term:`elementwise` on matrices of
doubles. doubles.
...@@ -55,22 +57,132 @@ Also note the call to ``dmatrices``. This is a shortcut, use it wisely ...@@ -55,22 +57,132 @@ Also note the call to ``dmatrices``. This is a shortcut, use it wisely
Computing gradients Computing gradients
=================== ===================
WRITEME Now let's use Theano for a slightly more sophisticated task: create a
function which computes the derivative of some expression ``e`` with
respect to its parameter ``x``. For instance, we can compute the
gradient of :math:`x^2` with respect to :math:`x`.
>>> x = T.dscalar('x')
>>> y = x**2
>>> gy = T.grad(y, x)
>>> f = function([x], gy)
>>> f(4)
array(8.0)
>>> f(94.2)
array(188.40000000000001)
We can also compute the gradient of complex expressions such as the
logistic function defined above:
>>> x = T.dmatrix('x')
>>> s = 1 / (1 + T.exp(-x))
>>> gs = T.grad(s, x)
>>> glogistic = function([x], gs)
The resulting function computes the gradient of its first argument
with respect to the second. It is pretty much equivalent in semantics
and in computational complexity as what you would obtain through an
`automatic differentiation`_ tool.
.. note::
In general, the result of ``T.grad`` has the same dimensions as the
second argument. This is exactly like the first derivative if the
first argument is a scalar or a tensor of size 1 but not if it is
larger. For more information on the semantics when the first
argument has a larger size and details about the implementation,
see the :ref:`gradient` section.
Setting a default value for an argument Setting a default value for an argument
======================================= =======================================
WRITEME Let's say you want to define a function that adds two numbers, except
that if you only provide one number, the other input is assumed to be
one. You can do it like this:
>>> x, y = T.dscalars('xy')
>>> z = x + y
>>> f = function([x, (y, 1)], z)
>>> f(33)
array(34.0)
>>> f(33, 2)
array(35.0)
The syntax is that if one of the elements in the list of inputs is a
pair, the input is the first element of the pair and the second
element is its default value. Here ``y``'s default value is set to 1.
.. _functionstateexample:
Making a function with state Making a function with state
============================ ============================
WRITEME It is also possible to make a function with an internal state. For
example, let's say we want to make an accumulator: at the beginning,
the state is initialized to zero, then on each function call the state
is incremented by the function's argument. We'll also make it so that
the increment has a default value of 1.
First let's define the accumulator function:
>>> inc = T.scalar('inc')
>>> state = T.scalar('state')
>>> new_state = state + inc
>>> accumulator = function([(inc, 1), ((state, new_state), 0)], new_state)
The first argument is a pair. As we saw in the previous section this
simply means that inc is an input with a default value of 1. The
second argument has a new syntax which creates an internal state or
closure. The syntax is ``((state_result, new_state_result),
initial_value)``. What this means is that every time ``accumulator``
will be called, the value of the internal ``state`` will be replaced
by the value computed as ``new_state``. In this case, the state will
be replaced by the result of incrementing it by ``inc``.
There is no limit to how many states you can have. You can add an
arbitrary number of elements to the input list which correspond to the
syntax described in the previous paragraph. You can name the states
however you like as long as the name does not conflict with the names
of other inputs.
Anyway, let's try it out! The state can be accessed using the square
brackets notation ``[]``. You may access the state either by putting
the :ref:`result` representing it or the name of that
:ref:`result`. In our example we can access the state either with the
``state`` object or the string 'state'.
>>> accumulator[state]
array(0.0)
>>> accumulator['state']
array(0.0)
Here we use the accumulator and check that the state is correct each
time:
>>> accumulator()
array(1.0)
>>> accumulator['state']
array(1.0)
>>> accumulator(300)
array(301.0)
>>> accumulator['state']
array(301.0)
It is of course possible to reset the state. This is done very
naturally by assigning to the state using the square brackets
notation:
>>> accumulator['state'] = 5
>>> accumulator(0.9)
array(5.9000000000000004)
>>> accumulator['state']
array(5.9000000000000004)
**Next:** `Using Module`_ **Next:** `Using Module`_
.. _Using Module: module.html .. _Using Module: module.html
.. _automatic differentiation: http://en.wikipedia.org/wiki/Automatic_differentiation
...@@ -36,6 +36,11 @@ Now we're ready for the tour: ...@@ -36,6 +36,11 @@ Now we're ready for the tour:
`Using Module`_ `Using Module`_
Getting serious Getting serious
WRITEME: using modes?
`Wrapping up`_
A guide to what to look at next
--------------------------------------- ---------------------------------------
...@@ -47,9 +52,12 @@ Now we're ready for the tour: ...@@ -47,9 +52,12 @@ Now we're ready for the tour:
adding adding
examples examples
module module
wrapup
.. _Adding two numbers together: adding.html .. _Adding two numbers together: adding.html
.. _More examples: examples.html .. _More examples: examples.html
.. _Using Module: module.html .. _Using Module: module.html
.. _Wrapping up: wrapup.html
...@@ -3,5 +3,270 @@ ...@@ -3,5 +3,270 @@
Using Module Using Module
============ ============
WRITEME Now that we're familiar with the basics, we can see Theano's more
advanced interface, Module. This interface allows you to define Theano
"objects" which can have many state variables and many methods sharing
these states. This is what you should use if you aim to use Theano to
define complex systems such as a neural network.
Remake of the "state" example
=============================
Let's use Module to re-implement :ref:`this example
<functionstateexample>`.
>>> m = Module()
>>> m.state = Member(T.dscalar())
>>> m.inc = External(T.dscalar())
>>> m.new_state = m.state + m.inc
>>> m.call = Method(m.inc, m.new_state, {m.state: m.new_state})
>>> acc = m.make(state = 0)
>>> acc.state
array(0.0)
>>> acc.call(2)
array(2.0)
>>> acc.state
array(2.0)
>>> acc.state = 39.99
>>> acc.call(0.01)
array(40.0)
>>> acc.state
array(40.0)
This deserves to be broken up a bit...
>>> m = Module()
Here we instantiate an empty Module.
>>> m.state = Member(T.dscalar())
Then we declare a variable for use with our Module. That variable will
be a :ref:`member` of the module, which means that it will be
accessible as a field of the object we will create later (for reading
and writing). It will also be accessible from any :ref:`method`
defined in our Module.
.. note::
There is no need to name the variable explicitly here. m.state will
be given the name 'state' automatically.
>>> m.inc = External(T.dscalar('inc'))
The inc variable doesn't need to be declared as a Member because it
will only serve as an input to the method we will define. This is why
it is defined as an :ref:`external` variable. Do note that it is
inconsequential if you do declare it as a Member - it is very unlikely
to cause you any problems.
.. note::
Both Member and External are optional. If you do the direct assignment:
>>> m.state = T.dscalar()
The default is to declare m.state as a Member. This is a sensible
default in most situations and can avoid clutter, so feel free to
omit explicit calls to Member.
>>> m.new_state = m.state + m.inc
This line describes how to compute the new state.
.. note::
Here new_state is implicitly declared as External since it is
illegal to declare a Result as a Member if it is the result of
previous computations.
>>> m.call = Method(m.inc, m.new_state, {m.state: m.new_state})
Here we declare a Method. The three arguments are as follow:
* **inputs**: a list of input Results
* **outputs**: a list of output Results
* **updates**: a dictionary mapping a Result declared as a Member to a
Result representing the computation of the next state of the member.
If possible, you may also give the updates as keyword arguments, as
in: ``Method(m.inc, m.new_state, state = m.new_state)``. This implies
that m.state's name is 'state'.
>>> acc = m.make(state = 0)
This line is what does the magic. Everything in m is symbolic. Once
the make method is called on it, an object is made which can do real
computation and whose fields contain real values.
The keyword arguments given to make are optional and are used to
assign initial values to each Member. If a Member is omitted, the
initial value is None.
>>> acc.state
array(0.0)
Since state was declared as a Member, we can access it easily using
'.state'.
>>> acc.call(2)
array(2.0)
>>> acc.state
array(2.0)
>>> acc.call(6)
array(8.0)
>>> acc.state
array(8.0)
When we call the ``call`` method, all the updates given to the
corresponding Method's ``updates`` field are performed. We only had
one update which mapped ``state`` to ``new_state`` and you can see
that it works as intended, adding the argument to the internal state
every time.
>>> acc.state = 39.99
>>> acc.call(0.01)
array(40.0)
>>> acc.state
array(40.0)
The state is also easy to set.
Using Inheritance
=================
A friendlier way to use Module is to implement your functionality as a
subclass of Module:
.. code-block:: python
class Accumulator(Module):
def __init__(self):
super(Accumulator, self).__init__() # don't forget this
self.inc = External(T.dscalar())
self.state = Member(T.dscalar())
self.new_state = self.inc + self.state
self.call = Method(inputs = self.inc,
outputs = self.new_state,
updates = {self.state: self.new_state})
m = Accumulator()
acc = m.make(state = 0)
This is just like the previous example except slightly fancier.
.. warning::
Do not forget to call the constructor of the parent class! (the
call to ``super().__init__`` in the previous code block) If you
forget it, you'll get strange behavior :(
Extending your Module with Python methods
=========================================
Let's say we want to add a method to our accumulator to print out the
state and we want to call it ``print_state``. All we need to do is to
give a method called ``_instance_print_state`` to our Module.
.. code-block:: python
class Accumulator(Module):
def __init__(self):
super(Accumulator, self).__init__() # don't forget this
self.inc = External(T.dscalar())
self.state = Member(T.dscalar())
self.new_state = self.inc + self.state
self.call = Method(inputs = self.inc,
outputs = self.new_state,
updates = {self.state: self.new_state})
def _instance_print_state(self, acc):
print '%s is: %s' % (self.state, acc.state)
m = Accumulator()
acc = m.make(state = 0)
acc.print_state() # --> prints "state is: 0.0"
Any method called like ``_instance_XXX`` will result in the object
obtained through a call to ``make`` to gain an ``XXX`` method. Note
that when we define ``_instance_print_state`` there are two "self"
arguments: ``self`` which is *symbolic* and ``obj`` which contains
*data*. Therefore, ``self.state`` is the symbolic state variable and
prints out as "state", whereas ``obj.state`` is the state's actual
value in the accumulator and prints out as "0.0".
Adding custom initialization
============================
As was said in the previous section, you can add functionality with
``_instance_XXX`` methods. One of these methods is actually special:
``_instance_initialize`` will be called with whatever arguments you
give to ``make``. There is a default behavior which we have used,
where we give the states' initial values with keyword arguments
(``acc.make(state = 0)``). If you want more personalized behavior, you
can override the default with your own method, which has to be called
``_instance_initialize``.
Here is an example where we take width and height arguments to
initialize a state with a matrix of zeros:
.. code-block:: python
import numpy
class MatrixAccumulator(Module):
def __init__(self):
super(MatrixAccumulator, self).__init__() # don't forget this
self.inc = External(T.dscalar())
self.state = Member(T.dmatrix())
self.new_state = self.inc + self.state
self.call = Method(inputs = self.inc,
outputs = self.new_state,
updates = {self.state: self.new_state})
def _instance_print_state(self, acc):
print '%s is: %s' % (self.state, acc.state)
def _instance_initialize(self, acc, nrows, ncols):
acc.state = numpy.zeros((nrows, ncols))
m = Accumulator()
acc = m.make(2, 5) # this calls m._instance_initialize(acc, 2, 5)
acc.print_state()
# OUTPUT:
# state is: [[ 0. 0. 0. 0. 0.]
# [ 0. 0. 0. 0. 0.]]
**Next:** `Wrapping up`_
.. _Wrapping up: wrapup.html
===========
Wrapping up
===========
WRITEME
...@@ -124,7 +124,7 @@ class BadDestroyMap(DebugModeError): ...@@ -124,7 +124,7 @@ class BadDestroyMap(DebugModeError):
print >> sio, " repr (old val):", repr(self.old_val) print >> sio, " repr (old val):", repr(self.old_val)
print >> sio, " repr (new val):", repr(self.new_val) print >> sio, " repr (new val):", repr(self.new_val)
print >> sio, "" print >> sio, ""
print >> sio, " Hint: this can be caused by a deficient values_eq_enough() or __eq__() implementation that compares node input values" print >> sio, " Hint: this can be caused by a deficient values_eq_approx() or __eq__() implementation that compares node input values"
return sio.getvalue() return sio.getvalue()
class StochasticOrder(DebugModeError): class StochasticOrder(DebugModeError):
...@@ -216,7 +216,7 @@ def _check_inputs(node, storage_map, r_vals, dr_vals, active_nodes): ...@@ -216,7 +216,7 @@ def _check_inputs(node, storage_map, r_vals, dr_vals, active_nodes):
destroyed_res_list = [node.inputs[i] for i in destroyed_idx_list] destroyed_res_list = [node.inputs[i] for i in destroyed_idx_list]
for r_idx, r in enumerate(node.inputs): for r_idx, r in enumerate(node.inputs):
if not r.type.values_eq_enough(r_vals[r], storage_map[r][0]): if not r.type.values_eq_approx(r_vals[r], storage_map[r][0]):
# some input node 'r' got changed by running the node # some input node 'r' got changed by running the node
# this may or may not be ok... # this may or may not be ok...
if r in destroyed_res_list: if r in destroyed_res_list:
...@@ -257,7 +257,7 @@ def _find_bad_optimizations0(order, reasons, r_vals): ...@@ -257,7 +257,7 @@ def _find_bad_optimizations0(order, reasons, r_vals):
r_val = r_vals[r] r_val = r_vals[r]
assert r.type == new_r.type assert r.type == new_r.type
if not r.type.values_eq_enough(r_val, new_r_val): if not r.type.values_eq_approx(r_val, new_r_val):
raise BadOptimization(old_r=r, raise BadOptimization(old_r=r,
new_r=new_r, new_r=new_r,
old_r_val=r_val, old_r_val=r_val,
...@@ -296,7 +296,7 @@ def _find_bad_optimizations1(order, reasons, r_vals): ...@@ -296,7 +296,7 @@ def _find_bad_optimizations1(order, reasons, r_vals):
new_r_val = r_vals[re] new_r_val = r_vals[re]
r_val = r_vals[re0] r_val = r_vals[re0]
assert re.type == re0.type assert re.type == re0.type
if not re.type.values_eq_enough(r_val, new_r_val): if not re.type.values_eq_approx(r_val, new_r_val):
equivalence_sets_broken[id(r_equiv)] = True equivalence_sets_broken[id(r_equiv)] = True
there_is_a_problem = True there_is_a_problem = True
re0 = re re0 = re
...@@ -667,7 +667,7 @@ class _Linker(gof.link.LocalLinker): ...@@ -667,7 +667,7 @@ class _Linker(gof.link.LocalLinker):
raise InvalidValueError(r, storage_map[r][0]) raise InvalidValueError(r, storage_map[r][0])
# compares the version from thunk_py (in r_vals) # compares the version from thunk_py (in r_vals)
# to the version produced by thunk_c (in storage_map) # to the version produced by thunk_c (in storage_map)
if not r.type.values_eq_enough(r_vals[r], storage_map[r][0]): if not r.type.values_eq_approx(r_vals[r], storage_map[r][0]):
raise BadClinkerOutput(r, val_py=r_vals[r], val_c=storage_map[r][0]) raise BadClinkerOutput(r, val_py=r_vals[r], val_c=storage_map[r][0])
storage_map[r][0] = None #clear the storage_map for the thunk_c storage_map[r][0] = None #clear the storage_map for the thunk_c
......
...@@ -221,7 +221,11 @@ class PureType(object): ...@@ -221,7 +221,11 @@ class PureType(object):
def is_valid_value(self, a): def is_valid_value(self, a):
"""Required: Return True for any python object `a` that would be a legal value for a Result of this Type""" """Required: Return True for any python object `a` that would be a legal value for a Result of this Type"""
raise AbstractFunctionError() try:
self.filter(a, True)
return True
except TypeError:
return False
def make_result(self, name = None): def make_result(self, name = None):
"""Return a new `Result` instance of Type `self`. """Return a new `Result` instance of Type `self`.
...@@ -246,8 +250,17 @@ class PureType(object): ...@@ -246,8 +250,17 @@ class PureType(object):
r.tag.trace = traceback.extract_stack()[:-1] r.tag.trace = traceback.extract_stack()[:-1]
return r return r
def values_eq_enough(self, a, b): def values_eq(self, a, b):
"""Return True if a and b can be considered equal as Op outputs, else False. """
Return True if a and b can be considered exactly equal.
a and b are assumed to be valid values of this Type.
"""
return a == b
def values_eq_approx(self, a, b):
"""
Return True if a and b can be considered approximately equal.
:param a: a potential value for a Result of this Type. :param a: a potential value for a Result of this Type.
...@@ -255,12 +268,14 @@ class PureType(object): ...@@ -255,12 +268,14 @@ class PureType(object):
:rtype: Bool :rtype: Bool
This function is used by theano debugging tools to decide whether two values are This function is used by theano debugging tools to decide
equivalent, admitting a certain amount of numerical instability. For example, whether two values are equivalent, admitting a certain amount
for floating-point numbers this function should be an approximate comparison. of numerical instability. For example, for floating-point
numbers this function should be an approximate comparison.
By default, this does an exact comparison.
""" """
return (a == b) return self.values_eq(a, b)
_nothing = """ _nothing = """
......
...@@ -52,13 +52,8 @@ class Scalar(Type): ...@@ -52,13 +52,8 @@ class Scalar(Type):
except Exception, e: except Exception, e:
raise TypeError("Could not convert %s (value=%s) to %s" % (type(data), data, self.dtype), e) raise TypeError("Could not convert %s (value=%s) to %s" % (type(data), data, self.dtype), e)
def values_eq_enough(self, a, b): def values_eq_approx(self, a, b, tolerance = 1e-4):
return abs(a - b) / (a+b) < 1e-4 return abs(a - b) / (a+b) < tolerance
def is_valid_value(self, a):
_a = numpy.asarray(a)
rval = (_a.ndim == 0) and (str(_a.dtype) == self.dtype)
return rval
def __eq__(self, other): def __eq__(self, other):
return type(self) == type(other) and other.dtype == self.dtype return type(self) == type(other) and other.dtype == self.dtype
......
...@@ -186,7 +186,8 @@ class Sparse(gof.Type): ...@@ -186,7 +186,8 @@ class Sparse(gof.Type):
def __repr__(self): def __repr__(self):
return "Sparse[%s, %s]" % (str(self.dtype), str(self.format)) return "Sparse[%s, %s]" % (str(self.dtype), str(self.format))
def values_eq_enough(self, a, b, eps=1e-6): def values_eq_approx(self, a, b, eps=1e-6):
# print "VEA", a, b, scipy.sparse.issparse(a), scipy.sparse.issparse(b), abs(a-b).sum(), abs(a-b).sum() < (1e-6 * a.nnz)
return scipy.sparse.issparse(a) \ return scipy.sparse.issparse(a) \
and scipy.sparse.issparse(b) \ and scipy.sparse.issparse(b) \
and abs(a-b).sum() < (1e-6 * a.nnz) and abs(a-b).sum() < (1e-6 * a.nnz)
......
...@@ -222,21 +222,10 @@ class Tensor(Type): ...@@ -222,21 +222,10 @@ class Tensor(Type):
"""Compare True iff other is the same kind of Tensor""" """Compare True iff other is the same kind of Tensor"""
return type(self) == type(other) and other.dtype == self.dtype and other.broadcastable == self.broadcastable return type(self) == type(other) and other.dtype == self.dtype and other.broadcastable == self.broadcastable
def values_eq_enough(self, a, b): def values_eq_approx(self, a, b):
return type(a) is numpy.ndarray and type(b) is numpy.ndarray \ return type(a) is numpy.ndarray and type(b) is numpy.ndarray \
and (a.shape == b.shape) and numpy.allclose(a, b) and (a.shape == b.shape) and numpy.allclose(a, b)
def is_valid_value(self, a):
rval = (type(a) is numpy.ndarray) and (self.ndim == a.ndim) \
and (str(a.dtype) == self.dtype) \
and all([((si == 1) or not bi) for si, bi in zip(a.shape, self.broadcastable)])
if not rval:
print type(a),(type(a) is numpy.ndarray)
print a.ndim, (self.ndim == a.ndim)
print a.dtype, (str(a.dtype) == self.dtype)
print a.shape, self.broadcastable, ([(shp_i == 1) for shp_i in a.shape] == self.broadcastable)
return rval
def __hash__(self): def __hash__(self):
"""Hash equal for same kinds of Tensor""" """Hash equal for same kinds of Tensor"""
return hash(self.dtype) ^ hash(self.broadcastable) return hash(self.dtype) ^ hash(self.broadcastable)
......
Markdown 格式
0%
您添加了 0 到此讨论。请谨慎行事。
请先完成此评论的编辑!
注册 或者 后发表评论