Skip to content

P
pytensor
提交 71021fec authored 作者: James Bergstra's avatar James Bergstra
浏览文件
操作

docs

上级 8d9e127e
隐藏空白字符变更
内嵌 并排
正在显示 包含 530 行增加 和 496 行删除
doc/library/compile/function.txt
浏览文件 @ 71021fec
.. _usingfunction:
=====================
===========================================
:mod:`function` - defines theano.function
===========================================
.. class:: Param
.. class:: In
This is for internal use. User code should use `Param`
.. function:: function(x)
TODO
.. _libdoc_compile_function:
compile.function
=====================
================
This page is about `theano.function`, the interface for compiling
graphs into callable objects.
The signature for this function is:
.. code-block:: python
def function(inputs, outputs, mode=None):
...
You've already seen example usage in the basic tutorial... something like this:
>>> x = theano.tensor.dscalar()
>>> f = theano.function([x], 2*x)
>>> print f(4) # prints 8.0
The idea here is that we've compiled the symbolic graph (``2*x``) into a function that can be called on a number and will do some computations.
.. _function_inputs:
Inputs
======
The ``inputs`` argument to ``theano.function`` is a list, containing the ``Variable`` instances for which values will be specified at the time of the function call. But inputs can be more than just Variables.
``In`` instances let us attach properties to ``Variables`` to tell function more about how to use them.
.. class:: In
.. method:: __init__(variable, name=None, value=None, update=None, mutable=False, strict=False, autoname=True, implicit=None)
``variable``: a Variable instance. This will be assigned a value
before running the function, not computed from its owner.
``name``: Any type. (If ``autoname_input==True``, defaults to
``variable.name``). If ``name`` is a valid Python identifier, this input
can be set by ``kwarg``, and its value can be accessed by
``self.<name>``. The default value is ``None``.
``value``: literal or ``Container``. The initial/default value for this
input. If update is`` None``, this input acts just like
an argument with a default value in Python. If update is not ``None``,
changes to this
value will "stick around", whether due to an update or a user's
explicit action.
``update``: Variable instance. This expression Variable will
replace ``value`` after each function call. The default value is
``None``, indicating that no update is to be done.
``mutable``: Bool (requires value). If ``True``, permit the
compiled function to modify the Python object being used as the
default value. The default value is ``False``.
``strict``: Bool (default: ``False`` ). ``True`` means that the value
you pass for this input must have exactly the right type. Otherwise, it
may be cast automatically to the proper type.
``autoname``: Bool. If set to ``True``, if ``name`` is ``None`` and
the Variable has a name, it will be taken as the input's
name. If autoname is set to ``False``, the name is the exact
value passed as the name parameter (possibly ``None``).
``implicit``: Bool or ``None`` (default: ``None``)
``True``: This input is implicit in the sense that the user is not allowed
to provide a value for it. Requires ``value`` to be set.
``False``: The user can provide a value for this input. Be careful
when ``value`` is a container, because providing an input value will
overwrite the content of this container.
``None``: Automatically choose between ``True`` or ``False`` depending on the
situation. It will be set to ``False`` in all cases except if
``value`` is a container (so that there is less risk of accidentally
overwriting its content without being aware of it).
Value: initial and default values
---------------------------------
A non-None `value` argument makes an In() instance an optional parameter
of the compiled function. For example, in the following code we are
defining an arity-2 function ``inc``.
>>> u, x, s = T.scalars('u', 'x', 's')
>>> inc = function([u, In(x, value=3), In(s, update=(s+x*u), value=10.0)], [])
Since we provided a ``value`` for ``s`` and ``x``, we can call it with just a value for ``u`` like this:
>>> inc(5) # update s with 10+3*5
[]
>>> print inc[s]
25.0
The effect of this call is to increment the storage associated to ``s`` in ``inc`` by 15.
If we pass two arguments to ``inc``, then we override the value associated to
``x``, but only for this one function call.
>>> inc(3, 4) # update s with 25 + 3*4
[]
>>> print inc[s]
37.0
>>> print inc[x] # the override value of 4 was only temporary
3.0
If we pass three arguments to ``inc``, then we override the value associated
with ``x`` and ``u`` and ``s``.
Since ``s``'s value is updated on every call, the old value of ``s`` will be ignored and then replaced.
>>> inc(3, 4, 7) # update s with 7 + 3*4
[]
>>> print inc[s]
19.0
We can also assign to ``inc[s]`` directly:
>>> inc[s] = 10
>>> inc[s]
array(10.0)
Advanced: Sharing Storage Between Functions
-------------------------------------------
``value`` can be a :api:`theano.gof.link.Container` as well as a literal.
This permits linking a value of a Variable in one function to the value of a Variable in another function.
By using a ``Container`` as a value we can implement shared variables between functions.
For example, consider the following program.
>>> x, s = T.scalars('xs')
>>> inc = function([x, In(s, update=(s+x), value=10.0)], [])
>>> dec = function([x, In(s, update=(s-x), value=inc.container[s])], [])
>>> dec(3)
[]
>>> print inc[s]
7.0
>>> inc(2)
[]
>>> print dec[s]
9.0
The functions ``inc`` and ``dec`` operate on a shared internal value for ``s``.
Theano's Module system uses this mechanism to share storage between Methods.
The container being shared doesn't have to correspond to the same Variable in both functions,
but that's usually how this mechanism is used.
Note that when an input's ``value`` parameter is a shared container, this
input is considered as implicit by default. This means it cannot be set by the
user.
If ``implicit`` is manually set to ``False``, then it can be set by the user,
but then it will overwrite the container's content, so one should be careful
when allowing this.
This is illustrated in the following example.
>>> dec(1, 0) # Try to manually set an implicit input
<type 'exceptions.TypeError'>: Tried to provide value for implicit input: s
>>> dec = function([x, In(s, update=(s-x), value=inc.container[s], implicit=False)], [])
>>> inc[s] = 2
>>> print dec[s] # Containers are shared
2.0
>>> dec(1)
[]
>>> print inc[s] # Calling dec decreased the value in inc's container
1.0
>>> dec(1, 0) # Update inc[s] with 0 - 1 = -1
[]
>>> print inc[s]
-1.0
>>> print dec[s] # Still shared
-1.0
Input Argument Restrictions
---------------------------
The following restrictions apply to the inputs to ``theano.function``:
- Every input list element must be a valid ``In`` instance, or must be
upgradable to a valid ``In`` instance. See the shortcut rules below.
- The same restrictions apply as in Python function definitions:
default arguments and keyword arguments must come at the end of
the list. Un-named mandatory arguments must come at the beginning of
the list.
- Names have to be unique within an input list. If multiple inputs
have the same name, then the function will raise an exception. [***Which
exception?**]
- Two ``In`` instances may not name the same Variable. I.e. you cannot
give the same parameter multiple times.
If no name is specified explicitly for an In instance, then its name
will be taken from the Variable's name. Note that this feature can cause
harmless-looking input lists to not satisfy the two conditions above.
In such cases, Inputs should be named explicitly to avoid problems
such as duplicate names, and named arguments preceding unnamed ones.
This automatic naming feature can be disabled by instantiating an In
instance explicitly with the ``autoname`` flag set to False.
Access to function values and containers
----------------------------------------
For each input, ``theano.function`` will create a ``Container`` if
``value`` was not already a ``Container`` (or if ``implicit`` was ``False``). At the time of a function call,
each of these containers must be filled with a value. Each input (but
especially ones with a default value or an update expression) may have a
value between calls. The function interface defines a way to get at
both the current value associated with an input, as well as the container
which will contain all future values:
- The ``value`` property accesses the current values. It is both readable
and writable, but assignments (writes) may be implemented by an internal
copy and/or casts.
- The ``container`` property accesses the corresponding container.
This property accesses is a read-only dictionary-like interface. It is
useful for fetching the container associated with a particular input to
share containers between functions, or to have a sort of pointer to an
always up-to-date value.
Both ``value`` and ``container`` properties provide dictionary-like access based on three types of keys:
- integer keys: you can look up a value/container by its position in the input list;
- name keys: you can look up a value/container by its name;
- Variable keys: you can look up a value/container by the Variable it corresponds to.
In addition to these access mechanisms, there is an even more convenient
method to access values by indexing a Function directly by typing
``fn[<name>]``, as in the examples above.
To show some examples of these access methods...
.. code-block:: python
a, b, c = T.scalars('xys') # set the internal names of graph nodes
# Note that the name of c is 's', not 'c'!
fn = function([a, b, ((c, c+a+b), 10.0)], [])
#the value associated with c is accessible in 3 ways
assert fn['s'] is fn.value[c]
assert fn['s'] is fn.container[c].value
assert fn['s'] == 10.0
fn(1, 2)
assert fn['s'] == 13.0
fn.s = 99.0
fn(1, 0)
assert fn['s'] == 100.0
fn.value[c] = 99.0
fn(1,0)
assert fn['s'] == 100.0
assert fn['s'] == fn.value[c]
assert fn['s'] == fn.container[c].value
Input Shortcuts
---------------
Every element of the inputs list will be upgraded to an In instance if necessary.
- a Variable instance ``r`` will be upgraded like ``In(r)``
- a tuple ``(name, r)`` will be ``In(r, name=name)``
- a tuple ``(r, val)`` will be ``In(r, value=value, autoname=True)``
- a tuple ``((r,up), val)`` will be ``In(r, value=value, update=up, autoname=True)``
- a tuple ``(name, r, val)`` will be ``In(r, name=name, value=value)``
- a tuple ``(name, (r,up), val)`` will be ``In(r, name=name, value=val, update=up, autoname=True)``
Example:
.. code-block:: python
import theano
from theano import tensor as T
from theano.compile.io import In
x = T.scalar()
y = T.scalar('y')
z = T.scalar('z')
w = T.scalar('w')
fn = theano.function(inputs = [x, y, In(z, value=42), ((w, w+x), 0)],
outputs = x + y + z)
# the first two arguments are required and the last two are
# optional and initialized to 42 and 0, respectively.
# The last argument, w, is updated with w + x each time the
# function is called.
fn(1) # illegal because there are two required arguments
fn(1, 2) # legal, z is 42, w goes 0 -> 1 (because w <- w + x), returns array(45.0)
fn(1, y = 2) # legal, z is 42, w goes 1 -> 2, returns array(45.0)
fn(x = 1, y = 2) # illegal because x was not named
fn(1, 2, 3) # legal, z is 3, w goes 2 -> 3, returns array(6.0)
fn(1, z = 3, y = 2) # legal, z is 3, w goes 3 -> 4, returns array(6.0)
fn(1, 2, w = 400) # legal, z is 42 again, w goes 400 -> 401, returns array(45.0)
fn(1, 2) # legal, z is 42, w goes 401 -> 402, returns array(45.0)
In the example above, ``z`` has value 42 when no value is explicitly given.
This default value is potentially used at every function invocation, because
``z`` has no ``update`` or storage associated with it.
.. _function_outputs:
Outputs
=======
The ``outputs`` argument to function can be one of
- ``None``, or
- a Variable or ``Out`` instance, or
- a list of Variables or ``Out`` instances.
An ``Out`` instance is a structure that lets us attach options to individual output ``Variable`` instances,
similarly to how ``In`` lets us attach options to individual input ``Variable`` instances.
**Out(variable, borrow=False)** returns an ``Out`` instance:
* ``borrow``
If ``True``, a reference to function's internal storage
is OK. A value returned for this output might be clobbered by running
the function again, but the function might be faster.
Default: ``False``
If a single ``Variable`` or ``Out`` instance is given as argument, then the compiled function will return a single value.
If a list of ``Variable`` or ``Out`` instances is given as argument, then the compiled function will return a list of their values.
.. code-block:: python
x, y, s = T.matrices('xys')
# print a list of 2 ndarrays
fn1 = theano.function([x], [x+x, Out((x+x).T, borrow=True)])
print fn1(numpy.asarray([[1,0],[0,1]]))
# print a list of 1 ndarray
fn2 = theano.function([x], [x+x])
print fn2(numpy.asarray([[1,0],[0,1]]))
# print an ndarray
fn3 = theano.function([x], outputs=x+x)
print fn3(numpy.asarray([[1,0],[0,1]]))
.. _function_mode:
Mode
====
The ``mode`` parameter to ``theano.function`` controls how the
inputs-to-outputs graph is transformed into a callable object.
Theano defines the following modes by name:
- ``FAST_COMPILE``: Apply just a few optimizations, but use C op implementations where possible.
- ``FAST_RUN``: Apply all optimizations, and use C op implementations where possible.
- ``DEBUG_MODE``: Verify the correctness of all optimizations, and compare C and python
implementations. This mode can take much longer than the other modes,
but can identify many kinds of problems.
The default mode is typically 'FAST_RUN', but it can be controlled via the
environment variable 'THEANO_DEFAULT_MODE', which can in turn be overridden by
setting ``theano.compile.mode.default_mode`` directly, which can in turn be
overridden by passing the keyword argument to ``theano.function``.
For a finer level of control over which optimizations are applied, and whether
C or python implementations are used, read :api:`compile.mode.Mode`.
.. _compile_debugMode:
DebugMode ??
doc/library/compile/index.txt
浏览文件 @ 71021fec
.. _libdoc_compile:
========================================================================
:mod:`compile` -- transforming expression graphs to functions [doc TODO]
========================================================================
.. class:: Param
.. class:: In
This is for internal use. User code should use `Param`
.. function:: function(x)
TODO
.. _libdoc_compile_function:
compile.function
================
This page is about `theano.function`, the interface for compiling
graphs into callable objects.
The signature for this function is:
.. code-block:: python
def function(inputs, outputs, mode=None):
...
You've already seen example usage in the basic tutorial... something like this:
>>> x = theano.tensor.dscalar()
>>> f = theano.function([x], 2*x)
>>> print f(4) # prints 8.0
The idea here is that we've compiled the symbolic graph (``2*x``) into a function that can be called on a number and will do some computations.
.. _function_inputs:
Inputs
======
The ``inputs`` argument to ``theano.function`` is a list, containing the ``Variable`` instances for which values will be specified at the time of the function call. But inputs can be more than just Variables.
``In`` instances let us attach properties to ``Variables`` to tell function more about how to use them.
.. class:: In
.. method:: __init__(variable, name=None, value=None, update=None, mutable=False, strict=False, autoname=True, implicit=None)
``variable``: a Variable instance. This will be assigned a value
before running the function, not computed from its owner.
``name``: Any type. (If ``autoname_input==True``, defaults to
``variable.name``). If ``name`` is a valid Python identifier, this input
can be set by ``kwarg``, and its value can be accessed by
``self.<name>``. The default value is ``None``.
``value``: literal or ``Container``. The initial/default value for this
input. If update is`` None``, this input acts just like
an argument with a default value in Python. If update is not ``None``,
changes to this
value will "stick around", whether due to an update or a user's
explicit action.
``update``: Variable instance. This expression Variable will
replace ``value`` after each function call. The default value is
``None``, indicating that no update is to be done.
``mutable``: Bool (requires value). If ``True``, permit the
compiled function to modify the Python object being used as the
default value. The default value is ``False``.
``strict``: Bool (default: ``False`` ). ``True`` means that the value
you pass for this input must have exactly the right type. Otherwise, it
may be cast automatically to the proper type.
``autoname``: Bool. If set to ``True``, if ``name`` is ``None`` and
the Variable has a name, it will be taken as the input's
name. If autoname is set to ``False``, the name is the exact
value passed as the name parameter (possibly ``None``).
``implicit``: Bool or ``None`` (default: ``None``)
``True``: This input is implicit in the sense that the user is not allowed
to provide a value for it. Requires ``value`` to be set.
``False``: The user can provide a value for this input. Be careful
when ``value`` is a container, because providing an input value will
overwrite the content of this container.
``None``: Automatically choose between ``True`` or ``False`` depending on the
situation. It will be set to ``False`` in all cases except if
``value`` is a container (so that there is less risk of accidentally
overwriting its content without being aware of it).
Value: initial and default values
---------------------------------
A non-None `value` argument makes an In() instance an optional parameter
of the compiled function. For example, in the following code we are
defining an arity-2 function ``inc``.
>>> u, x, s = T.scalars('u', 'x', 's')
>>> inc = function([u, In(x, value=3), In(s, update=(s+x*u), value=10.0)], [])
Since we provided a ``value`` for ``s`` and ``x``, we can call it with just a value for ``u`` like this:
>>> inc(5) # update s with 10+3*5
[]
>>> print inc[s]
25.0
The effect of this call is to increment the storage associated to ``s`` in ``inc`` by 15.
If we pass two arguments to ``inc``, then we override the value associated to
``x``, but only for this one function call.
>>> inc(3, 4) # update s with 25 + 3*4
[]
>>> print inc[s]
37.0
>>> print inc[x] # the override value of 4 was only temporary
3.0
If we pass three arguments to ``inc``, then we override the value associated
with ``x`` and ``u`` and ``s``.
Since ``s``'s value is updated on every call, the old value of ``s`` will be ignored and then replaced.
>>> inc(3, 4, 7) # update s with 7 + 3*4
[]
>>> print inc[s]
19.0
We can also assign to ``inc[s]`` directly:
>>> inc[s] = 10
>>> inc[s]
array(10.0)
Advanced: Sharing Storage Between Functions
-------------------------------------------
``value`` can be a :api:`theano.gof.link.Container` as well as a literal.
This permits linking a value of a Variable in one function to the value of a Variable in another function.
By using a ``Container`` as a value we can implement shared variables between functions.
For example, consider the following program.
>>> x, s = T.scalars('xs')
>>> inc = function([x, In(s, update=(s+x), value=10.0)], [])
>>> dec = function([x, In(s, update=(s-x), value=inc.container[s])], [])
>>> dec(3)
[]
>>> print inc[s]
7.0
>>> inc(2)
[]
>>> print dec[s]
9.0
The functions ``inc`` and ``dec`` operate on a shared internal value for ``s``.
Theano's Module system uses this mechanism to share storage between Methods.
The container being shared doesn't have to correspond to the same Variable in both functions,
but that's usually how this mechanism is used.
Note that when an input's ``value`` parameter is a shared container, this
input is considered as implicit by default. This means it cannot be set by the
user.
If ``implicit`` is manually set to ``False``, then it can be set by the user,
but then it will overwrite the container's content, so one should be careful
when allowing this.
This is illustrated in the following example.
>>> dec(1, 0) # Try to manually set an implicit input
<type 'exceptions.TypeError'>: Tried to provide value for implicit input: s
>>> dec = function([x, In(s, update=(s-x), value=inc.container[s], implicit=False)], [])
>>> inc[s] = 2
>>> print dec[s] # Containers are shared
2.0
>>> dec(1)
[]
>>> print inc[s] # Calling dec decreased the value in inc's container
1.0
>>> dec(1, 0) # Update inc[s] with 0 - 1 = -1
[]
>>> print inc[s]
-1.0
>>> print dec[s] # Still shared
-1.0
Input Argument Restrictions
---------------------------
The following restrictions apply to the inputs to ``theano.function``:
- Every input list element must be a valid ``In`` instance, or must be
upgradable to a valid ``In`` instance. See the shortcut rules below.
- The same restrictions apply as in Python function definitions:
default arguments and keyword arguments must come at the end of
the list. Un-named mandatory arguments must come at the beginning of
the list.
- Names have to be unique within an input list. If multiple inputs
have the same name, then the function will raise an exception. [***Which
exception?**]
- Two ``In`` instances may not name the same Variable. I.e. you cannot
give the same parameter multiple times.
If no name is specified explicitly for an In instance, then its name
will be taken from the Variable's name. Note that this feature can cause
harmless-looking input lists to not satisfy the two conditions above.
In such cases, Inputs should be named explicitly to avoid problems
such as duplicate names, and named arguments preceding unnamed ones.
This automatic naming feature can be disabled by instantiating an In
instance explicitly with the ``autoname`` flag set to False.
Access to function values and containers
----------------------------------------
For each input, ``theano.function`` will create a ``Container`` if
``value`` was not already a ``Container`` (or if ``implicit`` was ``False``). At the time of a function call,
each of these containers must be filled with a value. Each input (but
especially ones with a default value or an update expression) may have a
value between calls. The function interface defines a way to get at
both the current value associated with an input, as well as the container
which will contain all future values:
- The ``value`` property accesses the current values. It is both readable
and writable, but assignments (writes) may be implemented by an internal
copy and/or casts.
- The ``container`` property accesses the corresponding container.
This property accesses is a read-only dictionary-like interface. It is
useful for fetching the container associated with a particular input to
share containers between functions, or to have a sort of pointer to an
always up-to-date value.
Both ``value`` and ``container`` properties provide dictionary-like access based on three types of keys:
- integer keys: you can look up a value/container by its position in the input list;
- name keys: you can look up a value/container by its name;
- Variable keys: you can look up a value/container by the Variable it corresponds to.
In addition to these access mechanisms, there is an even more convenient
method to access values by indexing a Function directly by typing
``fn[<name>]``, as in the examples above.
To show some examples of these access methods...
.. code-block:: python
a, b, c = T.scalars('xys') # set the internal names of graph nodes
# Note that the name of c is 's', not 'c'!
fn = function([a, b, ((c, c+a+b), 10.0)], [])
#the value associated with c is accessible in 3 ways
assert fn['s'] is fn.value[c]
assert fn['s'] is fn.container[c].value
assert fn['s'] == 10.0
fn(1, 2)
assert fn['s'] == 13.0
fn.s = 99.0
fn(1, 0)
assert fn['s'] == 100.0
fn.value[c] = 99.0
fn(1,0)
assert fn['s'] == 100.0
assert fn['s'] == fn.value[c]
assert fn['s'] == fn.container[c].value
Input Shortcuts
---------------
Every element of the inputs list will be upgraded to an In instance if necessary.
- a Variable instance ``r`` will be upgraded like ``In(r)``
- a tuple ``(name, r)`` will be ``In(r, name=name)``
- a tuple ``(r, val)`` will be ``In(r, value=value, autoname=True)``
- a tuple ``((r,up), val)`` will be ``In(r, value=value, update=up, autoname=True)``
- a tuple ``(name, r, val)`` will be ``In(r, name=name, value=value)``
- a tuple ``(name, (r,up), val)`` will be ``In(r, name=name, value=val, update=up, autoname=True)``
Example:
.. code-block:: python
import theano
from theano import tensor as T
from theano.compile.io import In
x = T.scalar()
y = T.scalar('y')
z = T.scalar('z')
w = T.scalar('w')
fn = theano.function(inputs = [x, y, In(z, value=42), ((w, w+x), 0)],
outputs = x + y + z)
# the first two arguments are required and the last two are
# optional and initialized to 42 and 0, respectively.
# The last argument, w, is updated with w + x each time the
# function is called.
fn(1) # illegal because there are two required arguments
fn(1, 2) # legal, z is 42, w goes 0 -> 1 (because w <- w + x), returns array(45.0)
fn(1, y = 2) # legal, z is 42, w goes 1 -> 2, returns array(45.0)
fn(x = 1, y = 2) # illegal because x was not named
fn(1, 2, 3) # legal, z is 3, w goes 2 -> 3, returns array(6.0)
fn(1, z = 3, y = 2) # legal, z is 3, w goes 3 -> 4, returns array(6.0)
fn(1, 2, w = 400) # legal, z is 42 again, w goes 400 -> 401, returns array(45.0)
fn(1, 2) # legal, z is 42, w goes 401 -> 402, returns array(45.0)
In the example above, ``z`` has value 42 when no value is explicitly given.
This default value is potentially used at every function invocation, because
``z`` has no ``update`` or storage associated with it.
.. _function_outputs:
Outputs
=======
The ``outputs`` argument to function can be one of
- ``None``, or
- a Variable or ``Out`` instance, or
- a list of Variables or ``Out`` instances.
An ``Out`` instance is a structure that lets us attach options to individual output ``Variable`` instances,
similarly to how ``In`` lets us attach options to individual input ``Variable`` instances.
**Out(variable, borrow=False)** returns an ``Out`` instance:
* ``borrow``
If ``True``, a reference to function's internal storage
is OK. A value returned for this output might be clobbered by running
the function again, but the function might be faster.
Default: ``False``
If a single ``Variable`` or ``Out`` instance is given as argument, then the compiled function will return a single value.
If a list of ``Variable`` or ``Out`` instances is given as argument, then the compiled function will return a list of their values.
.. code-block:: python
x, y, s = T.matrices('xys')
# print a list of 2 ndarrays
fn1 = theano.function([x], [x+x, Out((x+x).T, borrow=True)])
print fn1(numpy.asarray([[1,0],[0,1]]))
# print a list of 1 ndarray
fn2 = theano.function([x], [x+x])
print fn2(numpy.asarray([[1,0],[0,1]]))
# print an ndarray
fn3 = theano.function([x], outputs=x+x)
print fn3(numpy.asarray([[1,0],[0,1]]))
.. _function_mode:
Mode
====
The ``mode`` parameter to ``theano.function`` controls how the
inputs-to-outputs graph is transformed into a callable object.
Theano defines the following modes by name:
- ``FAST_COMPILE``: Apply just a few optimizations, but use C op implementations where possible.
- ``FAST_RUN``: Apply all optimizations, and use C op implementations where possible.
- ``DEBUG_MODE``: Verify the correctness of all optimizations, and compare C and python
implementations. This mode can take much longer than the other modes,
but can identify many kinds of problems.
The default mode is typically 'FAST_RUN', but it can be controlled via the
environment variable 'THEANO_DEFAULT_MODE', which can in turn be overridden by
setting ``theano.compile.mode.default_mode`` directly, which can in turn be
overridden by passing the keyword argument to ``theano.function``.
For a finer level of control over which optimizations are applied, and whether
C or python implementations are used, read :api:`compile.mode.Mode`.
.. _compile_debugMode:
DebugMode ??
==============================================================
:mod:`compile` -- transforming expression graphs to functions
==============================================================
.. module:: compile
:platform: Unix, Windows
:synopsis: transforming expression graphs to functions
.. moduleauthor:: LISA
.. toctree::
:maxdepth: 1
function
module
debugmode
profilemode
doc/library/compile/profilemode.txt
浏览文件 @ 71021fec
.. _profilemode:
=========================================
ProfileMode
=========================================
================================================
:mod:`profilemode` -- profiling theano functions
================================================
Guide
=====
To profile a Theano graph, a special mode called ProfileMode, must be passed as
an argument when compiling your graph. Using ProfileMode is a three-step
......@@ -118,6 +121,34 @@ generates the following output:
"""
.. note::
***TODO***
The following text was recovered from a recent version of the source
file... hopefully things haven't gotten too out-of-sync!
The first show an Apply-wise summary, the second show an Op-wise summary, the third show an type-Op-wise summary.
The Apply-wise summary print the timing information for the worst
offending Apply nodes. This corresponds to individual Op applications
within your graph which take the longest to execute (so if you use dot
twice, you will see two entries there).
The Op-wise summary print the execution time of all Apply nodes
executing the same Op are grouped together and the total execution
time per Op is shown (so if you use dot twice, you will see only one
entry there corresponding to the sum of the time spent in each of
them). If two Op have different hash value, they will be separate.
The type-Op-wise summary group the result by type of op. So event if
two Op have different hash value, they will be merged.
Their is an hack with the Op-wise summary. Go see it if you want to know more.
The summary has two components to it. In the first section called the
Apply-wise summary, timing information is provided for the worst
offending Apply nodes. This corresponds to individual Op applications
......@@ -131,7 +162,43 @@ there corresponding to the sum of the time spent in each of them).
Note that the ProfileMode also shows which Ops were running a c
implementation.
Developers wishing to optimize the performance of their graph, should
focus on the worst offending Ops. If no C implementation exists for
this op, consider writing a C implementation yourself or use the
mailing list, to suggest that a C implementation be provided.
Developers wishing to optimize the performance of their graph should
focus on the worst offending Ops and Apply nodes -- either by optimizing an
implementation, providing a missing C implementation, or by writing a graph
optimization that eliminates the offending Op altogether.
You should strongly consider emailing one of our lists about your issue before
spending too much time on this.
Reference
=========
.. class:: ProfileMode(Mode)
.. method:: print_summary(n_apply_to_print=None, n_ops_to_print=None)
Print three summaries to stdout that show where cpu time is spent during theano function executions (for all functions using this object instance).
:param n_apply_to_print: the number of apply nodes to print.
The default 15, but can be configured via ``ProfileMode.n_ops_to_print`` in :ref:`THEANO_FLAGS`.
:param n_ops_to_print: the number of ops to print.
Default 20, or but can be configured via ``ProfileMode.n_apply_to_print`` in :ref:`THEANO_FLAGS`.
:returns: None
.. method:: print_diff_summary(self, other, n_apply_to_print=None, n_ops_to_print=None):
""" As print_summary, but print the difference on two different profile mode.
TODO: Also we don't print the Apply-wise summary as it don't work for now.
TODO: make comparaison with gpu code.
:param other: the other instance of ProfileMode that we want to be compared to.
:param n_apply_to_print: the number of apply nodes to print.
The default 15, but can be configured via ``ProfileMode.n_ops_to_print`` in :ref:`THEANO_FLAGS`.
:param n_ops_to_print: the number of ops to print.
Default 20, or but can be configured via ``ProfileMode.n_apply_to_print`` in :ref:`THEANO_FLAGS`.
:returns: None
doc/library/index.txt
浏览文件 @ 71021fec
......@@ -8,7 +8,7 @@ Library Documentation
This documentation covers Theano module-wise.
.. toctree::
:maxdepth: 2
:maxdepth: 1
tensor/index
gradient
......@@ -20,3 +20,9 @@ This documentation covers Theano module-wise.
scalar/index
gof/index
There are also some top-level imports that you might find more convenient:
* **theano.function** - :func:`compile.function.function`
* **theano.Param** - :class:`compile.function.Param`
* **theano.dot** - works like :func:`tensor.dot` for both sparse and dense matrix products
doc/sandbox/features.txt deleted 100644 → 0
浏览文件 @ 8d9e127e
.. _envfeaturelist:
====================
List of Env Features
====================
See :api:`gof.env.Env`.
WRITEME
.. _nodefinder:
NodeFinder
==========
See :api:`gof.toolbox.NodeFinder`.
WRITEME
doc/sandbox/numpy.txt deleted 100644 → 0
浏览文件 @ 8d9e127e
.. _numpy::
===============
NumPy refresher
===============
Give summary of type(x) vs x.type vs x.dtype
doc/sandbox/thinking_in_theano.txt deleted 100644 → 0
浏览文件 @ 8d9e127e
.. thinking_in_theano:
==================
Thinking in Theano
==================
Theano offers quite a bit of flexibility.
How should you write your algorithm to make the most of what Theano can do?
A few tips
----------
- Remember that your code builds a graph that theano compiles, and you cannot
literally put loops into that graph.
- Remember that Variables are symbolic of computations, not
storage. It does not make sense to *reassign* to a Variable.
Limitations
-----------
- Conditional control flow is possible but not efficient. In essence, both
sides of an if (see ``switch``) will be evaluated.
- Loops are not supported, but soon will be.
(A ``scan`` op is in the works, but not included yet.)
- Recursion is not supported within a graph.
A few examples
--------------
**DO WE WANT SOMETHING HERE?**
These are intended to illustrate good ways of formulating an algorithm for
Theano.
For complete, usable implementations of these algorithms see WRITEME.
- Short-time Fourier Transform
- Contrastive Divergence for Restricted Boltzmann Machine
- Kalman Filter
- Logistic Regression
- Training a neural network with sigmoidal hidden layer by backpropagation
- Learning an Echo-State Network
doc/tutorial/index.txt
浏览文件 @ 71021fec
......@@ -24,6 +24,7 @@ installation (see :ref:`install`).
adding
examples
loading_and_saving
remarks
debug_faq
tools
doc/tutorial/remarks.txt 0 → 100644
浏览文件 @ 71021fec
.. _tutorial_general_remarks:
=====================
Some general Remarks
=====================
Theano offers quite a bit of flexibility, but has some limitations too.
How should you write your algorithm to make the most of what Theano can do?
Limitations
-----------
- Conditional control flow is possible but currently not efficient. The current implementation will evaluate both sides of an ``if`` construct (see :func:`tensor.switch`).
- While- or for-Loops within an expression graph are not supported, but soon will be.
A ``scan`` op is in ``theano.sandbox``, but not quite ready for mainstream yet.
- Neither ``goto`` nor recursion is supported or planned within expression graphs.
A few tips
----------
* Remember that your code builds a graph that theano compiles, and you cannot
literally put loops into that graph.
* Remember that Variables are symbolic of computations, not
storage. It does not make sense to *reassign* to a Variable.
Markdown 格式
0% 或
您添加了 0 人 到此讨论。请谨慎行事。
请先完成此评论的编辑！
请 注册 或者 后发表评论