Skip to content

P
pytensor
提交 d9c7ae08 authored 作者: Pascal Lamblin's avatar Pascal Lamblin
浏览文件
操作

merge

上级 191a96e7 d6f86538
全部展开 隐藏空白字符变更
内嵌 并排
正在显示 包含 2058 行增加 和 1017 行删除
doc/advanced/ccodegen.txt deleted 100644 → 0
浏览文件 @ 191a96e7
.. _ccodegen:
=================
C Code Generation
=================
WRITEME
doc/advanced/features.txt deleted 100644 → 0
浏览文件 @ 191a96e7
.. _envfeaturelist:
====================
List of Env Features
====================
See :api:`gof.env.Env`.
WRITEME
.. _nodefinder:
NodeFinder
==========
See :api:`gof.toolbox.NodeFinder`.
WRITEME
doc/advanced/module.txt deleted 100644 → 0
浏览文件 @ 191a96e7
.. _moduleinterface:
================
Module Interface
================
A Theano Module is like Theano's version of a file.
When you instantiate a ``Module()``, you are creating a blank file.
Into this file you can put both symbolic and non-symbolic objects.
Non-symbolic objects are like constants (technically literals) in the file.
Symbolic objects are like variables and functions.
The functions in a Module are called Methods.
The variables in a Module (and submodules) are global.
Module Methods have access to all these global variables.
To use a Module, you need to compile it.
This is done by calling `Module.make()`.
The result of compiling a Module is a ModuleInstance, this is the compiled
version of your Theano file.
In the ModuleInstance, your symbolic variables have become containers (containing None),
and your Methods have become callable functions.
You should initialize the symbolic variables by calling
``ModuleInstance.initialize()`` (although make() will call it for you,
on the top-level ModuleInstance.)
You can compile a Module several times, to create multiple ModuleInstances.
Each of these will have its own copy of all program literals.
Module Graph
------------
Components can be grouped into a directed graph.
When we call `make`, this graph is replicated with ComponentInstances instead of
Components. Wheras Components are represent symbolic things (ie. Variables), ComponentInstances represent non-symbolic ones (ie. sparse matrices, ndarrays, callable functions).
.. index::
single: Component
single: component; Component
.. _component:
---------
Component
---------
All of the elements of what is called the "module system" or "modules" are
components.
A component subclass is represents a symbolic theano thing, and implements the
``build`` function.
The ``build`` function is responsible for converting the symbolic thing into a
non-symbolic thing.
Compiling with make
-------------------
Conversion from a Component graph to a ComponentInstance graph is performed by `Component.make`.
This method traverses the Component graph in multiple passes.
In the first pass (the allocate pass), it creates storage for all Variables that are contained in the graph (see
`Component.allocate`). These are the module variables.
In the second pass (the build pass), it creates functions that (in general) operate on these module variables.
This pass also serves to construct all ComponentInstance-derived instances as well, such as
`ModuleInstance`s. The objects that are returned from this second pass are the return value of
`Component.make`.
In the third pass (the initialize pass), is optional and not necessarily recursive through the
graph.
The purpose of the third pass is to call the initialize method of the ComponentInstances built
during the second pass.
During this pass the ComponentInstance graph is complete. It is a good time to fill storage
allocated in phase 1 with sensible values.
.. index::
single: External
single: component; External
.. _external:
--------
External
--------
WRITEME
.. index::
single: Member
single: component; Member
.. _member:
------
Member
------
WRITEME
.. index::
single: Method
single: component; Method
.. _method:
------
Method
------
WRITEME
.. index::
single: Module
single: component; Module
.. _module:
------
Module
------
A Module instance can contain objects as attributes.
This makes it something like a class in the way that Method is
analogous to a function.
A Module is meant to contain Components.
Attributes which are not Components themselves must at least be transform-able
into Components by :api:`compile.module.wrap`. If a Module contains something
that is not convertible into a Component, then it is not possible to compile
that Module with ``make``.
Old Text
--------
In the Module system, the analog of the file is the `Module`, the analog of the function is the
`Method`, and the analog of the variable is the `Member`. Module, Member, and Method all work
at the symbolic level. Once a graph of Modules, Members, and Methods is ready for use, it must
be compiled with a call to `make` which will return an isomorphic structure in which Modules
have become `ModuleInstances`, Members have become `Container`s, and Methods have become
`Function`s.
This structure contains numbers and functions, and is ready for computation.
doc/advanced_tutorial/index.txt deleted 100644 → 0
浏览文件 @ 191a96e7
.. _advtutorial:
=================
Advanced Tutorial
=================
Before tackling this tutorial, it is highly recommended to read the
:ref:`basictutorial`.
The advanced tutorial is meant to give the reader a greater
understanding of the building blocks of Theano. Through this tutorial
we are going to define one :ref:`type`, ``double``, and basic
arithmetic :ref:`operations <op>` on that Type. We will first define
them using a Python implementation and then we will add a C
implementation.
This tutorial should be of most use to users who want to extend Theano
with custom types and operations related to these types.
It is a good idea to read this tutorial as well since it provides
grounding for fundamental Theano concepts.
.. toctree::
theano_vs_c
graphstructures
type
op
inplace
ctype
cop
optimization
tips
doc/basic_tutorial/index.txt deleted 100644 → 0
浏览文件 @ 191a96e7
.. _basictutorial:
==============
Basic Tutorial
==============
Before doing anything in this tutorial, make sure that Theano is
installed on your system (see :ref:`install`).
Done? Alright!
Let's start an interactive session and import the package you just
installed:
>>> from theano import *
Many of symbols you will need to use are in the ``tensor`` subpackage
of theano. Let's import that subpackage under a handy name. I like
``T``.
>>> import theano.tensor as T
Now we're ready for the tour:
.. toctree::
adding
examples
tools
doc/conf.py
浏览文件 @ d9c7ae08
...@@ -39,7 +39,7 @@ templates_path = ['.templates'] ...@@ -39,7 +39,7 @@ templates_path = ['.templates']
source_suffix = '.txt' source_suffix = '.txt'
# The master toctree document. # The master toctree document.
master_doc = 'contents' master_doc = 'index'
# General substitutions. # General substitutions.
project = 'Theano' project = 'Theano'
...@@ -64,7 +64,7 @@ today_fmt = '%B %d, %Y' ...@@ -64,7 +64,7 @@ today_fmt = '%B %d, %Y'
# List of directories, relative to source directories, that shouldn't be searched # List of directories, relative to source directories, that shouldn't be searched
# for source files. # for source files.
exclude_dirs = ['images', 'scripts', 'trac'] exclude_dirs = ['images', 'scripts', 'sandbox']
# The reST default role (used for this markup: `text`) to use for all documents. # The reST default role (used for this markup: `text`) to use for all documents.
#default_role = None #default_role = None
...@@ -90,7 +90,8 @@ pygments_style = 'sphinx' ...@@ -90,7 +90,8 @@ pygments_style = 'sphinx'
# The style sheet to use for HTML and HTML Help pages. A file of that name # The style sheet to use for HTML and HTML Help pages. A file of that name
# must exist either in Sphinx' static/ path, or in one of the custom paths # must exist either in Sphinx' static/ path, or in one of the custom paths
# given in html_static_path. # given in html_static_path.
html_style = 'default.css' #html_style = 'default.css'
html_theme = 'sphinxdoc'
# The name for this set of Sphinx documents. If None, it defaults to # The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation". # "<project> v<release> documentation".
...@@ -101,7 +102,8 @@ html_style = 'default.css' ...@@ -101,7 +102,8 @@ html_style = 'default.css'
# The name of an image file (within the static path) to place at the top of # The name of an image file (within the static path) to place at the top of
# the sidebar. # the sidebar.
html_logo = 'images/theano_logo-200x67.png' #html_logo = 'images/theano_logo-200x67.png'
html_logo = 'images/theano_logo_allblue_200x46.png'
# The name of an image file (within the static path) to use as favicon of the # The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
......
doc/contents.txt deleted 100644 → 0
浏览文件 @ 191a96e7
.. _contents:
========
Contents
========
.. toctree::
:maxdepth: 2
introduction
LICENSE
install
basic_tutorial/index
advanced_tutorial/index
topics/index
indexes/index
glossary
links
internal/index
NEWS
.. examples/index
.. advanced/index
doc/topics/debug_faq.txt → doc/extending/debug_faq.txt
浏览文件 @ d9c7ae08
...@@ -9,7 +9,7 @@ There are many kinds of bugs that might come up in a computer program. ...@@ -9,7 +9,7 @@ There are many kinds of bugs that might come up in a computer program.
This page is structured as an FAQ. It should provide recipes to tackle common This page is structured as an FAQ. It should provide recipes to tackle common
problems, and introduce some of the tools that we use to find problems in our problems, and introduce some of the tools that we use to find problems in our
Theano code, and even (it happens) in Theano's internals, such as Theano code, and even (it happens) in Theano's internals, such as
:ref:`debugmode`. :ref:`using_debugmode`.
...@@ -49,7 +49,7 @@ I wrote a new Op/Type, and weird stuff is happening... ...@@ -49,7 +49,7 @@ I wrote a new Op/Type, and weird stuff is happening...
First, check the :ref:`op_contract` and the :ref:`type_contract` First, check the :ref:`op_contract` and the :ref:`type_contract`
and make sure you're following the rules. and make sure you're following the rules.
Then try running your program in :ref:`debugmode`. DebugMode might catch Then try running your program in :ref:`using_debugmode`. DebugMode might catch
something that you're not seeing. something that you're not seeing.
...@@ -65,8 +65,8 @@ I wrote a new optimization, and it changed my results even though I'm pretty sur ...@@ -65,8 +65,8 @@ I wrote a new optimization, and it changed my results even though I'm pretty sur
------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------
First, check the :ref:`op_contract` and make sure you're following the rules. First, check the :ref:`op_contract` and make sure you're following the rules.
Then try running your program in :ref:`debugmode`. DebugMode might catch Then try running your program in :ref:`using_debugmode`. DebugMode might
something that you're not seeing. catch something that you're not seeing.
The function I compiled is too slow, what's up? The function I compiled is too slow, what's up?
...@@ -77,7 +77,7 @@ First, make sure you're running in FAST_RUN mode, by passing ...@@ -77,7 +77,7 @@ First, make sure you're running in FAST_RUN mode, by passing
operations have excruciatingly slow Python implementations and that operations have excruciatingly slow Python implementations and that
can negatively effect the performance of FAST_COMPILE. can negatively effect the performance of FAST_COMPILE.
Second, try the theano :ref:`profilemode`. This will tell you which Second, try the theano :ref:`using_profilemode`. This will tell you which
Apply nodes, and which Ops are eating up your CPU cycles. Apply nodes, and which Ops are eating up your CPU cycles.
...@@ -110,9 +110,7 @@ put logic inside of the print_eval function that would, for example, only ...@@ -110,9 +110,7 @@ put logic inside of the print_eval function that would, for example, only
print something out if a certain kind of Op was used, at a certain program print something out if a certain kind of Op was used, at a certain program
position, or if a particular value shows up in one of the inputs or outputs. position, or if a particular value shows up in one of the inputs or outputs.
This can be a really powerful debugging tool. Read about more things you can .. TODO: documentation for link.WrapLinkerMany
do with :api:`link.WrapLinkerMany`.
Note well the call to ``fn`` inside the call to ``print_eval``; without it, This can be a really powerful debugging tool. Note the call to ``fn`` inside the call to ``print_eval``; without it, the graph wouldn't get computed at all!
the graph wouldn't get computed at all!
doc/extending/index.txt 0 → 100644
浏览文件 @ d9c7ae08
.. _extending:
================
Extending Theano
================
This documentation is for users who want to extend Theano with new Types, new
Operations (Ops), and new graph optimizations.
Along the way, it also introduces many aspects of how Theano works, so it is
also good for you if you are interested in getting more under the hood with
Theano itself.
Before tackling this tutorial, it is highly recommended to read the
:ref:`basictutorial`.
The first few pages will walk you through the definition of a new :ref:`type`,
``double``, and a basic arithmetic :ref:`operations <op>` on that Type. We
will start by defining them using a Python implementation and then we will add
a C implementation.
.. toctree::
pipeline
theano_vs_c
graphstructures
type
op
inplace
ctype
cop
optimization
tips
unittest
doc/images/theano_logo.svg deleted 100644 → 0
浏览文件 @ 191a96e7
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- Created with Inkscape (http://www.inkscape.org/) -->
<svg
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:cc="http://web.resource.org/cc/"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:svg="http://www.w3.org/2000/svg"
xmlns="http://www.w3.org/2000/svg"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
width="345.86591"
height="115.13724"
id="svg2"
sodipodi:version="0.32"
inkscape:version="0.45.1"
sodipodi:docbase="/home/olivier/hg/theano"
sodipodi:docname="theano_logo.svg"
inkscape:output_extension="org.inkscape.output.svg.inkscape"
version="1.0"
inkscape:export-filename="/home/olivier/hg/theano/theano_logo_big.png"
inkscape:export-xdpi="273.58655"
inkscape:export-ydpi="273.58655">
<defs
id="defs4" />
<sodipodi:namedview
id="base"
pagecolor="#ffffff"
bordercolor="#666666"
borderopacity="1.0"
gridtolerance="10000"
guidetolerance="10"
objecttolerance="10"
inkscape:pageopacity="0.0"
inkscape:pageshadow="2"
inkscape:zoom="1.979899"
inkscape:cx="248.50886"
inkscape:cy="97.530852"
inkscape:document-units="px"
inkscape:current-layer="layer1"
inkscape:window-width="1680"
inkscape:window-height="1030"
inkscape:window-x="0"
inkscape:window-y="0"
showguides="true"
inkscape:guide-bbox="true" />
<metadata
id="metadata7">
<rdf:RDF>
<cc:Work
rdf:about="">
<dc:format>image/svg+xml</dc:format>
<dc:type
rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
</cc:Work>
</rdf:RDF>
</metadata>
<g
inkscape:label="Layer 1"
inkscape:groupmode="layer"
id="layer1"
transform="translate(-219.06115,-88.23416)">
<path
id="path5572"
d="M 245.99986,202.38198 C 235.76172,199.76305 230.3317,195.18454 224.56469,184.30815 C 220.37775,176.41173 219.14676,170.92373 219.06742,159.80009 C 219.02952,154.48681 219.14363,153.33451 219.96737,150.71192 L 220.91072,147.70853 L 222.03485,150.91475 C 223.32792,154.60284 224.5932,157.2101 225.42491,157.90035 C 225.91931,158.31066 226.04839,157.45384 226.31509,151.99127 C 226.48664,148.47733 226.74177,144.59829 226.88203,143.3712 C 227.13637,141.14611 227.14306,141.13711 229.37079,140.02194 C 233.6165,137.89661 241.51289,137.62549 255.7355,139.11671 C 262.25557,139.80033 276.27711,139.65881 278.302,138.88894 C 280.15154,138.18575 280.55926,136.52884 280.07117,131.69921 C 279.49474,125.99537 279.0548,124.08561 277.22091,119.32634 C 272.4649,106.98367 264.75123,100.69911 254.31572,100.6648 C 244.91721,100.6339 237.20308,106.18784 232.64521,116.26692 C 228.63554,125.13371 226.84755,134.63837 225.79128,152.70119 L 225.49476,157.77183 L 224.6018,156.08339 C 220.32764,148.00176 218.55416,134.3005 220.39244,123.56361 C 221.81624,115.24763 224.72248,108.02444 229.43922,101.07873 C 233.51167,95.08179 239.33503,91.22689 247.37024,89.20891 C 252.54529,87.90924 256.08615,87.90924 261.2612,89.20891 C 269.29641,91.22689 275.11977,95.08179 279.19222,101.07873 C 283.85913,107.95107 286.81123,115.24029 288.1872,123.28884 C 289.11587,128.72102 289.26704,136.96138 288.48572,139.5625 C 287.80095,141.84221 282.75423,149.25874 282.58446,148.23482 C 282.51467,147.81394 282.66002,147.09129 282.90745,146.62895 C 283.60255,145.33016 282.97412,144.79606 281.91813,145.78812 C 281.09814,146.55845 280.95497,146.57992 280.4772,146.00425 C 279.46931,144.78981 279.09827,146.0508 280.02317,147.54731 C 281.09294,149.27824 281.11194,149.86163 280.09855,149.86163 C 279.6655,149.86163 279.2114,150.02307 279.08945,150.2204 C 278.12451,151.78171 263.15706,152.14918 251.27333,150.90331 C 242.48708,149.98217 235.49959,150.17874 233.86598,151.393 C 232.52086,152.39282 230.73981,155.92513 230.13832,158.78596 C 229.56685,161.50406 229.89814,169.75383 230.71167,173.06316 C 231.53272,176.40313 234.44347,181.26714 237.48117,184.37536 C 245.97324,193.06457 259.99042,193.16426 268.52866,184.59618 C 272.82158,180.28826 276.28725,173.36771 275.26986,171.13477 C 275.01206,170.56897 274.80113,169.46845 274.80113,168.68918 C 274.80113,167.27252 276.03299,164.34881 276.84003,163.85004 C 277.97809,163.14668 279.2633,160.34344 279.2633,158.56453 C 279.2633,156.50464 279.81574,155.1351 280.64665,155.1351 C 281.94053,155.1351 281.78744,149.84815 280.42796,147.58266 C 279.38328,145.84176 279.47773,145.48404 280.68309,146.61641 C 281.46075,147.34699 281.69721,147.42235 281.69721,146.93962 C 281.69721,146.59338 282.00521,146.05957 282.38164,145.75336 C 282.9932,145.2559 283.02559,145.28301 282.68588,146.00793 C 282.47678,146.45415 282.35906,148.62448 282.4243,150.8309 C 282.5319,154.47038 282.63024,154.91126 283.48431,155.58307 C 284.25335,156.18799 284.4647,156.82757 284.6386,159.07597 C 284.78839,161.01273 285.24037,162.64716 286.16384,164.59151 C 287.23183,166.84012 287.43789,167.69463 287.27043,169.18035 C 287.15459,170.2081 286.70684,171.3939 286.24597,171.89349 C 285.2295,172.99536 281.11174,180.12521 280.69642,181.50246 C 279.94371,183.99856 277.41503,189.23736 275.76462,191.71994 C 273.21329,195.55768 270.45935,197.86457 265.70147,200.14953 C 258.59319,203.56326 253.06615,204.18955 245.99986,202.38198 z "
style="fill:#000000;fill-opacity:1" />
<text
xml:space="preserve"
style="font-size:15.53327274px;font-style:normal;font-weight:normal;fill:#7799ee;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;font-family:Bitstream Vera Sans"
x="285.01266"
y="186.09427"
id="text5574"
transform="scale(1.0402212,0.961334)"><tspan
sodipodi:role="line"
id="tspan5576"
x="285.01266"
y="186.09427"
style="font-size:93.19962311px;font-weight:normal;fill:#7799ee;fill-opacity:1;font-family:MgOpen Modata"
dx="0 -4.2857141 -6.4285722 -5 -5.7142901 -6.0714293"
dy="0 0 -1.3672954 0.35714287 1.0101526 -1.0101526">Theano</tspan></text>
</g>
</svg>
doc/images/theano_logo_allblue.svg 0 → 100644
浏览文件 @ d9c7ae08
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- Created with Inkscape (http://www.inkscape.org/) -->
<svg
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:cc="http://creativecommons.org/ns#"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:svg="http://www.w3.org/2000/svg"
xmlns="http://www.w3.org/2000/svg"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
width="345.86591"
height="115.13724"
id="svg2"
sodipodi:version="0.32"
inkscape:version="0.46"
sodipodi:docbase="/home/olivier/hg/theano"
sodipodi:docname="theano_logo_allblue.svg"
inkscape:output_extension="org.inkscape.output.svg.inkscape"
version="1.0"
inkscape:export-filename="theano_logo_allblue_200x46.png"
inkscape:export-xdpi="53.889999"
inkscape:export-ydpi="53.889999">
<defs
id="defs4">
<inkscape:perspective
sodipodi:type="inkscape:persp3d"
inkscape:vp_x="0 : 57.568619 : 1"
inkscape:vp_y="0 : 1000 : 0"
inkscape:vp_z="345.86591 : 57.568619 : 1"
inkscape:persp3d-origin="172.93295 : 38.379079 : 1"
id="perspective10" />
</defs>
<sodipodi:namedview
id="base"
pagecolor="#ffffff"
bordercolor="#666666"
borderopacity="1.0"
gridtolerance="10000"
guidetolerance="10"
objecttolerance="10"
inkscape:pageopacity="0.0"
inkscape:pageshadow="2"
inkscape:zoom="1.4"
inkscape:cx="187.28443"
inkscape:cy="-18.63669"
inkscape:document-units="px"
inkscape:current-layer="layer1"
inkscape:window-width="1280"
inkscape:window-height="999"
inkscape:window-x="0"
inkscape:window-y="6"
showguides="true"
inkscape:guide-bbox="true"
showgrid="false" />
<metadata
id="metadata7">
<rdf:RDF>
<cc:Work
rdf:about="">
<dc:format>image/svg+xml</dc:format>
<dc:type
rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
</cc:Work>
</rdf:RDF>
</metadata>
<g
inkscape:label="Layer 1"
inkscape:groupmode="layer"
id="layer1"
transform="translate(-219.06115,-88.23416)">
<path
id="path5572"
d="M 361.71053,188.16928 C 354.76773,186.82023 350.70665,184.96687 346.79586,179.36429 C 343.95656,175.29673 343.50062,171.96473 343.44682,166.23478 C 343.42111,163.49783 343.49849,162.90428 344.0571,161.55334 L 344.69681,160.00626 L 345.45913,161.65784 C 346.336,163.55761 347.19403,164.90066 347.75802,165.25619 C 348.09328,165.46756 348.18083,165.02619 348.36169,162.21236 C 348.47801,160.40227 348.65102,158.40413 348.74614,157.77203 C 348.91863,156.62587 348.92314,156.62125 350.43384,156.04681 C 353.31299,154.95199 358.66778,154.81233 368.31255,155.58049 C 372.73402,155.93264 382.24935,156.1272 383.25884,155.25172 C 384.30569,154.34386 384.43282,153.93027 384.10183,151.44246 C 383.71094,148.50433 384.12609,147.83778 382.88247,145.3862 C 379.65727,139.02834 374.30749,136.3197 367.23085,136.30203 C 360.85744,136.28611 355.74516,138.6184 352.65434,143.81025 C 349.93524,148.37768 348.72275,153.27364 348.00646,162.57805 L 347.80539,165.18998 L 347.19983,164.32026 C 344.3014,160.1573 342.50418,152.88816 343.75077,147.35745 C 344.71627,143.07377 346.68707,139.35299 349.88566,135.77517 C 352.64732,132.68608 357.19089,130.91181 362.63982,129.87232 C 366.14917,129.20284 368.55034,129.20284 372.05972,129.87232 C 377.50862,130.91181 381.45763,132.89753 384.21929,135.98663 C 387.38405,139.52666 389.38596,143.28142 390.31905,147.42734 C 390.9488,150.22555 391.05131,154.47027 390.52149,155.81012 C 390.05713,156.98445 386.63479,160.80479 386.51965,160.27735 C 386.47234,160.06056 386.57091,159.6883 386.73869,159.45016 C 387.21007,158.78114 386.7839,158.506 386.0678,159.01703 C 385.51175,159.41383 385.41465,159.42491 385.09066,159.12836 C 384.40717,158.50279 384.15557,159.15234 384.78277,159.92321 C 385.50823,160.81483 385.52111,161.11534 384.8339,161.11534 C 384.54022,161.11534 384.23228,161.1985 384.14958,161.30014 C 383.49524,162.1044 373.34534,162.29368 365.28662,161.65193 C 359.32839,161.17745 354.58998,161.2787 353.48216,161.90416 C 352.57,162.41919 351.36222,164.23874 350.95433,165.71238 C 350.56679,167.11253 350.79145,171.3621 351.34313,173.06677 C 351.89991,174.78722 353.87377,177.29276 355.93375,178.89384 C 361.69246,183.36976 371.19795,183.42111 376.98801,179.00757 C 379.89915,176.78852 382.24934,173.22363 381.55941,172.07343 C 381.38459,171.78199 381.24154,171.2151 381.24154,170.81367 C 381.24154,170.08392 382.07691,168.57788 382.62419,168.32095 C 383.39593,167.95867 384.26748,166.51466 384.26748,165.59831 C 384.26748,164.53724 384.64213,163.83178 385.20558,163.83178 C 386.08301,163.83178 385.97918,161.1084 385.05727,159.94142 C 384.34884,159.04465 384.41288,158.86039 385.23029,159.44369 C 385.75764,159.82002 385.91798,159.85884 385.91798,159.61018 C 385.91798,159.43183 386.12685,159.15685 386.38212,158.99912 C 386.79684,158.74289 386.81881,158.75684 386.58845,159.13026 C 386.44664,159.3601 386.36682,160.4781 386.41105,161.61465 C 386.48402,163.48937 386.5507,163.71649 387.12987,164.06252 C 387.65138,164.37413 387.79471,164.70362 387.91265,165.86178 C 388.01423,166.85943 388.3207,167.70135 388.94694,168.70291 C 389.67118,169.86121 389.81092,170.30137 389.69736,171.06666 C 389.61881,171.59609 389.31517,172.20689 389.00266,172.46425 C 388.31336,173.03183 385.52096,176.70451 385.23933,177.41397 C 384.72888,178.69974 383.01411,181.39832 381.89492,182.67712 C 380.16478,184.65398 378.29724,185.84229 375.07078,187.01932 C 370.25045,188.77779 366.50239,189.10039 361.71053,188.16928 z"
style="fill:#11557c;fill-opacity:1;stroke:#ffffff;stroke-width:0.73518676000000005;stroke-opacity:1"
sodipodi:nodetypes="csscccsssssssssscccsssssssssssssssssssssssssssssssssssssssc" />
<text
xml:space="preserve"
style="font-size:96px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:100%;writing-mode:lr-tb;text-anchor:start;fill:#11557c;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono"
x="221.28195"
y="185.3815"
id="text2382"
sodipodi:linespacing="100%"><tspan
sodipodi:role="line"
id="tspan2386"
x="221.28195"
y="185.3815">th ano</tspan></text>
</g>
</svg>
doc/index.txt
浏览文件 @ d9c7ae08
=========== Welcome
Quick Start =======
===========
Theano is a Python library that allows you to define, optimize, and Theano is a Python library that allows you to define, optimize, and
efficiently evaluate mathematical expressions involving multi-dimensional evaluate mathematical expressions involving multi-dimensional
arrays. arrays efficiently. Theano features:
The latest release is version `0.1 * **tight integration with numpy**
<http://pylearn.org/theano/downloads/Theano-0.1.tar.gz>`_. * **near-transparent use of a GPU** to accelerate for intense calculations [JAN 2010].
You can download the latest `PDF documentation * **symbolic differentiation**
<http://pylearn.org/theano/theano.pdf>`_, rather than reading it online. * **speed and stability optimizations**: write ``log(1+exp(x))`` and get the right answer.
You can go to the :ref:`Table of Contents <contents>`. * **dynamic C code generation** for faster expression evaluation
News Download
---- ========
* 2009-04-01: Theano 0.1 released. See the :ref:`release notes <NEWS>`. In April 2009 we made declared the creation of a `0.1 release <http://pylearn.org/theano/downloads/Theano-0.1.tar.gz>`_.
Development has continued non-stop since then.
Choose your own adventure... The current version is available via::
----------------------------
hg clone http://hg.assembla.com/theano Theano
* You have no idea what Theano is and you read the :ref:`introduction
<introduction>`. The theano subfolder should be on your ``$PYTHONPATH``. For more information about
installation and configuration, see :ref:`installing Theano <install>`.
* Convinced by the Theano staff's rhetoric that it is worthy of your
time (you can hear them into the distance congratulating themselves Documentation
on the success of their dishonest ploy - a lone puppy's =============
heart-wrenching cry echoes behind you but is cut short by an
outworldly screech and what might be the dripping sound of a bloody You can download the latest `PDF documentation <http://pylearn.org/theano/theano.pdf>`_, rather than reading it online.
blade - you don't care, though, because Theano is wicked cool), you
:ref:`install Theano <install>` and :ref:`learn the basics * If you have no idea what Theano is read the :ref:`introduction <introduction>`.
<basictutorial>`. * :ref:`learn the basics <tutorial>`
* :ref:`library reference <libdoc>`
* Blinded by your newfound knowledge, you want to play God and * :ref:`extending Theano <extending>` with new Types, and Ops
:ref:`learn how to extend Theano <advtutorial>`. The Gods are * :ref:`internal docs <internal>`
pleased that they get to devour your soul for this heresy, but you
ascertain that it is worth it. .. toctree::
:maxdepth: 1
* Decidedly, your thirst for Theano-related information is :hidden:
unquenchable! So there you are, reading about Theano's compilation
:ref:`pipeline <pipeline>`. Little do you know that article could NEWS
not have been made without the tears of a thousand innocent kittens. introduction
install
* You descend into madness as you read detailed (or as-of-yet not tutorial/index
written, but our plans for world domination as well as the hourly library/index
sacrifices that our God requires leave us little time for writing) extending/index
articles about :ref:`op`, :ref:`type`, :ref:`function`, indexes/index
:ref:`module`, :ref:`compilation`, :ref:`optimization`, :ref:`env` glossary
and as you consult the list of :ref:`envfeaturelist`. Sometimes you links
check the :ref:`glossary` just in case you missed an important internal/index
concept. examples/index
proposals/index
* Requiring help in your dark endeavors, you register and post to the LICENSE
`theano-users`_ mailing list and curious to know who is behind this
witchery and how their wicked plans come into fruition, you read the
`theano-dev`_ mailing list. Community
=========
* Mistaking your feverish sweat for genuine excitation, you
investigate `Theano's Trac <trac/>`__ for tickets_ that your feeble * Register and post to `theano-users`_ if you want to talk to all Theano users.
mind might comprehend and your consumed brain is yearning to solve.
* Register and post to `theano-dev`_ if you want to talk to the developers.
* Bored a Friday evening, you browse `Theano's API <api/>`__. May
the Great Ones help you with that one. * We try to stay organized with `Theano's Trac <trac/>`__
* Come visit us in Montreal! Most of the developers are students in the LISA_ group at the `University of Montreal`_.
.. note::
May you be assured that this page was made solely under the
unsobering influence of boredom and that all of the unspeakable
things mentioned therein are merely artifacts of the author's
maddened delusions.
.. _theano-dev: http://groups.google.com/group/theano-dev .. _theano-dev: http://groups.google.com/group/theano-dev
.. _theano-users: http://groups.google.com/group/theano-users .. _theano-users: http://groups.google.com/group/theano-users
.. _tickets: http://pylearn.org/theano/trac/query?status=accepted&status=assigned&status=new&status=reopened&group=milestone&max=200&col=id&col=summary&col=status&col=owner&col=type&col=priority&col=component&col=time&report=9&order=priority .. _tickets: http://pylearn.org/theano/trac/query?status=accepted&status=assigned&status=new&status=reopened&group=milestone&max=200&col=id&col=summary&col=status&col=owner&col=type&col=priority&col=component&col=time&report=9&order=priority
.. _LISA: http://www.iro.umontreal.ca/~lisa
.. _University of Montreal: http://www.umontreal.ca
doc/indexes/index.txt deleted 100644 → 0
浏览文件 @ 191a96e7
=======
Indexes
=======
The following indexes are generated automatically and cover most
:ref:`Ops <op>`, :ref:`Types <type>` and :ref:`Optimizers <optimization>`
in Theano.
.. toctree::
:maxdepth: 1
oplist
typelist
doc/install.txt
浏览文件 @ d9c7ae08
...@@ -16,7 +16,7 @@ Requirements ...@@ -16,7 +16,7 @@ Requirements
In order to use Theano, the following libraries and software will need In order to use Theano, the following libraries and software will need
to be installed: to be installed:
Linux or OS-X operating system Linux, OS-X or Windows operating system
We develop mainly on 64-bit Linux machines. 32-bit architectures are We develop mainly on 64-bit Linux machines. 32-bit architectures are
not well-tested. not well-tested.
...@@ -66,6 +66,7 @@ want. Unpack the release, and type: ...@@ -66,6 +66,7 @@ want. Unpack the release, and type:
python setup.py test python setup.py test
python setup.py install python setup.py install
.. _install_bleeding_edge:
Bleeding Edge Bleeding Edge
-------------- --------------
...@@ -73,7 +74,6 @@ Bleeding Edge ...@@ -73,7 +74,6 @@ Bleeding Edge
Feeling lucky and want to run bleeding-edge code? Feeling lucky and want to run bleeding-edge code?
Then check out the :ref:`dev_start_guide` guide. Then check out the :ref:`dev_start_guide` guide.
I bet you also run with scissors.
Configuring the environment Configuring the environment
--------------------------- ---------------------------
...@@ -109,7 +109,7 @@ automatic code generation, but that way is much, much slower. ...@@ -109,7 +109,7 @@ automatic code generation, but that way is much, much slower.
- ``THEANO_DEFAULT_MODE``: - ``THEANO_DEFAULT_MODE``:
String value specifying the default mode to use when compiling Theano String value specifying the default mode to use when compiling Theano
graphs. This can be one of the strings defined in graphs. This can be one of the strings defined in
:api:`theano.compile.mode`. :ref:`using_modes`.
Possible values so far are: Possible values so far are:
- ``'FAST_COMPILE'`` - ``'FAST_COMPILE'``
...@@ -145,10 +145,102 @@ This advice has not been tested recently, so please inform us of your results. ...@@ -145,10 +145,102 @@ This advice has not been tested recently, so please inform us of your results.
Windows Windows
------- -------
As of now, the Windows platform is not supported. In fact, it has Running Theano under Windows is currently achieved by using the `MinGW
never even been tested, so feel free to explore this uncharted <http://www.mingw.org>`__ port of the GCC compiler.
territory and inform us of your progress! It could probably also run with `Cygwin <http://www.cygwin.com/>`__,
but this has not been tested yet.
- From `the MinGW files <http://sourceforge.net/projects/mingw/files/>`__,
download the latest version of the ``Automated MinGW Installer`` and install
it (keeping default options).
- From `the MinGW files <http://sourceforge.net/projects/mingw/files/>`__,
download the latest ``MSYS Base System`` executable file and run it
(note that the latest version of MSYS Base System may not contain an
executable file, in which case it is easier to just use an
older version, e.g. MSYS-1.0.11.exe).
This will install MSYS (you can keep the default install options).
It will also run a post-install script where it will ask you about the
location of MinGW (typically ``c:/MinGW``).
- From `the MinGW files <http://sourceforge.net/projects/mingw/files/>`__,
download the current version of ``GCC Version 4`` (full package with
binaries, e.g.
gcc-full-4.4.0-mingw32-bin-2.tar.lzma). Unpack it (you may use
`7-Zip <http://www.7-zip.org>`__ to unpack files with the
.lzma extension), copying the content into the root directory
of your MinGW installation (if you obtain a .tar file, make
sure you expand it as well, either with `7-Zip <http://www.7-zip.org>`__
or through the ``tar`` command on the MSYS command line).
- If you are familiar with vi, you may find useful to download and install
``MSYS vim`` (this is done in a similar way to GCC 4).
This is strictly optional and mostly helpful to edit configuration files
from within MSYS.
- Run MSYS (Start/Programs/MinGW/MSYS/MSYS) and check the installation
by verifying that the proper version of GCC is found:
.. code-block:: bash
gcc --version
You may also decide to change the location of your home directory by
adding a line at the beginning of msys.bat, that would look like
``set HOME=C:\My\Home\For\MinGW`` (you can also set a global ``HOME``
environment variable within Windows, but this could affect more programs).
- If you do not have them already, install the latest versions of
`Python 2.x <http://www.python.org/download/windows>`__ and
corresponding `Numpy <http://sourceforge.net/projects/numpy/files/>`__
then `SciPy <http://sourceforge.net/projects/scipy/files/>`__
packages (simply use the executable installers).
- Ensure that the Python installation directory and its ``Scripts`` sub-directory
are in your system path. This may be done by
modifying the global ``PATH`` Windows environment variables, or by creating a ``.profile`` file in
your MinGW home, containing the line
``export PATH=$PATH:/c/Python26:/c/Python26/Scripts``
(for Python 2.6).
- Install a ``BLAS`` library. Note that although the following instructions
will give you a generic way to build your own library, there may exist
better (more optimized) versions of BLAS available for your system, but
these have not been tested for Windows at this time.
To build BLAS, download the latest version of `LAPACK <http://www.netlib.org/lapack/>`__
(typically lapack.tgz), then issue the following commands in MSYS
(for LAPACK 3.2.1):
.. code-block:: bash
tar zxvf lapack.tgz
cd lapack-3.2.1
gfortran -shared -O3 -o libblas.dll BLAS/SRC/*.f
mv libblas.dll /mingw/lib
- Install `Mercurial <http://mercurial.selenic.com/downloads/>`__
(you can use the regular Windows release, you do not need TortoiseHg).
- In order to run Theano's test-suite, you will need `nose
<http://somethingaboutorange.com/mrl/projects/nose>`__.
After unpacking its source code, you can build and install it from within
the code directory by:
.. code-block:: bash
python setup.py install
- Install Theano using the above :ref:`install_bleeding_edge` installation instructions
(using ``easy_install`` will require additional packages and has not been
tested yet, while the latest official Theano release is also untested at this
time).
In particular, do not forget to make the Theano package accessible from
Python, e.g. by adding to your ``.profile`` a line like
``export PYTHONPATH=PYTHONPATH:$HOME/Theano``.
- Please note that at this time, some tests (launched using ``nosetests``) are
still failing under Windows.
We are working on fixing them.
Generating the documentation Generating the documentation
---------------------------- ----------------------------
......
doc/internal/dev_start_guide.txt
浏览文件 @ d9c7ae08
...@@ -6,10 +6,10 @@ Developer Start Guide ...@@ -6,10 +6,10 @@ Developer Start Guide
===================== =====================
- Learn some `non-basic python`_ to understand what's going on in some of the - Learn some non-basic python to understand what's going on in some of the
tricker files (like tensor.py). tricker files (like tensor.py).
- BasicNumpy_ essential things to know about numpy. - Roughly go through the numpy documentation.
- Learn to write reStructuredText_ for epydoc_. - Learn to write reStructuredText_ for epydoc_.
...@@ -41,13 +41,13 @@ As a developer, you should clone this repository like this: ...@@ -41,13 +41,13 @@ As a developer, you should clone this repository like this:
.. code-block:: bash .. code-block:: bash
hg clone 'http://username:password@pylearn.org/hg/Theano' hg clone 'http://username:password@hg.assembla.com/theano Theano'
You can also clone the code anonymously: You can also clone the code anonymously:
.. code-block:: bash .. code-block:: bash
hg clone http://pylearn.org/hg/Theano hg clone http://hg.assembla.com/theano Theano
Setting up your environment Setting up your environment
=========================== ===========================
...@@ -134,6 +134,7 @@ so there might be some latency in the process. ...@@ -134,6 +134,7 @@ so there might be some latency in the process.
For more detail :ref:`see <metadocumentation_nightly_build>`. For more detail :ref:`see <metadocumentation_nightly_build>`.
.. TODO: fix this links
.. _non-basic python: http://pylearn.org/theano/wiki/NonbasicPython .. _non-basic python: http://pylearn.org/theano/wiki/NonbasicPython
.. _reStructuredText: http://docutils.sourceforge.net/rst.html .. _reStructuredText: http://docutils.sourceforge.net/rst.html
......
doc/introduction.txt
浏览文件 @ d9c7ae08
.. _introduction: .. _introduction:
============ ==================
Introduction Theano at a Glance
============ ==================
Theano is a Python library that allows you to define, optimize, and Theano is a Python library that allows you to define, optimize, and evaluate
efficiently evaluate mathematical expressions involving mathematical expressions involving multi-dimensional arrays. Using Theano it is
multi-dimensional arrays. Using Theano, for problems involving large possible to attain speeds rivaling hand-crafted C implementations for problems
amounts of data, it is possible to attain speeds that are only a few involving large amounts of data. It can also surpass C on a CPU by many orders
percentage points slower than hand-crafted C implementations. of magnitude by taking advantage of recent GPUs.
Theano melds some aspects of a computer algebra system (CAS) with Theano melds some aspects of a computer algebra system (CAS) with
aspects of an optimizing compiler. It can even transform some or all aspects of an optimizing compiler. It can even transform some or all
...@@ -125,18 +125,16 @@ Getting started ...@@ -125,18 +125,16 @@ Getting started
:ref:`install` :ref:`install`
Instructions to download and install Theano on your system. Instructions to download and install Theano on your system.
:ref:`basictutorial` :ref:`tutorial`
Getting started with Theano's basic features. Go there if you are Getting started with Theano's basic features. Go here if you are
new! new!
:ref:`advtutorial` :ref:`libdoc`
This tutorial is for more advanced users who want to define their Details of what Theano provides. It is recommended to go through
own operations and optimizations. It is recommended to go through the :ref:`tutorial` first though.
the :ref:`basictutorial` first.
For a complete map of the documentation you may check the
:ref:`contents`. Also, a PDF version of the online documentation may A PDF version of the online documentation may be found `here <theano.pdf>`_.
be found `here <theano.pdf>`_.
Contact us Contact us
......
doc/library/compile/debugmode.txt 0 → 100644
浏览文件 @ d9c7ae08
.. _debugmode:
=================
:mod:`debugmode`
=================
.. module:: debugmode
:platform: Unix, Windows
:synopsis: defines DebugMode
.. moduleauthor:: LISA
Guide
=====
The DebugMode evaluation mode includes a number of self-checks and assertions
that can help to diagnose several kinds of programmer errors that can lead to
incorrect output.
It is much slower to evaluate a function or method with DebugMode than
it would be in ``'FAST_RUN'`` or even ``'FAST_COMPILE'``. We recommended you use
DebugMode during development, but not when you launch 1000 processes on
a cluster.
DebugMode can be used as follows:
.. code-block:: python
x = tensor.dvector('x')
f = theano.function([x], 10*x, mode='DEBUG_MODE')
f(5)
f(0)
f(7)
It can also be used by setting an environment variable ``THEANO_DEFAULT_MODE=DEBUG_MODE``.
It can also be used by passing a DebugMode instance as the mode, as in
>>> f = theano.function([x], 10*x, mode=DebugMode(check_c_code=False))
If any problem is detected, DebugMode will raise an exception according to
what went wrong, either at call time (``f(5)``) or compile time (
``f = theano.function(x, 10*x, mode='DEBUG_MODE')``). These exceptions
should *not* be ignored; talk to your local Theano guru or email the
users list if you cannot make the exception go away.
Some kinds of errors can only be detected for certain input value combinations.
In the example above, there is no way to guarantee that a future call to say,
``f(-1)`` won't cause a problem. DebugMode is not a silver bullet.
If you instantiate DebugMode using the constructor ``compile.DebugMode``
rather than the keyword ``DEBUG_MODE`` you can configure its behaviour via
constructor arguments.
Reference
==========
.. class:: DebugMode(Mode)
Evaluation Mode that detects internal theano errors.
This mode catches several kinds of internal error:
- inconsistent c_code and perform implementations (see `BadCLinkerOutput`)
- a variable replacing another when their runtime values don't match. This is a symptom of
an incorrect optimization step, or faulty Op implementation (raises `BadOptimization`)
- stochastic optimization ordering (raises `StochasticOrder`)
- incomplete `destroy_map` specification (raises `BadDestroyMap`)
- an op that returns an illegal value not matching the output Variable Type (raises
InvalidValueError)
Each of these exceptions inherits from the more generic `DebugModeError`.
If there are no internal errors, this mode behaves like FAST_RUN or FAST_COMPILE, but takes
a little longer and uses more memory.
If there are internal errors, this mode will raise an `DebugModeError` exception.
.. attribute:: stability_patience = config.THEANO_DEBUGMODE_PATIENCE
When checking for the stability of optimization, recompile the graph this many times.
Default 10.
.. attribute:: check_c_code = config.THEANO_DEBUGMODE_CHECK_C
Should we evaluate (and check) the `c_code` implementations?
``True`` -> yes, ``False`` -> no.
Default yes.
.. attribute:: check_py_code = config.THEANO_DEBUGMODE_CHECK_PY
Should we evaluate (and check) the `perform` implementations?
``True`` -> yes, ``False`` -> no.
Default yes.
.. attribute:: check_isfinite = config.THEANO_DEBUGMODE_CHECK_FINITE
Should we check for (and complain about) ``NaN``/``Inf`` ndarray elements?
``True`` -> yes, ``False`` -> no.
Default yes.
.. attribute:: require_matching_strides = config.THEANO_DEBUGMODE_CHECK_STRIDES
Check for (and complain about) Ops whose python and C
outputs are ndarrays with different strides. (This can catch bugs, but
is generally overly strict.)
0 -> no check, 1 -> warn, 2 -> err.
Default warn.
.. method:: __init__(self, optimizer='fast_run', stability_patience=None, check_c_code=None, check_py_code=None, check_isfinite=None, require_matching_strides=None, linker=None)
Initialize member variables.
If any of these arguments (except optimizer) is not None, it overrides the class default.
The linker arguments is not used. It is set their to allow Mode.requiring() and some other fct to work with DebugMode too.
The keyword version of DebugMode (which you get by using ``mode='DEBUG_MODE``)
is quite strict, and can raise several different Exception types.
There following are DebugMode exceptions you might encounter:
.. class:: DebugModeError(Exception)
This is a generic error. All the other exceptions inherit from this one.
This error is typically not raised directly.
However, you can use ``except DebugModeError: ...`` to catch any of the more
specific types of Exception.
.. class:: BadCLinkerOutput(DebugModeError)
This exception means that python (``perform``) and c (``c_code``) for an Op
didn't compute the same thing like they were supposed to.
The problem might be a bug in either ``perform`` or ``c_code`` (or both).
.. class:: BadOptimization(DebugModeError)
This exception indicates that an Optimization replaced one variable (say V1)
with another one (say V2) but at runtime, the values for V1 and V2 were
different. This is something that optimizations are not supposed to do.
It can be tricky to identify the one-true-cause of an optimization error, but
this exception provides a lot of guidance. Most of the time, the
exception object will indicate which optimization was at fault.
The exception object also contains information such as a snapshot of the
before/after graph where the optimization introduced the error.
.. class:: BadDestroyMap(DebugModeError)
This happens when an Op's ``perform()`` or ``c_code()`` modifies an input that it wasn't
supposed to. If either the ``perform`` or ``c_code`` implementation of an Op
might modify any input, it has to advertise that fact via the ``destroy_map``
attribute.
For detailed documentation on the ``destroy_map`` attribute, see :ref:`inplace`.
.. class:: BadViewMap(DebugModeError)
This happens when an Op's perform() or c_code() creates an alias or alias-like
dependency between an input and an output... and it didn't warn the
optimization system via the ``view_map`` attribute.
For detailed documentation on the ``view_map`` attribute, see :ref:`views`.
.. class:: StochasticOrder(DebugModeError)
This happens when an optimization does not perform the same graph operations
in the same order when run several times in a row. This can happen if any
steps are ordered by ``id(object)`` somehow, such as via the default object
hash function. A Stochastic optimization invalidates the pattern of work
whereby we debug in DEBUG_MODE and then run the full-size jobs in FAST_RUN.
.. class:: InvalidValueError(DebugModeError)
This happens when some Op's ``perform`` or ``c_code`` implementation computes
an output that is invalid with respect to the type of the corresponding output
variable. Like if it returned a complex-valued ndarray for a ``dscalar``
Type.
This can also be triggered when floating-point values such as NaN and Inf are
introduced into the computations. It indicates which Op created the first
NaN. These floating-point values can be allowed by passing the
``check_isfinite=False`` argument to DebugMode.
doc/library/compile/function.txt 0 → 100644
浏览文件 @ d9c7ae08
.. _usingfunction:
===========================================
:mod:`function` - defines theano.function
===========================================
.. module:: function
:platform: Unix, Windows
:synopsis: defines theano.function and related classes
.. moduleauthor:: LISA
Guide
=====
This module provides :func:`function`, commonly accessed as `theano.function`,
the interface for compiling graphs into callable objects.
You've already seen example usage in the basic tutorial... something like this:
>>> x = theano.tensor.dscalar()
>>> f = theano.function([x], 2*x)
>>> print f(4) # prints 8.0
The idea here is that we've compiled the symbolic graph (``2*x``) into a function that can be called on a number and will do some computations.
The behaviour of function can be controlled in several ways, such as
:class:`Param`, ``mode``, ``updates``, and ``givens``. These are covered
in the :ref:`tutorial examples <basictutexamples>` and :ref:`tutorial on modes <using_modes>`.
Reference
=========
.. class:: Out
A class for attaching information to function outputs
.. attribute:: variable
A variable in an expression graph to use as a compiled-function
output
.. attribute:: borrow
``True`` indicates that a reference to internal storage may be returned, and that the caller is aware that subsequent function evaluations might overwrite this memory.
.. method:: __init__(variable, borrow=False)
Initialize attributes from arguments.
.. class:: Param
A class for attaching information to function inputs.
.. attribute:: variable
A variable in an expression graph to use as a compiled-function parameter
.. attribute:: default
The default value to use at call-time (can also be a Container where
the function will find a value at call-time.)
.. attribute:: name
A string to identify an argument for this parameter in keyword arguments.
.. attribute:: mutable
``True`` means the compiled-function is allowed to modify this
argument. ``False`` means it is not allowed.
.. attribute:: strict
If ``False``, a function argument may be copied or cast to match the type
required by the parameter `variable`. If ``True``, a function argument
must exactly match the type required by `variable`.
.. method:: __init__(self, variable, default=None, name=None, mutable=False, strict=False)
Initialize object attributes.
.. function:: function(inputs, outputs, mode=None, updates=[], givens=[], accept_inplace=False, name=None)
Return a callable object that will calculate `outputs` from `inputs`.
:type params: list of either Variable or Param instances, but not shared
variables.
:param params: the returned :class:`Function` instance will have
parameters for these variables.
:type outputs: list of Variables or Out instances
:param outputs: expressions to compute.
:type mode: None, string or :class:`Mode` instance.
:param mode: compilation mode
:type updates: iterable over pairs (shared_variable, new_expression).
List, tuple or dict.
:param updates: expressions for new SharedVariable values
:type givens: iterable over pairs (Var1, Var2) of Variables.
List, tuple or dict. The Var1
and Var2 in each pair must have the same Type.
:param givens: specific substitutions to make in the
computation graph (Var2 replaces Var1).
:param name: an optional name for this function.
The profile mode will print the time spent in this function.
:rtype: Function instance
:returns: a callable object that will compute the outputs (given the inputs)
and update the implicit function arguments according to the `updates`.
Inputs can be given as variables or Param instances.
:class:`Param` instances also have a variable, but they attach some extra
information about how call-time arguments corresponding to that variable
should be used. Similarly, :class:`Out` instances can attach information
about how output variables should be returned.
The default is typically 'FAST_RUN' but this can be changed in
:doc:`theano.config <../config>` or via :envvar:`THEANO_DEFAULT_MODE`. The mode
argument controls the sort of optimizations that will be applied to the
graph, and the way the optimized graph will be evaluated.
After each function evaluation, the `updates` mechanism can replace the
value of any SharedVariable [implicit] inputs with new values computed
from the expressions in the `updates` list. An exception will be raised
if you give two update expressions for the same SharedVariable input (that
doesn't make sense).
Regarding givens: Be careful to make sure that these substitutions are
independent--behaviour when Var1 of one pair appears in the graph leading
to Var2 in another expression is undefined. Replacements specified with
givens are different from optimizations in that Var2 is not expected to be
equivalent to Var1.
doc/library/compile/index.txt 0 → 100644
浏览文件 @ d9c7ae08
.. _libdoc_compile:
==============================================================
:mod:`compile` -- transforming expression graphs to functions
==============================================================
.. module:: compile
:platform: Unix, Windows
:synopsis: transforming expression graphs to functions
.. moduleauthor:: LISA
.. toctree::
:maxdepth: 1
function
io
mode
module
debugmode
profilemode
doc/topics/function.txt → doc/library/compile/io.txt
浏览文件 @ d9c7ae08
.. _usingfunction: .. note::
===================== ***TODO*** Freshen up this old documentation
Using theano.function
=====================
This page is about :api:`theano.function .. _function_inputs:
<theano.compile.function_module.function>`, the interface for compiling
graphs into callable objects.
The signature for this function is:
.. code-block:: python
def function(inputs, outputs, mode=None):
...
You've already seen example usage in the basic tutorial... something like this:
>>> x = theano.tensor.dscalar() ===========================================
>>> f = theano.function([x], 2*x) :mod:`io` - defines theano.function [TODO]
>>> print f(4) # prints 8.0 ===========================================
The idea here is that we've compiled the symbolic graph (``2*x``) into a function that can be called on a number and will do some computations. .. module:: io
:platform: Unix, Windows
:synopsis: defines In and Out
.. moduleauthor:: LISA
.. _function_inputs:
Inputs Inputs
====== ======
...@@ -34,7 +23,7 @@ The ``inputs`` argument to ``theano.function`` is a list, containing the ``Varia ...@@ -34,7 +23,7 @@ The ``inputs`` argument to ``theano.function`` is a list, containing the ``Varia
``In`` instances let us attach properties to ``Variables`` to tell function more about how to use them. ``In`` instances let us attach properties to ``Variables`` to tell function more about how to use them.
.. class:: In .. class:: In(object)
.. method:: __init__(variable, name=None, value=None, update=None, mutable=False, strict=False, autoname=True, implicit=None) .. method:: __init__(variable, name=None, value=None, update=None, mutable=False, strict=False, autoname=True, implicit=None)
...@@ -364,28 +353,3 @@ If a list of ``Variable`` or ``Out`` instances is given as argument, then the co ...@@ -364,28 +353,3 @@ If a list of ``Variable`` or ``Out`` instances is given as argument, then the co
fn3 = theano.function([x], outputs=x+x) fn3 = theano.function([x], outputs=x+x)
print fn3(numpy.asarray([[1,0],[0,1]])) print fn3(numpy.asarray([[1,0],[0,1]]))
.. _function_mode:
Mode
====
The ``mode`` parameter to ``theano.function`` controls how the
inputs-to-outputs graph is transformed into a callable object.
Theano defines the following modes by name:
- ``FAST_COMPILE``: Apply just a few optimizations, but use C op implementations where possible.
- ``FAST_RUN``: Apply all optimizations, and use C op implementations where possible.
- ``DEBUG_MODE``: Verify the correctness of all optimizations, and compare C and python
implementations. This mode can take much longer than the other modes,
but can identify many kinds of problems.
The default mode is typically 'FAST_RUN', but it can be controlled via the
environment variable 'THEANO_DEFAULT_MODE', which can in turn be overridden by
setting ``theano.compile.mode.default_mode`` directly, which can in turn be
overridden by passing the keyword argument to ``theano.function``.
For a finer level of control over which optimizations are applied, and whether
C or python implementations are used, read :api:`compile.mode.Mode`.
doc/library/compile/mode.txt 0 → 100644
浏览文件 @ d9c7ae08
======================================
:mod:`mode` -- controlling compilation
======================================
.. module:: mode
:platform: Unix, Windows
:synopsis: controlling compilation
.. moduleauthor:: LISA
Guide
=====
The ``mode`` parameter to :func:`theano.function`` controls how the
inputs-to-outputs graph is transformed into a callable object.
Theano defines the following modes by name:
- ``FAST_COMPILE``: Apply just a few optimizations, but use C op implementations where possible.
- ``FAST_RUN``: Apply all optimizations, and use C op implementations where possible.
- ``DEBUG_MODE``: Verify the correctness of all optimizations, and compare C and python
implementations. This mode can take much longer than the other modes,
but can identify many kinds of problems.
The default mode is typically 'FAST_RUN', but it can be controlled via the
environment variable 'THEANO_DEFAULT_MODE', which can in turn be overridden by
setting ``theano.compile.mode.default_mode`` directly, which can in turn be
overridden by passing the keyword argument to ``theano.function``.
For a finer level of control over which optimizations are applied, and whether
C or python implementations are used, read :api:`compile.mode.Mode`.
Reference
=========
.. attribute:: FAST_COMPILE
.. attribute:: FAST_RUN
.. attribute:: DEBUG_MODE
.. attribute:: PROFILE_MODE
.. class:: Mode(object)
Compilation is controlled by two attributes: the `optimizer` controls how
an expression graph will be transformed; the `linker` controls how the
optimized expression graph will be evaluated.
.. attribute:: optimizer
An :class:`optimizer` instance.
.. attribute:: linker
A :class:`linker` instance.
.. method:: including(*tags)
Return a new Mode instance like this one, but with an
optimizer modified by including the given tags.
.. method:: excluding(*tags)
Return a new Mode instance like this one, but with an
optimizer modified by excluding the given tags.
.. method:: requiring(*tags)
Return a new Mode instance like this one, but with an
optimizer modified by requiring the given tags.
doc/topics/module.txt → doc/library/compile/module.txt
浏览文件 @ d9c7ae08
============ =======================================
Using Module :mod:`module` -- a theano object system
============ =======================================
.. note::
Module addresses similar needs to `shared`. New code is encouraged to
use `shared` variables.
Now that we're familiar with the basics, we introduce Theano's more Now that we're familiar with the basics, we introduce Theano's more
advanced interface, Module. This interface allows you to define Theano advanced interface, Module. This interface allows you to define Theano
......
doc/topics/profilemode.txt → doc/library/compile/profilemode.txt
浏览文件 @ d9c7ae08
.. _profilemode: .. _profilemode:
========================================= ================================================
ProfileMode :mod:`profilemode` -- profiling Theano functions
========================================= ================================================
.. module:: profilemode
:platform: Unix, Windows
:synopsis: profiling Theano functions with ProfileMode
.. moduleauthor:: LISA
Guide
=====
To profile a Theano graph, a special mode called ProfileMode, must be passed as To profile a Theano graph, a special mode called ProfileMode, must be passed as
an argument when compiling your graph. Using ProfileMode is a three-step an argument when compiling your graph. Using ProfileMode is a three-step
...@@ -118,6 +127,34 @@ generates the following output: ...@@ -118,6 +127,34 @@ generates the following output:
""" """
.. note::
***TODO***
The following text was recovered from a recent version of the source
file... hopefully things haven't gotten too out-of-sync!
The first show an Apply-wise summary, the second show an Op-wise summary, the third show an type-Op-wise summary.
The Apply-wise summary print the timing information for the worst
offending Apply nodes. This corresponds to individual Op applications
within your graph which take the longest to execute (so if you use dot
twice, you will see two entries there).
The Op-wise summary print the execution time of all Apply nodes
executing the same Op are grouped together and the total execution
time per Op is shown (so if you use dot twice, you will see only one
entry there corresponding to the sum of the time spent in each of
them). If two Op have different hash value, they will be separate.
The type-Op-wise summary group the result by type of op. So event if
two Op have different hash value, they will be merged.
Their is an hack with the Op-wise summary. Go see it if you want to know more.
The summary has two components to it. In the first section called the The summary has two components to it. In the first section called the
Apply-wise summary, timing information is provided for the worst Apply-wise summary, timing information is provided for the worst
offending Apply nodes. This corresponds to individual Op applications offending Apply nodes. This corresponds to individual Op applications
...@@ -131,7 +168,43 @@ there corresponding to the sum of the time spent in each of them). ...@@ -131,7 +168,43 @@ there corresponding to the sum of the time spent in each of them).
Note that the ProfileMode also shows which Ops were running a c Note that the ProfileMode also shows which Ops were running a c
implementation. implementation.
Developers wishing to optimize the performance of their graph, should Developers wishing to optimize the performance of their graph should
focus on the worst offending Ops. If no C implementation exists for focus on the worst offending Ops and Apply nodes -- either by optimizing an
this op, consider writing a C implementation yourself or use the implementation, providing a missing C implementation, or by writing a graph
mailing list, to suggest that a C implementation be provided. optimization that eliminates the offending Op altogether.
You should strongly consider emailing one of our lists about your issue before
spending too much time on this.
Reference
=========
.. class:: ProfileMode(Mode)
.. method:: print_summary(n_apply_to_print=None, n_ops_to_print=None)
Print three summaries to stdout that show where cpu time is spent during theano function executions (for all functions using this object instance).
:param n_apply_to_print: the number of apply nodes to print.
The default 15, but can be configured via ``ProfileMode.n_ops_to_print`` in :envvar:`THEANO_FLAGS`.
:param n_ops_to_print: the number of ops to print.
Default 20, or but can be configured via ``ProfileMode.n_apply_to_print`` in :envvar:`THEANO_FLAGS`.
:returns: None
.. method:: print_diff_summary(self, other, n_apply_to_print=None, n_ops_to_print=None):
""" As print_summary, but print the difference on two different profile mode.
TODO: Also we don't print the Apply-wise summary as it don't work for now.
TODO: make comparaison with gpu code.
:param other: the other instance of ProfileMode that we want to be compared to.
:param n_apply_to_print: the number of apply nodes to print.
The default 15, but can be configured via ``ProfileMode.n_ops_to_print`` in :envvar:`THEANO_FLAGS`.
:param n_ops_to_print: the number of ops to print.
Default 20, or but can be configured via ``ProfileMode.n_apply_to_print`` in :envvar:`THEANO_FLAGS`.
:returns: None
doc/library/config.txt 0 → 100644
浏览文件 @ d9c7ae08
.. _libdoc_config:
=======================================
:mod:`config` -- library configuration
=======================================
.. module:: config
:platform: Unix, Windows
:synopsis: library configuration
.. moduleauthor:: LISA
.. envvar:: THEANO_FLAGS
***TODO***
.. attribute:: floatx
***TODO*** what attributes are in here?
doc/topics/floatX.txt → doc/library/floatX.txt
浏览文件 @ d9c7ae08
.. _floatX:
======================== .. _libdoc_floatX:
Special data type floatX
========================
Intro =======================================================================
:mod:`floatX` -- easy switching between float32 and float64
=======================================================================
.. module:: floatx
:platform: Unix, Windows
:synopsis: easy switching between float32 and float64
.. moduleauthor:: LISA
Guide
===== =====
Their is a special data type called floatX. It is not a real datatype. It is never present in the theano graph, but their exist constructor and function that will change the floatX to float32 or float64(default) in your graph. You can change the value of floatX when you start the execution of python by setting the environement variables THEANO_GPU=floatX=float{32,64}(case sensitive). You can have the value of floatX with:: Their is a special data type called floatX. It is not a real datatype. It is never present in the theano graph, but their exist constructor and function that will change the floatX to float32 or float64(default) in your graph. You can change the value of floatX when you start the execution of python by setting the environement variables THEANO_GPU=floatX=float{32,64}(case sensitive). You can have the value of floatX with::
...@@ -35,3 +42,42 @@ HINT: linear algorythm are less affected by the different precision then non-lin ...@@ -35,3 +42,42 @@ HINT: linear algorythm are less affected by the different precision then non-lin
numpy.asarray(x,dtype=config.floatX) warn copy only if needed. numpy.asarray(x,dtype=config.floatX) warn copy only if needed.
WARNING: theano.floatx.set_floatX() exist for our test. Don't use it for something else. If you do, it will make code hard to read and it is a sign that their is something better for you then floatX. WARNING: theano.floatx.set_floatX() exist for our test. Don't use it for something else. If you do, it will make code hard to read and it is a sign that their is something better for you then floatX.
Reference
==========
.. function:: xscalar(name=None)
Alias for either :func:`dscalar` or :func:`fscalar`
.. function:: xvector(name=None)
Alias for either :func:`d...` or :func:`f---`
.. function:: xmatrix(name=None)
Alias for either :func:`d...` or :func:`f---`
.. function:: xrow(name=None)
Alias for either :func:`d...` or :func:`f---`
.. function:: xcol(name=None)
Alias for either :func:`d...` or :func:`f---`
.. function:: xtensor3(name=None)
Alias for either :func:`d...` or :func:`f---`
.. function:: xtensor4(name=None)
Alias for either :func:`d...` or :func:`f---`
.. function:: set_floatX(dtype=config.floatX)
Reset the :func:`xscalar`, ... :func:`xtensor4` aliases to return types
with given dtype.
doc/library/gof/index.txt 0 → 100644
浏览文件 @ d9c7ae08
.. _libdoc_gof:
================================================
:mod:`gof` -- theano internals [doc TODO]
================================================
doc/library/gradient.txt 0 → 100644
浏览文件 @ d9c7ae08
.. _libdoc_gradient:
===========================================
:mod:`gradient` -- symbolic differentiation
===========================================
.. module:: gradient
:platform: Unix, Windows
:synopsis: low-level automatic differentiation
.. moduleauthor:: LISA
Symbolic gradient is usually computed from :func:`tensor.grad`, which offers a
more convenient syntax for the common case of wanting the gradient in some
expressions with respect to a scalar cost. The :func:`grad_sources_inputs`
function does the underlying work, and is more flexible, but is also more
awkward to use when :func:`tensor.grad` can do the job.
.. function:: grad_sources_inputs(sources, graph_inputs, warn_type=True)
A gradient source is a pair (``r``, ``g_r``), in which ``r`` is a `Variable`, and ``g_r`` is a
`Variable` that is a gradient wrt ``r``.
This function traverses the graph backward from the ``r`` sources,
calling ``op.grad(...)`` for all ops with some non-None gradient on an output.
The ``op.grad(...)`` functions are called like this:
.. code-block:: python
op.grad(op.inputs[:], [total_gradient(v) for v in op.outputs])
This call to ``op.grad`` should return a list or tuple: one symbolic gradient per input.
If ``op`` has a single input, then ``op.grad`` should return a list or tuple of length 1.
For each input wrt to which ``op`` is not differentiable, it should return ``None`` instead
of a `Variable` instance.
If a source ``r`` receives a gradient from another source ``r2``, then the effective
gradient on ``r`` is the sum of both gradients.
:type sources: list of pairs of Variable: (v, gradient-on-v) to
initialize the total_gradient dictionary
:param sources: gradients to back-propagate using chain rule
:param warn_type: True will trigger warnings via the logging module when
the gradient on an expression has a different type than the original
expression
:type warn_type: bool
:type graph_inputs: list of Variable
:param graph_inputs: variables considered to be constant
(do not backpropagate through them)
:rtype: dictionary whose keys and values are of type `Variable`
:returns: mapping from each Variable encountered in the backward traversal to its [total] gradient.
doc/library/index.txt 0 → 100644
浏览文件 @ d9c7ae08
.. _libdoc:
=====================
Library Documentation
=====================
This documentation covers Theano module-wise.
.. toctree::
:maxdepth: 1
tensor/index
gradient
config
floatX
printing
compile/index
sparse/index
scalar/index
gof/index
There are also some top-level imports that you might find more convenient:
.. module:: theano
:platform: Unix, Windows
:synopsis: Theano top-level import
.. moduleauthor:: LISA
.. function:: function(...)
Alias for :func:`function.function`
.. class:: Param
Alias for :class:`function.Param`
.. function:: dot(x, y)
Works like :func:`tensor.dot` for both sparse and dense matrix products
doc/library/printing.txt 0 → 100644
浏览文件 @ d9c7ae08
.. _libdoc_printing:
================================================================================
:mod:`printing` -- graph printing and symbolic print statement [doc TODO]
================================================================================
doc/library/scalar/index.txt 0 → 100644
浏览文件 @ d9c7ae08
.. _libdoc_scalar:
==============================================================
:mod:`scalar` -- symbolic scalar types, ops [doc TODO]
==============================================================
doc/library/sparse/index.txt 0 → 100644
浏览文件 @ d9c7ae08
.. _libdoc_sparse:
===========================================================
:mod:`sparse` -- symbolic sparse matrices [doc TODO]
===========================================================
doc/library/tensor/basic.txt 0 → 100644
浏览文件 @ d9c7ae08
差异被折叠。
doc/library/tensor/bcast.svg 0 → 100644
浏览文件 @ d9c7ae08
差异被折叠。
doc/library/tensor/index.txt 0 → 100644
浏览文件 @ d9c7ae08
.. _libdoc_tensor:
==================================================
:mod:`tensor` -- types and ops for symbolic numpy
==================================================
.. module:: tensor
:platform: Unix, Windows
:synopsis: symbolic types and operations for n-dimensional arrays.
.. moduleauthor:: LISA
Theano's strength is in expressing symbolic calculations involving tensors.
There are many types of symbolic expressions for tensors. For everyone's
sanity, they are grouped into the following sections:
.. toctree::
:maxdepth: 1
basic
shared_randomstreams
nnet
signal
doc/library/tensor/nnet.txt 0 → 100644
浏览文件 @ d9c7ae08
.. _libdoc_tensor_nnet:
======================================================
:mod:`nnet` -- Ops for neural networks
======================================================
.. module:: nnet
:platform: Unix, Windows
:synopsis: Ops for neural networks
.. moduleauthor:: LISA
.. function:: sigmoid(x)
Returns the standard sigmoid nonlinearity applied to x
:Parameters: *x* - symbolic Tensor (or compatible)
:Return type: same as x
:Returns: element-wise sigmoid: :math:`sigmoid(x) = \frac{1}{1 + \exp(-x)}`.
Example:
.. code-block:: python
x,y,b = T.dvectors('x','y','b')
W = T.dmatrix('W')
y = T.nnet.sigmoid(T.dot(W,x) + b)
.. note:: The underlying code will return an exact 0 or 1 if an element of x is too small or too big.
.. function:: softplus(x)
Returns the softplus nonlinearity applied to x
:Parameter: *x* - symbolic Tensor (or compatible)
:Return type: same as x
:Returns: elementwise softplus: :math:`softplus(x) = \log_e{\left(1 + \exp(x)\right)}`.
.. note:: The underlying code will return an exact 0 if an element of x is too small.
.. code-block:: python
x,y,b = T.dvectors('x','y','b')
W = T.dmatrix('W')
y = T.nnet.softplus(T.dot(W,x) + b)
.. function:: softmax(x)
Returns the softmax function of x:
:Parameter: *x* symbolic **2D** Tensor (or compatible).
:Return type: same as x
:Returns: a symbolic 2D tensor whose ijth element is :math:`softmax_{ij}(x) = \frac{\exp{x_{ij}}}{\sum_k\exp(x_{ik})}`.
The softmax function will, when applied to a matrix, compute the softmax values row-wise.
.. code-block:: python
x,y,b = T.dvectors('x','y','b')
W = T.dmatrix('W')
y = T.nnet.softmax(T.dot(W,x) + b)
.. function:: binary_crossentropy(output,target)
Computes the binary cross-entropy between a target and an output:
:Parameters:
* *target* - symbolic Tensor (or compatible)
* *output* - symbolic Tensor (or compatible)
:Return type: same as target
:Returns: a symbolic tensor, where the following is applied elementwise :math:`crossentropy(t,o) = -(t\cdot log(o) + (1 - t) \cdot log(1 - o))`.
The following block implements a simple auto-associator with a sigmoid
nonlinearity and a reconstruction error which corresponds to the binary
cross-entropy (note that this assumes that x will contain values between 0 and
1):
.. code-block:: python
x,y,b = T.dvectors('x','y','b')
W = T.dmatrix('W')
h = T.nnet.sigmoid(T.dot(W,x) + b)
x_recons = T.nnet.sigmoid(T.dot(V,h) + c)
recon_cost = T.nnet.binary_crossentropy(x_recons,x).mean()
.. function:: categorical_crossentropy(coding_dist,true_dist)
Return the cross-entropy between an approximating distribution and a true distribution.
The cross entropy between two probability distributions measures the average number of bits
needed to identify an event from a set of possibilities, if a coding scheme is used based
on a given probability distribution q, rather than the "true" distribution p. Mathematically, this
function computes :math:`H(p,q) = - \sum_x p(x) \log(q(x))`, where
p=coding_dist and q=true_dist
:Parameters:
* *coding_dist* - symbolic 2D Tensor (or compatible). Each row
represents a distribution.
* *true_dist* - symbolic 2D Tensor **OR** symbolic vector of ints. In
the case of an integer vector argument, each element represents the
position of the '1' in a 1-of-N encoding (aka "one-hot" encoding)
:Return type: tensor of rank one-less-than `coding_dist`
.. note:: An application of the scenario where *true_dist* has a 1-of-N representation
is in classification with softmax outputs. If `coding_dist` is the output of
the softmax and `true_dist` is a vector of correct labels, then the function
will compute ``y_i = - \log(coding_dist[i, one_of_n[i]])``, which corresponds
to computing the neg-log-probability of the correct class (which is typically
the training criterion in classification settings).
.. code-block:: python
y = T.nnet.softmax(T.dot(W,x) + b)
cost = T.nnet.categorical_crossentropy(y,o)
# o is either the above-mentioned 1-of-N vector or 2D tensor
doc/topics/randomstreams.txt → doc/library/tensor/shared_randomstreams.txt
浏览文件 @ d9c7ae08
.. _randomstreams: .. _libdoc_tensor_shared_randomstreams:
==================== ======================================================
Using RandomStreams :mod:`shared_randomstreams` -- Friendly random numbers
==================== ======================================================
.. module:: shared_randomstreams
:platform: Unix, Windows
:synopsis: symbolic random variables
.. moduleauthor:: LISA
Guide
=====
Since Theano uses a functional design, producing pseudo-random numbers in a Since Theano uses a functional design, producing pseudo-random numbers in a
graph is not quite as straightforward as it is in numpy. But close. If you are using theano's graph is not quite as straightforward as it is in numpy. If you are using Theano's
shared variables, then a RandomStreams object is probably what you want. If you are shared variables, then a `RandomStreams` object is probably what you want. (If you are
using the function interface directly, or you are using Module then this tutorial will be useful but not exactly what you want. using Module then this tutorial will be useful but not exactly what you want.
Have a look at the :api:`RandomFunction` Op. Have a look at the :api:`RandomFunction` Op.)
The way to think about putting randomness into theano's computations is to The way to think about putting randomness into theano's computations is to
put random variables in your graph. Theano will allocate a numpy RandomState put random variables in your graph. Theano will allocate a numpy RandomState
...@@ -33,7 +41,7 @@ Here's a brief example. The setup code is: ...@@ -33,7 +41,7 @@ Here's a brief example. The setup code is:
Here, 'rv_u' represents a random stream of 2x2 matrices of draws from a uniform Here, 'rv_u' represents a random stream of 2x2 matrices of draws from a uniform
distribution. Likewise, 'rv_n' represenents a random stream of 2x2 matrices of distribution. Likewise, 'rv_n' represenents a random stream of 2x2 matrices of
draws from a normal distribution. The distributions that are implemented are draws from a normal distribution. The distributions that are implemented are
defined in :ref:`tensor.shared_randomstreams.RandomStreams`. defined in :class:`RandomStreams`.
Now let's use these things. If we call f(), we get random uniform numbers. Now let's use these things. If we call f(), we get random uniform numbers.
Since we are updating the internal state of the random number generator (via Since we are updating the internal state of the random number generator (via
...@@ -56,27 +64,6 @@ random variable appears three times in the output expression. ...@@ -56,27 +64,6 @@ random variable appears three times in the output expression.
>>> nearly_zeros = function([], rv_u + rv_u - 2 * rv_u, updates=[rv_u.update]) >>> nearly_zeros = function([], rv_u + rv_u - 2 * rv_u, updates=[rv_u.update])
Internal Attributes
--------------------
The random variables returned by methods like ``RandomStreams.uniform`` have
some useful internal attributes.
The one we used above is the ``update`` attribute. ``rv_u.update`` is a pair
whose first element is a shared variable whose value is a numpy RandomState,
and whose second element is an [symbolic] expression for the next value of that
RandomState after drawing samples.
The first element of the ``update`` attribute is also accessible as
``rv_u.rng``. A random variable can be re-seeded by seeding or assigning to
``rv_u.rng.value``.
The ``RandomStreams`` object itself has an ``updates()`` method that returns a
list of all the (state, new_state) update pairs from the random variables it
has returned. This can be a convenient shortcut to enumerating all
the random variables in a large graph in the ``update`` paramter of function.
Seedings Streams Seedings Streams
---------------- ----------------
...@@ -87,14 +74,13 @@ You can seed just one random variable by seeding or assigning to the ...@@ -87,14 +74,13 @@ You can seed just one random variable by seeding or assigning to the
>>> rv_u.rng.value.seed(89234) # seeds the generator for rv_u >>> rv_u.rng.value.seed(89234) # seeds the generator for rv_u
You can also seed *all* of the random variables allocated by a ``RandomStreams`` You can also seed *all* of the random variables allocated by a :class:`RandomStreams`
object by that object's ``seed`` method. This seed will be used to seed a object by that object's ``seed`` method. This seed will be used to seed a
temporary random number generator, that will in turn generate seeds for each temporary random number generator, that will in turn generate seeds for each
of the random variables. of the random variables.
>>> srng.seed(902340) # seeds rv_u and rv_n with different seeds each >>> srng.seed(902340) # seeds rv_u and rv_n with different seeds each
Sharing Streams between Functions Sharing Streams between Functions
--------------------------------- ---------------------------------
...@@ -111,9 +97,179 @@ For example: ...@@ -111,9 +97,179 @@ For example:
>>> v2 = f() # v2 != v1 >>> v2 = f() # v2 != v1
.. _Tools: tools.html
Reference
=========
.. class:: RandomStreams(object)
This is a symbolic stand-in for ``numpy.random.RandomState``. It has
methods such as `uniform` and `normal` that return symbolic random variables.
.. method:: updates()
:returns: a list of all the (state, new_state) update pairs from the
random variables it has returned. This can be a convenient shortcut
to enumerating all the random variables in a large graph in the
``update`` paramter of function.
.. method:: seed(meta_seed)
`meta_seed` will be used to seed a temporary random number generator,
that will in turn generate seeds for each of the random variables that
has been created by this object.
:returns: None
.. method:: binomial(self, size, n=1, p=0.5)
Symbolic stand-in for numpy.random.RandomState.binomial
:returns: :class:`RandomVariable` of float64 that will have `shape==size` at run-time.
.. method:: uniform(self, size, low=0.0, high=1.0)
Symbolic stand-in for numpy.random.RandomState.uniform
:returns: :class:`RandomVariable` of float64 that will have `shape==size` at run-time.
.. method:: normal(self, size, loc=0.0, std=1.0)
Symbolic stand-in for numpy.random.RandomState.normal
:returns: :class:`RandomVariable` of float64 that will have `shape==size` at run-time.
.. method:: random_integers(self, size, low=0, high=1)
Symbolic stand-in for numpy.random.RandomState.random_integers
:returns: :class:`RandomVariable` of int64 that will have `shape==size` at run-time.
.. class:: RandomVariable(object)
.. attribute:: rng
The shared variable whose ``.value`` is the numpy RandomState
generator feeding this random variable.
.. attribute:: update
A pair
whose first element is a shared variable whose value is a numpy RandomState,
and whose second element is an [symbolic] expression for the next value of that
RandomState after drawing samples.
Including this pair in the``updates`` list to function will cause the
function to update the random number generator feeding this variable.
.. _libdoc_tensor_raw_random:
=============================================
:mod:`raw_random` -- Low-level random numbers
=============================================
.. module:: raw_random
:platform: Unix, Windows
:synopsis: symbolic random variables
.. moduleauthor:: LISA
Raw random provides the random-number drawing functionality, that underlies
the :class:`RandomStreams` interface.
Reference
=========
.. class:: RandomStateType(gof.Type)
A `Type` for variables that will take ``numpy.random.RandomState`` values.
.. class:: RandomFunction(gof.Op)
Op that draws random numbers from a numpy.RandomState object. This Op is
parametrized to draw numbers from many distributions.
.. function:: random_function(fn, dtype, *rfargs, **rfkwargs)
Returns a wrapper around RandomFunction which automatically infers the number
of dimensions of the output from the given shape. If the shape cannot be inferred,
the user can give an integer as first argument, which will be interpreted as the
number of dimensions.
If the distribution is not scalar (e.g., a multinomial), the output will have
more dimensions than what the shape argument suggests. The "ndim_added" keyword
arguments allows to specify how many dimensions to add (for a multinomial, 1).
The number of dimensions for the following shape arguments can be inferred:
* shape(x)
* make_lvector(x, y, z, ...)
* ndarrays, constants
.. function:: uniform(random_state, size, low=0.0, high=1.0)
Sample from a uniform distribution between low and high.
If the size argument is ambiguous on the number of
dimensions, the first argument may be a plain integer
to supplement the missing information.
:returns: :class:`RandomVariable`, NewRandomState
.. function:: binomial(random_state, size, n=1, p=0.5)
Sample n times with probability of success prob for each trial,
return the number of successes.
If the size argument is ambiguous on the number of
dimensions, the first argument may be a plain integer
to supplement the missing information.
:returns: :class:`RandomVariable`, NewRandomState
.. function:: normal(random_state, size, avg=0.0, std=1.0)
Sample from a normal distribution centered on avg with
the specified standard deviation (std)
If the size argument is ambiguous on the number of
dimensions, the first argument may be a plain integer
to supplement the missing information.
:returns: :class:`RandomVariable`, NewRandomState
.. function:: random_integers(random_state, size, low=0, high=1)
Sample a random integer between low and high, both inclusive.
If the size argument is ambiguous on the number of
dimensions, the first argument may be a plain integer
to supplement the missing information.
:returns: :class:`RandomVariable`, NewRandomState
.. function:: permutation(random_state, size, n=1)
Returns permutations of the integers between 0 and n-1, as many times
as required by size. For instance, if size=(p,q), p*q permutations
will be generated, and the output shape will be (p,q,n), because each
permutation is of size n.
If the size argument is ambiguous on the number of dimensions, the first
argument may be a plain integer i, which should correspond to len(size).
Note that the output will then be of dimension i+1.
:returns: :class:`RandomVariable`, NewRandomState
.. function:: multinomial(random_state, size, p_vals=[0.5, 0.5])
Sample from a multinomial distribution defined by probabilities pvals,
as many times as required by size. For instance, if size=(p,q), p*q
samples will be drawn, and the output shape will be (p,q,len(pvals)).
If the size argument is ambiguous on the number of dimensions, the first
argument may be a plain integer i, which should correspond to len(size).
Note that the output will then be of dimension i+1.
:returns: :class:`RandomVariable`, NewRandomState
doc/library/tensor/signal.txt 0 → 100644
浏览文件 @ d9c7ae08
.. _libdoc_tensor_signal:
======================================================
:mod:`signal` -- Signal processing
======================================================
.. module:: signal
:platform: Unix, Windows
:synopsis: ops for signal processing
.. moduleauthor:: LISA
TODO: Give examples for how to use these things! They are pretty complicated.
.. function:: conv2D(*todo)
.. function:: downsample2D(*todo)
.. function:: fft(*todo)
doc/proposals/index.txt 0 → 100644
浏览文件 @ d9c7ae08
.. _proposals:
==================================
Proposals for new/revised features
==================================
.. toctree::
:maxdepth: 1
pfunc
noupdates
doc/proposals/noupdates.txt 0 → 100644
浏览文件 @ d9c7ae08
=================
Automatic updates
=================
.. note:
Proposed 2010 01 13
The Module version of RandomStreams could arrange for the automatic update of
certain inputs (such as the random number generators) at the time of make(), so
that certain *obvious* patterns would work:
>>> rs = RandomStreams()
>>> u = rs.uniform(...)
>>> f = theano.function([], u)
>>> assert not numpy.all(f() == f())
Unfortunately, with shared variables this does not work! Function needs to be
told which shared variables to update. The current workaround is to do this:
>>> theano.function([], u, updates=rs.updates())
or this:
>>> theano.function([], u, updates=[u.update])
But it is all too easy to forget to do either of these workarounds, and
accidentally run a program whose random numbers are the same in every call.
Proposal
========
Add an optional `default_update` attribute to Shared variables. This will be
consulted by function. If no update expression is given for this variable in
the updates list, then this default will be inserted. Note well: a value of None for the
default_update means to update with a value of None! To have no default update,
make sure that the default_update attribute is not defined.
Add an optional argument to function: `no_default_updates`. This argument defaults to
False, which results in the current semantics.
A True value here would mean "ignore all default_update expressions", and this
would be useful for disabling implicit behaviour.
A list of shared variables here would mean to ignore the
default_update_expressions in these specific variables.
Alternatives
============
Consider a singleton 'NOUPDATE' object that can be used as a pseudo-expression
in the update list. This doesn't introduce a new keyword argument, which makes
it slightly more awkward to document in theano.function. Really though, I have
no strong feelings between this and the no_updates paramter.
doc/proposals/pfunc.txt
浏览文件 @ d9c7ae08
====================================== =============================================
Proposal for pfunc Function Interface Proposal for pfunc Function Interface [DONE]
====================================== =============================================
.. note:: .. note::
......
doc/trac/ccodegen.txt → doc/sandbox/ccodegen.txt
浏览文件 @ d9c7ae08
...@@ -68,53 +68,51 @@ Failure ...@@ -68,53 +68,51 @@ Failure
Besides cleanup code, all code has access to the %(fail)s template. For three code blocks, the generated C code will pretty much look like this: Besides cleanup code, all code has access to the %(fail)s template. For three code blocks, the generated C code will pretty much look like this:
{{{ .. code-block::
int failure = 0; int failure = 0;
{
<code1>
{
<code2>
{ {
<code3> <code1>
label3: {
<cleanup3> <code2>
{
<code3>
label3:
<cleanup3>
}
label2:
<cleanup2>
}
label1:
<cleanup1>
} }
label2: return failure;
<cleanup2>
}
label1:
<cleanup1>
}
return failure;
}}}
And %(fail)s in the nth code block will take the value "{failure = n; goto label<n>;}". This means only the blocks executed up to the failure point are cleaned up and the return value indicates which block failed, which is handy for debugging. And %(fail)s in the nth code block will take the value "{failure = n; goto label<n>;}". This means only the blocks executed up to the failure point are cleaned up and the return value indicates which block failed, which is handy for debugging.
When compiling an Op, we want to sync the outputs so we can get the results from Python. In case of failure, we will not necessarily want to sync. Because of that, typical code will look like this: When compiling an Op, we want to sync the outputs so we can get the results from Python. In case of failure, we will not necessarily want to sync. Because of that, typical code will look like this:
{{{ .. code-block::
int failure = 0; int failure = 0;
<declare input> <declare input>
<declare output> <declare output>
{
<extract input>
{
<extract output>
{ {
<perform> <extract input>
label3: {
<clean up perform> <extract output>
{
<perform>
label3:
<clean up perform>
}
label2:
if (!failure)
<sync output>
<clean up output>
}
label1:
<clean up input>
} }
label2: return failure;
if (!failure)
<sync output>
<clean up output>
}
label1:
<clean up input>
}
return failure;
}}}
Furthermore, is not necessary to extract the output because we mean to overwrite it anyway. In that case, <extract output> will be a no-op, but of course we may still need to clean up or sync what <perform> will put in the declared outputs. Furthermore, is not necessary to extract the output because we mean to overwrite it anyway. In that case, <extract output> will be a no-op, but of course we may still need to clean up or sync what <perform> will put in the declared outputs.
...@@ -124,20 +122,19 @@ Example ResultBase ...@@ -124,20 +122,19 @@ Example ResultBase
The following ResultBase represents a double (we only care about the C part). The following ResultBase represents a double (we only care about the C part).
{{{ .. code-block::
class Double(ResultBase): class Double(ResultBase):
<snip> <snip>
def c_declare(self): def c_declare(self):
return "double %(name)s;" return "double %(name)s;"
def c_init(self): def c_init(self):
return "%(name)s = 0.0;" return "%(name)s = 0.0;"
def c_extract(self): def c_extract(self):
return "%(name)s = PyFloat_AsDouble(py_%(name)s);" return "%(name)s = PyFloat_AsDouble(py_%(name)s);"
def c_cleanup(self): def c_cleanup(self):
return "" # nothing to do return "" # nothing to do
def c_sync(self): def c_sync(self):
return "Py_XDECREF(py_%(name)s); py_%(name)s = PyFloat_FromDouble(%(name)s);" return "Py_XDECREF(py_%(name)s); py_%(name)s = PyFloat_FromDouble(%(name)s);"
}}}
Example Op Example Op
...@@ -145,71 +142,69 @@ Example Op ...@@ -145,71 +142,69 @@ Example Op
The following ResultBase represents addition of two nonnegative doubles (we only care about the C part). The following ResultBase represents addition of two nonnegative doubles (we only care about the C part).
{{{ .. code-block::
class Add(Op): class Add(Op):
<snip> <snip>
def c_var_names(self): def c_var_names(self):
return "[['x', 'y'], ['z']]" return "[['x', 'y'], ['z']]"
def c_validate_update(self): def c_validate_update(self):
return "if (%(x)s < 0 || %(y)s < 0) %(fail)s" # fail if x or y is negative return "if (%(x)s < 0 || %(y)s < 0) %(fail)s" # fail if x or y is negative
def c_validate_update_cleanup(self): def c_validate_update_cleanup(self):
return "" # nothing to do return "" # nothing to do
def c_code(self): def c_code(self):
return "%(z)s = %(x)s + %(y)s;" return "%(z)s = %(x)s + %(y)s;"
def c_code_cleanup(self): def c_code_cleanup(self):
return "" # nothing to do return "" # nothing to do
}}}
Generating a C function Generating a C function
======================= =======================
For the example Op, the generated C function will typically look like this: For the example Op, the generated C function will typically look like this:
{{{ .. code-block::
void add(PyObject* storage_x, PyObject* storage_y, PyObject* storage_z) { void add(PyObject* storage_x, PyObject* storage_y, PyObject* storage_z) {
PyObject* py_x = PyList_GET_ITEM(storage_x, 0); Py_XINCREF(py_x); // automatic PyObject* py_x = PyList_GET_ITEM(storage_x, 0); Py_XINCREF(py_x); // automatic
PyObject* py_y = PyList_GET_ITEM(storage_y, 0); Py_XINCREF(py_y); // automatic PyObject* py_y = PyList_GET_ITEM(storage_y, 0); Py_XINCREF(py_y); // automatic
PyObject* py_z = Py_None; // we don't care what's currently in storage_z PyObject* py_z = Py_None; // we don't care what's currently in storage_z
failure = 0 failure = 0
double x; // x.c_declare double x; // x.c_declare
double y; // y.c_declare double y; // y.c_declare
double z; // z.c_declare double z; // z.c_declare
{
x = PyFloat_AsDouble(py_x); // x.c_extract
{
y = PyFloat_AsDouble(py_y); // y.c_extract
{ {
# we don't need to use z.c_extract x = PyFloat_AsDouble(py_x); // x.c_extract
{ {
if (x < 0 || y < 0) { // add.validate_update y = PyFloat_AsDouble(py_y); // y.c_extract
// This is automatically inserted in place of %(fail)s
failure = 4;
goto label_add_validate_update_cleanup;
}
{ {
z = x + y; // add.c_code # we don't need to use z.c_extract
label_add_code_cleanup: {
if (x < 0 || y < 0) { // add.validate_update
// This is automatically inserted in place of %(fail)s
failure = 4;
goto label_add_validate_update_cleanup;
}
{
z = x + y; // add.c_code
label_add_code_cleanup:
}
label_add_validate_update_cleanup:
}
label_z_sync_or_cleanup:
if (!failure) {
Py_XDECREF(py_z); // z.c_sync
py_z = PyFloat_FromDouble(z); // z.c_sync, the result is now available from Python!
PyList_SET_ITEM(storage_z, 0, py_z); // always done after _.c_sync
}
Py_XDECREF(py_z); // always done after _.c_cleanup
} }
label_add_validate_update_cleanup: label_y_cleanup:
} Py_XDECREF(py_y); // always done after _.c_cleanup
label_z_sync_or_cleanup:
if (!failure) {
Py_XDECREF(py_z); // z.c_sync
py_z = PyFloat_FromDouble(z); // z.c_sync, the result is now available from Python!
PyList_SET_ITEM(storage_z, 0, py_z); // always done after _.c_sync
} }
Py_XDECREF(py_z); // always done after _.c_cleanup label_x_cleanup:
Py_XDECREF(py_x); // always done after _.c_cleanup
} }
label_y_cleanup: return failure;
Py_XDECREF(py_y); // always done after _.c_cleanup
} }
label_x_cleanup:
Py_XDECREF(py_x); // always done after _.c_cleanup
}
return failure;
}
}}}
Generating a C struct Generating a C struct
===================== =====================
...@@ -218,30 +213,29 @@ To accelerate processing a tad, a struct can be generated instead of a function. ...@@ -218,30 +213,29 @@ To accelerate processing a tad, a struct can be generated instead of a function.
Here is a sketch of the struct equivalent of the previous function: Here is a sketch of the struct equivalent of the previous function:
{{{ .. code-block::
struct add { struct add {
PyObject* storage_x; PyObject* storage_x;
PyObject* storage_y; PyObject* storage_y;
PyObject* storage_z; PyObject* storage_z;
double z; // z.c_declare double z; // z.c_declare
void init(PyObject* storage_x, PyObject* storage_y, PyObject* storage_z) { void init(PyObject* storage_x, PyObject* storage_y, PyObject* storage_z) {
<set the struct members of the same names> <set the struct members of the same names>
<init the struct members corresponding to z> <init the struct members corresponding to z>
} }
void cleanup(void) { void cleanup(void) {
<cleanup z> <cleanup z>
} }
void run(void) { void run(void) {
<same code as before minus z's cleanup> <same code as before minus z's cleanup>
} }
add() { this->init(); } add() { this->init(); }
~add() { this->cleanup(); } ~add() { this->cleanup(); }
}; };
}}}
Advantages of using a struct: Advantages of using a struct:
* Can be run several times even if we provide the storage only once. * Can be run several times even if we provide the storage only once.
......
doc/trac/elemwise_compiler.txt → doc/sandbox/elemwise_compiler.txt
浏览文件 @ d9c7ae08
...@@ -33,27 +33,26 @@ Question: does it make sense to apply the order to the loop, or is this broadcas ...@@ -33,27 +33,26 @@ Question: does it make sense to apply the order to the loop, or is this broadcas
Here is the loop for {{{order == c}}}. Check for errors! Here is the loop for {{{order == c}}}. Check for errors!
{{{ .. code-block::
<initialize iterators> <initialize iterators>
i1 = -1 i1 = -1
while (++i1 < dim1) { while (++i1 < dim1) {
i2 = -1 i2 = -1
rank_N-1_accumulator = init rank_N-1_accumulator = init
while (++i2 < dim2) { while (++i2 < dim2) {
... ...
iN = -1 iN = -1
while (++iN < dimN) { while (++iN < dimN) {
<accumulate rank N input> <accumulate rank N input>
<SET rank N output using broadcasted inputs> <SET rank N output using broadcasted inputs>
<NEXT rank N iterator> <NEXT rank N iterator>
}
...
}
<SET rank 1 output using accumulated inputs>
<NEXT rank 1 iterator>
} }
...
}
<SET rank 1 output using accumulated inputs>
<NEXT rank 1 iterator>
}
}}}
When {{{order == f}}}, the iterators ''ideally'' (but not necessarily) iterate in FORTRAN order, i.e. the while loops are on {{{dimN..dim1}}} instead of {{{dim1..dimN}}}. When {{{order == f}}}, the iterators ''ideally'' (but not necessarily) iterate in FORTRAN order, i.e. the while loops are on {{{dimN..dim1}}} instead of {{{dim1..dimN}}}.
......
doc/sandbox/module.txt
浏览文件 @ d9c7ae08
差异被折叠。
doc/sandbox/module2.txt 0 → 100644
浏览文件 @ d9c7ae08
差异被折叠。
doc/sandbox/numpy.txt deleted 100644 → 0
浏览文件 @ 191a96e7
.. _numpy::
===============
NumPy refresher
===============
Give summary of type(x) vs x.type vs x.dtype
doc/sandbox/thinking_in_theano.txt deleted 100644 → 0
浏览文件 @ 191a96e7
.. thinking_in_theano:
==================
Thinking in Theano
==================
Theano offers quite a bit of flexibility.
How should you write your algorithm to make the most of what Theano can do?
A few tips
----------
- Remember that your code builds a graph that theano compiles, and you cannot
literally put loops into that graph.
- Remember that Variables are symbolic of computations, not
storage. It does not make sense to *reassign* to a Variable.
Limitations
-----------
- Conditional control flow is possible but not efficient. In essence, both
sides of an if (see ``switch``) will be evaluated.
- Loops are not supported, but soon will be.
(A ``scan`` op is in the works, but not included yet.)
- Recursion is not supported within a graph.
A few examples
--------------
**DO WE WANT SOMETHING HERE?**
These are intended to illustrate good ways of formulating an algorithm for
Theano.
For complete, usable implementations of these algorithms see WRITEME.
- Short-time Fourier Transform
- Contrastive Divergence for Restricted Boltzmann Machine
- Kalman Filter
- Logistic Regression
- Training a neural network with sigmoidal hidden layer by backpropagation
- Learning an Echo-State Network
doc/scripts/docgen.py
浏览文件 @ d9c7ae08
...@@ -75,15 +75,17 @@ if __name__ == '__main__': ...@@ -75,15 +75,17 @@ if __name__ == '__main__':
options['--all'] = not (bool(options['--epydoc']) ^ bool(options['--rst'])) options['--all'] = not (bool(options['--epydoc']) ^ bool(options['--rst']))
import gen_oplist if 0:
print 'Generating oplist...' import gen_oplist
gen_oplist.print_file(open('%s/doc/indexes/oplist.txt' % throot, 'w')) print 'Generating oplist...'
print 'oplist done!' gen_oplist.print_file(open('%s/doc/indexes/oplist.txt' % throot, 'w'))
print 'oplist done!'
import gen_typelist
print 'Generating typelist...' if 0:
gen_typelist.print_file(open('%s/doc/indexes/typelist.txt' % throot, 'w')) import gen_typelist
print 'typelist done!' print 'Generating typelist...'
gen_typelist.print_file(open('%s/doc/indexes/typelist.txt' % throot, 'w'))
print 'typelist done!'
def mkdir(path): def mkdir(path):
try: try:
......
doc/topics/debugmode.txt deleted 100644 → 0
浏览文件 @ 191a96e7
.. _debugmode:
===============
Using DebugMode
===============
The DebugMode evaluation mode (available via ``mode='DEBUG_MODE'``,
:api:`DebugMode`) includes a number of self-checks and assertions that
can help to diagnose several kinds of programmer errors that can lead
to incorrect output.
It is much slower to evaluate a function or method in DEBUG_MODE than
it would be in FAST_RUN or even FAST_COMPILE. We recommended you use
DebugMode during development, but not when you launch 1000 processes on
a cluster.
DebugMode is used as follows:
.. code-block:: python
x = theano.dvector('x')
f = theano.function(x, 10*x, mode='DEBUG_MODE')
f(5)
f(0)
f(7)
If any problem is detected, DebugMode will raise an exception according to
what went wrong, either at call time (e.g. ``f(5)``) or compile time (e.g
``f = theano.function(x, 10*x, mode='DEBUG_MODE')``). These exceptions
should *not* be ignored; talk to your local Theano guru or email the
users list if you cannot make the exception go away.
Some kinds of errors can only be detected for certain input value combinations.
In the example above, there is no way to guarantee that a future call to say,
``f(-1)`` won't cause a problem. DebugMode is not a silver bullet.
If you instantiate DebugMode using the constructor ``compile.DebugMode``
rather than the keyword ``DEBUG_MODE`` you can configure its behaviour via
constructor arguments. See :api:`DebugMode` for details.
The keyword version of DebugMode (which you get by using ``mode='DEBUG_MODE``)
is quite strict, and can raise several different Exception types.
There following are DebugMode exceptions you might encounter:
DebugModeError
--------------
This is a generic error. All the other exceptions inherit from this one.
This error is typically not raised directly.
However, you can use ``except DebugModeError: ...`` to catch any of the more
specific types of Exception.
For detailed documentation see :api:`DebugModeError`.
BadCLinkerOutput
----------------
This exception means that python (``perform``) and c (``c_code``) for an Op
didn't compute the same thing like they were supposed to.
The problem might be a bug in either ``perform`` or ``c_code`` (or both).
For detailed documentation see :api:`BadCLinkerOutput`.
BadOptimization
---------------
This exception indicates that an Optimization replaced one variable (say V1)
with another one (say V2) but at runtime, the values for V1 and V2 were
different. This is something that optimizations are not supposed to do.
It can be tricky to identify the one-true-cause of an optimization error, but
this exception provides a lot of guidance. Most of the time, the
exception object will indicate which optimization was at fault.
The exception object also contains information such as a snapshot of the
before/after graph where the optimization introduced the error.
For detailed documentation see :api:`BadOptimization`.
BadDestroyMap
-------------
This happens when an Op's ``perform()`` or ``c_code()`` modifies an input that it wasn't
supposed to. If either the ``perform`` or ``c_code`` implementation of an Op
might modify any input, it has to advertise that fact via the ``destroy_map``
attribute.
For detailed documentation on the Exception, see :api:`BadDestroyMap`.
For detailed documentation on the ``destroy_map`` attribute, see :ref:`inplace`.
BadViewMap
----------
This happens when an Op's perform() or c_code() creates an alias or alias-like
dependency between an input and an output... and it didn't warn the
optimization system via the ``view_map`` attribute.
For detailed documentation on the Exception, see :api:`BadViewMap`.
For detailed documentation on the ``view_map`` attribute, see :ref:`views`.
StochasticOrder
---------------
This happens when an optimization does not perform the same graph operations
in the same order when run several times in a row. This can happen if any
steps are ordered by ``id(object)`` somehow, such as via the default object
hash function. A Stochastic optimization invalidates the pattern of work
whereby we debug in DEBUG_MODE and then run the full-size jobs in FAST_RUN.
For detailed documentation see :api:`StochasticOrder`.
InvalidValueError
-----------------
This happens when some Op's ``perform`` or ``c_code`` implementation computes
an output that is invalid with respect to the type of the corresponding output
variable. Like if it returned a complex-valued ndarray for a ``dscalar``
Type.
This can also be triggered when floating-point values such as NaN and Inf are
introduced into the computations. It indicates which Op created the first
NaN. These floating-point values can be allowed by passing the
``check_isfinite=False`` argument to DebugMode.
For detailed documentation see :api:`InvalidValueError`.
doc/topics/index.txt deleted 100644 → 0
浏览文件 @ 191a96e7
.. _topics:
======
Topics
======
.. toctree::
:maxdepth: 2
function
floatX
module
pipeline
unittest
profilemode
debugmode
debug_faq
randomstreams
doc/basic_tutorial/adding.txt → doc/tutorial/adding.txt
浏览文件 @ d9c7ae08
...@@ -8,8 +8,8 @@ Baby steps - Adding two numbers together ...@@ -8,8 +8,8 @@ Baby steps - Adding two numbers together
Adding two scalars Adding two scalars
================== ==================
So, to get us started and get a feel of what we're working with, let's So, to get us started with Theano and get a feel of what we're working with,
make a simple function: add two numbers together. Here is how you do let's make a simple function: add two numbers together. Here is how you do
it: it:
>>> x = T.dscalar('x') >>> x = T.dscalar('x')
...@@ -26,17 +26,31 @@ array(28.4) ...@@ -26,17 +26,31 @@ array(28.4)
Let's break this down into several steps. The first step is to define Let's break this down into several steps. The first step is to define
two symbols, or Variables, representing the quantities that you want two symbols (*Variables*) representing the quantities that you want
to add. Note that from now on, we will use the term :term:`Variable` to add. Note that from now on, we will use the term
to mean "symbol" (in other words, ``x``, ``y``, ``z`` are all Variable *Variable* to mean "symbol" (in other words,
objects). The output of the function ``f`` is a ``numpy.ndarray`` ``x``, ``y``, ``z`` are all *Variable* objects). The output of the function
with zero dimensions. ``f`` is a ``numpy.ndarray`` with zero dimensions.
If you are following along and typing into an interpreter, you may have If you are following along and typing into an interpreter, you may have
noticed that there was a slight delay in executing the ``function`` noticed that there was a slight delay in executing the ``function``
instruction. Behind the scenes, ``f`` was being compiled into C code. instruction. Behind the scenes, ``f`` was being compiled into C code.
.. TODO: help
.. note:
A *Variable* is the main data structure you work with when
using Theano. The symbolic inputs that you operate on are
*Variables* and what you get from applying various operations to
these inputs are also *Variables*. For example, when I type
>>> x = theano.tensor.ivector()
>>> y = -x
``x`` and ``y`` are both Variables, i.e. instances of the
``theano.gof.graph.Variable`` class. The
type of both ``x`` and ``y`` is ``theano.tensor.ivector``.
------------------------------------------- -------------------------------------------
...@@ -47,11 +61,11 @@ instruction. Behind the scenes, ``f`` was being compiled into C code. ...@@ -47,11 +61,11 @@ instruction. Behind the scenes, ``f`` was being compiled into C code.
In Theano, all symbols must be typed. In particular, ``T.dscalar`` In Theano, all symbols must be typed. In particular, ``T.dscalar``
is the type we assign to "0-dimensional arrays (`scalar`) of doubles is the type we assign to "0-dimensional arrays (`scalar`) of doubles
(`d`)". It is a Theano :term:`Type`. (`d`)". It is a Theano :ref:`type`.
``dscalar`` is not a class. Therefore, neither ``x`` nor ``y`` ``dscalar`` is not a class. Therefore, neither ``x`` nor ``y``
are actually instances of ``dscalar``. They are instances of are actually instances of ``dscalar``. They are instances of
:api:`TensorVariable <theano.tensor.basic.TensorVariable>`. ``x`` and ``y`` :ref:`TensorVariable <libdoc_tensor_type>`. ``x`` and ``y``
are, however, assigned the theano Type ``dscalar`` in their ``type`` are, however, assigned the theano Type ``dscalar`` in their ``type``
field, as you can see here: field, as you can see here:
...@@ -64,14 +78,14 @@ TensorType(float64, scalar) ...@@ -64,14 +78,14 @@ TensorType(float64, scalar)
>>> x.type == T.dscalar >>> x.type == T.dscalar
True True
You can learn more about the structures in Theano in You can learn more about the structures in Theano in :ref:`graphstructures`.
the :ref:`advtutorial` and in :ref:`graphstructures`.
By calling ``T.dscalar`` with a string argument, you create a By calling ``T.dscalar`` with a string argument, you create a
:term:`Variable` representing a floating-point scalar quantity with the *Variable* representing a floating-point scalar quantity with the
given name. If you provide no argument, the symbol will be unnamed. Names given name. If you provide no argument, the symbol will be unnamed. Names
are not required, but they can help debugging. are not required, but they can help debugging.
------------------------------------------- -------------------------------------------
**Step 2** **Step 2**
...@@ -80,8 +94,8 @@ The second step is to combine ``x`` and ``y`` into their sum ``z``: ...@@ -80,8 +94,8 @@ The second step is to combine ``x`` and ``y`` into their sum ``z``:
>>> z = x + y >>> z = x + y
``z`` is yet another :term:`Variable` which represents the addition of ``z`` is yet another *Variable* which represents the addition of
``x`` and ``y``. You can use the :api:`pp <theano.printing.pp>` ``x`` and ``y``. You can use the :ref:`pp <libdoc_printing>`
function to pretty-print out the computation associated to ``z``. function to pretty-print out the computation associated to ``z``.
>>> print pp(z) >>> print pp(z)
...@@ -96,7 +110,7 @@ and giving ``z`` as output: ...@@ -96,7 +110,7 @@ and giving ``z`` as output:
>>> f = function([x, y], z) >>> f = function([x, y], z)
The first argument to ``function`` is a list of :term:`Variables <Variable>` The first argument to :ref:`function <libdoc_compile_function>` is a list of Variables
that will be provided as inputs to the function. The second argument that will be provided as inputs to the function. The second argument
is a single Variable *or* a list of Variables. For either case, the second is a single Variable *or* a list of Variables. For either case, the second
argument is what we want to see as output when we apply the function. argument is what we want to see as output when we apply the function.
...@@ -133,18 +147,19 @@ array([[ 11., 22.], ...@@ -133,18 +147,19 @@ array([[ 11., 22.],
It is possible to add scalars to matrices, vectors to matrices, It is possible to add scalars to matrices, vectors to matrices,
scalars to vectors, etc. The behavior of these operations is defined scalars to vectors, etc. The behavior of these operations is defined
by :term:`broadcasting`. by :ref:`broadcasting <libdoc_tensor_broadcastable>`.
The following types are available: The following types are available:
* **byte**: bscalar, bvector, bmatrix * **byte**: bscalar, bvector, bmatrix, brow, bcol, btensor3, btensor4
* **32-bit integers**: iscalar, ivector, imatrix * **32-bit integers**: iscalar, ivector, imatrix, irow, icol, itensor3, itensor4
* **64-bit integers**: lscalar, lvector, lmatrix * **64-bit integers**: lscalar, lvector, lmatrix, lrow, lcol, ltensor3, ltensor4
* **float**: fscalar, fvector, fmatrix * **float**: fscalar, fvector, fmatrix, frow, fcol, ftensor3, ftensor4
* **double**: dscalar, dvector, dmatrix * **double**: dscalar, dvector, dmatrix, drow, dcol, dtensor3, dtensor4
* **complex**: cscalar, cvector, cmatrix, crow, ccol, ctensor3, ctensor4
The previous list is not exhaustive. A guide to all types compatible The previous list is not exhaustive. A guide to all types compatible
with numpy arrays may be found :ref:`here <predefinedtypes>`. with numpy arrays may be found :ref:`here <libdoc_tensor_creation>`.
.. note:: .. note::
......
doc/tutorial/debug_faq.txt 0 → 100644
浏览文件 @ d9c7ae08
.. _debug_faq:
=========================================
Debugging Theano: FAQ and Troubleshooting
=========================================
There are many kinds of bugs that might come up in a computer program.
This page is structured as an FAQ. It should provide recipes to tackle common
problems, and introduce some of the tools that we use to find problems in our
Theano code, and even (it happens) in Theano's internals, such as
:ref:`using_debugmode`.
How do I print an intermediate value in a Function/Method?
----------------------------------------------------------
Theano provides a 'Print' Op to do this.
.. code-block:: python
x = theano.tensor.dvector('x')
x_printed = theano.printing.Print('this is a very important value')(x)
f = theano.function([x], x * 5)
f_with_print = theano.function([x], x_printed * 5)
#this runs the graph without any printing
assert numpy.all( f([1,2,3]) == [5, 10, 15])
#this runs the graph with the message, and value printed
assert numpy.all( f_with_print([1,2,3]) == [5, 10, 15])
Since Theano runs your program in a topological order, you won't have precise
control over the order in which multiple Print() Ops are evaluted. For a more
precise inspection of what's being computed where, when, and how, see the
:ref:`faq_wraplinker`.
The function I compiled is too slow, what's up?
-----------------------------------------------
First, make sure you're running in FAST_RUN mode, by passing
``mode='FAST_RUN'`` to ``theano.function`` or ``theano.make``. Some
operations have excruciatingly slow Python implementations and that
can negatively effect the performance of FAST_COMPILE.
Second, try the theano :ref:`using_profilemode`. This will tell you which
Apply nodes, and which Ops are eating up your CPU cycles.
.. _faq_wraplinker:
How do I step through a compiled function with the WrapLinker?
--------------------------------------------------------------
This is not exactly an FAQ, but the doc is here for now...
It's pretty easy to roll-your-own evaluation mode.
Check out this one:
.. code-block:: python
class PrintEverythingMode(Mode):
def __init__(self):
def print_eval(i, node, fn):
print i, node, [input[0] for input in fn.inputs],
fn()
print [output[0] for output in fn.outputs]
wrap_linker = theano.gof.WrapLinkerMany([theano.gof.OpWiseCLinker()], [print_eval])
super(PrintEverythingMode, self).__init__(wrap_linker, optimizer='fast_run')
When you use ``mode=PrintEverythingMode()`` as the mode for Function or Method,
then you should see [potentially a lot of] output. Every Apply node will be printed out,
along with its position in the graph, the arguments to the ``perform`` or
``c_code`` and the output it computed.
>>> x = T.dscalar('x')
>>> f = function([x], [5*x], mode=PrintEverythingMode())
>>> f(3)
>>> # print: 0 Elemwise{mul,no_inplace}(5, x) [array(5, dtype=int8), array(3.0)] [array(15.0)]
>>> # print: [array(15.0)]
Admittedly, this may be a huge amount of
output to read through if you are using big tensors... but you can choose to
put logic inside of the print_eval function that would, for example, only
print something out if a certain kind of Op was used, at a certain program
position, or if a particular value shows up in one of the inputs or outputs.
Use your imagination :)
.. TODO: documentation for link.WrapLinkerMany
This can be a really powerful debugging tool.
Note the call to ``fn`` inside the call to ``print_eval``; without it, the graph wouldn't get computed at all!
doc/basic_tutorial/examples.txt → doc/tutorial/examples.txt
浏览文件 @ d9c7ae08
...@@ -22,7 +22,7 @@ the logistic curve, which is given by: ...@@ -22,7 +22,7 @@ the logistic curve, which is given by:
A plot of the logistic function, with x on the x-axis and s(x) on the A plot of the logistic function, with x on the x-axis and s(x) on the
y-axis. y-axis.
You want to compute the function :term:`elementwise` on matrices of You want to compute the function :ref:`elementwise <libdoc_tensor_elementwise>` on matrices of
doubles, which means that you want to apply this function to each doubles, which means that you want to apply this function to each
individual element of the matrix. individual element of the matrix.
...@@ -58,7 +58,7 @@ Computing more than one thing at the same time ...@@ -58,7 +58,7 @@ Computing more than one thing at the same time
============================================== ==============================================
Theano supports functions with multiple outputs. For example, we can Theano supports functions with multiple outputs. For example, we can
compute the :term:`elementwise` difference, absolute difference, and compute the :ref:`elementwise <libdoc_tensor_elementwise>` difference, absolute difference, and
squared difference between two matrices ``a`` and ``b`` at the same time: squared difference between two matrices ``a`` and ``b`` at the same time:
>>> a, b = T.dmatrices('a', 'b') >>> a, b = T.dmatrices('a', 'b')
...@@ -134,16 +134,17 @@ array([[ 0.25 , 0.19661193], ...@@ -134,16 +134,17 @@ array([[ 0.25 , 0.19661193],
The resulting function computes the gradient of its first argument The resulting function computes the gradient of its first argument
with respect to the second. In this way, Theano can be used for with respect to the second. In this way, Theano can be used for
`automatic differentiation`_. `automatic differentiation <http://en.wikipedia.org/wiki/Automatic_differentiation>`_.
.. note:: .. note::
The variable of ``T.grad`` has the same dimensions as the The second argument of ``T.grad`` can be a list, in which case the
second argument. This is exactly like the first derivative if the output is also a list. The order in both list is important, element
first argument is a scalar or a tensor of size 1 but not if it is *i* of the output list is the gradient of the first argument of
larger. For more information on the semantics when the first ``T.grad`` with respect to the *i*-th element of the list given as second argument.
argument has a larger size and details about the implementation, The first arguement of ``T.grad`` has to be a scalar (a tensor
see :api:`tensor.grad`. of size 1). For more information on the semantics of the arguments of
``T.grad`` and details about the implementation, see :ref:`this <libdoc_gradient>`.
Setting a default value for an argument Setting a default value for an argument
...@@ -273,9 +274,10 @@ shared variable, but you do *not* want to use its value. In this case, you can u ...@@ -273,9 +274,10 @@ shared variable, but you do *not* want to use its value. In this case, you can u
for the purpose of one particular function. for the purpose of one particular function.
>>> fn_of_state = state * 2 + inc >>> fn_of_state = state * 2 + inc
>>> non_shared_state = state.type() >>> foo = lscalar() # the type (lscalar) must match the shared variable we
>>> skip_shared = function([inc, non_shared_state], fn_of_state, >>> # are replacing with the ``givens`` list
givens=[(state, non_shared_state)]) >>> skip_shared = function([inc, foo], fn_of_state,
givens=[(state, foo)])
>>> skip_shared(1, 3) # we're using 3 for the state, not state.value >>> skip_shared(1, 3) # we're using 3 for the state, not state.value
array(7) array(7)
>>> state.value # old state still there, but we didn't use it >>> state.value # old state still there, but we didn't use it
...@@ -288,31 +290,5 @@ substitution to be co-dependent, the order of substitution is not defined, so ...@@ -288,31 +290,5 @@ substitution to be co-dependent, the order of substitution is not defined, so
the substitutions have to work in any order. the substitutions have to work in any order.
Mode
====
The ``mode`` parameter to :api:`theano.function` controls how the
inputs-to-outputs graph is transformed into a callable object.
Theano defines the following modes by name:
- ``FAST_COMPILE``: Apply just a few optimizations, but use C op implementations where possible.
- ``FAST_RUN``: Apply all optimizations, and use C op implementations where possible.
- ``DEBUG_MODE``: Verify the correctness of all optimizations, and compare C and python
implementations. This mode can take much longer than the other modes,
but can identify many kinds of problems.
The default mode is typically ``FAST_RUN``, but it can be controlled via
the environment variable ``THEANO_DEFAULT_MODE``, which can in turn be
overridden by setting :api:`theano.compile.mode.default_mode` directly,
which can in turn be overridden by passing the keyword argument to
:api:`theano.function`.
For a finer level of control over which optimizations are applied, and
whether C or python implementations are used, read
:api:`compile.mode.Mode`.
.. _automatic differentiation: http://en.wikipedia.org/wiki/Automatic_differentiation
doc/tutorial/index.txt 0 → 100644
浏览文件 @ d9c7ae08
.. _tutorial:
========
Tutorial
========
Let's start an interactive session and import Theano.
>>> from theano import *
Many of symbols you will need to use are in the ``tensor`` subpackage
of Theano. Let's import that subpackage under a handy name. I like
``T`` (and many tutorials use this convention).
>>> import theano.tensor as T
If that worked you're ready for the tutorial, otherwise check your
installation (see :ref:`install`).
.. toctree::
numpy
adding
examples
loading_and_saving
modes
remarks
debug_faq
doc/tutorial/loading_and_saving.txt 0 → 100644
浏览文件 @ d9c7ae08
.. tutorial_loadsave:
==================
Loading and Saving
==================
Many Theano objects can be serialized. However, you will want to consider different mechanisms
depending on the amount of time you anticipate between saving and reloading. For short-term
(such as temp files and network transfers) pickling is possible. For longer-term (such as
saving models from an experiment) you should not rely on pickled theano objects; we recommend
loading and saving the underlying shared objects as you would in the course of any other python
program.
pickling -- Short-term serialization
=====================================
Pickling and unpickling of functions. Caveats... basically don't do this for long-term storage.
***TODO***
not-pickling -- Long-term serialization
=======================================
***TODO***
Give a short example of how to add a __getstate__ and __setstate__ to a class. Point out to
use protocol=-1 for numpy ndarrays.
Point to the python docs for further reading.
doc/tutorial/modes.txt 0 → 100644
浏览文件 @ d9c7ae08
.. _using_modes:
===============================
Using different compiling modes
===============================
Mode
====
Everytime :ref:`theano.function <libdoc_compile_function>` is called
the symbolic relationships between the input and output Theano *variables*
are optimized and compiled. The way this compilation occurs
is controlled by the value of the ``mode`` parameter.
Theano defines the following modes by name:
- ``'FAST_COMPILE'``: Apply just a few graph optimizations, but use C implementations where possible.
- ``'FAST_RUN'``: Apply all optimizations, and use C implementations where possible.
- ``'DEBUG_MODE'``: Verify the correctness of all optimizations, and compare C and python
implementations. This mode can take much longer than the other modes,
but can identify many kinds of problems.
The default mode is typically ``FAST_RUN``, but it can be controlled via
the environment variable ``THEANO_DEFAULT_MODE``, which can in turn be
overridden by setting `theano.compile.mode.default_mode` directly,
which can in turn be overridden by passing the keyword argument to
:ref:`theano.function <libdoc_compile_function>`.
================= =============================================================== ===============================================================================
short name Full constructor What does it do?
================= =============================================================== ===============================================================================
(default) ``compile.mode.Mode(linker='py', optimizer=None)`` Python implementations with zero graph modifications.
FAST_COMPILE ``compile.mode.Mode(linker='c|py', optimizer='fast_compile')`` C implementations where available, quick and cheap graph transformations
FAST_RUN ``compile.mode.Mode(linker='c|py', optimizer='fast_run')`` C implementations where available, all available graph transformations.
DEBUG_MODE ``compile.debugmode.DebugMode()`` Both implementations where available, all available graph transformations.
================= =============================================================== ===============================================================================
.. _using_debugmode:
Using DebugMode
===============
While normally you should use the ``FAST_RUN`` or ``FAST_COMPILE`` mode,
it is useful at first (especially when you are defining new kinds of
expressions or new optimizations) to run your code using the DebugMode
(available via ``mode='DEBUG_MODE'``). The DebugMode is designed to
do several self-checks and assertations that can help to diagnose
possible programming errors that can lead to incorect output. Note that
``DEBUG_MODE`` is much slower then ``FAST_RUN`` or ``FAST_COMPILE`` so
use it only during development (not when you luch 1000 process on a
cluster!).
DebugMode is used as follows:
.. code-block:: python
x = T.dvector('x')
f = theano.function([x], 10*x, mode='DEBUG_MODE')
f(5)
f(0)
f(7)
If any problem is detected, DebugMode will raise an exception according to
what went wrong, either at call time (``f(5)``) or compile time (
``f = theano.function(x, 10*x, mode='DEBUG_MODE')``). These exceptions
should *not* be ignored; talk to your local Theano guru or email the
users list if you cannot make the exception go away.
Some kinds of errors can only be detected for certain input value combinations.
In the example above, there is no way to guarantee that a future call to say,
``f(-1)`` won't cause a problem. DebugMode is not a silver bullet.
If you instantiate DebugMode using the constructor (see :class:`DebugMode`)
rather than the keyword ``DEBUG_MODE`` you can configure its behaviour via
constructor arguments. See :ref:`DebugMode <debugMode>` for details.
The keyword version of DebugMode (which you get by using ``mode='DEBUG_MODE``)
is quite strict.
.. _using_profilemode:
ProfileMode
===========
Beside checking for errors, another important task is to profile your
code. For this Theano uses a special mode called ProfileMode which has
to be passed as an argument to :ref:`theano.function <libdoc_compile_function>`. Using the ProfileMode is a three-step process.
Creating a ProfileMode Instance
-------------------------------
First create a ProfileMode instance.
>>> from theano import ProfileMode
>>> profmode = theano.ProfileMode(optimizer='fast_run', linker=theano.gof.OpWiseCLinker())
The ProfileMode constructor takes as input an optimizer and a
linker. Which optimizer and linker to use will depend on the
application. For example, a user wanting to profile the Python
implementation only, should use the gof.PerformLinker (or "py" for
short). On the other hand, a user wanting to profile his graph using C
implementations wherever possible should use the ``gof.OpWiseCLinker``
(or "c|py"). For testing the speed of your code we would recommend
using the 'fast_run' optimizer and ``gof.OpWiseCLinker`` linker.
Compiling your Graph with ProfileMode
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Once the ProfileMode instance is created, simply compile your graph as you
would normally, by specifying the mode parameter.
>>> # with functions
>>> f = theano.function([input1,input2],[output1], mode=profmode)
>>> # with modules
>>> m = theano.Module()
>>> minst = m.make(mode=profmode)
Retrieving Timing Information
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Once your graph is compiled, simply run the program or operation you wish to
profile, then call ``profmode.print_summary()``. This will provide you with
the desired timing information, indicating where your graph is spending most
of its time.
This is best shown through an example.
Lets use the example of logistic
regression. (Code for this example is in the file
``benchmark/regression/regression.py``.)
Compiling the module with ProfileMode and calling ``profmode.print_summary()``
generates the following output:
.. code-block:: python
"""
ProfileMode.print_summary()
---------------------------
local_time 0.0749197006226 (Time spent running thunks)
Apply-wise summary: <fraction of local_time spent at this position> (<Apply position>, <Apply Op name>)
0.069 15 _dot22
0.064 1 _dot22
0.053 0 InplaceDimShuffle{x,0}
0.049 2 InplaceDimShuffle{1,0}
0.049 10 mul
0.049 6 Elemwise{ScalarSigmoid{output_types_preference=<theano.scalar.basic.transfer_type object at 0x171e650>}}[(0, 0)]
0.049 3 InplaceDimShuffle{x}
0.049 4 InplaceDimShuffle{x,x}
0.048 14 Sum{0}
0.047 7 sub
0.046 17 mul
0.045 9 sqr
0.045 8 Elemwise{sub}
0.045 16 Sum
0.044 18 mul
... (remaining 6 Apply instances account for 0.25 of the runtime)
Op-wise summary: <fraction of local_time spent on this kind of Op> <Op name>
0.139 * mul
0.134 * _dot22
0.092 * sub
0.085 * Elemwise{Sub{output_types_preference=<theano.scalar.basic.transfer_type object at 0x1779f10>}}[(0, 0)]
0.053 * InplaceDimShuffle{x,0}
0.049 * InplaceDimShuffle{1,0}
0.049 * Elemwise{ScalarSigmoid{output_types_preference=<theano.scalar.basic.transfer_type object at 0x171e650>}}[(0, 0)]
0.049 * InplaceDimShuffle{x}
0.049 * InplaceDimShuffle{x,x}
0.048 * Sum{0}
0.045 * sqr
0.045 * Sum
0.043 * Sum{1}
0.042 * Elemwise{Mul{output_types_preference=<theano.scalar.basic.transfer_type object at 0x17a0f50>}}[(0, 1)]
0.041 * Elemwise{Add{output_types_preference=<theano.scalar.basic.transfer_type object at 0x1736a50>}}[(0, 0)]
0.039 * Elemwise{Second{output_types_preference=<theano.scalar.basic.transfer_type object at 0x1736d90>}}[(0, 1)]
... (remaining 0 Ops account for 0.00 of the runtime)
(*) Op is running a c implementation
"""
The summary has two components to it. In the first section called the
Apply-wise summary, timing information is provided for the worst
offending Apply nodes. This corresponds to individual Op applications
within your graph which take the longest to execute (so if you use
``dot`` twice, you will see two entries there). In the second portion,
the Op-wise summary, the execution time of all Apply nodes executing
the same Op are grouped together and the total execution time per Op
is shown (so if you use ``dot`` twice, you will see only one entry
there corresponding to the sum of the time spent in each of them).
Note that the ProfileMode also shows which Ops were running a c
implementation.
doc/tutorial/remarks.txt 0 → 100644
浏览文件 @ d9c7ae08
差异被折叠。
theano/compile/function.py
浏览文件 @ d9c7ae08
差异被折叠。
theano/compile/pfunc.py
浏览文件 @ d9c7ae08
差异被折叠。
theano/floatx.py
浏览文件 @ d9c7ae08
差异被折叠。
theano/gradient.py
浏览文件 @ d9c7ae08
差异被折叠。
theano/sandbox/scan.py
浏览文件 @ d9c7ae08
差异被折叠。
theano/sandbox/solve.py 0 → 100644
浏览文件 @ d9c7ae08
差异被折叠。
theano/tensor/basic.py
浏览文件 @ d9c7ae08
差异被折叠。
theano/tensor/nnet.py
浏览文件 @ d9c7ae08
差异被折叠。
Markdown 格式
0% 或
您添加了 0 人 到此讨论。请谨慎行事。
请先完成此评论的编辑！
请 注册 或者 后发表评论