Skip to content

P
pytensor
提交 e8494632 authored 作者: Christof Angermueller's avatar Christof Angermueller
浏览文件
操作

Update printing and plotting sections in tutorials

上级 2907f95a
隐藏空白字符变更
内嵌 并排
正在显示 包含 259 行增加 和 255 行删除
doc/cifarSC2011/advanced_theano.txt
浏览文件 @ e8494632
......@@ -307,14 +307,17 @@ Consider the following logistic regression model:
We will now make use of Theano's printing features to compare the unoptimized
graph (``prediction``) to the optimized graph (``predict``).
- Pretty Printing
Pretty Printing
~~~~~~~~~~~~~~~
>>> theano.printing.pprint(prediction) # doctest: +NORMALIZE_WHITESPACE
'gt((TensorConstant{1} / (TensorConstant{1} + exp(((-(x \\dot w)) - b)))),
TensorConstant{0.5})'
- Debug Print
Debug Print
~~~~~~~~~~~
The graph before optimization:
......@@ -355,7 +358,8 @@ The graph after optimization:
|TensorConstant{(1,) of 0.5} [@K]
- Picture Printing of Graphs
Picture Printing of Graphs
~~~~~~~~~~~~~~~~~~~~~~~~~~
``pydotprint`` requires graphviz and pydot.
The graph before optimization:
......
doc/crei2013/advanced_theano.txt
浏览文件 @ e8494632
......@@ -53,7 +53,7 @@ Conditions
IfElse Op spend less time (about an half) than Switch since it computes only
one variable instead of both.
>>> python ifelse_switch.py
>>> python ifelse_switch.py # doctest: +SKIP
time spent evaluating both values 0.230000 sec
time spent evaluating one value 0.120000 sec
......@@ -108,68 +108,123 @@ Exercise 5
- Is there something we can do to speed up the GPU version?
Printing/Drawing Theano graphs
------------------------------
- Pretty Printing
``theano.printing.pprint(variable)``
>>> theano.printing.pprint(prediction)
gt((TensorConstant{1} / (TensorConstant{1} + exp(((-(x \\dot w)) - b)))),TensorConstant{0.5})
- Debug Print
``theano.printing.debugprint({fct, variable, list of variables})``
>>> theano.printing.debugprint(prediction)
Elemwise{gt,no_inplace} [@181772236] ''
|Elemwise{true_div,no_inplace} [@181746668] ''
| |InplaceDimShuffle{x} [@181746412] ''
| | |TensorConstant{1} [@181745836]
| |Elemwise{add,no_inplace} [@181745644] ''
| | |InplaceDimShuffle{x} [@181745420] ''
| | | |TensorConstant{1} [@181744844]
| | |Elemwise{exp,no_inplace} [@181744652] ''
| | | |Elemwise{sub,no_inplace} [@181744012] ''
| | | | |Elemwise{neg,no_inplace} [@181730764] ''
| | | | | |dot [@181729676] ''
| | | | | | |x [@181563948]
| | | | | | |w [@181729964]
| | | | |InplaceDimShuffle{x} [@181743788] ''
| | | | | |b [@181730156]
|InplaceDimShuffle{x} [@181771788] ''
| |TensorConstant{0.5} [@181771148]
>>> theano.printing.debugprint(predict)
Elemwise{Composite{neg,{sub,{{scalar_sigmoid,GT},neg}}}} [@183160204] '' 2
|dot [@183018796] '' 1
| |x [@183000780]
| |w [@183000812]
|InplaceDimShuffle{x} [@183133580] '' 0
| |b [@183000876]
|TensorConstant{[ 0.5]} [@183084108]
- Picture Printing of Graphs
>>> theano.printing.pydotprint_variables(prediction)
.. image:: ../hpcs2011_tutorial/pics/logreg_pydotprint_prediction.png
Consider the following logistic regression model:
>>> import numpy
>>> import theano
>>> import theano.tensor as T
>>> rng = numpy.random
>>> # Training data
>>> N = 400
>>> feats = 784
>>> D = (rng.randn(N, feats).astype(theano.config.floatX), rng.randint(size=N,low=0, high=2).astype(theano.config.floatX))
>>> training_steps = 10000
>>> # Declare Theano symbolic variables
>>> x = T.matrix("x")
>>> y = T.vector("y")
>>> w = theano.shared(rng.randn(feats).astype(theano.config.floatX), name="w")
>>> b = theano.shared(numpy.asarray(0., dtype=theano.config.floatX), name="b")
>>> x.tag.test_value = D[0]
>>> y.tag.test_value = D[1]
>>> # Construct Theano expression graph
>>> p_1 = 1 / (1 + T.exp(-T.dot(x, w)-b)) # Probability of having a one
>>> prediction = p_1 > 0.5 # The prediction that is done: 0 or 1
>>> # Compute gradients
>>> xent = -y*T.log(p_1) - (1-y)*T.log(1-p_1) # Cross-entropy
>>> cost = xent.mean() + 0.01*(w**2).sum() # The cost to optimize
>>> gw,gb = T.grad(cost, [w,b])
>>> # Training and prediction function
>>> train = theano.function(inputs=[x,y], outputs=[prediction, xent], updates=[[w, w-0.01*gw], [b, b-0.01*gb]], name = "train")
>>> predict = theano.function(inputs=[x], outputs=prediction, name = "predict")
We will now make use of Theano's printing features to compare the unoptimized
graph (``prediction``) to the optimized graph (``predict``).
Pretty Printing
~~~~~~~~~~~~~~~
>>> theano.printing.pprint(prediction) # doctest: +NORMALIZE_WHITESPACE
'gt((TensorConstant{1} / (TensorConstant{1} + exp(((-(x \\dot w)) - b)))),
TensorConstant{0.5})'
Debug Print
~~~~~~~~~~~
The graph before optimization:
>>> theano.printing.debugprint(prediction) # doctest: +NORMALIZE_WHITESPACE
Elemwise{gt,no_inplace} [@A] ''
|Elemwise{true_div,no_inplace} [@B] ''
| |DimShuffle{x} [@C] ''
| | |TensorConstant{1} [@D]
| |Elemwise{add,no_inplace} [@E] ''
| |DimShuffle{x} [@F] ''
| | |TensorConstant{1} [@D]
| |Elemwise{exp,no_inplace} [@G] ''
| |Elemwise{sub,no_inplace} [@H] ''
| |Elemwise{neg,no_inplace} [@I] ''
| | |dot [@J] ''
| | |x [@K]
| | |w [@L]
| |DimShuffle{x} [@M] ''
| |b [@N]
|DimShuffle{x} [@O] ''
|TensorConstant{0.5} [@P]
The graph after optimization:
>>> theano.printing.debugprint(predict) # doctest: +NORMALIZE_WHITESPACE
Elemwise{Composite{GT(scalar_sigmoid((-((-i0) - i1))), i2)}} [@A] '' 4
|CGemv{inplace} [@B] '' 3
| |Alloc [@C] '' 2
| | |TensorConstant{0.0} [@D]
| | |Shape_i{0} [@E] '' 1
| | |x [@F]
| |TensorConstant{1.0} [@G]
| |x [@F]
| |w [@H]
| |TensorConstant{0.0} [@D]
|InplaceDimShuffle{x} [@I] '' 0
| |b [@J]
|TensorConstant{(1,) of 0.5} [@K]
Picture Printing of Graphs
~~~~~~~~~~~~~~~~~~~~~~~~~~
``pydotprint`` requires graphviz and pydot.
The graph before optimization:
>>> theano.printing.pydotprint(prediction, outfile="pics/logreg_pydotprint_prediction.png", var_with_name_simple=True)
The output file is available at pics/logreg_pydotprint_prediction.png
.. image:: ./pics/logreg_pydotprint_prediction.png
:width: 800 px
All pydotprint* requires graphviz and pydot
The graph after optimization:
>>> theano.printing.pydotprint(predict)
>>> theano.printing.pydotprint(predict, outfile="pics/logreg_pydotprint_predict.png", var_with_name_simple=True)
The output file is available at pics/logreg_pydotprint_predict.png
.. image:: ../hpcs2011_tutorial/pics/logreg_pydotprint_predic.png
.. image:: ./pics/logreg_pydotprint_predict.png
:width: 800 px
>>> theano.printing.pydotprint(train) # This is a small train example!
The optimized training graph:
>>> theano.printing.pydotprint(train, outfile="pics/logreg_pydotprint_train.png", var_with_name_simple=True)
The output file is available at pics/logreg_pydotprint_train.png
.. image:: ../hpcs2011_tutorial/pics/logreg_pydotprint_train.png
.. image:: ./pics/logreg_pydotprint_train.png
:width: 1500 px
Debugging
---------
......
doc/tutorial/printing_drawing.txt
浏览文件 @ e8494632
......@@ -5,14 +5,13 @@
Printing/Drawing Theano graphs
==============================
.. TODO: repair the defective links in the next paragraph
Theano provides two functions (:func:`theano.pp` and
:func:`theano.printing.debugprint`) to print a graph to the terminal before or after
compilation. These two functions print expression graphs in different ways:
:func:`pp` is more compact and math-like, :func:`debugprint` is more verbose.
Theano also provides :func:`pydotprint` that creates a *png* image of the function.
You can read about them in :ref:`libdoc_printing`.
Theano provides the functions :func:`theano.printing.pprint` and
:func:`theano.printing.debugprint` to print a graph to the terminal before or
after compilation. :func:`pprint` is more compact and math-like,
:func:`debugprint` is more verbose. Theano also provides :func:`pydotprint`
that creates an image of the function. You can read about them in
:ref:`libdoc_printing`.
.. note::
......@@ -24,151 +23,109 @@ You can read about them in :ref:`libdoc_printing`.
real job execution, as this will make the graph slower and use more
memory.
Consider again the logistic regression but notice the additional printing instuctions.
The following output depicts the pre- and post- compilation graphs.
.. code-block:: python
import theano
import theano.tensor as T
import numpy
import os
rng = numpy.random
N = 400
feats = 784
D = (rng.randn(N, feats).astype(theano.config.floatX),
rng.randint(size=N,low=0, high=2).astype(theano.config.floatX))
training_steps = 10000
# Declare Theano symbolic variables
x = T.matrix("x")
y = T.vector("y")
w = theano.shared(rng.randn(feats).astype(theano.config.floatX), name="w")
b = theano.shared(numpy.asarray(0., dtype=theano.config.floatX), name="b")
x.tag.test_value = D[0]
y.tag.test_value = D[1]
#print "Initial model:"
#print w.get_value(), b.get_value()
# Construct Theano expression graph
p_1 = 1 / (1 + T.exp(-T.dot(x, w) - b)) # Probability of having a one
prediction = p_1 > 0.5 # The prediction that is done: 0 or 1
xent = -y * T.log(p_1) - (1 - y) * T.log(1 - p_1) # Cross-entropy
cost = xent.mean() + 0.01 * (w ** 2).sum() # The cost to optimize
gw,gb = T.grad(cost, [w, b])
# Compile expressions to functions
train = theano.function(
inputs=[x, y],
outputs=[prediction, xent],
updates=[(w, w - 0.01 * gw), (b, b - 0.01 * gb)],
name="train")
predict = theano.function(inputs=[x], outputs=prediction,
name="predict")
if any([x.op.__class__.__name__ in ['Gemv', 'CGemv'] for x in
train.maker.fgraph.toposort()]):
print 'Used the cpu'
elif any([x.op.__class__.__name__ == 'GpuGemv' for x in
train.maker.fgraph.toposort()]):
print 'Used the gpu'
else:
print 'ERROR, not able to tell if theano used the cpu or the gpu'
print train.maker.fgraph.toposort()
for i in range(training_steps):
pred, err = train(D[0], D[1])
#print "Final model:"
#print w.get_value(), b.get_value()
print "target values for D"
print D[1]
print "prediction on D"
print predict(D[0])
# Print the picture graphs
# after compilation
if not os.path.exists('pics'):
os.mkdir('pics')
theano.printing.pydotprint(predict,
outfile="pics/logreg_pydotprint_predic.png",
var_with_name_simple=True)
# before compilation
theano.printing.pydotprint_variables(prediction,
outfile="pics/logreg_pydotprint_prediction.png",
var_with_name_simple=True)
theano.printing.pydotprint(train,
outfile="pics/logreg_pydotprint_train.png",
var_with_name_simple=True)
Consider again the logistic regression example:
>>> import numpy
>>> import theano
>>> import theano.tensor as T
>>> rng = numpy.random
>>> # Training data
>>> N = 400
>>> feats = 784
>>> D = (rng.randn(N, feats).astype(theano.config.floatX), rng.randint(size=N,low=0, high=2).astype(theano.config.floatX))
>>> training_steps = 10000
>>> # Declare Theano symbolic variables
>>> x = T.matrix("x")
>>> y = T.vector("y")
>>> w = theano.shared(rng.randn(feats).astype(theano.config.floatX), name="w")
>>> b = theano.shared(numpy.asarray(0., dtype=theano.config.floatX), name="b")
>>> x.tag.test_value = D[0]
>>> y.tag.test_value = D[1]
>>> # Construct Theano expression graph
>>> p_1 = 1 / (1 + T.exp(-T.dot(x, w)-b)) # Probability of having a one
>>> prediction = p_1 > 0.5 # The prediction that is done: 0 or 1
>>> # Compute gradients
>>> xent = -y*T.log(p_1) - (1-y)*T.log(1-p_1) # Cross-entropy
>>> cost = xent.mean() + 0.01*(w**2).sum() # The cost to optimize
>>> gw,gb = T.grad(cost, [w,b])
>>> # Training and prediction function
>>> train = theano.function(inputs=[x,y], outputs=[prediction, xent], updates=[[w, w-0.01*gw], [b, b-0.01*gb]], name = "train")
>>> predict = theano.function(inputs=[x], outputs=prediction, name = "predict")
Pretty Printing
===============
``theano.printing.pprint(variable)``
>>> theano.printing.pprint(prediction) # (pre-compilation)
gt((TensorConstant{1} / (TensorConstant{1} + exp(((-(x \\dot w)) - b)))),TensorConstant{0.5})
Debug Printing
==============
``theano.printing.debugprint({fct, variable, list of variables})``
>>> theano.printing.debugprint(prediction) # (pre-compilation)
Elemwise{gt,no_inplace} [@181772236] ''
|Elemwise{true_div,no_inplace} [@181746668] ''
| |InplaceDimShuffle{x} [@181746412] ''
| | |TensorConstant{1} [@181745836]
| |Elemwise{add,no_inplace} [@181745644] ''
| | |InplaceDimShuffle{x} [@181745420] ''
| | | |TensorConstant{1} [@181744844]
| | |Elemwise{exp,no_inplace} [@181744652] ''
| | | |Elemwise{sub,no_inplace} [@181744012] ''
| | | | |Elemwise{neg,no_inplace} [@181730764] ''
| | | | | |dot [@181729676] ''
| | | | | | |x [@181563948]
| | | | | | |w [@181729964]
| | | | |InplaceDimShuffle{x} [@181743788] ''
| | | | | |b [@181730156]
|InplaceDimShuffle{x} [@181771788] ''
| |TensorConstant{0.5} [@181771148]
>>> theano.printing.debugprint(predict) # (post-compilation)
Elemwise{Composite{neg,{sub,{{scalar_sigmoid,GT},neg}}}} [@183160204] '' 2
|dot [@183018796] '' 1
| |x [@183000780]
| |w [@183000812]
|InplaceDimShuffle{x} [@183133580] '' 0
| |b [@183000876]
|TensorConstant{[ 0.5]} [@183084108]
Picture Printing
================
>>> theano.printing.pydotprint_variables(prediction) # (pre-compilation)
.. image:: ../hpcs2011_tutorial/pics/logreg_pydotprint_prediction.png
>>> theano.printing.pprint(prediction) # doctest: +NORMALIZE_WHITESPACE
'gt((TensorConstant{1} / (TensorConstant{1} + exp(((-(x \\dot w)) - b)))),
TensorConstant{0.5})'
Debug Print
===========
The pre-compilation graph:
>>> theano.printing.debugprint(prediction) # doctest: +NORMALIZE_WHITESPACE
Elemwise{gt,no_inplace} [@A] ''
|Elemwise{true_div,no_inplace} [@B] ''
| |DimShuffle{x} [@C] ''
| | |TensorConstant{1} [@D]
| |Elemwise{add,no_inplace} [@E] ''
| |DimShuffle{x} [@F] ''
| | |TensorConstant{1} [@D]
| |Elemwise{exp,no_inplace} [@G] ''
| |Elemwise{sub,no_inplace} [@H] ''
| |Elemwise{neg,no_inplace} [@I] ''
| | |dot [@J] ''
| | |x [@K]
| | |w [@L]
| |DimShuffle{x} [@M] ''
| |b [@N]
|DimShuffle{x} [@O] ''
|TensorConstant{0.5} [@P]
The post-compilation graph:
>>> theano.printing.debugprint(predict) # doctest: +NORMALIZE_WHITESPACE
Elemwise{Composite{GT(scalar_sigmoid((-((-i0) - i1))), i2)}} [@A] '' 4
|CGemv{inplace} [@B] '' 3
| |Alloc [@C] '' 2
| | |TensorConstant{0.0} [@D]
| | |Shape_i{0} [@E] '' 1
| | |x [@F]
| |TensorConstant{1.0} [@G]
| |x [@F]
| |w [@H]
| |TensorConstant{0.0} [@D]
|InplaceDimShuffle{x} [@I] '' 0
| |b [@J]
|TensorConstant{(1,) of 0.5} [@K]
Picture Printing of Graphs
==========================
The pre-compilation graph:
>>> theano.printing.pydotprint(prediction, outfile="pics/logreg_pydotprint_prediction.png", var_with_name_simple=True)
The output file is available at pics/logreg_pydotprint_prediction.png
.. image:: ./pics/logreg_pydotprint_prediction.png
:width: 800 px
Notice that ``pydotprint()`` requires *Graphviz* and Python's ``pydot``.
The post-compilation graph:
>>> theano.printing.pydotprint(predict) # (post-compilation)
>>> theano.printing.pydotprint(predict, outfile="pics/logreg_pydotprint_predict.png", var_with_name_simple=True)
The output file is available at pics/logreg_pydotprint_predict.png
.. image:: ../hpcs2011_tutorial/pics/logreg_pydotprint_predic.png
.. image:: ./pics/logreg_pydotprint_predict.png
:width: 800 px
>>> theano.printing.pydotprint(train) # This is a small train example!
The optimized training graph:
.. image:: ../hpcs2011_tutorial/pics/logreg_pydotprint_train.png
:width: 1500 px
>>> theano.printing.pydotprint(train, outfile="pics/logreg_pydotprint_train.png", var_with_name_simple=True)
The output file is available at pics/logreg_pydotprint_train.png
.. image:: ./pics/logreg_pydotprint_train.png
:width: 1500 px
doc/tutorial/shape_info.txt
浏览文件 @ e8494632
......@@ -20,17 +20,16 @@ Currently, information regarding shape is used in two ways in Theano:
Example:
.. code-block:: python
import theano
x = theano.tensor.matrix('x')
f = theano.function([x], (x ** 2).shape)
theano.printing.debugprint(f)
#MakeVector [@43860304] '' 2
# |Shape_i{0} [@43424912] '' 1
# | |x [@43423568]
# |Shape_i{1} [@43797968] '' 0
# | |x [@43423568]
>>> import theano
>>> x = theano.tensor.matrix('x')
>>> f = theano.function([x], (x ** 2).shape)
>>> theano.printing.debugprint(f) # doctest: +NORMALIZE_WHITESPACE
MakeVector [@A] '' 2
|Shape_i{0} [@B] '' 1
| |x [@C]
|Shape_i{1} [@D] '' 0
|x [@C]
The output of this compiled function does not contain any multiplication
or power. Theano has removed them to compute directly the shape of the
......@@ -42,48 +41,37 @@ Shape Inference Problem
Theano propagates information about shape in the graph. Sometimes this
can lead to errors. Consider this example:
.. code-block:: python
import numpy
import theano
x = theano.tensor.matrix('x')
y = theano.tensor.matrix('y')
z = theano.tensor.join(0, x, y)
xv = numpy.random.rand(5, 4)
yv = numpy.random.rand(3, 3)
f = theano.function([x,y], z.shape)
theano.printing.debugprint(f)
#MakeVector [@23910032] '' 4
# |Elemwise{Add{output_types_preference=transfer_type{0}}}[(0, 0)] [@24055120] '' 3
# | |Shape_i{0} [@23154000] '' 1
# | | |x [@23151760]
# | |Shape_i{0} [@23593040] '' 2
# | | |y [@23151888]
# |Shape_i{1} [@23531152] '' 0
# | |x [@23151760]
#MakeVector [@56338064] '' 4
# |Elemwise{Add{output_types_preference=transfer_type{0}}}[(0, 0)] [@56483152] '' 3
# | |Shape_i{0} [@55586128] '' 1
# | | |<TensorType(float64, matrix)> [@55583888]
# | |Shape_i{0} [@56021072] '' 2
# | | |<TensorType(float64, matrix)> [@55584016]
# |Shape_i{1} [@55959184] '' 0
# | |<TensorType(float64, matrix)> [@55583888]
print f(xv,yv)# DOES NOT RAISE AN ERROR AS SHOULD BE.
#[8,4]
f = theano.function([x,y], z)# Do not take the shape.
theano.printing.debugprint(f)
#Join [@44540496] '' 0
# |0 [@44540432]
# |x [@44540240]
# |y [@44540304]
f(xv,yv)
# Raises a dimensions mismatch error.
>>> import numpy
>>> import theano
>>> x = theano.tensor.matrix('x')
>>> y = theano.tensor.matrix('y')
>>> z = theano.tensor.join(0, x, y)
>>> xv = numpy.random.rand(5, 4)
>>> yv = numpy.random.rand(3, 3)
>>> f = theano.function([x,y], z.shape)
>>> theano.printing.debugprint(f) # doctest: +NORMALIZE_WHITESPACE
MakeVector [@A] '' 4
|Elemwise{Add}[(0, 0)] [@B] '' 3
| |Shape_i{0} [@C] '' 1
| | |x [@D]
| |Shape_i{0} [@E] '' 2
| |y [@F]
|Shape_i{1} [@G] '' 0
|x [@D]
print f(xv,yv)# DOES NOT RAISE AN ERROR AS SHOULD BE.
[8, 4]
>>> f = theano.function([x,y], z)# Do not take the shape.
>>> theano.printing.debugprint(f) # doctest: +NORMALIZE_WHITESPACE
Join [@A] '' 0
|TensorConstant{0} [@B]
|x [@C]
|y [@D]
>>> f(xv,yv) # doctest: +SKIP
>>> # Raises a dimensions mismatch error.
As you can see, when asking only for the shape of some computation (``join`` in the
example), an inferred shape is computed directly, without executing
......@@ -125,14 +113,14 @@ upgrade. Here is the current state of what can be done:
graph. This allows to perform some optimizations. In the following example,
this makes it possible to precompute the Theano function to a constant.
.. code-block:: python
import theano
x = theano.tensor.matrix()
x_specify_shape = theano.tensor.specify_shape(x, (2, 2))
f = theano.function([x], (x_specify_shape ** 2).shape)
theano.printing.debugprint(f)
# [2 2] [@72791376]
>>> import theano
>>> x = theano.tensor.matrix()
>>> x_specify_shape = theano.tensor.specify_shape(x, (2, 2))
>>> f = theano.function([x], (x_specify_shape ** 2).shape)
>>> theano.printing.debugprint(f) # doctest: +NORMALIZE_WHITESPACE
DeepCopyOp [@A] '' 0
|TensorConstant{(2,) of 2} [@B]
Future Plans
============
......
Markdown 格式
0% 或
您添加了 0 人 到此讨论。请谨慎行事。
请先完成此评论的编辑！
请 注册 或者 后发表评论