提交 e8494632 authored 作者: Christof Angermueller's avatar Christof Angermueller

Update printing and plotting sections in tutorials

上级 2907f95a
...@@ -307,14 +307,17 @@ Consider the following logistic regression model: ...@@ -307,14 +307,17 @@ Consider the following logistic regression model:
We will now make use of Theano's printing features to compare the unoptimized We will now make use of Theano's printing features to compare the unoptimized
graph (``prediction``) to the optimized graph (``predict``). graph (``prediction``) to the optimized graph (``predict``).
- Pretty Printing
Pretty Printing
~~~~~~~~~~~~~~~
>>> theano.printing.pprint(prediction) # doctest: +NORMALIZE_WHITESPACE >>> theano.printing.pprint(prediction) # doctest: +NORMALIZE_WHITESPACE
'gt((TensorConstant{1} / (TensorConstant{1} + exp(((-(x \\dot w)) - b)))), 'gt((TensorConstant{1} / (TensorConstant{1} + exp(((-(x \\dot w)) - b)))),
TensorConstant{0.5})' TensorConstant{0.5})'
- Debug Print Debug Print
~~~~~~~~~~~
The graph before optimization: The graph before optimization:
...@@ -355,7 +358,8 @@ The graph after optimization: ...@@ -355,7 +358,8 @@ The graph after optimization:
|TensorConstant{(1,) of 0.5} [@K] |TensorConstant{(1,) of 0.5} [@K]
- Picture Printing of Graphs Picture Printing of Graphs
~~~~~~~~~~~~~~~~~~~~~~~~~~
``pydotprint`` requires graphviz and pydot. ``pydotprint`` requires graphviz and pydot.
The graph before optimization: The graph before optimization:
......
...@@ -53,7 +53,7 @@ Conditions ...@@ -53,7 +53,7 @@ Conditions
IfElse Op spend less time (about an half) than Switch since it computes only IfElse Op spend less time (about an half) than Switch since it computes only
one variable instead of both. one variable instead of both.
>>> python ifelse_switch.py >>> python ifelse_switch.py # doctest: +SKIP
time spent evaluating both values 0.230000 sec time spent evaluating both values 0.230000 sec
time spent evaluating one value 0.120000 sec time spent evaluating one value 0.120000 sec
...@@ -108,68 +108,123 @@ Exercise 5 ...@@ -108,68 +108,123 @@ Exercise 5
- Is there something we can do to speed up the GPU version? - Is there something we can do to speed up the GPU version?
Printing/Drawing Theano graphs Printing/Drawing Theano graphs
------------------------------ ------------------------------
- Pretty Printing Consider the following logistic regression model:
``theano.printing.pprint(variable)`` >>> import numpy
>>> import theano
>>> theano.printing.pprint(prediction) >>> import theano.tensor as T
gt((TensorConstant{1} / (TensorConstant{1} + exp(((-(x \\dot w)) - b)))),TensorConstant{0.5}) >>> rng = numpy.random
>>> # Training data
>>> N = 400
- Debug Print >>> feats = 784
>>> D = (rng.randn(N, feats).astype(theano.config.floatX), rng.randint(size=N,low=0, high=2).astype(theano.config.floatX))
``theano.printing.debugprint({fct, variable, list of variables})`` >>> training_steps = 10000
>>> # Declare Theano symbolic variables
>>> theano.printing.debugprint(prediction) >>> x = T.matrix("x")
Elemwise{gt,no_inplace} [@181772236] '' >>> y = T.vector("y")
|Elemwise{true_div,no_inplace} [@181746668] '' >>> w = theano.shared(rng.randn(feats).astype(theano.config.floatX), name="w")
| |InplaceDimShuffle{x} [@181746412] '' >>> b = theano.shared(numpy.asarray(0., dtype=theano.config.floatX), name="b")
| | |TensorConstant{1} [@181745836] >>> x.tag.test_value = D[0]
| |Elemwise{add,no_inplace} [@181745644] '' >>> y.tag.test_value = D[1]
| | |InplaceDimShuffle{x} [@181745420] '' >>> # Construct Theano expression graph
| | | |TensorConstant{1} [@181744844] >>> p_1 = 1 / (1 + T.exp(-T.dot(x, w)-b)) # Probability of having a one
| | |Elemwise{exp,no_inplace} [@181744652] '' >>> prediction = p_1 > 0.5 # The prediction that is done: 0 or 1
| | | |Elemwise{sub,no_inplace} [@181744012] '' >>> # Compute gradients
| | | | |Elemwise{neg,no_inplace} [@181730764] '' >>> xent = -y*T.log(p_1) - (1-y)*T.log(1-p_1) # Cross-entropy
| | | | | |dot [@181729676] '' >>> cost = xent.mean() + 0.01*(w**2).sum() # The cost to optimize
| | | | | | |x [@181563948] >>> gw,gb = T.grad(cost, [w,b])
| | | | | | |w [@181729964] >>> # Training and prediction function
| | | | |InplaceDimShuffle{x} [@181743788] '' >>> train = theano.function(inputs=[x,y], outputs=[prediction, xent], updates=[[w, w-0.01*gw], [b, b-0.01*gb]], name = "train")
| | | | | |b [@181730156] >>> predict = theano.function(inputs=[x], outputs=prediction, name = "predict")
|InplaceDimShuffle{x} [@181771788] ''
| |TensorConstant{0.5} [@181771148] We will now make use of Theano's printing features to compare the unoptimized
>>> theano.printing.debugprint(predict) graph (``prediction``) to the optimized graph (``predict``).
Elemwise{Composite{neg,{sub,{{scalar_sigmoid,GT},neg}}}} [@183160204] '' 2
|dot [@183018796] '' 1 Pretty Printing
| |x [@183000780] ~~~~~~~~~~~~~~~
| |w [@183000812]
|InplaceDimShuffle{x} [@183133580] '' 0 >>> theano.printing.pprint(prediction) # doctest: +NORMALIZE_WHITESPACE
| |b [@183000876] 'gt((TensorConstant{1} / (TensorConstant{1} + exp(((-(x \\dot w)) - b)))),
|TensorConstant{[ 0.5]} [@183084108] TensorConstant{0.5})'
- Picture Printing of Graphs
Debug Print
>>> theano.printing.pydotprint_variables(prediction) ~~~~~~~~~~~
.. image:: ../hpcs2011_tutorial/pics/logreg_pydotprint_prediction.png The graph before optimization:
>>> theano.printing.debugprint(prediction) # doctest: +NORMALIZE_WHITESPACE
Elemwise{gt,no_inplace} [@A] ''
|Elemwise{true_div,no_inplace} [@B] ''
| |DimShuffle{x} [@C] ''
| | |TensorConstant{1} [@D]
| |Elemwise{add,no_inplace} [@E] ''
| |DimShuffle{x} [@F] ''
| | |TensorConstant{1} [@D]
| |Elemwise{exp,no_inplace} [@G] ''
| |Elemwise{sub,no_inplace} [@H] ''
| |Elemwise{neg,no_inplace} [@I] ''
| | |dot [@J] ''
| | |x [@K]
| | |w [@L]
| |DimShuffle{x} [@M] ''
| |b [@N]
|DimShuffle{x} [@O] ''
|TensorConstant{0.5} [@P]
The graph after optimization:
>>> theano.printing.debugprint(predict) # doctest: +NORMALIZE_WHITESPACE
Elemwise{Composite{GT(scalar_sigmoid((-((-i0) - i1))), i2)}} [@A] '' 4
|CGemv{inplace} [@B] '' 3
| |Alloc [@C] '' 2
| | |TensorConstant{0.0} [@D]
| | |Shape_i{0} [@E] '' 1
| | |x [@F]
| |TensorConstant{1.0} [@G]
| |x [@F]
| |w [@H]
| |TensorConstant{0.0} [@D]
|InplaceDimShuffle{x} [@I] '' 0
| |b [@J]
|TensorConstant{(1,) of 0.5} [@K]
Picture Printing of Graphs
~~~~~~~~~~~~~~~~~~~~~~~~~~
``pydotprint`` requires graphviz and pydot.
The graph before optimization:
>>> theano.printing.pydotprint(prediction, outfile="pics/logreg_pydotprint_prediction.png", var_with_name_simple=True)
The output file is available at pics/logreg_pydotprint_prediction.png
.. image:: ./pics/logreg_pydotprint_prediction.png
:width: 800 px :width: 800 px
All pydotprint* requires graphviz and pydot The graph after optimization:
>>> theano.printing.pydotprint(predict) >>> theano.printing.pydotprint(predict, outfile="pics/logreg_pydotprint_predict.png", var_with_name_simple=True)
The output file is available at pics/logreg_pydotprint_predict.png
.. image:: ../hpcs2011_tutorial/pics/logreg_pydotprint_predic.png .. image:: ./pics/logreg_pydotprint_predict.png
:width: 800 px :width: 800 px
>>> theano.printing.pydotprint(train) # This is a small train example! The optimized training graph:
>>> theano.printing.pydotprint(train, outfile="pics/logreg_pydotprint_train.png", var_with_name_simple=True)
The output file is available at pics/logreg_pydotprint_train.png
.. image:: ../hpcs2011_tutorial/pics/logreg_pydotprint_train.png .. image:: ./pics/logreg_pydotprint_train.png
:width: 1500 px :width: 1500 px
Debugging Debugging
--------- ---------
......
...@@ -5,14 +5,13 @@ ...@@ -5,14 +5,13 @@
Printing/Drawing Theano graphs Printing/Drawing Theano graphs
============================== ==============================
.. TODO: repair the defective links in the next paragraph
Theano provides two functions (:func:`theano.pp` and Theano provides the functions :func:`theano.printing.pprint` and
:func:`theano.printing.debugprint`) to print a graph to the terminal before or after :func:`theano.printing.debugprint` to print a graph to the terminal before or
compilation. These two functions print expression graphs in different ways: after compilation. :func:`pprint` is more compact and math-like,
:func:`pp` is more compact and math-like, :func:`debugprint` is more verbose. :func:`debugprint` is more verbose. Theano also provides :func:`pydotprint`
Theano also provides :func:`pydotprint` that creates a *png* image of the function. that creates an image of the function. You can read about them in
You can read about them in :ref:`libdoc_printing`. :ref:`libdoc_printing`.
.. note:: .. note::
...@@ -24,151 +23,109 @@ You can read about them in :ref:`libdoc_printing`. ...@@ -24,151 +23,109 @@ You can read about them in :ref:`libdoc_printing`.
real job execution, as this will make the graph slower and use more real job execution, as this will make the graph slower and use more
memory. memory.
Consider again the logistic regression but notice the additional printing instuctions. Consider again the logistic regression example:
The following output depicts the pre- and post- compilation graphs.
>>> import numpy
.. code-block:: python >>> import theano
>>> import theano.tensor as T
import theano >>> rng = numpy.random
import theano.tensor as T >>> # Training data
>>> N = 400
import numpy >>> feats = 784
>>> D = (rng.randn(N, feats).astype(theano.config.floatX), rng.randint(size=N,low=0, high=2).astype(theano.config.floatX))
import os >>> training_steps = 10000
>>> # Declare Theano symbolic variables
rng = numpy.random >>> x = T.matrix("x")
>>> y = T.vector("y")
N = 400 >>> w = theano.shared(rng.randn(feats).astype(theano.config.floatX), name="w")
feats = 784 >>> b = theano.shared(numpy.asarray(0., dtype=theano.config.floatX), name="b")
D = (rng.randn(N, feats).astype(theano.config.floatX), >>> x.tag.test_value = D[0]
rng.randint(size=N,low=0, high=2).astype(theano.config.floatX)) >>> y.tag.test_value = D[1]
training_steps = 10000 >>> # Construct Theano expression graph
>>> p_1 = 1 / (1 + T.exp(-T.dot(x, w)-b)) # Probability of having a one
# Declare Theano symbolic variables >>> prediction = p_1 > 0.5 # The prediction that is done: 0 or 1
x = T.matrix("x") >>> # Compute gradients
y = T.vector("y") >>> xent = -y*T.log(p_1) - (1-y)*T.log(1-p_1) # Cross-entropy
w = theano.shared(rng.randn(feats).astype(theano.config.floatX), name="w") >>> cost = xent.mean() + 0.01*(w**2).sum() # The cost to optimize
b = theano.shared(numpy.asarray(0., dtype=theano.config.floatX), name="b") >>> gw,gb = T.grad(cost, [w,b])
x.tag.test_value = D[0] >>> # Training and prediction function
y.tag.test_value = D[1] >>> train = theano.function(inputs=[x,y], outputs=[prediction, xent], updates=[[w, w-0.01*gw], [b, b-0.01*gb]], name = "train")
#print "Initial model:" >>> predict = theano.function(inputs=[x], outputs=prediction, name = "predict")
#print w.get_value(), b.get_value()
# Construct Theano expression graph
p_1 = 1 / (1 + T.exp(-T.dot(x, w) - b)) # Probability of having a one
prediction = p_1 > 0.5 # The prediction that is done: 0 or 1
xent = -y * T.log(p_1) - (1 - y) * T.log(1 - p_1) # Cross-entropy
cost = xent.mean() + 0.01 * (w ** 2).sum() # The cost to optimize
gw,gb = T.grad(cost, [w, b])
# Compile expressions to functions
train = theano.function(
inputs=[x, y],
outputs=[prediction, xent],
updates=[(w, w - 0.01 * gw), (b, b - 0.01 * gb)],
name="train")
predict = theano.function(inputs=[x], outputs=prediction,
name="predict")
if any([x.op.__class__.__name__ in ['Gemv', 'CGemv'] for x in
train.maker.fgraph.toposort()]):
print 'Used the cpu'
elif any([x.op.__class__.__name__ == 'GpuGemv' for x in
train.maker.fgraph.toposort()]):
print 'Used the gpu'
else:
print 'ERROR, not able to tell if theano used the cpu or the gpu'
print train.maker.fgraph.toposort()
for i in range(training_steps):
pred, err = train(D[0], D[1])
#print "Final model:"
#print w.get_value(), b.get_value()
print "target values for D"
print D[1]
print "prediction on D"
print predict(D[0])
# Print the picture graphs
# after compilation
if not os.path.exists('pics'):
os.mkdir('pics')
theano.printing.pydotprint(predict,
outfile="pics/logreg_pydotprint_predic.png",
var_with_name_simple=True)
# before compilation
theano.printing.pydotprint_variables(prediction,
outfile="pics/logreg_pydotprint_prediction.png",
var_with_name_simple=True)
theano.printing.pydotprint(train,
outfile="pics/logreg_pydotprint_train.png",
var_with_name_simple=True)
Pretty Printing Pretty Printing
=============== ===============
``theano.printing.pprint(variable)`` >>> theano.printing.pprint(prediction) # doctest: +NORMALIZE_WHITESPACE
'gt((TensorConstant{1} / (TensorConstant{1} + exp(((-(x \\dot w)) - b)))),
>>> theano.printing.pprint(prediction) # (pre-compilation) TensorConstant{0.5})'
gt((TensorConstant{1} / (TensorConstant{1} + exp(((-(x \\dot w)) - b)))),TensorConstant{0.5})
Debug Print
Debug Printing ===========
==============
The pre-compilation graph:
``theano.printing.debugprint({fct, variable, list of variables})``
>>> theano.printing.debugprint(prediction) # doctest: +NORMALIZE_WHITESPACE
>>> theano.printing.debugprint(prediction) # (pre-compilation) Elemwise{gt,no_inplace} [@A] ''
Elemwise{gt,no_inplace} [@181772236] '' |Elemwise{true_div,no_inplace} [@B] ''
|Elemwise{true_div,no_inplace} [@181746668] '' | |DimShuffle{x} [@C] ''
| |InplaceDimShuffle{x} [@181746412] '' | | |TensorConstant{1} [@D]
| | |TensorConstant{1} [@181745836] | |Elemwise{add,no_inplace} [@E] ''
| |Elemwise{add,no_inplace} [@181745644] '' | |DimShuffle{x} [@F] ''
| | |InplaceDimShuffle{x} [@181745420] '' | | |TensorConstant{1} [@D]
| | | |TensorConstant{1} [@181744844] | |Elemwise{exp,no_inplace} [@G] ''
| | |Elemwise{exp,no_inplace} [@181744652] '' | |Elemwise{sub,no_inplace} [@H] ''
| | | |Elemwise{sub,no_inplace} [@181744012] '' | |Elemwise{neg,no_inplace} [@I] ''
| | | | |Elemwise{neg,no_inplace} [@181730764] '' | | |dot [@J] ''
| | | | | |dot [@181729676] '' | | |x [@K]
| | | | | | |x [@181563948] | | |w [@L]
| | | | | | |w [@181729964] | |DimShuffle{x} [@M] ''
| | | | |InplaceDimShuffle{x} [@181743788] '' | |b [@N]
| | | | | |b [@181730156] |DimShuffle{x} [@O] ''
|InplaceDimShuffle{x} [@181771788] '' |TensorConstant{0.5} [@P]
| |TensorConstant{0.5} [@181771148]
>>> theano.printing.debugprint(predict) # (post-compilation) The post-compilation graph:
Elemwise{Composite{neg,{sub,{{scalar_sigmoid,GT},neg}}}} [@183160204] '' 2
|dot [@183018796] '' 1 >>> theano.printing.debugprint(predict) # doctest: +NORMALIZE_WHITESPACE
| |x [@183000780] Elemwise{Composite{GT(scalar_sigmoid((-((-i0) - i1))), i2)}} [@A] '' 4
| |w [@183000812] |CGemv{inplace} [@B] '' 3
|InplaceDimShuffle{x} [@183133580] '' 0 | |Alloc [@C] '' 2
| |b [@183000876] | | |TensorConstant{0.0} [@D]
|TensorConstant{[ 0.5]} [@183084108] | | |Shape_i{0} [@E] '' 1
| | |x [@F]
| |TensorConstant{1.0} [@G]
Picture Printing | |x [@F]
================ | |w [@H]
| |TensorConstant{0.0} [@D]
>>> theano.printing.pydotprint_variables(prediction) # (pre-compilation) |InplaceDimShuffle{x} [@I] '' 0
| |b [@J]
.. image:: ../hpcs2011_tutorial/pics/logreg_pydotprint_prediction.png |TensorConstant{(1,) of 0.5} [@K]
Picture Printing of Graphs
==========================
The pre-compilation graph:
>>> theano.printing.pydotprint(prediction, outfile="pics/logreg_pydotprint_prediction.png", var_with_name_simple=True)
The output file is available at pics/logreg_pydotprint_prediction.png
.. image:: ./pics/logreg_pydotprint_prediction.png
:width: 800 px :width: 800 px
Notice that ``pydotprint()`` requires *Graphviz* and Python's ``pydot``. The post-compilation graph:
>>> theano.printing.pydotprint(predict) # (post-compilation) >>> theano.printing.pydotprint(predict, outfile="pics/logreg_pydotprint_predict.png", var_with_name_simple=True)
The output file is available at pics/logreg_pydotprint_predict.png
.. image:: ../hpcs2011_tutorial/pics/logreg_pydotprint_predic.png .. image:: ./pics/logreg_pydotprint_predict.png
:width: 800 px :width: 800 px
>>> theano.printing.pydotprint(train) # This is a small train example! The optimized training graph:
.. image:: ../hpcs2011_tutorial/pics/logreg_pydotprint_train.png >>> theano.printing.pydotprint(train, outfile="pics/logreg_pydotprint_train.png", var_with_name_simple=True)
:width: 1500 px The output file is available at pics/logreg_pydotprint_train.png
.. image:: ./pics/logreg_pydotprint_train.png
:width: 1500 px
...@@ -20,17 +20,16 @@ Currently, information regarding shape is used in two ways in Theano: ...@@ -20,17 +20,16 @@ Currently, information regarding shape is used in two ways in Theano:
Example: Example:
.. code-block:: python >>> import theano
>>> x = theano.tensor.matrix('x')
import theano >>> f = theano.function([x], (x ** 2).shape)
x = theano.tensor.matrix('x') >>> theano.printing.debugprint(f) # doctest: +NORMALIZE_WHITESPACE
f = theano.function([x], (x ** 2).shape) MakeVector [@A] '' 2
theano.printing.debugprint(f) |Shape_i{0} [@B] '' 1
#MakeVector [@43860304] '' 2 | |x [@C]
# |Shape_i{0} [@43424912] '' 1 |Shape_i{1} [@D] '' 0
# | |x [@43423568] |x [@C]
# |Shape_i{1} [@43797968] '' 0
# | |x [@43423568]
The output of this compiled function does not contain any multiplication The output of this compiled function does not contain any multiplication
or power. Theano has removed them to compute directly the shape of the or power. Theano has removed them to compute directly the shape of the
...@@ -42,48 +41,37 @@ Shape Inference Problem ...@@ -42,48 +41,37 @@ Shape Inference Problem
Theano propagates information about shape in the graph. Sometimes this Theano propagates information about shape in the graph. Sometimes this
can lead to errors. Consider this example: can lead to errors. Consider this example:
.. code-block:: python >>> import numpy
>>> import theano
import numpy >>> x = theano.tensor.matrix('x')
import theano >>> y = theano.tensor.matrix('y')
x = theano.tensor.matrix('x') >>> z = theano.tensor.join(0, x, y)
y = theano.tensor.matrix('y') >>> xv = numpy.random.rand(5, 4)
z = theano.tensor.join(0, x, y) >>> yv = numpy.random.rand(3, 3)
xv = numpy.random.rand(5, 4)
yv = numpy.random.rand(3, 3) >>> f = theano.function([x,y], z.shape)
>>> theano.printing.debugprint(f) # doctest: +NORMALIZE_WHITESPACE
f = theano.function([x,y], z.shape) MakeVector [@A] '' 4
theano.printing.debugprint(f) |Elemwise{Add}[(0, 0)] [@B] '' 3
#MakeVector [@23910032] '' 4 | |Shape_i{0} [@C] '' 1
# |Elemwise{Add{output_types_preference=transfer_type{0}}}[(0, 0)] [@24055120] '' 3 | | |x [@D]
# | |Shape_i{0} [@23154000] '' 1 | |Shape_i{0} [@E] '' 2
# | | |x [@23151760] | |y [@F]
# | |Shape_i{0} [@23593040] '' 2 |Shape_i{1} [@G] '' 0
# | | |y [@23151888] |x [@D]
# |Shape_i{1} [@23531152] '' 0
# | |x [@23151760] print f(xv,yv)# DOES NOT RAISE AN ERROR AS SHOULD BE.
[8, 4]
#MakeVector [@56338064] '' 4
# |Elemwise{Add{output_types_preference=transfer_type{0}}}[(0, 0)] [@56483152] '' 3 >>> f = theano.function([x,y], z)# Do not take the shape.
# | |Shape_i{0} [@55586128] '' 1 >>> theano.printing.debugprint(f) # doctest: +NORMALIZE_WHITESPACE
# | | |<TensorType(float64, matrix)> [@55583888] Join [@A] '' 0
# | |Shape_i{0} [@56021072] '' 2 |TensorConstant{0} [@B]
# | | |<TensorType(float64, matrix)> [@55584016] |x [@C]
# |Shape_i{1} [@55959184] '' 0 |y [@D]
# | |<TensorType(float64, matrix)> [@55583888]
>>> f(xv,yv) # doctest: +SKIP
print f(xv,yv)# DOES NOT RAISE AN ERROR AS SHOULD BE. >>> # Raises a dimensions mismatch error.
#[8,4]
f = theano.function([x,y], z)# Do not take the shape.
theano.printing.debugprint(f)
#Join [@44540496] '' 0
# |0 [@44540432]
# |x [@44540240]
# |y [@44540304]
f(xv,yv)
# Raises a dimensions mismatch error.
As you can see, when asking only for the shape of some computation (``join`` in the As you can see, when asking only for the shape of some computation (``join`` in the
example), an inferred shape is computed directly, without executing example), an inferred shape is computed directly, without executing
...@@ -125,14 +113,14 @@ upgrade. Here is the current state of what can be done: ...@@ -125,14 +113,14 @@ upgrade. Here is the current state of what can be done:
graph. This allows to perform some optimizations. In the following example, graph. This allows to perform some optimizations. In the following example,
this makes it possible to precompute the Theano function to a constant. this makes it possible to precompute the Theano function to a constant.
.. code-block:: python
import theano >>> import theano
x = theano.tensor.matrix() >>> x = theano.tensor.matrix()
x_specify_shape = theano.tensor.specify_shape(x, (2, 2)) >>> x_specify_shape = theano.tensor.specify_shape(x, (2, 2))
f = theano.function([x], (x_specify_shape ** 2).shape) >>> f = theano.function([x], (x_specify_shape ** 2).shape)
theano.printing.debugprint(f) >>> theano.printing.debugprint(f) # doctest: +NORMALIZE_WHITESPACE
# [2 2] [@72791376] DeepCopyOp [@A] '' 0
|TensorConstant{(2,) of 2} [@B]
Future Plans Future Plans
============ ============
......
Markdown 格式
0%
您添加了 0 到此讨论。请谨慎行事。
请先完成此评论的编辑!
注册 或者 后发表评论