1600B [(400,)] i GpuElemwise{Sub}[(0, 1)](CudaNdarrayConstant{[ 1.]}, GpuFromHost.0)
1600B [(400,)] c HostFromGpu(GpuElemwise{Sub}[(0, 1)].0)
1600B [(400,)] c GpuFromHost(y)
... (remaining 7 Apply account for 448B/3800192B ((0.01%)) of the Apply with dense outputs sizes)
<created/inplace/view> is taken from the Op's declaration.
Apply nodes marked 'inplace' or 'view' may actually allocate memory, this is not reported here. If you use DebugMode, warnings will be emitted in those cases.
Here are tips to potentially make your code run faster
(if you think of new ones, suggest them on the mailing list).
Test them first, as they are not guaranteed to always provide a speedup.
Sorry, no tip for today.
"""
"""
Exercise 5
Exercise 5
-----------
-----------
- In the last exercises, do you see a speed up with the GPU?
- In the last exercises, do you see a speed up with the GPU?
- Where does it come from? (Use ProfileMode)
- Where does it come from? (Use profile=True)
- Is there something we can do to speed up the GPU version?
- Is there something we can do to speed up the GPU version?
...
@@ -427,4 +532,3 @@ Known limitations
...
@@ -427,4 +532,3 @@ Known limitations
- A few hundreds nodes is fine
- A few hundreds nodes is fine
- Disabling a few optimizations can speed up compilation
- Disabling a few optimizations can speed up compilation
- Usually too many nodes indicates a problem with the graph
- Usually too many nodes indicates a problem with the graph
functions using either of the following two options:
functions using either of the following two options:
1. Use Theano flag :attr:`config.profile` to enable profiling.
1. Use Theano flag :attr:`config.profile` to enable profiling.
- To enable the memory profiler use the Theano flag:
- To enable the memory profiler use the Theano flag:
:attr:`config.profile_memory` in addition to :attr:`config.profile`.
:attr:`config.profile_memory` in addition to :attr:`config.profile`.
- Moreover, to enable the profiling of Theano optimization phase,
- Moreover, to enable the profiling of Theano optimization phase,
...
@@ -30,8 +30,8 @@ functions using either of the following two options:
...
@@ -30,8 +30,8 @@ functions using either of the following two options:
2. Pass the argument :attr:`profile=True` to the function :func:`theano.function <function.function>`. And then call :attr:`f.profile.print_summary()` for a single function.
2. Pass the argument :attr:`profile=True` to the function :func:`theano.function <function.function>`. And then call :attr:`f.profile.print_summary()` for a single function.
- Use this option when you want to profile not all the
- Use this option when you want to profile not all the
functions but one or more specific function(s).
functions but one or more specific function(s).
- You can also combine the profile of many functions:
- You can also combine the profile of many functions:
.. testcode::
.. testcode::
profile = theano.compile.ProfileStats()
profile = theano.compile.ProfileStats()
...
@@ -68,6 +68,15 @@ compare equal, if their parameters differ (the scalar being
...
@@ -68,6 +68,15 @@ compare equal, if their parameters differ (the scalar being
executed). So the class section will merge more Apply nodes then the
executed). So the class section will merge more Apply nodes then the
Ops section.
Ops section.
Note that the profile also shows which Ops were running a c implementation.
Developers wishing to optimize the performance of their graph should
focus on the worst offending Ops and Apply nodes – either by optimizing
an implementation, providing a missing C implementation, or by writing
a graph optimization that eliminates the offending Op altogether.
You should strongly consider emailing one of our lists about your
issue before spending too much time on this.
Here is an example output when we disable some Theano optimizations to
Here is an example output when we disable some Theano optimizations to
give you a better idea of the difference between sections. With all
give you a better idea of the difference between sections. With all
optimizations enabled, there would be only one op left in the graph.
optimizations enabled, there would be only one op left in the graph.
print('Other time since import %.3fs %.1f%%'%(other_time,other_time/total_time*100))
print('%i Theano fct call, %.3fs per call'%(total_fct_call,time_per_call))
print()
print("List of apply that don't have float64 as input but have float64 in outputs. Usefull to know if we forgot some cast when using floatX=float32 or gpu code.")
print("Here are tips to potentially make your code run faster (if you think of new ones, suggest them on the mailing list). Test them first as they are not guaranteed to always provide a speedup.")
scal.Sqrt,scal.Abs,scal.Cos,scal.Sin,scal.Tan,scal.Tanh,scal.Cosh,scal.Sinh,T.nnet.sigm.ScalarSigmoid,T.nnet.sigm.ScalarSoftplus]# Abs, Mod in float{32,64} only
print(" - With the default gcc libm, exp in float32 is slower than in float64! Try Theano flags floatX=float64 or install amdlibm and set the theano flags lib.amdlibm=True")
