Comparing with the Python language, :term:`Op` is theano's
version of a function definition where an :term:`Apply` instance
is theano's version of a function call (or expression instance).
These subclasses are meant to be instantiated, and I'll call
an instance a theano Op instance (TOI). A TOI has several
important responsabilities:
* making :term:`Apply` instances, which mean "apply this TOI to some particular inputs" (via the ``make_node``),
* performing the calculation of outputs from given inputs (via the ``perform``),
* producing c code to perform calculation of outputs from inputs (via ``c_code, c_code_cleanup, c_support_code, c_headers, c_libraries, c_compile_args, c_no_compile_args``)
* [optionally] building gradient-calculating graphs (via ``grad``).
See :ref:`intro_to_ops`.
Optimization
graph transformation for faster execution
An optimization is a type of :term:`graph transformation`.
WRITEME.
Variable
a Type-related graph node (a variable)
A Variable
(`Variable API <http://pylearn.org/epydoc/theano.gof.graph.Variable-class.html>`_)
is theano's variable. It symbolically represents a value (which
can be a number, vector, matrix, tensor, etc.).
The inputs and outputs of every :term:`Op` are Variable instances.
The input and output arguments to create a :term:`function` are also Variables.
A Variable is like a strongly-typed variable in some other languages; each Variable contains a reference to a :term:`TTI` (Theano Type Instance) that defines the kind of value that can be associated to the Variable by a :term:`function`.
A Variable is a container for four important fields:
type
a :term:`TTI` defining the kind of value this Variable can have,
owner
either None (for graph roots) or the :term:`Apply` instance (i.e. variable of applying an :term:`Op`) of which ``self`` is an output,
index
the integer such that ``owner.outputs[index] is this_variable`` (ignored if ``owner`` is None)
name
a string to use in pretty-printing and debugging.
There are two subclasses related to Variable:
:term:`Value`
a Variable with a data field.
:term:`Constant`
like ``Value``, but the data it contains cannot be modified.
See :ref:`intro_to_types`.
Code Example:
.. code-block:: python
#!python
import theano
from theano import tensor
a = tensor.constant(1.5) # declare a symbolic constant
b = tensor.fscalar() # declare a symbolic floating-point scalar
c = a + b # create a simple expression
f = theano.function([b], [c]) # this works because a has a value associated with it already
assert 4.0 == f(2.5)[0] # bind 2.5 to an internal copy of b and evaluate an internal c
theano.function([a], [c]) # compilation error because b (required by c) is undefined
theano.function([a,b], [c]) # compilation error because a is constant, it can't be an input
d = tensor.value(1.5) # create a value similar to the constant 'a'
e = d + b
theano.function([d,b], [e]) # this works. d's default value of 1.5 is ignored.
The python variables ``a,b,c`` all refer to instances of type Variable.
The Variable refered to by ``a`` is also an instance of ``Constant``.
Theano.:term:`function` uses the :term:`Apply` instances' ``inputs`` field together with each Variable's ``owner`` field to determine which inputs are necessary to compute the function's outputs.
Scalar
[overloaded term]
The scalar we use in the :ref:`tutorial` is a zero-dimensional tensor.
The second type of scalar is for advanced users and developers.
It is actually a scalar, and is the primitive that is contained by a tensor.
WRITEME: More information.
Stabilization
graph transformation for numeric stability
Some graph transformations improve numerical stability. For example ``log(x+1)`` has a special implementation which is accurate for a wider range of ``x`` values than a naive implementation in which ``log`` is applied to a temporary variable storing ``1+x``. Theano includes many graph transformations aimed at recognizing patterns like this one and replacing them with compound expressions.
Stabilizations are like :term:`optimizations <Optimization>` in the sense that they are often pattern-based sub-graph substitutions.
Stabilizations are unlike :term:`optimizations <Optimization>` in that
- they are typically applied even when intermediate variables in the subgraph have external :term:`clients`,
- they are typically prioritized over transformations which improve run-time speed, and
- they are typically not faster than the naive implementation.
Tensor
Type for vectors, matrices, etc.
Tensor (`Tensor API <http://pylearn.org/epydoc/theano.tensor.Tensor-class.html doc>`_) is a class
that derives from :term:`Type`. It is the symbolic type for ``numpy.ndarray``.
The main properties that distinguish one Tensor instance from another are:
* ndim - the rank of the tensor
* :term:`broadcastable <Broadcasting>` - which dimensions are broadcastable
* :term:`dtype` - what kind of elements will the tensor contain
See also :term:`TensorVariable`.
TensorVariable
:term:`Variables <Variable>` of type :term:`Tensor` are of class
TensorVariable (`TensorVariable API <http://pylearn.org/epydoc/theano.tensor.TensorVariable-class.html>`_).
``TensorVariable`` adds operator overloading so that ``TensorVariable`` instances can be used
in mathematical expressions. When any input to an expression is a ``TensorVariable`` then the
expression will evaluate to an ``TensorVariable`` and a :term:`graph` corresponding to
the expression.
Many shortcuts exist for creating ``TensorVariable`` instances:
* ``<t>scalar`` - create a tensor of rank 0
* ``<t>vector`` - create a tensor of rank 1
* ``<t>matrix`` - create a tensor of rank 2
* ``<t>row`` - create a "row vector" (tensor of rank 2) that is :term:`broadcastable <Broadcasting>` over rows
* ``<t>col`` - create a "column vector" (tensor of rank 2) that is :term:`broadcastable <Broadcasting>` over columns
In each shortcut the ``<t>`` is a letter from ``fdbil`` that indicates the :term:`DType`
of the tensor:
* ``f`` - float
* ``d`` - double
* ``b`` - byte
* ``i`` - int
* ``l`` - long
TTI
An object instance whose type is a subclass of :term:`Type` is a theano type instance (TTI).
In the following code:
.. code-block:: python
# tensor.fvector is a TTI
# declare a symbolic floating-point vector using __call__
b = tensor.fvector()
# create a second Variable with the same TTI
c = tensor.fvector()
(``tensor.fvector``) is a TTI because
(``tensor.fvector``) is an instance of the (``theano.tensor.Tensor``) class, which is a subclass of (``theano.Type``).
Whenever you create a variable in theano (technically, a :term:`Variable`) it will contain a reference to a TTI.
That reference is typically constant during the lifetime of the Variable.
Many variables can refer to a single TTI, as do ``b`` and ``c`` above.
The TTI defines the kind of value which might end up in that variable when executing a :term:`function`.
In this sense, theano is like a strongly-typed language.
In our example above, ``b`` is a variable which is guaranteed to corresond to a ``numpy.ndarray`` of rank 1 when we try to do some computations with it.
Many :term:`Ops <Op>` will raise an exception if their inputs do not have the correct types (TTI references).
TTI references are also useful to do type-checking in pattern-based optimizations.
Type
:term:`Variables <Variable>` are strongly typed by :term:`Type` instances
