.. currentmodule:: pytensor.tensor

.. _libdoc_basic_tensor:

===========================
Basic Tensor Functionality
===========================

.. testsetup::

   import numpy as np
   import pytensor
   import pytensor.tensor as pt
   from pytensor.tensor.type import scalar, iscalar, TensorType, dmatrix, ivector, fmatrix
   from pytensor.tensor import set_subtensor, inc_subtensor, batched_dot
   from pytensor import shared

PyTensor supports symbolic tensor expressions.  When you type,

>>> import pytensor.tensor as pt
>>> x = pt.fmatrix()

the ``x`` is a :class:`TensorVariable` instance.

The ``pt.fmatrix`` object itself is an instance of :class:`TensorType`.
PyTensor knows what type of variable ``x`` is because ``x.type``
points back to ``pt.fmatrix``.

This section explains the various ways in which a tensor variable can be created,
the attributes and methods of :class:`TensorVariable` and :class:`TensorType`,
and various basic symbolic math and arithmetic that PyTensor supports for
tensor variables.

In general, PyTensor's API tries to mirror NumPy's, so, in most cases, it's safe
to assume that the basic NumPy array functions and methods will be available.

.. _libdoc_tensor_creation:

Creation
========

PyTensor provides a list of predefined tensor types that can be used
to create a tensor variables.  Variables can be named to facilitate debugging,
and all of these constructors accept an optional ``name`` argument.
For example, the following each produce a `TensorVariable` instance that stands
for a 0-dimensional `ndarray` of integers with the name ``'myvar'``:

>>> x = pt.scalar('myvar', dtype='int32')
>>> x = pt.iscalar('myvar')
>>> x = pt.tensor(dtype='int32', shape=(), name='myvar')
>>> from pytensor.tensor.type import TensorType
>>> x = TensorType(dtype='int32', shape=())('myvar')

Constructors with optional dtype
--------------------------------

These are the simplest and often-preferred methods for creating symbolic
variables in your code.  By default, they produce floating-point variables
(with dtype determined by `pytensor.config.floatX`) so if you use
these constructors it is easy to switch your code between different levels of
floating-point precision.

.. function:: scalar(name=None, dtype=config.floatX)

    Return a `Variable` for a 0-dimensional `ndarray`

.. function:: vector(name=None, dtype=config.floatX)

    Return a `Variable` for a 1-dimensional `ndarray`

.. function:: row(name=None, dtype=config.floatX)

    Return a `Variable` for a 2-dimensional `ndarray`
    in which the number of rows is guaranteed to be 1.

.. function:: col(name=None, dtype=config.floatX)

    Return a `Variable` for a 2-dimensional `ndarray`
    in which the number of columns is guaranteed to be 1.

.. function:: matrix(name=None, dtype=config.floatX)

    Return a `Variable` for a 2-dimensional `ndarray`

.. function:: tensor3(name=None, dtype=config.floatX)

    Return a `Variable` for a 3-dimensional `ndarray`

.. function:: tensor4(name=None, dtype=config.floatX)

    Return a `Variable` for a 4-dimensional `ndarray`

.. function:: tensor5(name=None, dtype=config.floatX)

    Return a `Variable` for a 5-dimensional `ndarray`

.. function:: tensor6(name=None, dtype=config.floatX)

    Return a `Variable` for a 6-dimensional `ndarray`

.. function:: tensor7(name=None, dtype=config.floatX)

    Return a `Variable` for a 7-dimensional `ndarray`

.. #COMMENT
    Each of the types described above can be constructed by two methods:
    a singular version (e.g., :ref:`dmatrix <libdoc_tensor_creation>`)
    and a plural version (:ref:`dmatrices <libdoc_tensor_creation>`).
    When called, the singular version takes a single
    argument which is the name of the *Variable* we want to make and it
    makes a single Variable of that type. The plural version can either take


All Fully-Typed Constructors
----------------------------

The following `TensorType` instances are provided in the `pytensor.tensor` module.
They are all callable, and accept an optional ``name`` argument.  So for example:

.. testcode:: constructors

   x = pt.dmatrix()        # creates one Variable with no name
   x = pt.dmatrix('x')     # creates one Variable with name 'x'
   xyz = pt.dmatrix('xyz') # creates one Variable with name 'xyz'

.. #COMMENT
    table generated by
    $ python PyTensor/doc/generate_dtype_tensor_table.py

============ =========== ==== ================ ===================================
Constructor  dtype       ndim shape            broadcastable
============ =========== ==== ================ ===================================
bscalar      int8        0    ()               ()
bvector      int8        1    (?,)             (False,)
brow         int8        2    (1,?)            (True, False)
bcol         int8        2    (?,1)            (False, True)
bmatrix      int8        2    (?,?)            (False, False)
btensor3     int8        3    (?,?,?)          (False, False, False)
btensor4     int8        4    (?,?,?,?)        (False, False, False, False)
btensor5     int8        5    (?,?,?,?,?)      (False, False, False, False, False)
btensor6     int8        6    (?,?,?,?,?,?)    (False,) * 6
btensor7     int8        7    (?,?,?,?,?,?,?)  (False,) * 7
wscalar      int16       0    ()               ()
wvector      int16       1    (?,)             (False,)
wrow         int16       2    (1,?)            (True, False)
wcol         int16       2    (?,1)            (False, True)
wmatrix      int16       2    (?,?)            (False, False)
wtensor3     int16       3    (?,?,?)          (False, False, False)
wtensor4     int16       4    (?,?,?,?)        (False, False, False, False)
wtensor5     int16       5    (?,?,?,?,?)      (False, False, False, False, False)
wtensor6     int16       6    (?,?,?,?,?,?)    (False,) * 6
wtensor7     int16       7    (?,?,?,?,?,?,?)  (False,) * 7
iscalar      int32       0    ()               ()
ivector      int32       1    (?,)             (False,)
irow         int32       2    (1,?)            (True, False)
icol         int32       2    (?,1)            (False, True)
imatrix      int32       2    (?,?)            (False, False)
itensor3     int32       3    (?,?,?)          (False, False, False)
itensor4     int32       4    (?,?,?,?)        (False, False, False, False)
itensor5     int32       5    (?,?,?,?,?)      (False, False, False, False, False)
itensor6     int32       6    (?,?,?,?,?,?)    (False,) * 6
itensor7     int32       7    (?,?,?,?,?,?,?)  (False,) * 7
lscalar      int64       0    ()               ()
lvector      int64       1    (?,)             (False,)
lrow         int64       2    (1,?)            (True, False)
lcol         int64       2    (?,1)            (False, True)
lmatrix      int64       2    (?,?)            (False, False)
ltensor3     int64       3    (?,?,?)          (False, False, False)
ltensor4     int64       4    (?,?,?,?)        (False, False, False, False)
ltensor5     int64       5    (?,?,?,?,?)      (False, False, False, False, False)
ltensor6     int64       6    (?,?,?,?,?,?)    (False,) * 6
ltensor7     int64       7    (?,?,?,?,?,?,?)  (False,) * 7
dscalar      float64     0    ()               ()
dvector      float64     1    (?,)             (False,)
drow         float64     2    (1,?)            (True, False)
dcol         float64     2    (?,1)            (False, True)
dmatrix      float64     2    (?,?)            (False, False)
dtensor3     float64     3    (?,?,?)          (False, False, False)
dtensor4     float64     4    (?,?,?,?)        (False, False, False, False)
dtensor5     float64     5    (?,?,?,?,?)      (False, False, False, False, False)
dtensor6     float64     6    (?,?,?,?,?,?)    (False,) * 6
dtensor7     float64     7    (?,?,?,?,?,?,?)  (False,) * 7
fscalar      float32     0    ()               ()
fvector      float32     1    (?,)             (False,)
frow         float32     2    (1,?)            (True, False)
fcol         float32     2    (?,1)            (False, True)
fmatrix      float32     2    (?,?)            (False, False)
ftensor3     float32     3    (?,?,?)          (False, False, False)
ftensor4     float32     4    (?,?,?,?)        (False, False, False, False)
ftensor5     float32     5    (?,?,?,?,?)      (False, False, False, False, False)
ftensor6     float32     6    (?,?,?,?,?,?)    (False,) * 6
ftensor7     float32     7    (?,?,?,?,?,?,?)  (False,) * 7
cscalar      complex64   0    ()               ()
cvector      complex64   1    (?,)             (False,)
crow         complex64   2    (1,?)            (True, False)
ccol         complex64   2    (?,1)            (False, True)
cmatrix      complex64   2    (?,?)            (False, False)
ctensor3     complex64   3    (?,?,?)          (False, False, False)
ctensor4     complex64   4    (?,?,?,?)        (False, False, False, False)
ctensor5     complex64   5    (?,?,?,?,?)      (False, False, False, False, False)
ctensor6     complex64   6    (?,?,?,?,?,?)    (False,) * 6
ctensor7     complex64   7    (?,?,?,?,?,?,?)  (False,) * 7
zscalar      complex128  0    ()               ()
zvector      complex128  1    (?,)             (False,)
zrow         complex128  2    (1,?)            (True, False)
zcol         complex128  2    (?,1)            (False, True)
zmatrix      complex128  2    (?,?)            (False, False)
ztensor3     complex128  3    (?,?,?)          (False, False, False)
ztensor4     complex128  4    (?,?,?,?)        (False, False, False, False)
ztensor5     complex128  5    (?,?,?,?,?)      (False, False, False, False, False)
ztensor6     complex128  6    (?,?,?,?,?,?)    (False,) * 6
ztensor7     complex128  7    (?,?,?,?,?,?,?)  (False,) * 7
============ =========== ==== ================ ===================================

Plural Constructors
-------------------

There are several constructors that can produce multiple variables at once.
These are not frequently used in practice, but often used in tutorial examples to save space!

.. function:: iscalars, lscalars, fscalars, dscalars

    Return one or more scalar variables.

.. function:: ivectors, lvectors, fvectors, dvectors

    Return one or more vector variables.

.. function:: irows, lrows, frows, drows

    Return one or more row variables.

.. function:: icols, lcols, fcols, dcols

    Return one or more col variables.

.. function:: imatrices, lmatrices, fmatrices, dmatrices

    Return one or more matrix variables.

Each of these plural constructors accepts
an integer or several strings. If an integer is provided, the method
will return that many `Variables` and if strings are provided, it will
create one `Variable` for each string, using the string as the `Variable`'s
name. For example:

.. testcode:: constructors

   # Creates three matrix `Variable`s with no names
   x, y, z = pt.dmatrices(3)
   # Creates three matrix `Variables` named 'x', 'y' and 'z'
   x, y, z = pt.dmatrices('x', 'y', 'z')


Custom tensor types
-------------------

If you would like to construct a tensor variable with a non-standard
broadcasting pattern, or a larger number of dimensions you'll need to create
your own :class:`TensorType` instance.  You create such an instance by passing
the dtype and broadcasting pattern to the constructor.  For example, you
can create your own 8-dimensional tensor type

>>> dtensor8 = TensorType(dtype='float64', shape=(None,)*8)
>>> x = dtensor8()
>>> z = dtensor8('z')

You can also redefine some of the provided types and they will interact
correctly:

>>> my_dmatrix = TensorType('float64', shape=(None,)*2)
>>> x = my_dmatrix()  # allocate a matrix variable
>>> my_dmatrix == dmatrix
True

See :class:`TensorType` for more information about creating new types of
tensors.


Converting from Python Objects
-------------------------------

Another way of creating a `TensorVariable` (a `TensorSharedVariable` to be
precise) is by calling :func:`pytensor.shared`

.. testcode::

    x = pytensor.shared(np.random.standard_normal((3, 4)))

This will return a :term:`shared variable <shared variable>` whose ``.value`` is
a NumPy `ndarray`.  The number of dimensions and dtype of the `Variable` are
inferred from the `ndarray` argument.  The argument to `shared` *will not be
copied*, and subsequent changes will be reflected in ``x.value``.

For additional information, see the :func:`shared() <shared.shared>` documentation.

.. _libdoc_tensor_autocasting:

Finally, when you use a NumPy `ndarray` or a Python number together with
:class:`TensorVariable` instances in arithmetic expressions, the result is a
:class:`TensorVariable`. What happens to the `ndarray` or the number?
PyTensor requires that the inputs to all expressions be `Variable` instances, so
PyTensor automatically wraps them in a :class:`TensorConstant`.

.. note::

    PyTensor makes a copy of any `ndarray` that is used in an expression, so
    subsequent changes to that `ndarray` will not have any effect on the PyTensor
    expression in which they're contained.

For NumPy `ndarrays` the dtype is given, but the static shape/broadcastable pattern must be
inferred.  The `TensorConstant` is given a type with a matching dtype,
and a static shape/broadcastable pattern with a ``1``\/``True`` for every shape
dimension that is one and ``None``\/``False`` for every dimension with an unknown
shape.

For Python numbers, the static shape/broadcastable pattern is ``()`` but the dtype must be
inferred.  Python integers are stored in the smallest dtype that can hold
them, so small constants like ``1`` are stored in a `bscalar`.
Likewise, Python floats are stored in an `fscalar` if `fscalar` suffices to hold
them perfectly, but a `dscalar` otherwise.

.. note::

    When ``config.floatX == float32`` (see :mod:`config`), then Python floats
    are stored instead as single-precision floats.

    For fine control of this rounding policy, see
    `pytensor.tensor.basic.autocast_float`.

.. function:: as_tensor_variable(x, name=None, ndim=None)

    Turn an argument `x` into a `TensorVariable` or `TensorConstant`.

    Many tensor `Op`\s run their arguments through this function as
    pre-processing.  It passes through `TensorVariable` instances, and tries to
    wrap other objects into `TensorConstant`.

    When `x` is a Python number, the dtype is inferred as described above.

    When `x` is a `list` or `tuple` it is passed through `np.asarray`

    If the `ndim` argument is not ``None``, it must be an integer and the output
    will be broadcasted if necessary in order to have this many dimensions.

    :rtype: :class:`TensorVariable` or :class:`TensorConstant`


`TensorType` and `TensorVariable`
=================================

.. class:: TensorType(Type)

    The `Type` class used to mark Variables that stand for `numpy.ndarray`
    values.  `numpy.memmap`, which is a subclass of `numpy.ndarray`, is also
    allowed.  Recalling to the tutorial, the purple box in
    :ref:`the tutorial's graph-structure figure <tutorial-graphfigure>` is an instance of this class.

    .. attribute:: shape
        A tuple of ``None`` and integer values representing the static shape associated with this
        `Type`.  ``None`` values represent unknown/non-fixed shape values.

        .. note::

            Broadcastable tuples/values are an old Theano construct that are
            being phased-out in PyTensor.

    .. attribute:: broadcastable

        A tuple of ``True``\/``False`` values, one for each dimension.  ``True`` in
        position ``i`` indicates that at evaluation-time, the `ndarray` will have
        size one in that ``i``-th dimension.  Such a dimension is called a
        *broadcastable dimension* (see :ref:`tutbroadcasting`).

        The broadcastable pattern indicates both the number of dimensions and
        whether a particular dimension must have length one.

        Here is a table mapping some broadcastable patterns to what they
        mean:

        ===================== =================================
        pattern               interpretation
        ===================== =================================
        []                    scalar
        [True]                1D scalar (vector of length 1)
        [True, True]          2D scalar (1x1 matrix)
        [False]               vector
        [False, False]        matrix
        [False] * n           nD tensor
        [True, False]         row (1xN matrix)
        [False, True]         column (Mx1 matrix)
        [False, True, False]  A Mx1xP tensor (a)
        [True, False, False]  A 1xNxP tensor (b)
        [False, False, False] A MxNxP tensor (pattern of a + b)
        ===================== =================================

        For dimensions in which broadcasting is ``False``, the length of this
        dimension can be one or more.  For dimensions in which broadcasting is ``True``,
        the length of this dimension must be one.

        When two arguments to an element-wise operation (like addition or
        subtraction) have a different number of dimensions, the broadcastable
        pattern is *expanded to the left*, by padding with ``True``. For example,
        a vector's pattern, ``[False]``, could be expanded to ``[True, False]``, and
        would behave like a row (1xN matrix). In the same way, a matrix (``[False,
        False]``) would behave like a 1xNxP tensor (``[True, False, False]``).

        If we wanted to create a `TensorType` representing a matrix that would
        broadcast over the middle dimension of a 3-dimensional tensor when
        adding them together, we would define it like this:

        >>> middle_broadcaster = TensorType('complex64', shape=(None, 1, None))

    .. attribute:: ndim

        The number of dimensions that a `Variable`'s value will have at
        evaluation-time.  This must be known when we are building the
        expression graph.

    .. attribute:: dtype

        A string indicating the numerical type of the `ndarray` for which a
        `Variable` of this `Type` represents.

        .. _dtype_list:

        The :attr:`dtype` attribute of a `TensorType` instance can be any of the
        following strings.

        ================= =================== =================
        dtype             domain              bits
        ================= =================== =================
        ``'int8'``        signed integer      8
        ``'int16'``       signed integer      16
        ``'int32'``       signed integer      32
        ``'int64'``       signed integer      64
        ``'uint8'``       unsigned integer    8
        ``'uint16'``      unsigned integer    16
        ``'uint32'``      unsigned integer    32
        ``'uint64'``      unsigned integer    64
        ``'float32'``     floating point      32
        ``'float64'``     floating point      64
        ``'complex64'``   complex             64 (two float32)
        ``'complex128'``  complex             128 (two float64)
        ================= =================== =================

    .. method:: __init__(self, dtype, broadcastable)

        If you wish to use a `Type` that is not already available (for example,
        a 5D tensor), you can build an appropriate `Type` by instantiating
        :class:`TensorType`.


`TensorVariable`
----------------

.. class:: TensorVariable(Variable, _tensor_py_operators)

    A `Variable` type that represents symbolic tensors.

    See :class:`_tensor_py_operators` for most of the attributes and methods
    you'll want to call.

.. class:: TensorConstant(Variable, _tensor_py_operators)

    Python and NumPy numbers are wrapped in this type.

    See :class:`_tensor_py_operators` for most of the attributes and methods
    you'll want to call.

.. class:: TensorSharedVariable(Variable, _tensor_py_operators)

    This type is returned by :func:`shared` when the value to share is a NumPy
    ndarray.

    See :class:`_tensor_py_operators` for most of the attributes and methods
    you'll want to call.


.. autoclass:: pytensor.tensor.variable._tensor_py_operators
   :members:

    This mix-in class adds convenient attributes, methods, and support
    to `TensorVariable`, `TensorConstant` and `TensorSharedVariable` for
    Python operators (see :ref:`tensor_operator_support`).

    .. attribute:: type

        A reference to the :class:`TensorType` instance describing the sort of
        values that might be associated with this variable.

    .. attribute:: ndim
       :noindex:

        The number of dimensions of this tensor.  Aliased to
        :attr:`TensorType.ndim`.

    .. attribute:: dtype
       :noindex:

        The numeric type of this tensor. Aliased to
        :attr:`TensorType.dtype`.

    .. method:: reshape(shape, ndim=None)
       :noindex:

        Returns a view of this tensor that has been reshaped as in
        `numpy.reshape`.  If the shape is a `Variable` argument, then you might
        need to use the optional `ndim` parameter to declare how many elements
        the shape has, and therefore how many dimensions the reshaped Variable
        will have.

        See :func:`reshape`.

    .. method:: dimshuffle(*pattern)
       :noindex:

        Returns a view of this tensor with permuted dimensions.  Typically the
        pattern will include the integers ``0, 1, ... ndim-1``, and any number of
        ``'x'`` characters in dimensions where this tensor should be broadcasted.

        A few examples of patterns and their effect:

            * ``('x',)``: make a 0d (scalar) into a 1d vector
            * ``(0, 1)``: identity for 2d vectors
            * ``(1, 0)``: inverts the first and second dimensions
            * ``('x', 0)``: make a row out of a 1d vector (N to 1xN)
            * ``(0, 'x')``: make a column out of a 1d vector (N to Nx1)
            * ``(2, 0, 1)``: AxBxC to CxAxB
            * ``(0, 'x', 1)``: AxB to Ax1xB
            * ``(1, 'x', 0)``: AxB to Bx1xA
            * ``(1,)``: This removes the dimension at index 0. It must be a broadcastable dimension.

    .. method:: flatten(ndim=1)

        Returns a view of this tensor with `ndim` dimensions, whose shape for the first
        ``ndim-1`` dimensions will be the same as ``self``, and shape in the
        remaining dimension will be expanded to fit in all the data from ``self``.

        See :func:`flatten`.

    .. method:: ravel()

        return `flatten`. For NumPy compatibility.

    .. attribute:: T

        Transpose of this tensor.

        >>> x = pt.zmatrix()
        >>> y = 3+.2j * x.T

    .. method:: {any,all}(axis=None, keepdims=False)
    .. method:: {sum,prod,mean}(axis=None, dtype=None, keepdims=False, acc_dtype=None)
    .. method:: {var,std,min,max,argmin,argmax}(axis=None, keepdims=False),
    .. method:: diagonal(offset=0, axis1=0, axis2=1)
    .. method:: astype(dtype)
    .. method:: take(indices, axis=None, mode='raise')
    .. method:: copy()
       :noindex:

        Return a new symbolic variable that is a copy of the variable. Does not copy the tag.

    .. method:: norm(L, axis=None)
    .. method:: nonzero(self, return_matrix=False)
       :noindex:
    .. method:: nonzero_values(self)
       :noindex:
    .. method:: sort(self, axis=-1, kind='quicksort', order=None)
       :noindex:
    .. method:: argsort(self, axis=-1, kind='quicksort', order=None)
       :noindex:
    .. method:: clip(self, a_min, a_max) with a_min <= a_max
    .. method:: conf()
    .. method:: repeat(repeats, axis=None)
       :noindex:
    .. method:: round(mode="half_away_from_zero")
       :noindex:
    .. method:: trace()
    .. method:: get_underlying_scalar_constant_value()
    .. method:: zeros_like(model, dtype=None)

       All the above methods are equivalent to NumPy for PyTensor on the current tensor.

    .. method:: __{abs,neg,lt,le,gt,ge,invert,and,or,add,sub,mul,div,truediv,floordiv}__

       Those elemwise operation are supported via Python syntax.

Shaping and Shuffling
=====================

To re-order the dimensions of a variable, to insert or remove broadcastable
dimensions, see :meth:`_tensor_py_operators.dimshuffle`.

.. function:: shape(x)

    Returns an `lvector` representing the shape of `x`.

.. function:: reshape(x, newshape, ndim=None)
   :noindex:

    :type x: any `TensorVariable` (or compatible)
    :param x: variable to be reshaped

    :type newshape: `lvector` (or compatible)
    :param newshape: the new shape for `x`

    :param ndim: optional - the length that `newshape`'s value will have.
        If this is ``None``, then `reshape` will infer it from `newshape`.

    :rtype: variable with `x`'s dtype, but `ndim` dimensions

    .. note::

        This function can infer the length of a symbolic `newshape` value in
        some cases, but if it cannot and you do not provide the `ndim`, then
        this function will raise an Exception.


.. function:: shape_padleft(x, n_ones=1)

    Reshape `x` by left padding the shape with `n_ones` 1s.
    All new dimensions will be broadcastable.

    :param x: variable to be reshaped
    :type x: any `TensorVariable` (or compatible)

    :type n_ones: int
    :type n_ones: number of dimension to be added to `x`



.. function:: shape_padright(x, n_ones=1)

    Reshape `x` by right padding the shape with `n_ones` ones.
    All new dimensions will be broadcastable.

    :param x: variable to be reshaped
    :type x: any TensorVariable (or compatible)

    :type n_ones: int
    :type n_ones: number of dimension to be added to `x`


.. function:: shape_padaxis(t, axis)

    Reshape `t` by inserting ``1`` at the dimension `axis`.
    All new dimensions will be broadcastable.

    :type x: any `TensorVariable` (or compatible)
    :param x: variable to be reshaped

    :type axis: int
    :param  axis: axis where to add the new dimension to `x`

    Example:

    >>> tensor = pytensor.tensor.type.tensor3()
    >>> pytensor.tensor.shape_padaxis(tensor, axis=0)
    InplaceDimShuffle{x,0,1,2}.0
    >>> pytensor.tensor.shape_padaxis(tensor, axis=1)
    InplaceDimShuffle{0,x,1,2}.0
    >>> pytensor.tensor.shape_padaxis(tensor, axis=3)
    InplaceDimShuffle{0,1,2,x}.0
    >>> pytensor.tensor.shape_padaxis(tensor, axis=-1)
    InplaceDimShuffle{0,1,2,x}.0

.. autofunction:: specify_shape(x, shape)

.. function:: flatten(x, ndim=1)

    Similar to :func:`reshape`, but the shape is inferred from the shape of `x`.

    :param x: variable to be flattened
    :type x: any `TensorVariable` (or compatible)

    :type ndim: int
    :param ndim: the number of dimensions in the returned variable

    :rtype: variable with same dtype as `x` and `ndim` dimensions
    :returns: variable with the same shape as `x` in the leading `ndim-1`
        dimensions, but with all remaining dimensions of `x` collapsed into
        the last dimension.

    For example, if we flatten a tensor of shape ``(2, 3, 4, 5)`` with ``flatten(x,
    ndim=2)``, then we'll have the same (i.e. ``2-1=1``) leading dimensions
    ``(2,)``, and the remaining dimensions are collapsed, so the output in this
    example would have shape ``(2, 60)``.


.. function:: tile(x, reps, ndim=None)

    Construct an array by repeating the input `x` according to `reps`
    pattern.

    Tiles its input according to `reps`. The length of `reps` is the
    number of dimension of `x` and contains the number of times to
    tile `x` in each dimension.

    :see: `numpy.tile
        <http://docs.scipy.org/doc/numpy/reference/generated/numpy.tile.html>`_
        documentation for examples.

    :see: :func:`pytensor.tensor.extra_ops.repeat
        <pytensor.tensor.extra_ops.repeat>`

    :note: Currently, `reps` must be a constant, `x.ndim` and
        ``len(reps)`` must be equal and, if specified, `ndim` must be
        equal to both.

.. autofunction:: roll

Creating Tensors
================


.. function:: zeros_like(x, dtype=None)

    :param x: tensor that has the same shape as output
    :param dtype: data-type, optional
                  By default, it will be x.dtype.

    Returns a tensor the shape of `x` filled with zeros of the type of `dtype`.


.. function:: ones_like(x)


    :param x: tensor that has the same shape as output
    :param dtype: data-type, optional
                  By default, it will be `x.dtype`.

    Returns a tensor the shape of `x` filled with ones of the type of `dtype`.


.. function:: zeros(shape, dtype=None)

    :param shape: a tuple/list of scalar with the shape information.
    :param dtype: the dtype of the new tensor. If ``None``, will use ``"floatX"``.

    Returns a tensor filled with zeros of the provided shape.

.. function:: ones(shape, dtype=None)

    :param shape: a tuple/list of scalar with the shape information.
    :param dtype: the dtype of the new tensor. If ``None``, will use ``"floatX"``.

    Returns a tensor filled with ones of the provided shape.

.. function:: fill(a,b)

    :param a: tensor that has same shape as output
    :param b: PyTensor scalar or value with which you want to fill the output

    Create a matrix by filling the shape of `a` with `b`.

.. function:: alloc(value, *shape)

    :param value: a value with which to fill the output
    :param shape: the dimensions of the returned array
    :returns: an N-dimensional tensor initialized by `value` and having the specified shape.

.. function:: eye(n, m=None, k=0, dtype=pytensor.config.floatX)

    :param n: number of rows in output (value or PyTensor scalar)
    :param m: number of columns in output (value or PyTensor scalar)
    :param k: Index of the diagonal: ``0`` refers to the main diagonal,
              a positive value refers to an upper diagonal, and a
              negative value to a lower diagonal. It can be an PyTensor
              scalar.
    :returns: An array where all elements are equal to zero, except for the `k`-th
              diagonal, whose values are equal to one.

.. function:: identity_like(x, dtype=None)

    :param x: tensor
    :param dtype: The dtype of the returned tensor. If `None`, default to dtype of `x`
    :returns: A tensor of same shape as `x` that is filled with zeros everywhere
              except for the main diagonal, whose values are equal to one. The output
              will have same dtype as `x` unless overridden in `dtype`.

.. function:: stack(tensors, axis=0)

    Stack tensors in sequence on given axis (default is ``0``).

    Take a sequence of tensors and stack them on given axis to make a single
    tensor. The size in dimension `axis` of the result will be equal to the number
    of tensors passed.

    :param tensors: a list or a tuple of one or more tensors of the same rank.
    :param axis: the axis along which the tensors will be stacked. Default value is ``0``.
    :returns: A tensor such that ``rval[0] == tensors[0]``, ``rval[1] == tensors[1]``, etc.

    Examples:

    >>> a = pytensor.tensor.type.scalar()
    >>> b = pytensor.tensor.type.scalar()
    >>> c = pytensor.tensor.type.scalar()
    >>> x = pytensor.tensor.stack([a, b, c])
    >>> x.ndim # x is a vector of length 3.
    1
    >>> a = pytensor.tensor.type.tensor4()
    >>> b = pytensor.tensor.type.tensor4()
    >>> c = pytensor.tensor.type.tensor4()
    >>> x = pytensor.tensor.stack([a, b, c])
    >>> x.ndim # x is a 5d tensor.
    5
    >>> rval = x.eval(dict((t, np.zeros((2, 2, 2, 2))) for t in [a, b, c]))
    >>> rval.shape # 3 tensors are stacked on axis 0
    (3, 2, 2, 2, 2)

    We can also specify different axis than default value ``0``:

    >>> x = pytensor.tensor.stack([a, b, c], axis=3)
    >>> x.ndim
    5
    >>> rval = x.eval(dict((t, np.zeros((2, 2, 2, 2))) for t in [a, b, c]))
    >>> rval.shape # 3 tensors are stacked on axis 3
    (2, 2, 2, 3, 2)
    >>> x = pytensor.tensor.stack([a, b, c], axis=-2)
    >>> x.ndim
    5
    >>> rval = x.eval(dict((t, np.zeros((2, 2, 2, 2))) for t in [a, b, c]))
    >>> rval.shape # 3 tensors are stacked on axis -2
    (2, 2, 2, 3, 2)

.. function:: stack(*tensors)
   :noindex:

    .. warning::

        The interface `stack(*tensors)` is deprecated!  Use
        `stack(tensors, axis=0)` instead.

    Stack tensors in sequence vertically (row wise).

    Take a sequence of tensors and stack them vertically to make a single
    tensor.

    :param tensors: one or more tensors of the same rank
    :returns: A tensor such that ``rval[0] == tensors[0]``, ``rval[1] == tensors[1]``, etc.

    >>> x0 = pt.scalar()
    >>> x1 = pt.scalar()
    >>> x2 = pt.scalar()
    >>> x = pt.stack(x0, x1, x2)
    >>> x.ndim # x is a vector of length 3.
    1

.. function:: concatenate(tensor_list, axis=0)

    :type tensor_list: a list or tuple of Tensors that all have the same shape in the axes
                        *not* specified by the `axis` argument.
    :param tensor_list: one or more Tensors to be concatenated together into one.
    :type axis: literal or symbolic integer
    :param axis: Tensors will be joined along this axis, so they may have different
        ``shape[axis]``

    >>> x0 = pt.fmatrix()
    >>> x1 = pt.ftensor3()
    >>> x2 = pt.fvector()
    >>> x = pt.concatenate([x0, x1[0], pt.shape_padright(x2)], axis=1)
    >>> x.ndim
    2

.. function:: stacklists(tensor_list)

    :type tensor_list: an iterable that contains either tensors or other
        iterables of the same type as `tensor_list` (in other words, this
        is a tree whose leaves are tensors).
    :param tensor_list: tensors to be stacked together.

    Recursively stack lists of tensors to maintain similar structure.

    This function can create a tensor from a shaped list of scalars:

    >>> from pytensor.tensor import stacklists, scalars, matrices
    >>> from pytensor import function
    >>> a, b, c, d = scalars('abcd')
    >>> X = stacklists([[a, b], [c, d]])
    >>> f = function([a, b, c, d], X)
    >>> f(1, 2, 3, 4)
    array([[ 1.,  2.],
           [ 3.,  4.]])

    We can also stack arbitrarily shaped tensors. Here we stack matrices into
    a 2 by 2 grid:

    >>> from numpy import ones
    >>> a, b, c, d = matrices('abcd')
    >>> X = stacklists([[a, b], [c, d]])
    >>> f = function([a, b, c, d], X)
    >>> x = ones((4, 4), 'float32')
    >>> f(x, x, x, x).shape
    (2, 2, 4, 4)

.. autofunction:: pytensor.tensor.basic.choose

Reductions
==========


.. function:: max(x, axis=None, keepdims=False)

    :Parameter: *x* -  symbolic Tensor (or compatible)
    :Parameter: *axis* - axis or axes along which to compute the maximum
    :Parameter: *keepdims* - (boolean) If this is set to True, the axes which are reduced are
        left in the result as dimensions with size one. With this option, the result
        will broadcast correctly against the original tensor.
    :Returns: maximum of *x* along *axis*

    axis can be:
     * *None* - in which case the maximum is computed along all axes (like NumPy)
     * an *int* - computed along this axis
     * a *list of ints* - computed along these axes

.. function:: argmax(x, axis=None, keepdims=False)

    :Parameter: *x* - symbolic Tensor (or compatible)
    :Parameter: *axis* - axis along which to compute the index of the maximum
    :Parameter: *keepdims* - (boolean) If this is set to True, the axis which is reduced is
        left in the result as a dimension with size one. With this option, the result
        will broadcast correctly against the original tensor.
    :Returns: the index of the maximum value along a given axis

    if ``axis == None``, `argmax` over the flattened tensor (like NumPy)

.. function:: max_and_argmax(x, axis=None, keepdims=False)

    :Parameter: *x* - symbolic Tensor (or compatible)
    :Parameter: *axis* - axis along which to compute the maximum and its index
    :Parameter: *keepdims* - (boolean) If this is set to True, the axis which is reduced is
        left in the result as a dimension with size one. With this option, the result
        will broadcast correctly against the original tensor.
    :Returns: the maximum value along a given axis and its index.

    if ``axis == None``, `max_and_argmax` over the flattened tensor (like NumPy)

.. function:: min(x, axis=None, keepdims=False)

    :Parameter: *x* -  symbolic Tensor (or compatible)
    :Parameter: *axis* - axis or axes along which to compute the minimum
    :Parameter: *keepdims* - (boolean) If this is set to True, the axes which are reduced are
        left in the result as dimensions with size one. With this option, the result
        will broadcast correctly against the original tensor.
    :Returns: minimum of *x* along *axis*

    `axis` can be:
     * ``None`` - in which case the minimum is computed along all axes (like NumPy)
     * an *int* - computed along this axis
     * a *list of ints* - computed along these axes

.. function:: argmin(x, axis=None, keepdims=False)

    :Parameter: *x* - symbolic Tensor (or compatible)
    :Parameter: *axis* - axis along which to compute the index of the minimum
    :Parameter: *keepdims* - (boolean) If this is set to True, the axes which are reduced are
        left in the result as dimensions with size one. With this option, the result
        will broadcast correctly against the original tensor.
    :Returns: the index of the minimum value along a given axis

    if ``axis == None``, `argmin` over the flattened tensor (like NumPy)

.. function:: sum(x, axis=None, dtype=None, keepdims=False, acc_dtype=None)

    :Parameter: *x* -  symbolic Tensor (or compatible)
    :Parameter: *axis* - axis or axes along which to compute the sum
    :Parameter: *dtype* - The dtype of the returned tensor.
        If None, then we use the default dtype which is the same as
        the input tensor's dtype except when:

        - the input dtype is a signed integer of precision < 64 bit, in
          which case we use int64
        - the input dtype is an unsigned integer of precision < 64 bit, in
          which case we use uint64

        This default dtype does _not_ depend on the value of "acc_dtype".

    :Parameter: *keepdims* - (boolean) If this is set to True, the axes which are reduced are
        left in the result as dimensions with size one. With this option, the result
        will broadcast correctly against the original tensor.

    :Parameter: *acc_dtype* -  The dtype of the internal accumulator.
        If None (default), we use the dtype in the list below,
        or the input dtype if its precision is higher:

        - for int dtypes, we use at least int64;
        - for uint dtypes, we use at least uint64;
        - for float dtypes, we use at least float64;
        - for complex dtypes, we use at least complex128.

    :Returns: sum of *x* along *axis*

    `axis` can be:
     * ``None`` - in which case the sum is computed along all axes (like NumPy)
     * an int - computed along this axis
     * a list of ints - computed along these axes

.. function:: prod(x, axis=None, dtype=None, keepdims=False, acc_dtype=None, no_zeros_in_input=False)

    :Parameter: *x* -  symbolic Tensor (or compatible)
    :Parameter: *axis* - axis or axes along which to compute the product
    :Parameter: *dtype* - The dtype of the returned tensor.
        If None, then we use the default dtype which is the same as
        the input tensor's dtype except when:

        - the input dtype is a signed integer of precision < 64 bit, in
          which case we use int64
        - the input dtype is an unsigned integer of precision < 64 bit, in
          which case we use uint64

        This default dtype does _not_ depend on the value of "acc_dtype".

    :Parameter: *keepdims* - (boolean) If this is set to True, the axes which are reduced are
        left in the result as dimensions with size one. With this option, the result
        will broadcast correctly against the original tensor.

    :Parameter: *acc_dtype* -  The dtype of the internal accumulator.
        If None (default), we use the dtype in the list below,
        or the input dtype if its precision is higher:

        - for int dtypes, we use at least int64;
        - for uint dtypes, we use at least uint64;
        - for float dtypes, we use at least float64;
        - for complex dtypes, we use at least complex128.

    :Parameter: *no_zeros_in_input* - The grad of prod is complicated
         as we need to handle 3 different cases: without zeros in the
         input reduced group, with 1 zero or with more zeros.

    This could slow you down, but more importantly, we currently
    don't support the second derivative of the 3 cases. So you
    cannot take the second derivative of the default prod().

    To remove the handling of the special cases of 0 and so get
    some small speed up and allow second derivative set
    ``no_zeros_in_inputs`` to ``True``. It defaults to ``False``.

    **It is the user responsibility to make sure there are no zeros
    in the inputs. If there are, the grad will be wrong.**

    :Returns: product of every term in *x* along *axis*

    `axis` can be:
     * ``None`` - in which case the sum is computed along all axes (like NumPy)
     * an int - computed along this axis
     * a list of ints - computed along these axes

.. function:: mean(x, axis=None, dtype=None, keepdims=False, acc_dtype=None)

    :Parameter: *x* -  symbolic Tensor (or compatible)
    :Parameter: *axis* - axis or axes along which to compute the mean
    :Parameter: *dtype* - The dtype to cast the result of the inner summation into.
        For instance, by default, a sum of a float32 tensor will be
        done in float64 (acc_dtype would be float64 by default),
        but that result will be casted back in float32.
    :Parameter: *keepdims* - (boolean) If this is set to True, the axes which are reduced are
        left in the result as dimensions with size one. With this option, the result
        will broadcast correctly against the original tensor.
    :Parameter: *acc_dtype* -  The dtype of the internal accumulator of the
        inner summation. This will not necessarily be the dtype of the
        output (in particular if it is a discrete (int/uint) dtype, the
        output will be in a float type).  If None, then we use the same
        rules as :func:`sum`.
    :Returns: mean value of *x* along *axis*

    `axis` can be:
     * ``None`` - in which case the mean is computed along all axes (like NumPy)
     * an int - computed along this axis
     * a list of ints - computed along these axes

.. function:: var(x, axis=None, keepdims=False)

    :Parameter: *x* -  symbolic Tensor (or compatible)
    :Parameter: *axis* - axis or axes along which to compute the variance
    :Parameter: *keepdims* - (boolean) If this is set to True, the axes which are reduced are
        left in the result as dimensions with size one. With this option, the result
        will broadcast correctly against the original tensor.
    :Returns: variance of *x* along *axis*

    `axis` can be:
     * ``None`` - in which case the variance is computed along all axes (like NumPy)
     * an int - computed along this axis
     * a list of ints - computed along these axes

.. function:: std(x, axis=None, keepdims=False)

    :Parameter: *x* -  symbolic Tensor (or compatible)
    :Parameter: *axis* - axis or axes along which to compute the standard deviation
    :Parameter: *keepdims* - (boolean) If this is set to True, the axes which are reduced are
        left in the result as dimensions with size one. With this option, the result
        will broadcast correctly against the original tensor.
    :Returns: variance of *x* along *axis*

    `axis` can be:
     * ``None`` - in which case the standard deviation is computed along all axes (like NumPy)
     * an int - computed along this axis
     * a list of ints - computed along these axes

.. function:: all(x, axis=None, keepdims=False)

    :Parameter: *x* -  symbolic Tensor (or compatible)
    :Parameter: *axis* - axis or axes along which to apply 'bitwise and'
    :Parameter: *keepdims* - (boolean) If this is set to True, the axes which are reduced are
        left in the result as dimensions with size one. With this option, the result
        will broadcast correctly against the original tensor.
    :Returns: bitwise and of *x* along *axis*

    `axis` can be:
     * ``None`` - in which case the 'bitwise and' is computed along all axes (like NumPy)
     * an int - computed along this axis
     * a list of ints - computed along these axes

.. function:: any(x, axis=None, keepdims=False)

    :Parameter: *x* -  symbolic Tensor (or compatible)
    :Parameter: *axis* - axis or axes along which to apply bitwise or
    :Parameter: *keepdims* - (boolean) If this is set to True, the axes which are reduced are
        left in the result as dimensions with size one. With this option, the result
        will broadcast correctly against the original tensor.
    :Returns: bitwise or of *x* along *axis*

    `axis` can be:
     * ``None`` - in which case the 'bitwise or' is computed along all axes (like NumPy)
     * an int - computed along this axis
     * a list of ints - computed along these axes

.. function:: ptp(x, axis = None)

    Range of values (maximum - minimum) along an axis.
    The name of the function comes from the acronym for peak to peak.

    :Parameter: *x* Input tensor.

    :Parameter: *axis* Axis along which to find the peaks. By default,
                flatten the array.

    :Returns: A new array holding the result.

.. _indexing:

Indexing
========

Like NumPy, PyTensor distinguishes between *basic* and *advanced* indexing.
PyTensor fully supports basic indexing
(see `NumPy's indexing  <https://numpy.org/doc/stable/user/basics.indexing.html>`_)
and `integer advanced indexing
<https://numpy.org/doc/stable/user/basics.indexing.html#integer-array-indexing>`_.

Index-assignment is *not* supported.  If you want to do something like ``a[5]
= b`` or ``a[5]+=b``, see :func:`pytensor.tensor.subtensor.set_subtensor` and
:func:`pytensor.tensor.subtensor.inc_subtensor` below.

.. autofunction:: pytensor.tensor.subtensor.set_subtensor

.. autofunction:: pytensor.tensor.subtensor.inc_subtensor

.. _tensor_operator_support:

Operator Support
================

Many Python operators are supported.

>>> a, b = pt.itensor3(), pt.itensor3() # example inputs

Arithmetic
--------------

.. doctest::
   :options: +SKIP

   >>> a + 3      # pt.add(a, 3) -> itensor3
   >>> 3 - a      # pt.sub(3, a)
   >>> a * 3.5    # pt.mul(a, 3.5) -> ftensor3 or dtensor3 (depending on casting)
   >>> 2.2 / a    # pt.truediv(2.2, a)
   >>> 2.2 // a   # pt.intdiv(2.2, a)
   >>> 2.2**a     # pt.pow(2.2, a)
   >>> b % a      # pt.mod(b, a)

Bitwise
-------------

.. doctest::
   :options: +SKIP

   >>> a & b      # pt.and_(a,b)    bitwise and (alias pt.bitwise_and)
   >>> a ^ 1      # pt.xor(a,1)     bitwise xor (alias pt.bitwise_xor)
   >>> a | b      # pt.or_(a,b)     bitwise or (alias pt.bitwise_or)
   >>> ~a         # pt.invert(a)    bitwise invert (alias pt.bitwise_not)

Inplace
-------

In-place operators are *not* supported.  PyTensor's graph rewrites
will determine which intermediate values to use for in-place
computations.  If you would like to update the value of a
:term:`shared variable`, consider using the ``updates`` argument to
:func:`PyTensor.function`.

.. _libdoc_tensor_elemwise:

:class:`Elemwise`
=================

Casting
-------

.. function:: cast(x, dtype)

    Cast any tensor `x` to a tensor of the same shape, but with a different
    numerical type `dtype`.

    This is not a reinterpret cast, but a coercion `cast`, similar to
    ``numpy.asarray(x, dtype=dtype)``.

    .. testcode:: cast

        import pytensor.tensor as pt
        x = pt.matrix()
        x_as_int = pt.cast(x, 'int32')

    Attempting to casting a complex value to a real value is ambiguous and
    will raise an exception.  Use `real`, `imag`, `abs`, or `angle`.

.. function:: real(x)

    Return the real (not imaginary) components of tensor `x`.
    For non-complex `x` this function returns `x`.

.. function:: imag(x)

    Return the imaginary components of tensor `x`.
    For non-complex `x` this function returns ``zeros_like(x)``.


Comparisons
-----------

The six usual equality and inequality operators share the same interface.
  :Parameter:  *a* - symbolic Tensor (or compatible)
  :Parameter:  *b* - symbolic Tensor (or compatible)
  :Return type: symbolic Tensor
  :Returns: a symbolic tensor representing the application of the logical :class:`Elemwise` operator.

  .. note::

    PyTensor has no boolean dtype.  Instead, all boolean tensors are represented
    in ``'int8'``.

  Here is an example with the less-than operator.

  .. testcode:: oper

    import pytensor.tensor as pt
    x,y = pt.dmatrices('x','y')
    z = pt.le(x,y)

.. function:: lt(a, b)

    Returns a symbolic ``'int8'`` tensor representing the result of logical less-than (a<b).

    Also available using syntax ``a < b``

.. function:: gt(a, b)

    Returns a symbolic ``'int8'`` tensor representing the result of logical greater-than (a>b).

    Also available using syntax ``a > b``

.. function:: le(a, b)

    Returns a variable representing the result of logical less than or equal (a<=b).

    Also available using syntax ``a <= b``

.. function:: ge(a, b)

    Returns a variable representing the result of logical greater or equal than (a>=b).

    Also available using syntax ``a >= b``

.. function:: eq(a, b)

    Returns a variable representing the result of logical equality (a==b).

.. function:: neq(a, b)

    Returns a variable representing the result of logical inequality (a!=b).

.. function:: greater(a, b)

    Alias for `gt`. greater is the NumPy name.

.. function:: greater_equal(a, b)

    Alias for `ge`. greater_equal is the NumPy name.

.. function:: less(a, b)

    Alias for `lt`. less is the NumPy name.

.. function:: less_equal(a, b)

    Alias for `le`. less_equal is the NumPy name.

.. function:: equal(a, b)

    Alias for `eq`. equal is the NumPy name.

.. function:: not_equal(a, b)

    Alias for `neq`. not_equal is the NumPy name.

.. function:: isnan(a)

    Returns a variable representing the comparison of ``a`` elements with nan.

    This is equivalent to ``numpy.isnan``.

.. function:: isinf(a)

    Returns a variable representing the comparison of ``a`` elements
    with inf or -inf.

    This is equivalent to ``numpy.isinf``.

.. function:: isclose(a, b, rtol=1e-05, atol=1e-08, equal_nan=False)

    Returns a symbolic ``'int8'`` tensor representing where two tensors are equal
    within a tolerance.

    The tolerance values are positive, typically very small numbers.
    The relative difference `(rtol * abs(b))` and the absolute difference `atol` are
    added together to compare against the absolute difference between `a` and `b`.

    For finite values, isclose uses the following equation to test whether two
    floating point values are equivalent:
    ``|a - b| <= (atol + rtol * |b|)``

    For infinite values, isclose checks if both values are the same signed inf value.

    If equal_nan is True, isclose considers NaN values in the same position to be close.
    Otherwise, NaN values are not considered close.

    This is equivalent to ``numpy.isclose``.

.. function:: allclose(a, b, rtol=1e-05, atol=1e-08, equal_nan=False)

    Returns a symbolic ``'int8'`` value representing if all elements in two tensors are equal
    within a tolerance.

    See notes in `isclose` for determining values equal within a tolerance.

    This is equivalent to ``numpy.allclose``.

Condition
---------

.. function:: switch(cond, ift, iff)

    Returns a variable representing a switch between ift (i.e. "if true") and iff (i.e. "if false")
    based on the condition cond. This is the PyTensor equivalent of `numpy.where`.

      :Parameter:  *cond* - symbolic Tensor (or compatible)
      :Parameter:  *ift* - symbolic Tensor (or compatible)
      :Parameter:  *iff* - symbolic Tensor (or compatible)
      :Return type: symbolic Tensor

    .. testcode:: switch

      import pytensor.tensor as pt
      a,b = pt.dmatrices('a','b')
      x,y = pt.dmatrices('x','y')
      z = pt.switch(pt.lt(a,b), x, y)

.. function:: where(cond, ift, iff)

   Alias for `switch`. where is the NumPy name.

.. function:: clip(x, min, max)

    Return a variable representing `x`, but with all elements greater than
    `max` clipped to `max` and all elements less than `min` clipped to `min`.

    Normal broadcasting rules apply to each of `x`, `min`, and `max`.

    Note that there is no warning for inputs that are the wrong way round
    (`min > max`), and that results in this case may differ from `numpy.clip`.

Bit-wise
--------


The bitwise operators possess this interface:
    :Parameter:  *a* - symbolic tensor of integer type.
    :Parameter:  *b* - symbolic tensor of integer type.

    .. note::

        The bitwise operators must have an integer type as input.

        The bit-wise not (invert) takes only one parameter.

    :Return type: symbolic tensor with corresponding dtype.

.. function:: and_(a, b)

    Returns a variable representing the result of the bitwise and.

.. function:: or_(a, b)

    Returns a variable representing the result of the bitwise or.

.. function:: xor(a, b)

    Returns a variable representing the result of the bitwise xor.

.. function:: invert(a)

    Returns a variable representing the result of the bitwise not.

.. function:: bitwise_and(a, b)

   Alias for `and_`. bitwise_and is the NumPy name.

.. function:: bitwise_or(a, b)

   Alias for `or_`. bitwise_or is the NumPy name.

.. function:: bitwise_xor(a, b)

   Alias for `xor_`. bitwise_xor is the NumPy name.

.. function:: bitwise_not(a, b)

   Alias for invert. invert is the NumPy name.

Here is an example using the bit-wise ``and_`` via the ``&`` operator:

.. testcode:: bitwise

    import pytensor.tensor as pt
    x,y = pt.imatrices('x','y')
    z = x & y


Mathematical
------------

.. function:: abs(a)

    Returns a variable representing the absolute of ``a``, i.e. ``|a|``.

    .. note:: Can also be accessed using `builtins.abs`: i.e. ``abs(a)``.

.. function:: angle(a)

    Returns a variable representing angular component of complex-valued Tensor ``a``.

.. function:: exp(a)

    Returns a variable representing the exponential of ``a``.

.. function:: maximum(a, b)

   Returns a variable representing the maximum element by element of a and b

.. function:: minimum(a, b)

   Returns a variable representing the minimum element by element of a and b

.. function:: neg(a)

    Returns a variable representing the negation of ``a`` (also ``-a``).

.. function:: reciprocal(a)

    Returns a variable representing the inverse of a, ie 1.0/a. Also called reciprocal.

.. function:: log(a), log2(a), log10(a)

    Returns a variable representing the base e, 2 or 10 logarithm of a.

.. function:: sign(a)

    Returns a variable representing the sign of a.

.. function:: ceil(a)

    Returns a variable representing the ceiling of a (for example ceil(2.1) is 3).

.. function:: floor(a)

    Returns a variable representing the floor of a (for example floor(2.9) is 2).

.. function:: round(a, mode="half_away_from_zero")
   :noindex:

    Returns a variable representing the rounding of a in the same dtype as a. Implemented rounding mode are half_away_from_zero and half_to_even.

.. function:: iround(a, mode="half_away_from_zero")

    Short hand for cast(round(a, mode),'int64').

.. function:: sqr(a)

    Returns a variable representing the square of a, ie a^2.

.. function:: sqrt(a)

    Returns a variable representing the of a, ie a^0.5.

.. function:: cos(a), sin(a), tan(a)

    Returns a variable representing the trigonometric functions of a (cosine, sine and tangent).

.. function:: cosh(a), sinh(a), tanh(a)

    Returns a variable representing the hyperbolic trigonometric functions of a (hyperbolic cosine, sine and tangent).

.. function:: erf(a), erfc(a)

    Returns a variable representing the error function or the complementary error function. `wikipedia <http://en.wikipedia.org/wiki/Error_function>`__

.. function:: erfinv(a), erfcinv(a)

    Returns a variable representing the inverse error function or the inverse complementary error function. `wikipedia <http://en.wikipedia.org/wiki/Error_function#Inverse_functions>`__

.. function:: gamma(a)

   Returns a variable representing the gamma function.

.. function:: gammaln(a)

   Returns a variable representing the logarithm of the gamma function.

.. function:: psi(a)

   Returns a variable representing the derivative of the logarithm of
   the gamma function (also called the digamma function).

.. function:: chi2sf(a, df)

   Returns a variable representing the survival function (1-cdf —
   sometimes more accurate).

   C code is provided in the Theano_lgpl repository.
   This makes it faster.

   https://github.com/Theano/Theano_lgpl.git


.. _libdoc_tensor_broadcastable:

You can find more information about Broadcasting in the :ref:`tutbroadcasting` tutorial.

Linear Algebra
==============

.. function:: dot(X, Y)

     For 2-D arrays it is equivalent to matrix multiplication, and for
     1-D arrays to inner product of vectors (without complex
     conjugation). For N dimensions it is a sum product over the last
     axis of a and the second-to-last of b:

    :param X: left term
    :param Y: right term
    :type X: symbolic tensor
    :type Y: symbolic tensor
    :rtype: symbolic matrix or vector
    :return: the inner product of `X` and `Y`.

.. function:: outer(X, Y)

    :param X: left term
    :param Y: right term
    :type X: symbolic vector
    :type Y: symbolic vector
    :rtype: symbolic matrix

    :return: vector-vector outer product

.. function:: tensordot(a, b, axes=2)

    Given two tensors a and b,tensordot computes a generalized dot product over
    the provided axes. PyTensor's implementation reduces all expressions to
    matrix or vector dot products and is based on code from Tijmen Tieleman's
    `gnumpy` (http://www.cs.toronto.edu/~tijmen/gnumpy.html).

    :param a: the first tensor variable
    :type a: symbolic tensor

    :param b: the second tensor variable
    :type b: symbolic tensor

    :param axes: an integer or array. If an integer, the number of axes
                 to sum over. If an array, it must have two array
                 elements containing the axes to sum over in each tensor.

                 Note that the default value of 2 is not guaranteed to work
                 for all values of a and b, and an error will be raised if
                 that is the case. The reason for keeping the default is to
                 maintain the same signature as NumPy's tensordot function
                 (and np.tensordot raises analogous errors for non-compatible
                 inputs).

                 If an integer i, it is converted to an array containing
                 the last i dimensions of the first tensor and the first
                 i dimensions of the second tensor:

                     axes = [range(a.ndim - i, b.ndim), range(i)]

                 If an array, its two elements must contain compatible axes
                 of the two tensors. For example, [[1, 2], [2, 0]] means sum
                 over the 2nd and 3rd axes of a and the 3rd and 1st axes of b.
                 (Remember axes are zero-indexed!) The 2nd axis of a and the
                 3rd axis of b must have the same shape; the same is true for
                 the 3rd axis of a and the 1st axis of b.
    :type axes: int or array-like of length 2

    :returns: a tensor with shape equal to the concatenation of a's shape
              (less any dimensions that were summed over) and b's shape
              (less any dimensions that were summed over).
    :rtype: symbolic tensor

    It may be helpful to consider an example to see what tensordot does.
    PyTensor's implementation is identical to NumPy's. Here a has shape (2, 3, 4)
    and b has shape (5, 6, 4, 3). The axes to sum over are [[1, 2], [3, 2]] --
    note that a.shape[1] == b.shape[3] and a.shape[2] == b.shape[2]; these axes
    are compatible. The resulting tensor will have shape (2, 5, 6) -- the
    dimensions that are not being summed:

    .. testcode:: tensordot

        import numpy as np

        a = np.random.random((2,3,4))
        b = np.random.random((5,6,4,3))

        c = np.tensordot(a, b, [[1,2],[3,2]])

        a0, a1, a2 = a.shape
        b0, b1, _, _ = b.shape
        cloop = np.zeros((a0,b0,b1))

        # Loop over non-summed indices--these exist in the tensor product
        for i in range(a0):
            for j in range(b0):
                for k in range(b1):
                    # Loop over summed indices--these don't exist in the tensor product
                    for l in range(a1):
                        for m in range(a2):
                            cloop[i,j,k] += a[i,l,m] * b[j,k,m,l]

        assert np.allclose(c, cloop)

    This specific implementation avoids a loop by transposing a and b such that
    the summed axes of a are last and the summed axes of b are first. The
    resulting arrays are reshaped to 2 dimensions (or left as vectors, if
    appropriate) and a matrix or vector dot product is taken. The result is
    reshaped back to the required output dimensions.

    In an extreme case, no axes may be specified. The resulting tensor
    will have shape equal to the concatenation of the shapes of a and b:

    .. doctest:: tensordot

        >>> c = np.tensordot(a, b, 0)
        >>> a.shape
        (2, 3, 4)
        >>> b.shape
        (5, 6, 4, 3)
        >>> print(c.shape)
        (2, 3, 4, 5, 6, 4, 3)

    :note: See the documentation of `numpy.tensordot <http://docs.scipy.org/doc/numpy/reference/generated/numpy.tensordot.html>`_ for more examples.

.. function:: batched_dot(X, Y)

    :param x: A Tensor with sizes e.g.: for  3D (dim1, dim3, dim2)
    :param y: A Tensor with sizes e.g.: for 3D (dim1, dim2, dim4)

    This function computes the dot product between the two tensors, by iterating
    over the first dimension using scan.
    Returns a tensor of size e.g. if it is 3D: (dim1, dim3, dim4)
    Example:

    >>> first = pt.tensor3('first')
    >>> second = pt.tensor3('second')
    >>> result = batched_dot(first, second)

    :note:  This is a subset of `numpy.einsum`, but we do not provide it for now.

    :param X: left term
    :param Y: right term
    :type X: symbolic tensor
    :type Y: symbolic tensor

    :return: tensor of products

.. function:: batched_tensordot(X, Y, axes=2)

    :param x: A Tensor with sizes e.g.: for 3D (dim1, dim3, dim2)
    :param y: A Tensor with sizes e.g.: for 3D (dim1, dim2, dim4)
    :param axes: an integer or array. If an integer, the number of axes
                 to sum over. If an array, it must have two array
                 elements containing the axes to sum over in each tensor.

                 If an integer i, it is converted to an array containing
                 the last i dimensions of the first tensor and the first
                 i dimensions of the second tensor (excluding the first
                 (batch) dimension)::

                     axes = [range(a.ndim - i, b.ndim), range(1,i+1)]

                 If an array, its two elements must contain compatible axes
                 of the two tensors. For example, [[1, 2], [2, 4]] means sum
                 over the 2nd and 3rd axes of a and the 3rd and 5th axes of b.
                 (Remember axes are zero-indexed!) The 2nd axis of a and the
                 3rd axis of b must have the same shape; the same is true for
                 the 3rd axis of a and the 5th axis of b.
    :type axes: int or array-like of length 2

    :returns: a tensor with shape equal to the concatenation of a's shape
              (less any dimensions that were summed over) and b's shape
              (less first dimension and any dimensions that were summed over).
    :rtype: tensor of tensordots

    A hybrid of batch_dot and tensordot, this function computes the
    tensordot product between the two tensors, by iterating over the
    first dimension using scan to perform a sequence of tensordots.

    :note: See :func:`tensordot` and :func:`batched_dot` for
        supplementary documentation.

.. function:: mgrid

    :returns: an instance which returns a dense (or fleshed out) mesh-grid
              when indexed, so that each returned argument has the same shape.
              The dimensions and number of the output arrays are equal to the
              number of indexing dimensions. If the step length is not a complex
              number, then the stop is not inclusive.

    Example:

    >>> a = pt.mgrid[0:5, 0:3]
    >>> a[0].eval()
    array([[0, 0, 0],
           [1, 1, 1],
           [2, 2, 2],
           [3, 3, 3],
           [4, 4, 4]])
    >>> a[1].eval()
    array([[0, 1, 2],
           [0, 1, 2],
           [0, 1, 2],
           [0, 1, 2],
           [0, 1, 2]])

.. function:: ogrid

    :returns: an instance which returns an open (i.e. not fleshed out) mesh-grid
              when indexed, so that only one dimension of each returned array is
              greater than 1. The dimension and number of the output arrays are
              equal to the number of indexing dimensions. If the step length is
              not a complex number, then the stop is not inclusive.

    Example:

    >>> b = pt.ogrid[0:5, 0:3]
    >>> b[0].eval()
    array([[0],
           [1],
           [2],
           [3],
           [4]])
    >>> b[1].eval()
    array([[0, 1, 2]])


Gradient / Differentiation
==========================

.. automodule:: pytensor.gradient
    :members: grad
    :noindex:

