Add new RandomVariable Op and implementations

doc/library/scan.txt

@@ -273,7 +273,7 @@ the following:
    bvis = theano.shared(bvis_values)
    bhid = theano.shared(bhid_values)
 
-   trng = tt.shared_randomstreams.RandomStreams(1234)
+   trng = tt.random.utils.RandomStream(1234)
 
    def OneStep(vsample) :
        hmean = tt.nnet.sigmoid(theano.dot(vsample, W) + bhid)
@@ -354,7 +354,7 @@ updated:
     bvis = theano.shared(bvis_values)
     bhid = theano.shared(bhid_values)
 
-    trng = tt.shared_randomstreams.RandomStreams(1234)
+    trng = tt.random.utils.RandomStream(1234)
 
     # OneStep, with explicit use of the shared variables (W, bvis, bhid)
     def OneStep(vsample, W, bvis, bhid):

doc/library/tensor/index.txt

@@ -9,8 +9,8 @@
    :synopsis: symbolic types and operations for n-dimensional arrays.
 .. moduleauthor:: LISA
 
-Theano's strength is in expressing symbolic calculations involving tensors.  
-There are many types of symbolic expressions for tensors. 
+Theano's strength is in expressing symbolic calculations involving tensors.
+There are many types of symbolic expressions for tensors.
 They are grouped into the following sections:
 
 
@@ -19,8 +19,7 @@ They are grouped into the following sections:
 
     basic
     nnet/index
-    raw_random
-    shared_randomstreams
+    random/index
     signal/index
     utils
     elemwise

doc/library/tensor/random/basic.txt

+
+.. _libdoc_tensor_random:
+
+=============================================
+:mod:`random` -- Low-level random numbers
+=============================================
+
+.. module:: theano.tensor.random
+   :synopsis: symbolic random variables
+.. moduleauthor:: pymc-team
+
+
+The `theano.tensor.random` module provides random-number drawing functionality
+that closely resembles the `numpy.random` module.
+
+Reference
+=========
+
+.. class:: RandomStream()
+
+   A helper class that tracks changes in a shared ``numpy.random.RandomState``
+   and behaves like ``numpy.random.RandomState`` by managing access
+   to `RandomVariable`s.  For example:
+
+   .. testcode:: constructors
+
+      from theano.tensor.random.utils import RandomStream
+
+      rng = RandomStream()
+      sample = rng.normal(0, 1, size=(2, 2))
+
+.. class:: RandomStateType(gof.Type)
+
+    A `Type` for variables that will take ``numpy.random.RandomState``
+    values.
+
+.. function:: random_state_type(name=None)
+
+    Return a new Variable whose ``.type`` is ``random_state_type``.
+
+.. class:: RandomVariable(gof.Op)
+
+    `Op` that draws random numbers from a `numpy.random.RandomState` object.
+    This `Op` is parameterized to draw numbers from many possible
+    distributions.

doc/library/tensor/shared_randomstreams.txt → doc/library/tensor/random/utils.txt

-.. _libdoc_tensor_shared_randomstreams:
+.. _libdoc_tensor_random_utils:
 
 ======================================================
-:mod:`shared_randomstreams` -- Friendly random numbers
+:mod:`utils` -- Friendly random numbers
 ======================================================
 
-.. module:: theano.tensor.shared_randomstreams
+.. module:: theano.tensor.random.utils
    :platform: Unix, Windows
    :synopsis: symbolic random variables
 .. moduleauthor:: LISA
@@ -27,17 +27,15 @@ For an example of how to use random numbers, see
 Reference
 =========
 
-.. class:: RandomStreams(raw_random.RandomStreamsBase)
+.. class:: RandomStream()
 
-    This is a symbolic stand-in for ``numpy.random.RandomState``. 
-    Random variables of various distributions are instantiated by calls to
-    parent class :class:`raw_random.RandomStreamsBase`.
+    This is a symbolic stand-in for ``numpy.random.RandomState``.
 
     .. method:: updates()
 
         :returns: a list of all the (state, new_state) update pairs for the
           random variables created by this object
-          
+
         This can be a convenient shortcut to enumerating all the random
         variables in a large graph in the ``update`` parameter of function.
 
@@ -60,21 +58,4 @@ Reference
 
     .. method:: uniform, normal, binomial, multinomial, random_integers, ...
 
-        See :class:`raw_random.RandomStreamsBase`.
-
-.. class:: RandomVariable(object)
-
-    .. attribute:: rng
-
-        The shared variable whose ``.value`` is the numpy RandomState
-        generator feeding this random variable.
-
-    .. attribute:: update
-        
-        A pair
-        whose first element is a shared variable whose value is a numpy RandomState,
-        and whose second element is an [symbolic] expression for the next value of that
-        RandomState after drawing samples.
-        Including this pair in the``updates`` list to function will cause the
-        function to update the random number generator feeding this variable.
-
+        See :class:`basic.RandomVariable`.

doc/library/tensor/raw_random.txt

-
-.. _libdoc_tensor_raw_random:
-
-=============================================
-:mod:`raw_random` -- Low-level random numbers
-=============================================
-
-.. module:: theano.tensor.raw_random
-   :synopsis: symbolic random variables
-.. moduleauthor:: LISA
-
-
-Raw random provides the random-number drawing functionality, that underlies
-the friendlier :class:`RandomStreams` interface.
-
-Reference
-=========
-
-.. class:: RandomStreamsBase(object)
-    
-    This is the interface for the
-    :class:`theano.tensor.shared_randomstreams.RandomStreams` subclass
-
-    .. method:: binomial(self, size=(), n=1, p=0.5, ndim=None):
-
-        Sample ``n`` times with probability of success ``p`` for each
-        trial and return the number of successes.
-
-        If ``size`` is ambiguous on the number of dimensions, ``ndim``
-        may be a plain integer to supplement the missing information.
-
-	This wraps the numpy implementation, so it has the same
-	behavior.
-
-    .. method:: uniform(self,  size=(), low=0.0, high=1.0, ndim=None):
-
-        Sample a tensor of the given size whose elements come from a
-        uniform distribution between low and high.
-
-        If ``size`` is ambiguous on the number of dimensions, ``ndim``
-        may be a plain integer to supplement the missing information.
-
-	This wraps the numpy implementation, so it has the same
-        bounds: [``low``, ``high``\[.
-
-    .. method:: normal(self, size=(), avg=0.0, std=1.0, ndim=None):
-
-        Sample from a normal distribution centered on ``avg`` with the
-        specified standard deviation (``std``)
-
-        If ``size`` is ambiguous on the number of dimensions, ``ndim``
-        may be a plain integer to supplement the missing information.
-
-	This wrap numpy implementation, so it have the same behavior.
-
-    .. method:: random_integers(self, size=(), low=0, high=1, ndim=None):
-
-        Sample a random integer between low and high, both inclusive.
-
-        If ``size`` is ambiguous on the number of dimensions, ``ndim``
-        may be a plain integer to supplement the missing information.
-
-	This is a generalization of :py:func:`numpy.random.random_integers`
-        to the case where low and high are tensors. Otherwise it
-        behaves the same.
-
-    .. method:: choice(self, size=(), a=2, replace=True, p=None, ndim=None, dtype='int64'):
-
-        Choose values from ``a`` with or without replacement. ``a``
-        can be a 1-D array or a positive scalar. If ``a`` is a scalar,
-        the samples are drawn from the range [0, ``a``\[.
-
-        If ``size`` is ambiguous on the number of dimensions, ``ndim``
-        may be a plain integer to supplement the missing information.
-
-	This wraps the numpy implementation so it has the same behavior.
-
-    .. method:: poisson(self, size=(), lam=None, ndim=None, dtype='int64'):
-
-        Draw samples from a Poisson distribution.
-  
-        The Poisson distribution is the limit of the Binomial
-        distribution for large N.
-
-        If ``size`` is ambiguous on the number of dimensions, ``ndim``
-        may be a plain integer to supplement the missing information.
-
-	This wraps the numpy implementation so it has the same behavior.
-
-    .. method:: permutation(self, size=(), n=1, ndim=None):
-
-        Returns permutations of the integers between 0 and ``n-1``, as
-        many times as required by ``size``. For instance, if
-        ``size=(p,q)``, ``p*q`` permutations will be generated, and
-        the output shape will be ``(p,q,n)``, because each permutation
-        is of size ``n``.
-
-        Theano tries to infer the number of dimensions from the length
-        of ``size``, but you may always specify it with ``ndim``.
-
-        .. note::
-            The output will have ``ndim+1`` dimensions.
-
-        This is a generalization of :py:func:`numpy.random.permutation` to
-        tensors. Otherwise it behaves the same.
-
-    .. method:: multinomial(self, size=(), n=1, pvals=[0.5, 0.5], ndim=None):
-
-        Sample n times from a multinomial distribution defined by
-        probabilities ``pvals``, as many times as required by
-        ``size``. For instance, if ``size=(p,q)``, ``p*q`` samples
-        will be drawn, and the output shape will be
-        ``(p,q,len(pvals))``.
-
-        Theano tries to infer the number of dimensions from the length
-        of ``size``, but you may always specify it with ``ndim``.
-
-        .. note::
-            The output will have ``ndim+1`` dimensions.
-
-	This is a generalization of :py:func:`numpy.random.multinomial`
-        to the case where ``n`` and ``pvals`` are tensors. Otherwise
-        it behaves the same.
-
-    .. method:: shuffle_row_elements(self, input):
-
-        Return a variable with every row (rightmost index) shuffled.
-
-        This uses a permutation random variable internally, available
-        via the ``.permutation`` attribute of the return value.
-
-.. class:: RandomStateType(gof.Type)
-
-    A `Type` for variables that will take ``numpy.random.RandomState``
-    values.
-
-.. function:: random_state_type(name=None)
-
-    Return a new Variable whose ``.type`` is ``random_state_type``.
-
-.. class:: RandomFunction(gof.Op)
-
-    Op that draws random numbers from a numpy.RandomState object.
-    This Op is parametrized to draw numbers from many possible
-    distributions.
-
-.. function:: uniform(random_state, size=None, low=0.0, high=1.0, ndim=None, dtype=None)
-
-    Sample from a uniform distribution between low and high.
-
-    If the size argument is ambiguous on the number of
-    dimensions, the first argument may be a plain integer
-    to supplement the missing information.
-
-    :returns: :class:`RandomVariable`, NewRandomState
-
-.. function:: binomial(random_state, size=None, n=1, p=0.5, ndim=None, dtype='int64')
-
-    Sample ``n`` times with probability of success ``p`` for each
-    trial and return the number of successes.
-
-    If ``size`` is ambiguous on the number of dimensions, ``ndim`` may
-    be a plain integer to supplement the missing information.
-
-    :returns: :class:`RandomVariable`, NewRandomState
-
-.. function:: normal(random_state, size=None, avg=0.0, std=1.0, ndim=None, dtype=None)
-
-    Sample from a normal distribution centered on ``avg`` with the
-    specified standard deviation (``std``).
-
-    If ``size`` is ambiguous on the number of dimensions, ``ndim`` may
-    be a plain integer to supplement the missing information.
-
-    :returns: :class:`RandomVariable`, NewRandomState
-
-.. function:: random_integers(random_state, size=None, low=0, high=1, ndim=None, dtype='int64')
-
-    Sample random integers in [``low``, ``high``] to fill up ``size``.
-
-    If ``size`` is ambiguous on the number of dimensions, ``ndim`` may
-    be a plain integer to supplement the missing information.
-
-    :returns: :class:`RandomVariable`, NewRandomState
-
-.. function:: permutation(random_state, size=None, n=1, ndim=None, dtype='int64')
-
-    Returns permutations of the integers in [0, ``n``\[, as many times
-    as required by ``size``. For instance, if ``size=(p,q)``, ``p*q``
-    permutations will be generated, and the output shape will be
-    ``(p,q,n)``, because each permutation is of size ``n``.
-
-    If ``size`` is ambiguous on the number of dimensions, ``ndim``
-    may be a plain integer, which should correspond to ``len(size)``.
-
-    .. note::
-        The output will have ``ndim+1`` dimensions.
-
-    :returns: :class:`RandomVariable`, NewRandomState
-
-.. function:: multinomial(random_state, size=None, p_vals=[0.5, 0.5], ndim=None, dtype='int64')
-
-    Sample from a multinomial distribution defined by probabilities
-    ``pvals``, as many times as required by ``size``. For instance, if
-    ``size=(p,q)``, ``p*q`` samples will be drawn, and the output
-    shape will be ``(p,q,len(pvals))``.
-
-    If ``size`` is ambiguous on the number of dimensions, ``ndim``
-    may be a plain integer, which should correspond to ``len(size)``.
-
-    .. note::
-        The output will have ``ndim+1`` dimensions.
-
-    :returns: :class:`RandomVariable`, NewRandomState
-

doc/nextml2015/presentation.tex

@@ -866,7 +866,7 @@ X = tt.matrix("X")
 b_sym = tt.vector("b_sym")
 
 # define shared random stream
-trng = tt.shared_randomstreams.RandomStreams(1234)
+trng = tt.random.utils.RandomStream(1234)
 d=trng.binomial(size=W[1].shape)
 \end{lstlisting}
 \end{frame}

doc/tutorial/examples.txt

@@ -343,13 +343,13 @@ NumPy, though also not too complicated.
 
 The way to think about putting randomness into Theano's computations is
 to put random variables in your graph. Theano will allocate a NumPy
-RandomStream object (a random number generator) for each such
+`RandomStream` object (a random number generator) for each such
 variable, and draw from it as necessary. We will call this sort of
 sequence of random numbers a *random stream*. *Random streams* are at
 their core shared variables, so the observations on shared variables
 hold here as well. Theano's random objects are defined and implemented in
-:ref:`RandomStreams<libdoc_tensor_shared_randomstreams>` and, at a lower level,
-in :ref:`RandomStreamsBase<libdoc_tensor_raw_random>`.
+:ref:`RandomStream<libdoc_tensor_random_utils>` and, at a lower level,
+in :ref:`RandomVariable<libdoc_tensor_random_basic>`.
 
 Brief Example
 -------------
@@ -361,11 +361,11 @@ Here's a brief example.  The setup code is:
 
 .. testcode::
 
-    from theano.tensor.shared_randomstreams import RandomStreams
+    from theano.tensor.random.utils import RandomStream
     from theano import function
-    srng = RandomStreams(seed=234)
-    rv_u = srng.uniform((2,2))
-    rv_n = srng.normal((2,2))
+    srng = RandomStream(seed=234)
+    rv_u = srng.uniform(0, 1, size=(2,2))
+    rv_n = srng.normal(0, 1, size=(2,2))
     f = function([], rv_u)
     g = function([], rv_n, no_default_updates=True)    #Not updating rv_n.rng
     nearly_zeros = function([], rv_u + rv_u - 2 * rv_u)
@@ -373,8 +373,8 @@ Here's a brief example.  The setup code is:
 Here, 'rv_u' represents a random stream of 2x2 matrices of draws from a uniform
 distribution.  Likewise,  'rv_n' represents a random stream of 2x2 matrices of
 draws from a normal distribution.  The distributions that are implemented are
-defined in :class:`RandomStreams` and, at a lower level,
-in :ref:`raw_random<libdoc_tensor_raw_random>`. They only work on CPU.
+defined as :class:`RandomVariable`s
+in :ref:`basic<libdoc_tensor_random_basic>`. They only work on CPU.
 See `Other Implementations`_ for GPU version.
 
 
@@ -412,7 +412,7 @@ You can seed just one random variable by seeding or assigning to the
 >>> rng_val.seed(89234)                         # seeds the generator
 >>> rv_u.rng.set_value(rng_val, borrow=True)    # Assign back seeded rng
 
-You can also seed *all* of the random variables allocated by a :class:`RandomStreams`
+You can also seed *all* of the random variables allocated by a :class:`RandomStream`
 object by that object's ``seed`` method.  This seed will be used to seed a
 temporary random number generator, that will in turn generate seeds for each
 of the random variables.
@@ -447,11 +447,11 @@ number generators associated with a given theano graph (e.g. g1, with compiled
 function f1 below) to a second graph (e.g. g2, with function f2). This might
 arise for example if you are trying to initialize the state of a model, from
 the parameters of a pickled version of a previous model. For
-:class:`theano.tensor.shared_randomstreams.RandomStreams` and
-:class:`theano.sandbox.rng_mrg.MRG_RandomStreams`
+:class:`theano.tensor.random.utils.RandomStream` and
+:class:`theano.sandbox.rng_mrg.MRG_RandomStream`
 this can be achieved by copying elements of the `state_updates` parameter.
 
-Each time a random variable is drawn from a RandomStreams object, a tuple is
+Each time a random variable is drawn from a `RandomStream` object, a tuple is
 added to the `state_updates` list. The first element is a shared variable,
 which represents the state of the random number generator associated with this
 *particular* variable, while the second represents the theano graph
@@ -464,12 +464,12 @@ to another is shown below.
 >>> import theano
 >>> import numpy
 >>> import theano.tensor as tt
->>> from theano.sandbox.rng_mrg import MRG_RandomStreams
->>> from theano.tensor.shared_randomstreams import RandomStreams
+>>> from theano.sandbox.rng_mrg import MRG_RandomStream
+>>> from theano.tensor.random.utils import RandomStream
 
 >>> class Graph():
 ...     def __init__(self, seed=123):
-...         self.rng = RandomStreams(seed)
+...         self.rng = RandomStream(seed)
 ...         self.y = self.rng.uniform(size=(1,))
 
 >>> g1 = Graph(seed=123)
@@ -485,7 +485,7 @@ array([ 0.72803009])
 array([ 0.55056769])
 
 >>> def copy_random_state(g1, g2):
-...     if isinstance(g1.rng, MRG_RandomStreams):
+...     if isinstance(g1.rng, MRG_RandomStream):
 ...         g2.rng.rstate = g1.rng.rstate
 ...     for (su1, su2) in zip(g1.rng.state_updates, g2.rng.state_updates):
 ...         su2[0].set_value(su1[0].get_value())
@@ -501,7 +501,7 @@ array([ 0.59044123])
 Other Random Distributions
 --------------------------
 
-There are :ref:`other distributions implemented <libdoc_tensor_raw_random>`.
+There are :ref:`other distributions implemented <libdoc_tensor_random_basic>`.
 
 .. _example_other_random:
 
@@ -510,7 +510,7 @@ Other Implementations
 
 There is another implementations based on :ref:`MRG31k3p
 <libdoc_rng_mrg>`.
-The RandomStream only work on the CPU, MRG31k3p work on the CPU and GPU.
+The `RandomStream` only work on the CPU, MRG31k3p work on the CPU and GPU.
 
 .. note::
 
@@ -518,7 +518,7 @@ The RandomStream only work on the CPU, MRG31k3p work on the CPU and GPU.
 
         .. code-block:: python
 
-            from theano.sandbox.rng_mrg import MRG_RandomStreams as RandomStreams
+            from theano.sandbox.rng_mrg import MRG_RandomStream as RandomStream
 
 .. _logistic_regression:
 

doc/tutorial/loop.txt

@@ -329,7 +329,7 @@ Note that we need to iterate over the indices of ``y`` and not over the elements
   b_sym = tt.vector("b_sym")
 
   # define shared random stream
-  trng = tt.shared_randomstreams.RandomStreams(1234)
+  trng = tt.random.utils.RandomStream(1234)
   d=trng.binomial(size=W[1].shape)
 
   results, updates = theano.scan(lambda v: tt.tanh(tt.dot(v, W) + b_sym) * d, sequences=X)

requirements.txt

@@ -15,3 +15,4 @@ jaxlib; python_version > '3.6'
 diff-cover
 pre-commit
 isort
+pypolyagamma

tests/compile/test_builders.py

@@ -11,7 +11,7 @@ from theano.compile.builders import OpFromGraph
 from theano.compile.function import function
 from theano.gof.null_type import NullType
 from theano.gradient import DisconnectedType
-from theano.tensor.shared_randomstreams import RandomStreams
+from theano.tensor.random.utils import RandomStream
 
 
 class TestOpFromGraph(unittest_tools.InferShapeTester):
@@ -359,7 +359,7 @@ class TestOpFromGraph(unittest_tools.InferShapeTester):
         assert results == expect_result
 
         # Inner graph where some computation doesn't rely on explicit inputs
-        srng = RandomStreams(seed=234)
+        srng = RandomStream(seed=234)
         rv_u = srng.uniform((2, 2))
         x, y = tt.matrices("xy")
         out1 = x + rv_u

tests/gof/test_graph.py

@@ -4,7 +4,7 @@ from itertools import count
 import numpy as np
 import pytest
 
-from theano import shared, sparse, tensor
+from theano import shared, tensor
 from theano.gof.graph import (
     Apply,
     Variable,
@@ -287,34 +287,6 @@ class TestAutoName:
         assert r2.auto_name == "auto_" + str(autoname_id + 1)
         assert r3.auto_name == "auto_" + str(autoname_id + 2)
 
-    @pytest.mark.skipif(
-        not sparse.enable_sparse, reason="Optional package SciPy not installed"
-    )
-    def test_sparsevariable(self):
-        # Get counter value
-        autoname_id = next(Variable.__count__)
-        Variable.__count__ = count(autoname_id)
-        r1 = sparse.csc_matrix(name="x", dtype="float32")
-        r2 = sparse.dense_from_sparse(r1)
-        r3 = sparse.csc_from_dense(r2)
-        assert r1.auto_name == "auto_" + str(autoname_id)
-        assert r2.auto_name == "auto_" + str(autoname_id + 1)
-        assert r3.auto_name == "auto_" + str(autoname_id + 2)
-
-    def test_randomvariable(self):
-        # Get counter value
-        autoname_id = next(Variable.__count__)
-        Variable.__count__ = count(autoname_id)
-        mytype = tensor.TensorType(dtype="int32", broadcastable=())
-        r1 = tensor.shared_randomstreams.RandomStateSharedVariable(
-            name="x", type=mytype, value=1, strict=False
-        )
-        r2 = tensor.shared_randomstreams.RandomStateSharedVariable(
-            name="x", type=mytype, value=1, strict=False
-        )
-        assert r1.auto_name == "auto_" + str(autoname_id)
-        assert r2.auto_name == "auto_" + str(autoname_id + 1)
-
     def test_clone(self):
         # Get counter value
         autoname_id = next(Variable.__count__)

tests/gpuarray/test_multinomial.py

@@ -10,7 +10,7 @@ from theano.gpuarray.multinomial import (
     GPUAMultinomialFromUniform,
 )
 from theano.sandbox import multinomial
-from theano.sandbox.rng_mrg import MRG_RandomStreams as RandomStreams
+from theano.sandbox.rng_mrg import MRG_RandomStream as RandomStream
 
 
 def test_multinomial_output_dtype():
@@ -262,7 +262,7 @@ class TestFunctionWor:
     def test_select_distinct(self):
         # Tests that multinomial_wo_replacement always selects distinct elements
 
-        th_rng = RandomStreams(12345)
+        th_rng = RandomStream(12345)
 
         p = tensor.fmatrix()
         n = tensor.iscalar()
@@ -285,7 +285,7 @@ class TestFunctionWor:
         # Tests that multinomial_wo_replacement fails when asked to sample more
         # elements than the actual number of elements
 
-        th_rng = RandomStreams(12345)
+        th_rng = RandomStream(12345)
 
         p = tensor.fmatrix()
         n = tensor.iscalar()
@@ -305,7 +305,7 @@ class TestFunctionWor:
         # Tests that multinomial_wo_replacement selects elements, on average,
         # proportional to the their probabilities
 
-        th_rng = RandomStreams(12345)
+        th_rng = RandomStream(12345)
 
         p = tensor.fmatrix()
         n = tensor.iscalar()

tests/gpuarray/test_rng_mrg.py

@@ -11,7 +11,7 @@ from theano import config, tensor
 from theano.gpuarray.rng_mrg import GPUA_mrg_uniform
 from theano.gpuarray.type import gpuarray_shared_constructor
 from theano.sandbox import rng_mrg
-from theano.sandbox.rng_mrg import MRG_RandomStreams
+from theano.sandbox.rng_mrg import MRG_RandomStream
 
 
 utt.seed_rng()
@@ -42,7 +42,7 @@ def test_consistency_GPUA_serial():
             rstate.default_update = new_rstate
 
             # Not really necessary, just mimicking
-            # rng_mrg.MRG_RandomStreams' behavior
+            # rng_mrg.MRG_RandomStream' behavior
             sample.rstate = rstate
             sample.update = (rstate, new_rstate)
 
@@ -89,7 +89,7 @@ def test_consistency_GPUA_parallel():
         rstate.default_update = new_rstate
 
         # Not really necessary, just mimicking
-        # rng_mrg.MRG_RandomStreams' behavior
+        # rng_mrg.MRG_RandomStream' behavior
         sample.rstate = rstate
         sample.update = (rstate, new_rstate)
 
@@ -117,7 +117,7 @@ def test_GPUA_full_fill():
     # This needs to be large to trigger the problem on GPU
     size = (10, 1000)
 
-    R = MRG_RandomStreams(234)
+    R = MRG_RandomStream(234)
     uni = R.uniform(size, nstreams=60 * 256)
     f_cpu = theano.function([], uni)
 
@@ -177,7 +177,7 @@ def test_f16_nonzero():
 
 
 def test_cpu_target_with_shared_variable():
-    srng = MRG_RandomStreams()
+    srng = MRG_RandomStream()
     s = np.random.rand(2, 3).astype("float32")
     x = gpuarray_shared_constructor(s, name="x")
     try:

tests/gpuarray/test_scan.py

@@ -231,7 +231,7 @@ class TestScan:
             dtype="float32",
         )
         vsample = theano.shared(v_vsample)
-        trng = theano.sandbox.rng_mrg.MRG_RandomStreams(utt.fetch_seed())
+        trng = theano.sandbox.rng_mrg.MRG_RandomStream(utt.fetch_seed())
 
         def f(vsample_tm1):
             return (
@@ -513,7 +513,7 @@ class ScanGpuTests:
             dtype="float32",
         )
         vsample = theano.shared(v_vsample)
-        trng = theano.sandbox.rng_mrg.MRG_RandomStreams(utt.fetch_seed())
+        trng = theano.sandbox.rng_mrg.MRG_RandomStream(utt.fetch_seed())
 
         def f(vsample_tm1):
             return (

tests/misc/test_pkl_utils.py

@@ -6,7 +6,7 @@ import numpy as np
 
 import theano
 from theano.misc.pkl_utils import StripPickler, dump, load
-from theano.sandbox.rng_mrg import MRG_RandomStreams
+from theano.sandbox.rng_mrg import MRG_RandomStream
 
 
 class TestDumpLoad:
@@ -23,7 +23,7 @@ class TestDumpLoad:
             shutil.rmtree(self.tmpdir)
 
     def test_dump_load_mrg(self):
-        rng = MRG_RandomStreams()
+        rng = MRG_RandomStream()
 
         with open("test", "wb") as f:
             dump(rng, f)
@@ -31,7 +31,7 @@ class TestDumpLoad:
         with open("test", "rb") as f:
             rng = load(f)
 
-        assert type(rng) == MRG_RandomStreams
+        assert type(rng) == MRG_RandomStream
 
     def test_dump_zip_names(self):
         foo_1 = theano.shared(0, name="foo")

tests/sandbox/test_jax.py

@@ -330,7 +330,7 @@ def test_jax_scan_multiple_output():
     delta = tt.scalar("delta")
 
     # TODO: Use random streams when their JAX conversions are implemented.
-    # trng = tt.shared_randomstreams.RandomStreams(1234)
+    # trng = tt.random.RandomStream(1234)
 
     def seir_one_step(ct0, dt0, st0, et0, it0, logp_c, logp_d, beta, gamma, delta):
         # bt0 = trng.binomial(n=st0, p=beta)

tests/sandbox/test_multinomial_wo_replacement.py

@@ -3,7 +3,7 @@ import pytest
 
 from theano import config, function, tensor
 from theano.sandbox import multinomial
-from theano.sandbox.rng_mrg import MRG_RandomStreams as RandomStreams
+from theano.sandbox.rng_mrg import MRG_RandomStream as RandomStream
 
 
 class TestOP:
@@ -146,7 +146,7 @@ class TestFunction:
     def test_select_distinct(self):
         # Tests that multinomial_wo_replacement always selects distinct elements
 
-        th_rng = RandomStreams(12345)
+        th_rng = RandomStream(12345)
 
         p = tensor.fmatrix()
         n = tensor.iscalar()
@@ -169,7 +169,7 @@ class TestFunction:
         # Tests that multinomial_wo_replacement fails when asked to sample more
         # elements than the actual number of elements
 
-        th_rng = RandomStreams(12345)
+        th_rng = RandomStream(12345)
 
         p = tensor.fmatrix()
         n = tensor.iscalar()
@@ -189,7 +189,7 @@ class TestFunction:
         # Tests that multinomial_wo_replacement selects elements, on average,
         # proportional to the their probabilities
 
-        th_rng = RandomStreams(12345)
+        th_rng = RandomStream(12345)
 
         p = tensor.fmatrix()
         n = tensor.iscalar()

tests/sandbox/test_rng_mrg.py

@@ -9,10 +9,10 @@ import theano
 from tests import unittest_tools as utt
 from theano import config, tensor
 from theano.sandbox import rng_mrg
-from theano.sandbox.rng_mrg import MRG_RandomStreams
+from theano.sandbox.rng_mrg import MRG_RandomStream
 
 
-# TODO: test MRG_RandomStreams
+# TODO: test MRG_RandomStream
 # Partly done in test_consistency_randomstreams
 
 # TODO: test optimizer mrg_random_make_inplace
@@ -35,7 +35,7 @@ def test_deterministic():
     seed = utt.fetch_seed()
     sample_size = (10, 20)
 
-    R = MRG_RandomStreams(seed=seed)
+    R = MRG_RandomStream(seed=seed)
     u = R.uniform(size=sample_size)
     f = theano.function([], u)
 
@@ -43,7 +43,7 @@ def test_deterministic():
     fsample2 = f()
     assert not np.allclose(fsample1, fsample2)
 
-    R2 = MRG_RandomStreams(seed=seed)
+    R2 = MRG_RandomStream(seed=seed)
     u2 = R2.uniform(size=sample_size)
     g = theano.function([], u2)
     gsample1 = g()
@@ -53,7 +53,7 @@ def test_deterministic():
 
 
 def test_consistency_randomstreams():
-    # Verify that the random numbers generated by MRG_RandomStreams
+    # Verify that the random numbers generated by MRG_RandomStream
     # are the same as the reference (Java) implementation by L'Ecuyer et al.
     seed = 12345
     n_samples = 5
@@ -61,7 +61,7 @@ def test_consistency_randomstreams():
     n_substreams = 7
 
     samples = []
-    rng = MRG_RandomStreams(seed=seed)
+    rng = MRG_RandomStream(seed=seed)
     for i in range(n_streams):
         stream_samples = []
         u = rng.uniform(size=(n_substreams,), nstreams=n_substreams)
@@ -82,7 +82,7 @@ def test_get_substream_rstates():
         n_streams = 100
 
         dtype = "float32"
-        rng = MRG_RandomStreams(np.random.randint(2147462579))
+        rng = MRG_RandomStream(np.random.randint(2147462579))
 
         rng.get_substream_rstates(n_streams, dtype)
 
@@ -107,7 +107,7 @@ def test_consistency_cpu_serial():
                 rstate, ndim=None, dtype=config.floatX, size=(1,)
             )
             # Not really necessary, just mimicking
-            # rng_mrg.MRG_RandomStreams' behavior
+            # rng_mrg.MRG_RandomStream' behavior
             sample.rstate = rstate
             sample.update = (rstate, new_rstate)
 
@@ -151,7 +151,7 @@ def test_consistency_cpu_parallel():
             rstate, ndim=None, dtype=config.floatX, size=(n_substreams,)
         )
         # Not really necessary, just mimicking
-        # rng_mrg.MRG_RandomStreams' behavior
+        # rng_mrg.MRG_RandomStream' behavior
         sample.rstate = rstate
         sample.update = (rstate, new_rstate)
 
@@ -272,7 +272,7 @@ def test_uniform():
         # TEST CPU IMPLEMENTATION
         # The python and C implementation are tested with DebugMode
         x = tensor.matrix()
-        R = MRG_RandomStreams(234)
+        R = MRG_RandomStream(234)
         # Note: we specify `nstreams` to avoid a warning.
         # TODO Look for all occurrences of `guess_n_streams` and `30 * 256`
         # for such situations: it would be better to instead filter the
@@ -294,7 +294,7 @@ def test_uniform():
             steps_ = steps
         check_basics(f, steps_, const_size, prefix="mrg cpu", inputs=input)
 
-        RR = theano.tensor.shared_randomstreams.RandomStreams(234)
+        RR = theano.tensor.random.utils.RandomStream(234)
 
         uu = RR.uniform(size=size)
         ff = theano.function(var_input, uu)
@@ -305,7 +305,7 @@ def test_uniform():
 
 
 def test_broadcastable():
-    R = MRG_RandomStreams(234)
+    R = MRG_RandomStream(234)
     x = tensor.matrix()
     size1 = (10, 1)
     size2 = (x.shape[0], 1)
@@ -343,7 +343,7 @@ def test_broadcastable():
 
 
 def check_binomial(mean, size, const_size, var_input, input, steps, rtol):
-    R = MRG_RandomStreams(234)
+    R = MRG_RandomStream(234)
     u = R.binomial(size=size, p=mean)
     f = theano.function(var_input, u)
     f(*input)
@@ -364,9 +364,9 @@ def check_binomial(mean, size, const_size, var_input, input, steps, rtol):
         mean_rtol=rtol,
     )
 
-    RR = theano.tensor.shared_randomstreams.RandomStreams(234)
+    RR = theano.tensor.random.utils.RandomStream(234)
 
-    uu = RR.binomial(size=size, p=mean)
+    uu = RR.binomial(1, mean, size=size)
     ff = theano.function(var_input, uu)
     # It's not our problem if numpy generates 0 or 1
     check_basics(
@@ -470,7 +470,7 @@ def test_normal0():
     ]
 
     for size, const_size, var_input, input, avg, rtol, std_tol in test_cases:
-        R = MRG_RandomStreams(234)
+        R = MRG_RandomStream(234)
         # Note: we specify `nstreams` to avoid a warning.
         n = R.normal(
             size=size,
@@ -501,9 +501,9 @@ def test_normal0():
 
         sys.stdout.flush()
 
-        RR = theano.tensor.shared_randomstreams.RandomStreams(234)
+        RR = theano.tensor.random.utils.RandomStream(234)
 
-        nn = RR.normal(size=size, avg=avg, std=std)
+        nn = RR.normal(avg, std, size=size)
         ff = theano.function(var_input, nn)
 
         check_basics(
@@ -579,7 +579,7 @@ def test_normal_truncation():
     ]
 
     for size, const_size, var_input, input, avg, rtol, std_tol in test_cases:
-        R = MRG_RandomStreams(234)
+        R = MRG_RandomStream(234)
         # Note: we specify `nstreams` to avoid a warning.
         n = R.normal(
             size=size,
@@ -679,7 +679,7 @@ def test_truncated_normal():
     ]
 
     for size, const_size, var_input, input, avg, rtol, std_tol in test_cases:
-        R = MRG_RandomStreams(234)
+        R = MRG_RandomStream(234)
         # Note: we specify `nstreams` to avoid a warning.
         n = R.truncated_normal(
             size=size,
@@ -751,7 +751,7 @@ def test_multinomial():
 
     pvals = np.asarray(np.random.uniform(size=sample_size))
     pvals = np.apply_along_axis(lambda row: row / np.sum(row), 1, pvals)
-    R = MRG_RandomStreams(234)
+    R = MRG_RandomStream(234)
     # Note: we specify `nstreams` to avoid a warning.
     m = R.multinomial(pvals=pvals, dtype=config.floatX, nstreams=30 * 256)
     f = theano.function([], m)
@@ -771,7 +771,7 @@ def test_multinomial_n_samples():
 
     pvals = np.asarray(np.random.uniform(size=sample_size))
     pvals = np.apply_along_axis(lambda row: row / np.sum(row), 1, pvals)
-    R = MRG_RandomStreams(234)
+    R = MRG_RandomStream(234)
 
     for n_samples, steps in zip([5, 10, 100, 1000], [20, 10, 1, 1]):
         m = R.multinomial(
@@ -785,7 +785,7 @@ def test_multinomial_n_samples():
 class TestMRG:
     def test_bad_size(self):
 
-        R = MRG_RandomStreams(234)
+        R = MRG_RandomStream(234)
 
         for size in [
             (0, 100),
@@ -812,8 +812,8 @@ def test_multiple_rng_aliasing():
     # meant to detect a previous bug where state_updates was initialized as a
     # class-attribute, instead of the __init__ function.
 
-    rng1 = MRG_RandomStreams(1234)
-    rng2 = MRG_RandomStreams(2392)
+    rng1 = MRG_RandomStream(1234)
+    rng2 = MRG_RandomStream(2392)
     assert rng1.state_updates is not rng2.state_updates
 
 
@@ -822,7 +822,7 @@ def test_random_state_transfer():
 
     class Graph:
         def __init__(self, seed=123):
-            self.rng = MRG_RandomStreams(seed)
+            self.rng = MRG_RandomStream(seed)
             self.y = self.rng.uniform(size=(1,))
 
     g1 = Graph(seed=123)
@@ -840,7 +840,7 @@ def test_random_state_transfer():
 def test_gradient_scan():
     # Test for a crash when using MRG inside scan and taking the gradient
     # See https://groups.google.com/d/msg/theano-dev/UbcYyU5m-M8/UO9UgXqnQP0J
-    theano_rng = MRG_RandomStreams(10)
+    theano_rng = MRG_RandomStream(10)
     w = theano.shared(np.ones(1, dtype="float32"))
 
     def one_step(x):
@@ -854,7 +854,7 @@ def test_gradient_scan():
 
 
 def test_simple_shared_mrg_random():
-    theano_rng = MRG_RandomStreams(10)
+    theano_rng = MRG_RandomStream(10)
 
     values, updates = theano.scan(
         lambda: theano_rng.uniform((2,), -1, 1),
@@ -912,7 +912,7 @@ def test_seed_fn():
     idx = tensor.ivector()
 
     for new_seed, same in [(234, True), (None, True), (23, False)]:
-        random = MRG_RandomStreams(234)
+        random = MRG_RandomStream(234)
         fn1 = theano.function([], random.uniform((2, 2), dtype="float32"))
         fn2 = theano.function([], random.uniform((3, 3), nstreams=2, dtype="float32"))
         fn3 = theano.function(
@@ -961,7 +961,7 @@ def rng_mrg_overflow(sizes, fct, mode, should_raise_error):
 @pytest.mark.slow
 def test_overflow_cpu():
     # run with THEANO_FLAGS=mode=FAST_RUN,device=cpu,floatX=float32
-    rng = MRG_RandomStreams(np.random.randint(1234))
+    rng = MRG_RandomStream(np.random.randint(1234))
     fct = rng.uniform
     with config.change_flags(compute_test_value="off"):
         # should raise error as the size overflows
@@ -984,7 +984,7 @@ def test_overflow_cpu():
 
 
 def test_undefined_grad():
-    srng = MRG_RandomStreams(seed=1234)
+    srng = MRG_RandomStream(seed=1234)
 
     # checking uniform distribution
     low = tensor.scalar()
@@ -1068,7 +1068,7 @@ def test_undefined_grad():
 
 
 def test_f16_nonzero(mode=None, op_to_check=rng_mrg.mrg_uniform):
-    srng = MRG_RandomStreams(seed=utt.fetch_seed())
+    srng = MRG_RandomStream(seed=utt.fetch_seed())
     m = srng.uniform(size=(1000, 1000), dtype="float16")
     assert m.dtype == "float16", m.type
     f = theano.function([], m, mode=mode)
@@ -1079,7 +1079,7 @@ def test_f16_nonzero(mode=None, op_to_check=rng_mrg.mrg_uniform):
 
 @pytest.mark.slow
 def test_target_parameter():
-    srng = MRG_RandomStreams()
+    srng = MRG_RandomStream()
     pvals = np.array([[0.98, 0.01, 0.01], [0.01, 0.49, 0.50]])
 
     def basic_target_parameter_test(x):
@@ -1099,3 +1099,24 @@ def test_target_parameter():
     basic_target_parameter_test(
         srng.multinomial_wo_replacement(pvals=pvals.astype("float32"), target="cpu")
     )
+
+
+@config.change_flags(compute_test_value="off")
+def test_undefined_grad_opt():
+    # Make sure that undefined grad get removed in optimized graph.
+    random = MRG_RandomStream(np.random.randint(1, 2147462579))
+    pvals = theano.shared(np.random.rand(10, 20).astype(theano.config.floatX))
+    pvals = pvals / pvals.sum(axis=1)
+    pvals = theano.gradient.zero_grad(pvals)
+    samples = random.multinomial(pvals=pvals, n=1)
+    samples = theano.tensor.cast(samples, pvals.dtype)
+    samples = theano.gradient.zero_grad(samples)
+    cost = theano.tensor.sum(samples + pvals)
+    grad = theano.tensor.grad(cost, samples)
+    f = theano.function([], grad)
+    assert not any(
+        [
+            isinstance(node.op, theano.gradient.UndefinedGrad)
+            for node in f.maker.fgraph.apply_nodes
+        ]
+    )

tests/scan/test_basic.py

@@ -155,14 +155,14 @@ class multiple_outputs_numeric_grad:
 # verify_grad method so that other ops with multiple outputs can be tested.
 # DONE - rp
 def scan_project_sum(*args, **kwargs):
-    rng = theano.tensor.shared_randomstreams.RandomStreams(123)
+    rng = theano.tensor.random.utils.RandomStream(123)
     scan_outputs, updates = scan(*args, **kwargs)
     if type(scan_outputs) not in [list, tuple]:
         scan_outputs = [scan_outputs]
     # we should ignore the random-state updates so that
     # the uniform numbers are the same every evaluation and on every call
     rng.add_default_updates = False
-    factors = [rng.uniform(size=s.shape, low=0.1, high=0.9) for s in scan_outputs]
+    factors = [rng.uniform(0.1, 0.9, size=s.shape) for s in scan_outputs]
     # Random values (?)
     return (sum([(s * f).sum() for s, f in zip(scan_outputs, factors)]), updates)
 
@@ -407,7 +407,7 @@ class TestScan:
         )
         # get random initial values
         rng = np.random.RandomState(utt.fetch_seed())
-        v_u = rng.uniform(size=(4,), low=-5.0, high=5.0)
+        v_u = rng.uniform(-5.0, 5.0, size=(4,))
         v_x0 = rng.uniform()
         W = rng.uniform()
         W_in = rng.uniform()
@@ -446,7 +446,7 @@ class TestScan:
         )
         # get random initial values
 
-        v_u = rng.uniform(size=(4,), low=-5.0, high=5.0)
+        v_u = rng.uniform(-5.0, 5.0, size=(4,))
         v_x0 = rng.uniform()
         # compute the output i numpy
         v_out = np.zeros((4,))
@@ -461,13 +461,13 @@ class TestScan:
     # dimension instead of scalars/vectors
     def test_multiple_inputs_multiple_outputs(self):
         rng = np.random.RandomState(utt.fetch_seed())
-        vW_in2 = asarrayX(rng.uniform(size=(2,), low=-5.0, high=5.0))
-        vW = asarrayX(rng.uniform(size=(2, 2), low=-5.0, high=5.0))
-        vWout = asarrayX(rng.uniform(size=(2,), low=-5.0, high=5.0))
-        vW_in1 = asarrayX(rng.uniform(size=(2, 2), low=-5.0, high=5.0))
-        v_u1 = asarrayX(rng.uniform(size=(3, 2), low=-5.0, high=5.0))
-        v_u2 = asarrayX(rng.uniform(size=(3,), low=-5.0, high=5.0))
-        v_x0 = asarrayX(rng.uniform(size=(2,), low=-5.0, high=5.0))
+        vW_in2 = asarrayX(rng.uniform(-5.0, 5.0, size=(2,)))
+        vW = asarrayX(rng.uniform(-5.0, 5.0, size=(2, 2)))
+        vWout = asarrayX(rng.uniform(-5.0, 5.0, size=(2,)))
+        vW_in1 = asarrayX(rng.uniform(-5.0, 5.0, size=(2, 2)))
+        v_u1 = asarrayX(rng.uniform(-5.0, 5.0, size=(3, 2)))
+        v_u2 = asarrayX(rng.uniform(-5.0, 5.0, size=(3,)))
+        v_x0 = asarrayX(rng.uniform(-5.0, 5.0, size=(2,)))
         v_y0 = asarrayX(rng.uniform())
 
         W_in2 = theano.shared(vW_in2, name="win2")
@@ -517,13 +517,14 @@ class TestScan:
     def test_multiple_outs_taps(self):
         l = 5
         rng = np.random.RandomState(utt.fetch_seed())
-        vW_in2 = asarrayX(rng.uniform(size=(2,), low=-0.2, high=0.2))
-        vW = asarrayX(rng.uniform(size=(2, 2), low=-0.2, high=0.2))
-        vWout = asarrayX(rng.uniform(size=(2,), low=-0.2, high=0.2))
-        vW_in1 = asarrayX(rng.uniform(size=(2, 2), low=-0.2, high=0.2))
-        v_u1 = asarrayX(rng.uniform(size=(l, 2), low=-0.2, high=0.2))
-        v_u2 = asarrayX(rng.uniform(size=(l + 2, 2), low=-0.2, high=0.2))
-        v_x0 = asarrayX(rng.uniform(size=(2,), low=-0.2, high=0.2))
+
+        vW_in2 = asarrayX(rng.uniform(-2.0, 2.0, size=(2,)))
+        vW = asarrayX(rng.uniform(-2.0, 2.0, size=(2, 2)))
+        vWout = asarrayX(rng.uniform(-2.0, 2.0, size=(2,)))
+        vW_in1 = asarrayX(rng.uniform(-2.0, 2.0, size=(2, 2)))
+        v_u1 = asarrayX(rng.uniform(-2.0, 2.0, size=(l, 2)))
+        v_u2 = asarrayX(rng.uniform(-2.0, 2.0, size=(l + 2, 2)))
+        v_x0 = asarrayX(rng.uniform(-2.0, 2.0, size=(2,)))
         v_y0 = asarrayX(rng.uniform(size=(3,)))
 
         W_in2 = theano.shared(vW_in2, name="win2")
@@ -806,8 +807,8 @@ class TestScan:
         rng = np.random.RandomState(utt.fetch_seed())
         vW = asarrayX(rng.uniform())
         vW_in = asarrayX(rng.uniform())
-        vu = asarrayX(rng.uniform(size=(4,), low=-5.0, high=5.0))
-        vx0 = asarrayX(rng.uniform(size=(2,), low=-5.0, high=5.0))
+        vu = asarrayX(rng.uniform(-5.0, 5.0, size=(4,)))
+        vx0 = asarrayX(rng.uniform(-5.0, 5.0, size=(2,)))
 
         u = theano.tensor.vector("u")
         x0 = theano.tensor.vector("x0")
@@ -852,8 +853,8 @@ class TestScan:
         rng = np.random.RandomState(utt.fetch_seed())
         vW = asarrayX(rng.uniform())
         vW_in = asarrayX(rng.uniform())
-        vu = asarrayX(rng.uniform(size=(6,), low=-5.0, high=5.0))
-        vx0 = asarrayX(rng.uniform(size=(2,), low=-5.0, high=5.0))
+        vu = asarrayX(rng.uniform(-5.0, 5.0, size=(6,)))
+        vx0 = asarrayX(rng.uniform(-5.0, 5.0, size=(2,)))
 
         u = theano.tensor.vector("u")
         x0 = theano.tensor.vector("x0")
@@ -891,9 +892,9 @@ class TestScan:
         rng = np.random.RandomState(utt.fetch_seed())
         vW = asarrayX(np.random.uniform())
         vW_in = asarrayX(np.random.uniform())
-        vu0 = asarrayX(rng.uniform(size=(3,), low=-5.0, high=5.0))
-        vu1 = asarrayX(rng.uniform(size=(3,), low=-5.0, high=5.0))
-        vu2 = asarrayX(rng.uniform(size=(3,), low=-5.0, high=5.0))
+        vu0 = asarrayX(rng.uniform(-5.0, 5.0, size=(3,)))
+        vu1 = asarrayX(rng.uniform(-5.0, 5.0, size=(3,)))
+        vu2 = asarrayX(rng.uniform(-5.0, 5.0, size=(3,)))
         vx0 = asarrayX(rng.uniform())
         vx1 = asarrayX(rng.uniform())
 
@@ -958,9 +959,9 @@ class TestScan:
         rng = np.random.RandomState(utt.fetch_seed())
         vW = asarrayX(np.random.uniform())
         vW_in = asarrayX(np.random.uniform())
-        vu0 = asarrayX(rng.uniform(size=(3,), low=-5.0, high=5.0))
-        vu1 = asarrayX(rng.uniform(size=(4,), low=-5.0, high=5.0))
-        vu2 = asarrayX(rng.uniform(size=(5,), low=-5.0, high=5.0))
+        vu0 = asarrayX(rng.uniform(-5.0, 5.0, size=(3,)))
+        vu1 = asarrayX(rng.uniform(-5.0, 5.0, size=(4,)))
+        vu2 = asarrayX(rng.uniform(-5.0, 5.0, size=(5,)))
         vx0 = asarrayX(rng.uniform())
         vx1 = asarrayX(rng.uniform())
 
@@ -1142,10 +1143,10 @@ class TestScan:
         f(np.int32(0), np.float32(1.0), np.float32(0.5))
 
     def test_simple_shared_random(self):
-        theano_rng = theano.tensor.shared_randomstreams.RandomStreams(utt.fetch_seed())
+        theano_rng = theano.tensor.random.utils.RandomStream(utt.fetch_seed())
 
         values, updates = scan(
-            lambda: theano_rng.uniform((2,), -1, 1),
+            lambda: theano_rng.uniform(-1, 1, size=(2,)),
             [],
             [],
             [],
@@ -1184,20 +1185,20 @@ class TestScan:
         bhid = theano.shared(v_bhid, "vbhid")
         bvis = theano.shared(v_bvis, "vbvis")
         vsample = theano.tensor.matrix(dtype="float32")
-        trng = theano.tensor.shared_randomstreams.RandomStreams(utt.fetch_seed())
+        trng = theano.tensor.random.utils.RandomStream(utt.fetch_seed())
 
         def f(vsample_tm1):
             hmean_t = theano.tensor.nnet.sigmoid(
                 theano.tensor.dot(vsample_tm1, W) + bhid
             )
             hsample_t = theano.tensor.cast(
-                trng.binomial(hmean_t.shape, 1, hmean_t), dtype="float32"
+                trng.binomial(1, hmean_t, size=hmean_t.shape), dtype="float32"
             )
             vmean_t = theano.tensor.nnet.sigmoid(
                 theano.tensor.dot(hsample_t, W.T) + bvis
             )
             return theano.tensor.cast(
-                trng.binomial(vmean_t.shape, 1, vmean_t), dtype="float32"
+                trng.binomial(1, vmean_t, size=vmean_t.shape), dtype="float32"
             )
 
         theano_vsamples, updates = scan(
@@ -1265,7 +1266,7 @@ class TestScan:
         f2 = theano.function([u], outputs, updates=updates, allow_input_downcast=True)
         rng = np.random.RandomState(utt.fetch_seed())
 
-        v_u = rng.uniform(size=(5,), low=-5.0, high=5.0)
+        v_u = rng.uniform(-5.0, 5.0, size=(5,))
         numpy_result = v_u + 3
         theano_result = f2(v_u)
         utt.assert_allclose(theano_result, numpy_result)
@@ -1281,7 +1282,7 @@ class TestScan:
         )
 
         rng = np.random.RandomState(utt.fetch_seed())
-        vals = rng.uniform(size=(10,), low=-5.0, high=5.0)
+        vals = rng.uniform(-5.0, 5.0, size=(10,))
         abs_vals = abs(vals)
         theano_vals = f(vals)
         utt.assert_allclose(abs_vals, theano_vals)
@@ -1310,7 +1311,7 @@ class TestScan:
         )
         # get random initial values
         rng = np.random.RandomState(utt.fetch_seed())
-        v_u = rng.uniform(size=(4,), low=-5.0, high=5.0)
+        v_u = rng.uniform(-5.0, 5.0, size=(4,))
         v_x0 = rng.uniform()
         W = rng.uniform()
         W_in = rng.uniform()
@@ -1331,7 +1332,7 @@ class TestScan:
 
         f = theano.function([v, s], result, updates=updates, allow_input_downcast=True)
         rng = np.random.RandomState(utt.fetch_seed())
-        v_v = rng.uniform(size=(5,), low=-5.0, high=5.0)
+        v_v = rng.uniform(-5.0, 5.0, size=(5,))
         assert abs(np.sum(v_v) - f(v_v, 0.0)) < 1e-3
 
     def test_grad_one_output(self):
@@ -1370,10 +1371,11 @@ class TestScan:
 
         # get random initial values
         rng = np.random.RandomState(utt.fetch_seed())
-        v_u = np.array(rng.uniform(size=(10,), low=-0.5, high=0.5), dtype=config.floatX)
+        v_u = np.array(rng.uniform(-0.5, 0.5, size=(10,)), dtype=config.floatX)
         v_x0 = np.array(rng.uniform(), dtype=config.floatX)
         W = np.array(rng.uniform(), dtype=config.floatX)
         W_in = np.array(rng.uniform(), dtype=config.floatX)
+
         analytic_grad = grad_fn(v_u, v_x0, W_in, W)
 
         num_grad = multiple_outputs_numeric_grad(cost_fn, [v_u, v_x0, W_in, W])
@@ -1382,13 +1384,13 @@ class TestScan:
 
     def test_grad_multiple_outs(self):
         rng = np.random.RandomState(utt.fetch_seed())
-        vW_in2 = asarrayX(rng.uniform(size=(2,), low=-0.1, high=0.1))
-        vW = asarrayX(rng.uniform(size=(2, 2), low=-0.1, high=0.1))
-        vWout = asarrayX(rng.uniform(size=(2,), low=-0.1, high=0.1))
-        vW_in1 = asarrayX(rng.uniform(size=(2, 2), low=-0.1, high=0.1))
-        v_u1 = asarrayX(rng.uniform(size=(7, 2), low=-0.1, high=0.1))
-        v_u2 = asarrayX(rng.uniform(size=(7,), low=-0.1, high=0.1))
-        v_x0 = asarrayX(rng.uniform(size=(2,), low=-0.1, high=0.1))
+        vW_in2 = asarrayX(rng.uniform(-0.1, 0.1, size=(2,)))
+        vW = asarrayX(rng.uniform(-0.1, 0.1, size=(2, 2)))
+        vWout = asarrayX(rng.uniform(-0.1, 0.1, size=(2,)))
+        vW_in1 = asarrayX(rng.uniform(-0.1, 0.1, size=(2, 2)))
+        v_u1 = asarrayX(rng.uniform(-0.1, 0.1, size=(7, 2)))
+        v_u2 = asarrayX(rng.uniform(-0.1, 0.1, size=(7,)))
+        v_x0 = asarrayX(rng.uniform(0.1, 0.1, size=(2,)))
         v_y0 = asarrayX(rng.uniform())
 
         W_in2 = theano.shared(vW_in2, name="win2")
@@ -1447,13 +1449,15 @@ class TestScan:
     def test_grad_multiple_outs_taps(self):
         n = 5
         rng = np.random.RandomState(utt.fetch_seed())
-        vW_in2 = asarrayX(rng.uniform(size=(2,), low=-0.2, high=0.2))
-        vW = asarrayX(rng.uniform(size=(2, 2), low=-0.2, high=0.2))
-        vWout = asarrayX(rng.uniform(size=(2,), low=-0.2, high=0.2))
-        vW_in1 = asarrayX(rng.uniform(size=(2, 2), low=-0.2, high=0.2))
-        v_u1 = asarrayX(rng.uniform(size=(n, 2), low=-0.2, high=0.2))
-        v_u2 = asarrayX(rng.uniform(size=(n + 2, 2), low=-0.2, high=0.2))
-        v_x0 = asarrayX(rng.uniform(size=(2,), low=-0.2, high=0.2))
+
+        vW_in2 = asarrayX(rng.uniform(-0.2, 0.2, size=(2,)))
+        vW = asarrayX(rng.uniform(-0.2, 0.2, size=(2, 2)))
+        vWout = asarrayX(rng.uniform(-0.2, 0.2, size=(2,)))
+        vW_in1 = asarrayX(rng.uniform(-0.2, 0.2, size=(2, 2)))
+        v_u1 = asarrayX(rng.uniform(-0.2, 0.2, size=(n, 2)))
+        v_u2 = asarrayX(rng.uniform(-0.2, 0.2, size=(n + 2, 2)))
+        v_x0 = asarrayX(rng.uniform(0.2, 0.2, size=(2,)))
+
         v_y0 = asarrayX(rng.uniform(size=(3,)))
 
         W_in2 = theano.shared(vW_in2, name="win2")
@@ -1498,185 +1502,6 @@ class TestScan:
             )
             params = [u1, u2, x0, y0, W_in1]
             gparams = theano.tensor.grad(cost, params)
-            # Test the output including names
-            output_str = theano.printing.debugprint(cost, file="str")
-            lines = output_str.split("\n")
-            expected_output = """Elemwise{add,no_inplace} [id A] ''
- |Elemwise{add,no_inplace} [id B] ''
- | |Elemwise{add,no_inplace} [id C] ''
- | | |TensorConstant{0} [id D]
- | | |Sum{acc_dtype=float64} [id E] ''
- | |   |Elemwise{mul,no_inplace} [id F] ''
- | |     |Subtensor{int64::} [id G] ''
- | |     | |for{cpu,scan_fn}.1 [id H] ''
- | |     | | |Elemwise{minimum,no_inplace} [id I] ''
- | |     | | | |Elemwise{minimum,no_inplace} [id J] ''
- | |     | | | | |Elemwise{minimum,no_inplace} [id K] ''
- | |     | | | | | |Subtensor{int64} [id L] ''
- | |     | | | | | | |Shape [id M] ''
- | |     | | | | | | | |Subtensor{int64::} [id N] 'u1[0:]'
- | |     | | | | | | |   |u1 [id O]
- | |     | | | | | | |   |Constant{0} [id P]
- | |     | | | | | | |Constant{0} [id Q]
- | |     | | | | | |Subtensor{int64} [id R] ''
- | |     | | | | |   |Shape [id S] ''
- | |     | | | | |   | |Subtensor{int64:int64:} [id T] 'u2[0:-2]'
- | |     | | | | |   |   |u2 [id U]
- | |     | | | | |   |   |Constant{0} [id V]
- | |     | | | | |   |   |Constant{-2} [id W]
- | |     | | | | |   |Constant{0} [id X]
- | |     | | | | |Subtensor{int64} [id Y] ''
- | |     | | | |   |Shape [id Z] ''
- | |     | | | |   | |Subtensor{int64:int64:} [id BA] 'u2[1:-1]'
- | |     | | | |   |   |u2 [id U]
- | |     | | | |   |   |Constant{1} [id BB]
- | |     | | | |   |   |Constant{-1} [id BC]
- | |     | | | |   |Constant{0} [id BD]
- | |     | | | |Subtensor{int64} [id BE] ''
- | |     | | |   |Shape [id BF] ''
- | |     | | |   | |Subtensor{int64::} [id BG] 'u2[2:]'
- | |     | | |   |   |u2 [id U]
- | |     | | |   |   |Constant{2} [id BH]
- | |     | | |   |Constant{0} [id BI]
- | |     | | |Subtensor{:int64:} [id BJ] ''
- | |     | | | |Subtensor{int64::} [id N] 'u1[0:]'
- | |     | | | |ScalarFromTensor [id BK] ''
- | |     | | |   |Elemwise{minimum,no_inplace} [id I] ''
- | |     | | |Subtensor{:int64:} [id BL] ''
- | |     | | | |Subtensor{int64:int64:} [id T] 'u2[0:-2]'
- | |     | | | |ScalarFromTensor [id BM] ''
- | |     | | |   |Elemwise{minimum,no_inplace} [id I] ''
- | |     | | |Subtensor{:int64:} [id BN] ''
- | |     | | | |Subtensor{int64:int64:} [id BA] 'u2[1:-1]'
- | |     | | | |ScalarFromTensor [id BO] ''
- | |     | | |   |Elemwise{minimum,no_inplace} [id I] ''
- | |     | | |Subtensor{:int64:} [id BP] ''
- | |     | | | |Subtensor{int64::} [id BG] 'u2[2:]'
- | |     | | | |ScalarFromTensor [id BQ] ''
- | |     | | |   |Elemwise{minimum,no_inplace} [id I] ''
- | |     | | |IncSubtensor{Set;:int64:} [id BR] ''
- | |     | | | |AllocEmpty{dtype='%(float)s'} [id BS] ''
- | |     | | | | |Elemwise{add,no_inplace} [id BT] ''
- | |     | | | |   |Elemwise{minimum,no_inplace} [id I] ''
- | |     | | | |   |Subtensor{int64} [id BU] ''
- | |     | | | |     |Shape [id BV] ''
- | |     | | | |     | |Subtensor{:int64:} [id BW] ''
- | |     | | | |     |   |y0 [id BX]
- | |     | | | |     |   |Constant{3} [id BY]
- | |     | | | |     |Constant{0} [id BZ]
- | |     | | | |Subtensor{:int64:} [id BW] ''
- | |     | | | |ScalarFromTensor [id CA] ''
- | |     | | |   |Subtensor{int64} [id BU] ''
- | |     | | |IncSubtensor{Set;:int64:} [id CB] ''
- | |     | | | |AllocEmpty{dtype='%(float)s'} [id CC] ''
- | |     | | | | |Elemwise{add,no_inplace} [id CD] ''
- | |     | | | | | |Elemwise{minimum,no_inplace} [id I] ''
- | |     | | | | | |Subtensor{int64} [id CE] ''
- | |     | | | | |   |Shape [id CF] ''
- | |     | | | | |   | |Rebroadcast{0} [id CG] ''
- | |     | | | | |   |   |InplaceDimShuffle{x,0} [id CH] ''
- | |     | | | | |   |     |x0 [id CI]
- | |     | | | | |   |Constant{0} [id CJ]
- | |     | | | | |Subtensor{int64} [id CK] ''
- | |     | | | |   |Shape [id CL] ''
- | |     | | | |   | |Rebroadcast{0} [id CG] ''
- | |     | | | |   |Constant{1} [id CM]
- | |     | | | |Rebroadcast{0} [id CG] ''
- | |     | | | |ScalarFromTensor [id CN] ''
- | |     | | |   |Subtensor{int64} [id CE] ''
- | |     | | |Elemwise{minimum,no_inplace} [id I] ''
- | |     | | |win2 [id CO]
- | |     | | |w [id CP]
- | |     | | |wout [id CQ]
- | |     | | |win [id CR]
- | |     | |Constant{1} [id CS]
- | |     |RandomFunction{uniform}.1 [id CT] ''
- | |       |<RandomStateType> [id CU]
- | |       |Shape [id CV] ''
- | |       | |Subtensor{int64::} [id G] ''
- | |       |TensorConstant{0.1} [id CW]
- | |       |TensorConstant{0.9} [id CX]
- | |Sum{acc_dtype=float64} [id CY] ''
- |   |Elemwise{mul,no_inplace} [id CZ] ''
- |     |Subtensor{int64::} [id DA] ''
- |     | |for{cpu,scan_fn}.0 [id H] ''
- |     | |Constant{3} [id DB]
- |     |RandomFunction{uniform}.1 [id DC] ''
- |       |<RandomStateType> [id DD]
- |       |Shape [id DE] ''
- |       | |Subtensor{int64::} [id DA] ''
- |       |TensorConstant{0.1} [id DF]
- |       |TensorConstant{0.9} [id DG]
- |Sum{acc_dtype=float64} [id DH] ''
-   |Elemwise{mul,no_inplace} [id DI] ''
-     |for{cpu,scan_fn}.2 [id H] ''
-     |RandomFunction{uniform}.1 [id DJ] ''
-       |<RandomStateType> [id DK]
-       |Shape [id DL] ''
-       | |for{cpu,scan_fn}.2 [id H] ''
-       |TensorConstant{0.1} [id DM]
-       |TensorConstant{0.9} [id DN]
-
-Inner graphs of the scan ops:
-
-for{cpu,scan_fn}.1 [id H] ''
- >Elemwise{Composite{((i0 + i1) * i2)}} [id DO] ''
- > |y0[t-1] [id DP] -> [id BR]
- > |y0[t-3] [id DQ] -> [id BR]
- > |InplaceDimShuffle{} [id DR] ''
- >   |CGemv{inplace} [id DS] ''
- >     |AllocEmpty{dtype='%(float)s'} [id DT] ''
- >     | |TensorConstant{1} [id DU]
- >     |TensorConstant{1.0} [id DV]
- >     |InplaceDimShuffle{x,0} [id DW] ''
- >     | |wout_copy [id DX] -> [id CQ]
- >     |x0[t-1] [id DY] -> [id CB]
- >     |TensorConstant{0.0} [id DZ]
- >Elemwise{Composite{(i0 + ((i1 + (i2 * i3)) * i4) + i5)}} [id EA] ''
- > |CGemv{no_inplace} [id EB] ''
- > | |AllocEmpty{dtype='%(float)s'} [id EC] ''
- > | | |Shape_i{1} [id ED] ''
- > | |   |win_copy [id EE] -> [id CR]
- > | |TensorConstant{1.0} [id DV]
- > | |InplaceDimShuffle{1,0} [id EF] 'win_copy.T'
- > | | |win_copy [id EE] -> [id CR]
- > | |u1[t] [id EG] -> [id BJ]
- > | |TensorConstant{0.0} [id DZ]
- > |u2[t] [id EH] -> [id BN]
- > |u2[t-1] [id EI] -> [id BL]
- > |u2[t+1] [id EJ] -> [id BP]
- > |win2_copy [id EK] -> [id CO]
- > |CGemv{inplace} [id EL] ''
- >   |AllocEmpty{dtype='%(float)s'} [id EM] ''
- >   | |Shape_i{1} [id EN] ''
- >   |   |w_copy [id EO] -> [id CP]
- >   |TensorConstant{1.0} [id DV]
- >   |InplaceDimShuffle{1,0} [id EP] 'w_copy.T'
- >   | |w_copy [id EO] -> [id CP]
- >   |x0[t-1] [id DY] -> [id CB]
- >   |TensorConstant{0.0} [id DZ]
- >CGemv{no_inplace} [id EB] ''
-
-for{cpu,scan_fn}.0 [id H] ''
- >Elemwise{Composite{((i0 + i1) * i2)}} [id DO] ''
- >Elemwise{Composite{(i0 + ((i1 + (i2 * i3)) * i4) + i5)}} [id EA] ''
- >CGemv{no_inplace} [id EB] ''
-
-for{cpu,scan_fn}.2 [id H] ''
- >Elemwise{Composite{((i0 + i1) * i2)}} [id DO] ''
- >Elemwise{Composite{(i0 + ((i1 + (i2 * i3)) * i4) + i5)}} [id EA] ''
- >CGemv{no_inplace} [id EB] ''
-
-for{cpu,scan_fn}.2 [id H] ''
- >Elemwise{Composite{((i0 + i1) * i2)}} [id DO] ''
- >Elemwise{Composite{(i0 + ((i1 + (i2 * i3)) * i4) + i5)}} [id EA] ''
- >CGemv{no_inplace} [id EB] ''
-""" % {
-                "float": config.floatX
-            }
-            if config.mode != "FAST_COMPILE" and config.floatX == "float64":
-                for truth, out in zip(expected_output.split("\n"), lines):
-                    assert truth.strip() == out.strip(), (truth, out)
 
             cost_fn = theano.function(
                 [u1, u2, x0, y0, W_in1],
@@ -1705,13 +1530,13 @@ for{cpu,scan_fn}.2 [id H] ''
     def test_grad_multiple_outs_taps_backwards(self):
         n = 5
         rng = np.random.RandomState(utt.fetch_seed())
-        vW_in2 = asarrayX(rng.uniform(size=(2,), low=-0.2, high=0.2))
-        vW = asarrayX(rng.uniform(size=(2, 2), low=-0.2, high=0.2))
-        vWout = asarrayX(rng.uniform(size=(2,), low=-0.2, high=0.2))
-        vW_in1 = asarrayX(rng.uniform(size=(2, 2), low=-0.2, high=0.2))
-        v_u1 = asarrayX(rng.uniform(size=(n, 2), low=-0.2, high=0.2))
-        v_u2 = asarrayX(rng.uniform(size=(n + 2, 2), low=-0.2, high=0.2))
-        v_x0 = asarrayX(rng.uniform(size=(2,), low=-0.2, high=0.2))
+        vW_in2 = asarrayX(rng.uniform(-0.2, 0.2, size=(2,)))
+        vW = asarrayX(rng.uniform(-0.2, 0.2, size=(2, 2)))
+        vWout = asarrayX(rng.uniform(-0.2, 0.2, size=(2,)))
+        vW_in1 = asarrayX(rng.uniform(-0.2, 0.2, size=(2, 2)))
+        v_u1 = asarrayX(rng.uniform(-0.2, 0.2, size=(n, 2)))
+        v_u2 = asarrayX(rng.uniform(-0.2, 0.2, size=(n + 2, 2)))
+        v_x0 = asarrayX(rng.uniform(-0.2, 0.2, size=(2,)))
         v_y0 = asarrayX(rng.uniform(size=(3,)))
 
         W_in2 = theano.shared(vW_in2, name="win2")
@@ -1767,20 +1592,18 @@ for{cpu,scan_fn}.2 [id H] ''
 
     def test_grad_multiple_outs_some_uncomputable(self):
         rng = np.random.RandomState(utt.fetch_seed())
-        vW_in = asarrayX(rng.uniform(size=(2, 2), low=-3.0, high=3.0))
-        v_u = asarrayX(rng.uniform(size=(5, 2), low=-3.0, high=3.0))
+        vW_in = asarrayX(rng.uniform(-3.0, 3.0, size=(2, 2)))
+        v_u = asarrayX(rng.uniform(-3.0, 3.0, size=(5, 2)))
         v_u2 = np.array([1, 3, 4, 6, 8], dtype="int32")
-        v_x0 = asarrayX(rng.uniform(size=(2,), low=-3.0, high=3.0))
+        v_x0 = asarrayX(rng.uniform(-3.0, 3.0, size=(2,)))
 
         W_in = theano.tensor.matrix("win")
         u = theano.tensor.matrix("u1")
         u2 = theano.tensor.ivector("u2")
         x0 = theano.tensor.vector("x0", dtype=config.floatX)
-        # trng  = theano.tensor.shared_randomstreams.RandomStreams(
-        #                                               utt.fetch_seed())
 
         def f_rnn_cmpl(u_t, u2_t, x_tm1, W_in):
-            trng1 = theano.tensor.shared_randomstreams.RandomStreams(123)
+            trng1 = theano.tensor.random.utils.RandomStream(123)
             x_t = (
                 theano.tensor.cast(u2_t, config.floatX)
                 + theano.tensor.dot(u_t, W_in)
@@ -1852,19 +1675,19 @@ for{cpu,scan_fn}.2 [id H] ''
 
     def test_grad_multiple_outs_some_truncate(self):
         rng = np.random.RandomState(utt.fetch_seed())
-        vW_in = asarrayX(rng.uniform(size=(2, 2), low=-0.1, high=0.1))
-        v_u = asarrayX(rng.uniform(size=(5, 2), low=-0.1, high=0.1))
-        v_x0 = asarrayX(rng.uniform(size=(2,), low=-0.1, high=0.1))
+        vW_in = asarrayX(rng.uniform(-0.1, 0.1, size=(2, 2)))
+        v_u = asarrayX(rng.uniform(-0.1, 0.1, size=(5, 2)))
+        v_x0 = asarrayX(rng.uniform(-0.1, 0.1, size=(2,)))
 
         W_in = theano.tensor.matrix("win")
         u = theano.tensor.matrix("u1")
         x0 = theano.tensor.vector("x0")
-        # trng  = theano.tensor.shared_randomstreams.RandomStreams(
+        # trng  = theano.tensor.random.utils.RandomStream(
         #                                               utt.fetch_seed())
 
         def f_rnn_cmpl(u_t, x_tm1, W_in):
-            trng1 = theano.tensor.shared_randomstreams.RandomStreams(123)
-            rnd_nb = trng1.uniform(low=-0.1, high=0.1)
+            trng1 = theano.tensor.random.utils.RandomStream(123)
+            rnd_nb = trng1.uniform(-0.1, 0.1)
             x_t = theano.tensor.dot(u_t, W_in) + x_tm1 + rnd_nb
             x_t = theano.tensor.cast(x_t, dtype=config.floatX)
             return x_t
@@ -1933,12 +1756,12 @@ for{cpu,scan_fn}.2 [id H] ''
         n_in = 1
         n_out = 1
 
-        W_hh_v = asarrayX(rng.uniform(size=(n_hid, n_hid), low=-1, high=1))
-        h0_v = asarrayX(rng.uniform(size=(2, n_hid), low=-1, high=1))
-        b_h_v = asarrayX(rng.uniform(size=(n_hid), low=-0.01, high=0.01))
-        W_ih_v = asarrayX(rng.uniform(size=(n_in, n_hid), low=-1, high=1))
-        W_ho_v = asarrayX(rng.uniform(size=(n_hid, n_out), low=-1, high=1))
-        b_o_v = asarrayX(rng.uniform(size=(n_out), low=-0.01, high=0.01))
+        W_hh_v = asarrayX(rng.uniform(-1, 1, size=(n_hid, n_hid)))
+        h0_v = asarrayX(rng.uniform(-1, 1, size=(2, n_hid)))
+        b_h_v = asarrayX(rng.uniform(-0.01, 0.01, size=(n_hid)))
+        W_ih_v = asarrayX(rng.uniform(-1, 1, size=(n_in, n_hid)))
+        W_ho_v = asarrayX(rng.uniform(-1, 1, size=(n_hid, n_out)))
+        b_o_v = asarrayX(rng.uniform(-0.01, 0.01, size=(n_out)))
 
         # parameters of the rnn
         b_h = theano.shared(b_h_v, name="b_h")
@@ -2003,10 +1826,10 @@ for{cpu,scan_fn}.2 [id H] ''
         return cost
 
     def test_draw_as_input_to_scan(self):
-        trng = theano.tensor.shared_randomstreams.RandomStreams(123)
+        trng = theano.tensor.random.utils.RandomStream(123)
 
         x = theano.tensor.matrix("x")
-        y = trng.binomial(size=x.shape, p=x)
+        y = trng.binomial(1, x, size=x.shape)
         z, updates = scan(lambda a: a, non_sequences=y, n_steps=2)
 
         f = theano.function([x], [y, z], updates=updates, allow_input_downcast=True)
@@ -2236,13 +2059,14 @@ for{cpu,scan_fn}.2 [id H] ''
     # dimension instead of scalars/vectors
     def test_reordering(self):
         rng = np.random.RandomState(utt.fetch_seed())
-        vW_in2 = asarrayX(rng.uniform(size=(2,), low=-5.0, high=5.0))
-        vW = asarrayX(rng.uniform(size=(2, 2), low=-5.0, high=5.0))
-        vWout = asarrayX(rng.uniform(size=(2,), low=-5.0, high=5.0))
-        vW_in1 = asarrayX(rng.uniform(size=(2, 2), low=-5.0, high=5.0))
-        v_u1 = asarrayX(rng.uniform(size=(3, 2), low=-5.0, high=5.0))
-        v_u2 = asarrayX(rng.uniform(size=(3,), low=-5.0, high=5.0))
-        v_x0 = asarrayX(rng.uniform(size=(2,), low=-5.0, high=5.0))
+
+        vW_in2 = asarrayX(rng.uniform(-0.5, 0.5, size=(2,)))
+        vW = asarrayX(rng.uniform(-0.5, 0.5, size=(2, 2)))
+        vWout = asarrayX(rng.uniform(-0.5, 0.5, size=(2,)))
+        vW_in1 = asarrayX(rng.uniform(-0.5, 0.5, size=(2, 2)))
+        v_u1 = asarrayX(rng.uniform(-0.5, 0.5, size=(3, 2)))
+        v_u2 = asarrayX(rng.uniform(-0.5, 0.5, size=(3,)))
+        v_x0 = asarrayX(rng.uniform(-0.5, 0.5, size=(2,)))
         v_y0 = asarrayX(rng.uniform(size=(3,)))
 
         W_in2 = theano.shared(vW_in2, name="win2")
@@ -2320,13 +2144,14 @@ for{cpu,scan_fn}.2 [id H] ''
 
     def test_save_mem(self):
         rng = np.random.RandomState(utt.fetch_seed())
-        vW_in2 = asarrayX(rng.uniform(size=(2,), low=-5.0, high=5.0))
-        vW = asarrayX(rng.uniform(size=(2, 2), low=-5.0, high=5.0))
-        vWout = asarrayX(rng.uniform(size=(2,), low=-5.0, high=5.0))
-        vW_in1 = asarrayX(rng.uniform(size=(2, 2), low=-5.0, high=5.0))
-        v_u1 = asarrayX(rng.uniform(size=(8, 2), low=-5.0, high=5.0))
-        v_u2 = asarrayX(rng.uniform(size=(8,), low=-5.0, high=5.0))
-        v_x0 = asarrayX(rng.uniform(size=(2,), low=-5.0, high=5.0))
+
+        vW_in2 = asarrayX(rng.uniform(-0.5, 0.5, size=(2,)))
+        vW = asarrayX(rng.uniform(-0.5, 0.5, size=(2, 2)))
+        vWout = asarrayX(rng.uniform(-0.5, 0.5, size=(2,)))
+        vW_in1 = asarrayX(rng.uniform(-0.5, 0.5, size=(2, 2)))
+        v_u1 = asarrayX(rng.uniform(-0.5, 0.5, size=(8, 2)))
+        v_u2 = asarrayX(rng.uniform(-0.5, 0.5, size=(8,)))
+        v_x0 = asarrayX(rng.uniform(-0.5, 0.5, size=(2,)))
         v_y0 = asarrayX(rng.uniform(size=(3,)))
 
         W_in2 = theano.shared(vW_in2, name="win2")
@@ -2437,7 +2262,7 @@ for{cpu,scan_fn}.2 [id H] ''
         )
         # get random initial values
         rng = np.random.RandomState(utt.fetch_seed())
-        v_u = rng.uniform(size=(20,), low=-5.0, high=5.0)
+        v_u = rng.uniform(-5.0, 5.0, size=(20,))
 
         # compute the output in numpy
         tx1, tx2, tx3, tx4, tx5, tx6, tx7 = f2(v_u, 3, 15)
@@ -2496,7 +2321,7 @@ for{cpu,scan_fn}.2 [id H] ''
 
         # get random initial values
         rng = np.random.RandomState(utt.fetch_seed())
-        v_u = rng.uniform(size=(20,), low=-5.0, high=5.0)
+        v_u = rng.uniform(-5.0, 5.0, size=(20,))
 
         # compute the output in numpy
         tx1, tx2, tx3, tx4, tx5 = f2(v_u, [0, 0], 0, [0, 0], 0)
@@ -2590,7 +2415,7 @@ for{cpu,scan_fn}.2 [id H] ''
         # an until condition and random sampling in the inner function.
 
         x = tensor.scalar()
-        srng = theano.tensor.shared_randomstreams.RandomStreams(0)
+        srng = theano.tensor.random.utils.RandomStream(0)
 
         def inner_fct(previous_val):
             new_val = previous_val + srng.uniform()
@@ -2656,9 +2481,9 @@ for{cpu,scan_fn}.2 [id H] ''
         x = theano.tensor.vector("x")
 
         def lm(m):
-            trng = theano.tensor.shared_randomstreams.RandomStreams(utt.fetch_seed())
+            trng = theano.tensor.random.utils.RandomStream(utt.fetch_seed())
             return [
-                2 * m + trng.uniform(low=-1.1, high=1.1, dtype=config.floatX),
+                2 * m + trng.uniform(-1.1, 1.1, dtype=config.floatX),
                 m + trng.uniform(size=[3]),
             ]
 
@@ -3065,7 +2890,7 @@ for{cpu,scan_fn}.2 [id H] ''
 
         def rnn_fn(_u, _y, _W):
 
-            srng = theano.tensor.shared_randomstreams.RandomStreams(seed)
+            srng = theano.tensor.random.utils.RandomStream(seed)
             tmp_val = (
                 _u + _y + srng.uniform(size=v_h0.shape) * np.asarray(1e-6, dtype=floatX)
             )
@@ -3524,7 +3349,7 @@ for{cpu,scan_fn}.2 [id H] ''
         # the intermediary states of the random number generators used in the
         # test. The test case was modified from the original for simplicity
 
-        rand_stream = tensor.shared_randomstreams.RandomStreams()
+        rand_stream = tensor.random.utils.RandomStream()
         inp = tensor.matrix()
         norm_inp = inp / tensor.sum(inp, axis=0)
 
@@ -3533,7 +3358,7 @@ for{cpu,scan_fn}.2 [id H] ''
                 # sample the input matrix for each column according to the
                 # column values
                 pvals = norm_inp.T
-                sample = rand_stream.multinomial(n=1, pvals=pvals)
+                sample = rand_stream.multinomial(1, pvals)
                 return inp + sample
 
             pooled, updates_inner = theano.scan(
@@ -3541,7 +3366,7 @@ for{cpu,scan_fn}.2 [id H] ''
             )
 
             # randomly add stuff to units
-            rand_nums = rand_stream.binomial(size=pooled.shape)
+            rand_nums = rand_stream.binomial(1, 0.5, size=pooled.shape)
             return pooled + rand_nums, updates_inner
 
         out, updates_outer = theano.scan(
@@ -3652,7 +3477,7 @@ for{cpu,scan_fn}.2 [id H] ''
         # Test the mapping produces by
         # ScanOp.get_oinp_iinp_iout_oout_mappings()
 
-        rng = theano.tensor.shared_randomstreams.RandomStreams(123)
+        rng = theano.tensor.random.utils.RandomStream(123)
 
         def inner_fct(seq, mitsot, sitsot, nitsot, nseq):
             random_scalar = rng.uniform((1,))[0]
@@ -3889,13 +3714,14 @@ for{cpu,scan_fn}.2 [id H] ''
 
     def test_return_steps(self):
         rng = np.random.RandomState(utt.fetch_seed())
-        vW_in2 = asarrayX(rng.uniform(size=(2,), low=-5.0, high=5.0))
-        vW = asarrayX(rng.uniform(size=(2, 2), low=-5.0, high=5.0))
-        vWout = asarrayX(rng.uniform(size=(2,), low=-5.0, high=5.0))
-        vW_in1 = asarrayX(rng.uniform(size=(2, 2), low=-5.0, high=5.0))
-        v_u1 = asarrayX(rng.uniform(size=(8, 2), low=-5.0, high=5.0))
-        v_u2 = asarrayX(rng.uniform(size=(8,), low=-5.0, high=5.0))
-        v_x0 = asarrayX(rng.uniform(size=(2,), low=-5.0, high=5.0))
+
+        vW_in2 = asarrayX(rng.uniform(-0.5, 0.5, size=(2,)))
+        vW = asarrayX(rng.uniform(-0.5, 0.5, size=(2, 2)))
+        vWout = asarrayX(rng.uniform(-0.5, 0.5, size=(2,)))
+        vW_in1 = asarrayX(rng.uniform(-0.5, 0.5, size=(2, 2)))
+        v_u1 = asarrayX(rng.uniform(-0.5, 0.5, size=(8, 2)))
+        v_u2 = asarrayX(rng.uniform(-0.5, 0.5, size=(8,)))
+        v_x0 = asarrayX(rng.uniform(-0.5, 0.5, size=(2,)))
         v_y0 = asarrayX(rng.uniform(size=(3,)))
 
         W_in2 = theano.shared(vW_in2, name="win2")
@@ -4053,14 +3879,12 @@ for{cpu,scan_fn}.2 [id H] ''
         # and the numeric differentiation becomes unstable.
         # To fix this issue I ensure we are sampling numbers larger in
         # absolute value than 1.
-        v_x = np.array(
-            rng.uniform(size=(5, 2, 2), low=1.0, high=3.0), dtype=config.floatX
-        )
+        v_x = np.array(rng.uniform(1.0, 3.0, size=(5, 2, 2)), dtype=config.floatX)
         # Making some entries to be negative.
-        pos = rng.uniform(size=(5, 2, 2), low=0.0, high=1) < 0.5
+        pos = rng.uniform(0.0, 1, size=(5, 2, 2)) < 0.5
         v_x[pos] = -1 * v_x[pos]
-        v_w = np.array(rng.uniform(size=(2, 2), low=1.0, high=3.0), dtype=config.floatX)
-        pos = rng.uniform(size=(2, 2), low=0.0, high=1.0) < 0.5
+        v_w = np.array(rng.uniform(1.0, 3.0, size=(2, 2)), dtype=config.floatX)
+        pos = rng.uniform(0.0, 1.0, size=(2, 2)) < 0.5
         v_w[pos] = -1 * v_w[pos]
         analytic_grad = grad_fn(v_x, v_w)
         num_grad = multiple_outputs_numeric_grad(cost_fn, [v_x, v_w])

tests/tensor/random/test_basic.py

+import pickle
+from functools import partial
+
+import numpy as np
+import scipy.stats as stats
+from pytest import fixture, importorskip, raises
+
+import theano.tensor as tt
+from theano import change_flags, config
+from theano.gof.fg import FunctionGraph
+from theano.gof.graph import Variable
+from theano.gof.graph import inputs as tt_inputs
+from theano.gof.op import get_test_value
+from theano.tensor.random.basic import (
+    bernoulli,
+    beta,
+    betabinom,
+    binomial,
+    categorical,
+    cauchy,
+    choice,
+    dirichlet,
+    exponential,
+    gamma,
+    halfcauchy,
+    halfnormal,
+    invgamma,
+    multinomial,
+    multivariate_normal,
+    nbinom,
+    normal,
+    permutation,
+    poisson,
+    polyagamma,
+    randint,
+    truncexpon,
+    uniform,
+)
+
+
+@fixture(scope="module", autouse=True)
+def set_theano_flags():
+    with change_flags(cxx="", compute_test_value="raise"):
+        yield
+
+
+def rv_numpy_tester(rv, *params, **kwargs):
+    """Test for correspondence between `RandomVariable` and NumPy shape and
+    broadcast dimensions.
+    """
+    test_fn = kwargs.pop("test_fn", None)
+
+    if test_fn is None:
+        name = getattr(rv, "name", None)
+
+        if name is None:
+            name = rv.__name__
+
+        test_fn = getattr(np.random, name)
+
+    theano_res = rv(*params, **kwargs)
+
+    param_vals = [get_test_value(p) if isinstance(p, Variable) else p for p in params]
+    kwargs_vals = {
+        k: get_test_value(v) if isinstance(v, Variable) else v
+        for k, v in kwargs.items()
+    }
+
+    if "size" in kwargs:
+        kwargs["size"] = get_test_value(kwargs["size"])
+
+    numpy_res = np.asarray(test_fn(*param_vals, **kwargs_vals))
+
+    assert theano_res.type.numpy_dtype.kind == numpy_res.dtype.kind
+
+    numpy_shape = np.shape(numpy_res)
+    numpy_bcast = [s == 1 for s in numpy_shape]
+    np.testing.assert_array_equal(theano_res.type.broadcastable, numpy_bcast)
+
+    theano_res_val = theano_res.get_test_value()
+    np.testing.assert_array_equal(theano_res_val.shape, numpy_res.shape)
+
+
+def test_uniform_samples():
+
+    rv_numpy_tester(uniform)
+    rv_numpy_tester(uniform, size=())
+
+    test_low = np.array(10, dtype=config.floatX)
+    test_high = np.array(20, dtype=config.floatX)
+
+    rv_numpy_tester(uniform, test_low, test_high)
+    rv_numpy_tester(uniform, test_low, test_high, size=[3])
+
+
+def test_beta_samples():
+
+    test_a = np.array(0.5, dtype=config.floatX)
+    test_b = np.array(0.5, dtype=config.floatX)
+
+    rv_numpy_tester(beta, test_a, test_b)
+    rv_numpy_tester(beta, test_a, test_b, size=[3])
+
+
+def test_normal_infer_shape():
+    M_tt = tt.iscalar("M")
+    M_tt.tag.test_value = 3
+    sd_tt = tt.scalar("sd")
+    sd_tt.tag.test_value = np.array(1.0, dtype=config.floatX)
+
+    test_params = [
+        ([tt.as_tensor_variable(np.array(1.0, dtype=config.floatX)), sd_tt], None),
+        ([tt.as_tensor_variable(np.array(1.0, dtype=config.floatX)), sd_tt], (M_tt,)),
+        ([tt.as_tensor_variable(np.array(1.0, dtype=config.floatX)), sd_tt], (2, M_tt)),
+        ([tt.zeros((M_tt,)), sd_tt], None),
+        ([tt.zeros((M_tt,)), sd_tt], (M_tt,)),
+        ([tt.zeros((M_tt,)), sd_tt], (2, M_tt)),
+        ([tt.zeros((M_tt,)), tt.ones((M_tt,))], None),
+        ([tt.zeros((M_tt,)), tt.ones((M_tt,))], (2, M_tt)),
+        (
+            [
+                np.array([[-1, 20], [300, -4000]], dtype=config.floatX),
+                np.array([[1e-6, 2e-6]], dtype=config.floatX),
+            ],
+            (3, 2, 2),
+        ),
+        (
+            [np.array([1], dtype=config.floatX), np.array([10], dtype=config.floatX)],
+            (1, 2),
+        ),
+    ]
+    for args, size in test_params:
+        rv = normal(*args, size=size)
+        rv_shape = tuple(normal._infer_shape(size or (), args, None))
+        assert tuple(get_test_value(rv_shape)) == tuple(get_test_value(rv).shape)
+
+
+def test_normal_ShapeFeature():
+    M_tt = tt.iscalar("M")
+    M_tt.tag.test_value = 3
+    sd_tt = tt.scalar("sd")
+    sd_tt.tag.test_value = np.array(1.0, dtype=config.floatX)
+
+    d_rv = normal(tt.ones((M_tt,)), sd_tt, size=(2, M_tt))
+    d_rv.tag.test_value
+
+    fg = FunctionGraph(
+        [i for i in tt_inputs([d_rv]) if not isinstance(i, tt.Constant)],
+        [d_rv],
+        clone=False,
+        features=[tt.opt.ShapeFeature()],
+    )
+    s1, s2 = fg.shape_feature.shape_of[d_rv]
+
+    assert get_test_value(s1) == get_test_value(d_rv).shape[0]
+    assert get_test_value(s2) == get_test_value(d_rv).shape[1]
+
+
+def test_normal_samples():
+
+    rv_numpy_tester(normal)
+
+    test_mean = np.array(0, dtype=config.floatX)
+    test_stddev = np.array(1, dtype=config.floatX)
+
+    rv_numpy_tester(normal, test_mean, test_stddev)
+    rv_numpy_tester(normal, test_mean, test_stddev, size=[3])
+
+    # Broadcast sd over independent means...
+    test_mean = np.array([0, 1, 2], dtype=config.floatX)
+    test_stddev = np.array(1, dtype=config.floatX)
+    rv_numpy_tester(normal, test_mean, test_stddev)
+    rv_numpy_tester(normal, test_mean, test_stddev, size=[3, 3])
+
+    test_mean = np.array([0], dtype=config.floatX)
+    test_stddev = np.array([1], dtype=config.floatX)
+    rv_numpy_tester(normal, test_mean, test_stddev, size=[1])
+    rv_numpy_tester(normal, tt.as_tensor(test_mean), test_stddev, size=[1])
+    rv_numpy_tester(
+        normal,
+        tt.as_tensor_variable(test_mean),
+        test_stddev,
+        size=tt.as_tensor_variable([1]),
+    )
+
+
+def test_halfnormal_samples():
+    test_mean = np.array(0, dtype=config.floatX)
+    test_stddev = np.array(1, dtype=config.floatX)
+
+    rv_numpy_tester(halfnormal, test_fn=stats.halfnorm.rvs)
+    rv_numpy_tester(halfnormal, test_mean, test_stddev, test_fn=stats.halfnorm.rvs)
+    rv_numpy_tester(
+        halfnormal,
+        test_mean,
+        test_stddev,
+        size=[2, 3],
+        test_fn=stats.halfnorm.rvs,
+    )
+
+
+def test_gamma_samples():
+    test_a = np.array(0.5, dtype=config.floatX)
+    test_b = np.array(0.5, dtype=config.floatX)
+
+    rv_numpy_tester(gamma, test_a, test_b, test_fn=stats.gamma.rvs)
+    rv_numpy_tester(gamma, test_a, test_b, size=[2, 3], test_fn=stats.gamma.rvs)
+
+
+def test_exponential_samples():
+
+    rv_numpy_tester(exponential)
+
+    test_lambda = np.array(10, dtype=config.floatX)
+
+    rv_numpy_tester(exponential, test_lambda)
+    rv_numpy_tester(exponential, test_lambda, size=[2, 3])
+
+
+def test_mvnormal_samples():
+    def test_fn(mean=None, cov=None, size=None, rng=None):
+        if mean is None:
+            mean = np.array([0.0], dtype=config.floatX)
+        if cov is None:
+            cov = np.array([[1.0]], dtype=config.floatX)
+        if size is None:
+            size = ()
+        return multivariate_normal.rng_fn(rng, mean, cov, size)
+
+    rv_numpy_tester(multivariate_normal, test_fn=test_fn)
+
+    test_mean = np.array([0], dtype=config.floatX)
+    test_covar = np.diag(np.array([1], dtype=config.floatX))
+    rv_numpy_tester(multivariate_normal, test_mean, test_covar, test_fn=test_fn)
+    rv_numpy_tester(
+        multivariate_normal, test_mean, test_covar, size=[1], test_fn=test_fn
+    )
+    rv_numpy_tester(
+        multivariate_normal, test_mean, test_covar, size=[4], test_fn=test_fn
+    )
+    rv_numpy_tester(
+        multivariate_normal, test_mean, test_covar, size=[4, 1], test_fn=test_fn
+    )
+    rv_numpy_tester(
+        multivariate_normal, test_mean, test_covar, size=[4, 1, 1], test_fn=test_fn
+    )
+    rv_numpy_tester(
+        multivariate_normal, test_mean, test_covar, size=[1, 5, 8], test_fn=test_fn
+    )
+    test_mean = np.array(
+        [0, 1, 2],
+        dtype=config.floatX,
+    )
+    test_covar = np.diag(np.array([1, 10, 100], dtype=config.floatX))
+    rv_numpy_tester(multivariate_normal, test_mean, test_covar, test_fn=test_fn)
+
+    # Test parameter broadcasting
+    rv_numpy_tester(
+        multivariate_normal,
+        np.array([[0, 1, 2], [4, 5, 6]], dtype=config.floatX),
+        test_covar,
+        test_fn=test_fn,
+    )
+
+    test_covar = np.stack([test_covar, test_covar * 10.0])
+    rv_numpy_tester(
+        multivariate_normal,
+        np.array([0, 1, 2], dtype=config.floatX),
+        test_covar,
+        size=[2, 3],
+        test_fn=test_fn,
+    )
+
+    test_covar = np.stack([test_covar, test_covar * 10.0])
+    rv_numpy_tester(
+        multivariate_normal,
+        np.array([[0, 1, 2]], dtype=config.floatX),
+        test_covar,
+        size=[2, 3],
+        test_fn=test_fn,
+    )
+
+    rv_numpy_tester(
+        multivariate_normal,
+        np.array([[0], [10], [100]], dtype=config.floatX),
+        np.diag(np.array([1e-6], dtype=config.floatX)),
+        size=[2, 3],
+        test_fn=test_fn,
+    )
+
+
+def test_mvnormal_ShapeFeature():
+    M_tt = tt.iscalar("M")
+    M_tt.tag.test_value = 2
+
+    d_rv = multivariate_normal(tt.ones((M_tt,)), tt.eye(M_tt), size=2)
+
+    fg = FunctionGraph(
+        [i for i in tt_inputs([d_rv]) if not isinstance(i, tt.Constant)],
+        [d_rv],
+        clone=False,
+        features=[tt.opt.ShapeFeature()],
+    )
+
+    s1, s2 = fg.shape_feature.shape_of[d_rv]
+
+    assert get_test_value(s1) == 2
+    assert M_tt in tt_inputs([s2])
+
+    # Test broadcasted shapes
+    mean = tt.tensor(config.floatX, [True, False])
+    mean.tag.test_value = np.array([[0, 1, 2]], dtype=config.floatX)
+
+    test_covar = np.diag(np.array([1, 10, 100], dtype=config.floatX))
+    test_covar = np.stack([test_covar, test_covar * 10.0])
+    cov = tt.as_tensor(test_covar).type()
+    cov.tag.test_value = test_covar
+
+    d_rv = multivariate_normal(mean, cov, size=[2, 3])
+
+    fg = FunctionGraph(
+        [i for i in tt_inputs([d_rv]) if not isinstance(i, tt.Constant)],
+        [d_rv],
+        clone=False,
+        features=[tt.opt.ShapeFeature()],
+    )
+
+    s1, s2, s3, s4 = fg.shape_feature.shape_of[d_rv]
+
+    assert s1.get_test_value() == 2
+    assert s2.get_test_value() == 3
+    assert s3.get_test_value() == 2
+    assert s4.get_test_value() == 3
+
+
+def test_dirichlet_samples():
+
+    alphas = np.array([[100, 1, 1], [1, 100, 1], [1, 1, 100]], dtype=config.floatX)
+
+    res = get_test_value(dirichlet(alphas))
+    assert np.all(np.diag(res) >= res)
+
+    res = get_test_value(dirichlet(alphas, size=2))
+    assert res.shape == (2, 3, 3)
+    assert all(np.all(np.diag(r) >= r) for r in res)
+
+    for i in range(alphas.shape[0]):
+        res = get_test_value(dirichlet(alphas[i]))
+        assert np.all(res[i] > np.delete(res, [i]))
+
+        res = get_test_value(dirichlet(alphas[i], size=2))
+        assert res.shape == (2, 3)
+        assert all(np.all(r[i] > np.delete(r, [i])) for r in res)
+
+    rng_state = np.random.RandomState(np.random.MT19937(np.random.SeedSequence(1234)))
+
+    alphas = np.array([[1000, 1, 1], [1, 1000, 1], [1, 1, 1000]], dtype=config.floatX)
+
+    assert dirichlet.rng_fn(rng_state, alphas, None).shape == alphas.shape
+    assert dirichlet.rng_fn(rng_state, alphas, size=10).shape == (10,) + alphas.shape
+    assert (
+        dirichlet.rng_fn(rng_state, alphas, size=(10, 2)).shape
+        == (10, 2) + alphas.shape
+    )
+
+
+def test_dirichlet_infer_shape():
+    M_tt = tt.iscalar("M")
+    M_tt.tag.test_value = 3
+
+    test_params = [
+        ([tt.ones((M_tt,))], None),
+        ([tt.ones((M_tt,))], (M_tt + 1,)),
+        ([tt.ones((M_tt,))], (2, M_tt)),
+        ([tt.ones((M_tt, M_tt + 1))], None),
+        ([tt.ones((M_tt, M_tt + 1))], (M_tt + 2,)),
+        ([tt.ones((M_tt, M_tt + 1))], (2, M_tt + 2, M_tt + 3)),
+    ]
+    for args, size in test_params:
+        rv = dirichlet(*args, size=size)
+        rv_shape = tuple(dirichlet._infer_shape(size or (), args, None))
+        assert tuple(get_test_value(rv_shape)) == tuple(get_test_value(rv).shape)
+
+
+def test_dirichlet_ShapeFeature():
+    """Make sure `RandomVariable.infer_shape` works with `ShapeFeature`."""
+    M_tt = tt.iscalar("M")
+    M_tt.tag.test_value = 2
+    N_tt = tt.iscalar("N")
+    N_tt.tag.test_value = 3
+
+    d_rv = dirichlet(tt.ones((M_tt, N_tt)), name="Gamma")
+
+    fg = FunctionGraph(
+        [i for i in tt_inputs([d_rv]) if not isinstance(i, tt.Constant)],
+        [d_rv],
+        clone=False,
+        features=[tt.opt.ShapeFeature()],
+    )
+
+    s1, s2 = fg.shape_feature.shape_of[d_rv]
+
+    assert M_tt in tt_inputs([s1])
+    assert N_tt in tt_inputs([s2])
+
+
+def test_poisson_samples():
+
+    rv_numpy_tester(poisson)
+    rv_numpy_tester(poisson, size=tt.as_tensor((2, 3)))
+
+    test_lambda = np.array(10, dtype="int64")
+
+    rv_numpy_tester(poisson, test_lambda)
+    rv_numpy_tester(poisson, test_lambda, size=[2, 3])
+
+
+def test_cauchy_samples():
+    rv_numpy_tester(cauchy, test_fn=stats.cauchy.rvs)
+
+    test_loc = np.array(10, dtype=config.floatX)
+    test_scale = np.array(0.1, dtype=config.floatX)
+
+    rv_numpy_tester(cauchy, test_loc, test_scale, test_fn=stats.cauchy.rvs)
+    rv_numpy_tester(cauchy, test_loc, test_scale, size=[2, 3], test_fn=stats.cauchy.rvs)
+
+
+def test_halfcauchy_samples():
+    rv_numpy_tester(halfcauchy, test_fn=stats.halfcauchy.rvs)
+
+    test_loc = np.array(10, dtype=config.floatX)
+    test_scale = np.array(0.1, dtype=config.floatX)
+
+    rv_numpy_tester(halfcauchy, test_loc, test_scale, test_fn=stats.halfcauchy.rvs)
+    rv_numpy_tester(
+        halfcauchy,
+        test_loc,
+        test_scale,
+        size=[2, 3],
+        test_fn=stats.halfcauchy.rvs,
+    )
+
+
+def test_invgamma_samples():
+    test_loc = np.array(2, dtype=config.floatX)
+    test_scale = np.array(2, dtype=config.floatX)
+
+    rv_numpy_tester(
+        invgamma, test_loc, test_scale, test_fn=partial(invgamma.rng_fn, None)
+    )
+    rv_numpy_tester(
+        invgamma,
+        test_loc,
+        test_scale,
+        size=[2, 3],
+        test_fn=partial(invgamma.rng_fn, None),
+    )
+
+
+def test_truncexpon_samples():
+    test_b = np.array(5, dtype=config.floatX)
+    test_loc = np.array(0, dtype=config.floatX)
+    test_scale = np.array(1, dtype=config.floatX)
+
+    rv_numpy_tester(
+        truncexpon,
+        test_b,
+        test_loc,
+        test_scale,
+        test_fn=partial(truncexpon.rng_fn, None),
+    )
+    rv_numpy_tester(
+        truncexpon,
+        test_b,
+        test_loc,
+        test_scale,
+        size=[2, 3],
+        test_fn=partial(truncexpon.rng_fn, None),
+    )
+
+
+def test_bernoulli_samples():
+    test_p = np.array(0.5, dtype=config.floatX)
+
+    rv_numpy_tester(bernoulli, test_p, test_fn=partial(bernoulli.rng_fn, None))
+    rv_numpy_tester(
+        bernoulli,
+        test_p,
+        size=[2, 3],
+        test_fn=partial(bernoulli.rng_fn, None),
+    )
+
+
+def test_binomial_samples():
+    test_M = np.array(10, dtype="int64")
+    test_p = np.array(0.5, dtype=config.floatX)
+
+    rv_numpy_tester(binomial, test_M, test_p)
+    rv_numpy_tester(binomial, test_M, test_p, size=[2, 3])
+
+
+def test_nbinom_samples():
+    test_M = np.array(10, dtype="int64")
+    test_p = np.array(0.5, dtype=config.floatX)
+
+    rv_numpy_tester(nbinom, test_M, test_p, test_fn=partial(nbinom.rng_fn, None))
+    rv_numpy_tester(
+        nbinom,
+        test_M,
+        test_p,
+        size=[2, 3],
+        test_fn=partial(nbinom.rng_fn, None),
+    )
+
+
+def test_betabinom_samples():
+    test_M = np.array(10, dtype="int64")
+    test_a = np.array(0.5, dtype=config.floatX)
+    test_b = np.array(0.5, dtype=config.floatX)
+
+    rv_numpy_tester(
+        betabinom, test_M, test_a, test_b, test_fn=partial(betabinom.rng_fn, None)
+    )
+    rv_numpy_tester(
+        betabinom,
+        test_M,
+        test_a,
+        test_b,
+        size=[2, 3],
+        test_fn=partial(betabinom.rng_fn, None),
+    )
+
+
+def test_multinomial_samples():
+    test_M = np.array(10, dtype="int64")
+    test_p = np.array([0.7, 0.3], dtype=config.floatX)
+
+    rv_numpy_tester(multinomial, test_M, test_p)
+    rv_numpy_tester(
+        multinomial,
+        test_M,
+        test_p,
+        size=[2, 3],
+    )
+
+
+def test_categorical_samples():
+
+    rng_state = np.random.RandomState(np.random.MT19937(np.random.SeedSequence(1234)))
+
+    p = np.array([[100000, 1, 1], [1, 100000, 1], [1, 1, 100000]], dtype=config.floatX)
+    p = p / p.sum(axis=-1)
+
+    assert categorical.rng_fn(rng_state, p, size=None).shape == p.shape[:-1]
+
+    with raises(ValueError):
+        categorical.rng_fn(rng_state, p, size=10)
+
+    assert categorical.rng_fn(rng_state, p, size=(10, 3)).shape == (10, 3)
+    assert categorical.rng_fn(rng_state, p, size=(10, 2, 3)).shape == (10, 2, 3)
+
+    res = categorical(p)
+    assert np.array_equal(get_test_value(res), np.arange(3))
+
+    res = categorical(p, size=(10, 3))
+    exp_res = np.tile(np.arange(3), (10, 1))
+    assert np.array_equal(get_test_value(res), exp_res)
+
+    res = categorical(p, size=(10, 2, 3))
+    exp_res = np.tile(np.arange(3), (10, 2, 1))
+    assert np.array_equal(get_test_value(res), exp_res)
+
+
+def test_polyagamma_samples():
+
+    _ = importorskip("pypolyagamma")
+
+    # Sampled values should be scalars
+    a = np.array(1.1, dtype=config.floatX)
+    b = np.array(-10.5, dtype=config.floatX)
+    pg_rv = polyagamma(a, b)
+    assert get_test_value(pg_rv).shape == ()
+
+    pg_rv = polyagamma(a, b, size=[1])
+    assert get_test_value(pg_rv).shape == (1,)
+
+    pg_rv = polyagamma(a, b, size=[2, 3])
+    bcast_smpl = get_test_value(pg_rv)
+    assert bcast_smpl.shape == (2, 3)
+    # Make sure they're not all equal
+    assert np.all(np.abs(np.diff(bcast_smpl.flat)) > 0.0)
+
+    a = np.array([1.1, 3], dtype=config.floatX)
+    b = np.array(-10.5, dtype=config.floatX)
+    pg_rv = polyagamma(a, b)
+    bcast_smpl = get_test_value(pg_rv)
+    assert bcast_smpl.shape == (2,)
+    assert np.all(np.abs(np.diff(bcast_smpl.flat)) > 0.0)
+
+    pg_rv = polyagamma(a, b, size=(3, 2))
+    bcast_smpl = get_test_value(pg_rv)
+    assert bcast_smpl.shape == (3, 2)
+    assert np.all(np.abs(np.diff(bcast_smpl.flat)) > 0.0)
+
+
+def test_random_integer_samples():
+
+    rv_numpy_tester(randint, 10, None)
+    rv_numpy_tester(randint, 0, 1)
+    rv_numpy_tester(randint, 0, 1, size=[3])
+    rv_numpy_tester(randint, [0, 1, 2], 5)
+    rv_numpy_tester(randint, [0, 1, 2], 5, size=[3, 3])
+    rv_numpy_tester(randint, [0], [5], size=[1])
+    rv_numpy_tester(randint, tt.as_tensor_variable([-1]), [1], size=[1])
+    rv_numpy_tester(
+        randint, tt.as_tensor_variable([-1]), [1], size=tt.as_tensor_variable([1])
+    )
+
+
+def test_choice_samples():
+    with raises(NotImplementedError):
+        choice._shape_from_params(np.asarray(5))
+
+    rv_numpy_tester(choice, np.asarray(5))
+    rv_numpy_tester(choice, np.array([1.0, 5.0], dtype=config.floatX))
+    rv_numpy_tester(choice, np.asarray(5), 3)
+
+    with raises(ValueError):
+        rv_numpy_tester(choice, np.array([[1, 2], [3, 4]]))
+
+    rv_numpy_tester(choice, [1, 2, 3], 1)
+    rv_numpy_tester(choice, [1, 2, 3], 1, p=tt.as_tensor([1 / 3.0, 1 / 3.0, 1 / 3.0]))
+    rv_numpy_tester(choice, [1, 2, 3], (10, 2), replace=True)
+    rv_numpy_tester(choice, tt.as_tensor_variable([1, 2, 3]), 2, replace=True)
+
+
+def test_permutation_samples():
+    rv_numpy_tester(
+        permutation, np.asarray(5), test_fn=lambda x: np.random.permutation(x.item())
+    )
+    rv_numpy_tester(permutation, [1, 2, 3])
+    rv_numpy_tester(permutation, [[1, 2], [3, 4]])
+    rv_numpy_tester(permutation, np.array([1.0, 2.0, 3.0], dtype=config.floatX))
+
+
+@change_flags(compute_test_value="off")
+def test_pickle():
+    # This is an interesting `Op` case, because it has `None` types and a
+    # conditional dtype
+    sample_a = choice(5, size=(2, 3))
+
+    a_pkl = pickle.dumps(sample_a)
+    a_unpkl = pickle.loads(a_pkl)
+
+    assert a_unpkl.owner.op._props() == sample_a.owner.op._props()

tests/tensor/random/test_op.py

+import numpy as np
+from pytest import fixture, raises
+
+import theano.tensor as tt
+from theano import change_flags, config
+from theano.gradient import NullTypeGradError
+from theano.tensor.opt import Assert
+from theano.tensor.random.basic import normal
+from theano.tensor.random.op import RandomVariable, default_shape_from_params, observed
+from theano.tensor.type_other import NoneTypeT
+
+
+@fixture(scope="module", autouse=True)
+def set_theano_flags():
+    with change_flags(cxx="", compute_test_value="raise"):
+        yield
+
+
+def test_default_shape_from_params():
+    with raises(ValueError, match="^ndim_supp*"):
+        default_shape_from_params(0, (np.array([1, 2]), 0))
+
+    res = default_shape_from_params(1, (np.array([1, 2]), np.eye(2)), rep_param_idx=0)
+    assert res == (2,)
+
+    res = default_shape_from_params(1, (np.array([1, 2]), 0), param_shapes=((2,), ()))
+    assert res == (2,)
+
+    with raises(ValueError, match="^Reference parameter*"):
+        default_shape_from_params(1, (np.array(1),), rep_param_idx=0)
+
+    res = default_shape_from_params(
+        2, (np.array([1, 2]), np.ones((2, 3, 4))), rep_param_idx=1
+    )
+    assert res == (3, 4)
+
+
+def test_RandomVariable():
+
+    str_res = str(
+        RandomVariable(
+            "normal",
+            0,
+            [0, 0],
+            "normal",
+            inplace=True,
+        )
+    )
+
+    assert str_res == "normal_rv"
+
+    # `ndims_params` should be a `Sequence` type
+    with raises(TypeError, match="^Parameter ndims_params*"):
+        RandomVariable(
+            "normal",
+            0,
+            0,
+            config.floatX,
+            inplace=True,
+        )
+
+    # `size` should be a `Sequence` type
+    with raises(TypeError, match="^Parameter size*"):
+        RandomVariable(
+            "normal",
+            0,
+            [0, 0],
+            config.floatX,
+            inplace=True,
+        )(0, 1, size={1, 2})
+
+    # No dtype
+    with raises(TypeError, match="^dtype*"):
+        RandomVariable(
+            "normal",
+            0,
+            [0, 0],
+            inplace=True,
+        )(0, 1)
+
+    # Confirm that `inplace` works
+    rv = RandomVariable(
+        "normal",
+        0,
+        [0, 0],
+        "normal",
+        inplace=True,
+    )
+
+    assert rv.inplace
+    assert rv.destroy_map == {0: [3]}
+
+    # A no-params `RandomVariable`
+    rv = RandomVariable(name="test_rv", ndim_supp=0, ndims_params=())
+
+    with raises(TypeError):
+        rv.make_node(rng=1)
+
+    # `RandomVariable._infer_shape` should handle no parameters
+    rv_shape = rv._infer_shape(tt.constant([]), (), [])
+    assert rv_shape.equals(tt.constant([], dtype="int64"))
+
+    # Integer-specificed `dtype`
+    dtype_1 = tt.all_dtypes[1]
+    rv_node = rv.make_node(None, None, 1)
+    rv_out = rv_node.outputs[1]
+    rv_out.tag.test_value = 1
+
+    assert rv_out.dtype == dtype_1
+
+    with raises(NullTypeGradError):
+        tt.grad(rv_out, [rv_node.inputs[0]])
+
+    rv = RandomVariable("normal", 0, [0, 0], config.floatX, inplace=True)
+
+    mu = tt.tensor(config.floatX, [True, False, False])
+    mu.tag.test_value = np.zeros((1, 2, 3)).astype(config.floatX)
+    sd = tt.tensor(config.floatX, [False, False])
+    sd.tag.test_value = np.ones((2, 3)).astype(config.floatX)
+
+    s1 = tt.iscalar()
+    s1.tag.test_value = 1
+    s2 = tt.iscalar()
+    s2.tag.test_value = 2
+    s3 = tt.iscalar()
+    s3.tag.test_value = 3
+    s3 = Assert("testing")(s3, tt.eq(s1, 1))
+
+    res = rv.compute_bcast([mu, sd], (s1, s2, s3))
+    assert res == [False] * 3
+
+
+def test_observed():
+    rv_var = normal(0, 1, size=3)
+    obs_var = observed(rv_var, np.array([0.2, 0.1, -2.4], dtype=config.floatX))
+
+    assert obs_var.owner.inputs[0] is rv_var
+
+    with raises(TypeError):
+        observed(rv_var, np.array([1, 2], dtype=int))
+
+    with raises(TypeError):
+        observed(rv_var, np.array([[1.0, 2.0]], dtype=rv_var.dtype))
+
+    obs_rv = observed(None, np.array([0.2, 0.1, -2.4], dtype=config.floatX))
+
+    assert isinstance(obs_rv.owner.inputs[0].type, NoneTypeT)
+
+    rv_val = tt.vector()
+    rv_val.tag.test_value = np.array([0.2, 0.1, -2.4], dtype=config.floatX)
+
+    obs_var = observed(rv_var, rv_val)
+
+    with raises(NullTypeGradError):
+        tt.grad(obs_var.sum(), [rv_val])

tests/tensor/random/test_opt.py

+import numpy as np
+
+from theano.compile.function import function
+from theano.compile.mode import Mode
+from theano.gof.optdb import Query
+from theano.tensor.random.basic import normal
+
+
+opts = Query(include=["random_make_inplace"], exclude=[])
+inplace_mode = Mode("py", opts)
+
+
+def test_inplace_optimization():
+
+    out = normal(0, 1)
+
+    assert out.owner.op.inplace is False
+
+    f = function(
+        [],
+        out,
+        mode=inplace_mode,
+    )
+
+    (new_out,) = f.maker.fgraph.outputs
+    assert new_out.type == out.type
+    assert isinstance(new_out.owner.op, type(out.owner.op))
+    assert new_out.owner.op.inplace is True
+    assert all(
+        np.array_equal(a.data, b.data)
+        for a, b in zip(new_out.owner.inputs[1:], out.owner.inputs[1:])
+    )

tests/tensor/random/test_type.py

+import pickle
+import sys
+
+import numpy as np
+import pytest
+
+from theano import shared
+from theano.compile.ops import ViewOp
+from theano.tensor.random.type import RandomStateType, random_state_type
+
+
+# @pytest.mark.skipif(
+#     not config.cxx, reason="G++ not available, so we need to skip this test."
+# )
+def test_view_op_c_code():
+    # TODO: It might be good to make sure that the registered C code works
+    # (even though it's basically copy-paste from other registered `Op`s).
+    # from theano.compile.ops import view_op
+    # from theano.gof.cc import CLinker
+    # rng_var = random_state_type()
+    # rng_view = view_op(rng_var)
+    # function(
+    #     [rng_var],
+    #     rng_view,
+    #     mode=Mode(optimizer=None, linker=CLinker()),
+    # )
+
+    assert ViewOp.c_code_and_version[RandomStateType]
+
+
+class TestRandomStateType:
+    def test_pickle(self):
+        rng_r = random_state_type()
+
+        rng_pkl = pickle.dumps(rng_r)
+        rng_unpkl = pickle.loads(rng_pkl)
+
+        assert isinstance(rng_unpkl, type(rng_r))
+        assert isinstance(rng_unpkl.type, type(rng_r.type))
+
+    def test_repr(self):
+        assert repr(random_state_type) == "RandomStateType"
+
+    def test_filter(self):
+
+        rng_type = random_state_type
+
+        rng = np.random.RandomState()
+        assert rng_type.filter(rng) is rng
+
+        with pytest.raises(TypeError):
+            rng_type.filter(1)
+
+    def test_values_eq(self):
+
+        rng_type = random_state_type
+
+        rng_a = np.random.RandomState(12)
+        rng_b = np.random.RandomState(12)
+        rng_c = np.random.RandomState(123)
+
+        bg = np.random.PCG64()
+        rng_d = np.random.RandomState(bg)
+        rng_e = np.random.RandomState(bg)
+
+        bg_2 = np.random.Philox()
+        rng_f = np.random.RandomState(bg_2)
+        rng_g = np.random.RandomState(bg_2)
+
+        assert rng_type.values_eq(rng_a, rng_b)
+        assert not rng_type.values_eq(rng_a, rng_c)
+
+        assert not rng_type.values_eq(rng_a, rng_d)
+        assert not rng_type.values_eq(rng_d, rng_a)
+
+        assert not rng_type.values_eq(rng_a, rng_d)
+        assert rng_type.values_eq(rng_d, rng_e)
+
+        assert rng_type.values_eq(rng_f, rng_g)
+        assert not rng_type.values_eq(rng_g, rng_a)
+        assert not rng_type.values_eq(rng_e, rng_g)
+
+    def test_get_shape_info(self):
+        rng = np.random.RandomState(12)
+        rng_a = shared(rng)
+
+        assert isinstance(
+            random_state_type.get_shape_info(rng_a), np.random.RandomState
+        )
+
+    def test_get_size(self):
+        rng = np.random.RandomState(12)
+        rng_a = shared(rng)
+        shape_info = random_state_type.get_shape_info(rng_a)
+        size = random_state_type.get_size(shape_info)
+        assert size == sys.getsizeof(rng.get_state(legacy=False))
+
+    def test_may_share_memory(self):
+        rng_a = np.random.RandomState(12)
+        bg = np.random.PCG64()
+        rng_b = np.random.RandomState(bg)
+
+        rng_var_a = shared(rng_a, borrow=True)
+        rng_var_b = shared(rng_b, borrow=True)
+        shape_info_a = random_state_type.get_shape_info(rng_var_a)
+        shape_info_b = random_state_type.get_shape_info(rng_var_b)
+
+        assert random_state_type.may_share_memory(shape_info_a, shape_info_b) is False
+
+        rng_c = np.random.RandomState(bg)
+        rng_var_c = shared(rng_c, borrow=True)
+        shape_info_c = random_state_type.get_shape_info(rng_var_c)
+
+        assert random_state_type.may_share_memory(shape_info_b, shape_info_c) is True

tests/tensor/random/test_utils.py

+import numpy as np
+import pytest
+
+import theano.tensor as tt
+from tests import unittest_tools as utt
+from theano import change_flags, config, function
+from theano.compile.mode import Mode
+from theano.gof.optdb import Query
+from theano.tensor.random.utils import RandomStream, broadcast_params
+
+
+@pytest.fixture(scope="module", autouse=True)
+def set_theano_flags():
+    opts = Query(include=[None], exclude=[])
+    py_mode = Mode("py", opts)
+    with change_flags(mode=py_mode, compute_test_value="warn"):
+        yield
+
+
+def test_broadcast_params():
+
+    ndims_params = [0, 0]
+
+    mean = np.array([0, 1, 2])
+    cov = np.array(1e-6)
+    params = [mean, cov]
+    res = broadcast_params(params, ndims_params)
+    assert np.array_equal(res[0], mean)
+    assert np.array_equal(res[1], np.broadcast_to(cov, (3,)))
+
+    ndims_params = [1, 2]
+
+    mean = np.r_[1, 2, 3]
+    cov = np.stack([np.eye(3) * 1e-5, np.eye(3) * 1e-4])
+    params = [mean, cov]
+    res = broadcast_params(params, ndims_params)
+    assert np.array_equal(res[0], np.broadcast_to(mean, (2, 3)))
+    assert np.array_equal(res[1], cov)
+
+    mean = np.stack([np.r_[0, 0, 0], np.r_[1, 1, 1]])
+    cov = np.arange(3 * 3).reshape((3, 3))
+    params = [mean, cov]
+    res = broadcast_params(params, ndims_params)
+    assert np.array_equal(res[0], mean)
+    assert np.array_equal(res[1], np.broadcast_to(cov, (2, 3, 3)))
+
+    mean = np.stack([np.r_[0, 0, 0], np.r_[1, 1, 1]])
+    cov = np.stack(
+        [np.arange(3 * 3).reshape((3, 3)), np.arange(3 * 3).reshape((3, 3)) * 10]
+    )
+    params = [mean, cov]
+    res = broadcast_params(params, ndims_params)
+    assert np.array_equal(res[0], mean)
+    assert np.array_equal(res[1], cov)
+
+    mean = np.array([[1, 2, 3]])
+    cov = np.stack([np.eye(3) * 1e-5, np.eye(3) * 1e-4])
+    params = [mean, cov]
+    res = broadcast_params(params, ndims_params)
+    assert np.array_equal(res[0], np.array([[1, 2, 3], [1, 2, 3]]))
+    assert np.array_equal(res[1], cov)
+
+    mean = np.array([[0], [10], [100]])
+    cov = np.diag(np.array([1e-6]))
+    params = [mean, cov]
+    res = broadcast_params(params, ndims_params)
+    assert np.array_equal(res[0], mean)
+    assert np.array_equal(res[1], np.broadcast_to(cov, (3, 1, 1)))
+
+    # Try it in Theano
+    with change_flags(compute_test_value="raise"):
+        mean = tt.tensor(config.floatX, [False, True])
+        mean.tag.test_value = np.array([[0], [10], [100]], dtype=config.floatX)
+        cov = tt.matrix()
+        cov.tag.test_value = np.diag(np.array([1e-6], dtype=config.floatX))
+        params = [mean, cov]
+        res = broadcast_params(params, ndims_params)
+        assert np.array_equal(res[0].get_test_value(), mean.get_test_value())
+        assert np.array_equal(
+            res[1].get_test_value(), np.broadcast_to(cov.get_test_value(), (3, 1, 1))
+        )
+
+
+class TestSharedRandomStream:
+    def setup_method(self):
+        utt.seed_rng()
+
+    def test_tutorial(self):
+        srng = RandomStream(seed=234)
+        rv_u = srng.uniform(0, 1, size=(2, 2))
+        rv_n = srng.normal(0, 1, size=(2, 2))
+
+        f = function([], rv_u)
+        # Disabling `default_updates` means that we have to pass
+        # `srng.state_updates` to `function` manually, if we want the shared
+        # state to change
+        g = function([], rv_n, no_default_updates=True)
+        nearly_zeros = function([], rv_u + rv_u - 2 * rv_u)
+
+        assert np.all(f() != f())
+        assert np.all(g() == g())
+        assert np.all(abs(nearly_zeros()) < 1e-5)
+        assert isinstance(rv_u.rng.get_value(borrow=True), np.random.RandomState)
+
+    def test_basics(self):
+        random = RandomStream(seed=utt.fetch_seed())
+
+        with pytest.raises(TypeError):
+            random.uniform(0, 1, size=(2, 2), rng=np.random.RandomState(23))
+
+        with pytest.raises(AttributeError):
+            random.blah
+
+        with pytest.raises(AttributeError):
+            np_random = RandomStream(namespace=np)
+            np_random.ndarray
+
+        fn = function([], random.uniform(0, 1, size=(2, 2)), updates=random.updates())
+
+        fn_val0 = fn()
+        fn_val1 = fn()
+
+        rng_seed = np.random.RandomState(utt.fetch_seed()).randint(2 ** 30)
+        rng = np.random.RandomState(int(rng_seed))  # int() is for 32bit
+
+        numpy_val0 = rng.uniform(0, 1, size=(2, 2))
+        numpy_val1 = rng.uniform(0, 1, size=(2, 2))
+
+        assert np.allclose(fn_val0, numpy_val0)
+        assert np.allclose(fn_val1, numpy_val1)
+
+    def test_seed(self):
+        init_seed = 234
+        random = RandomStream(init_seed)
+
+        ref_state = np.random.RandomState(init_seed).get_state()
+        random_state = random.gen_seedgen.get_state()
+        assert random.default_instance_seed == init_seed
+        assert np.array_equal(random_state[1], ref_state[1])
+        assert random_state[0] == ref_state[0]
+        assert random_state[2:] == ref_state[2:]
+
+        new_seed = 43298
+        random.seed(new_seed)
+
+        ref_state = np.random.RandomState(new_seed).get_state()
+        random_state = random.gen_seedgen.get_state()
+        assert np.array_equal(random_state[1], ref_state[1])
+        assert random_state[0] == ref_state[0]
+        assert random_state[2:] == ref_state[2:]
+
+        random.seed()
+        ref_state = np.random.RandomState(init_seed).get_state()
+        random_state = random.gen_seedgen.get_state()
+        assert random.default_instance_seed == init_seed
+        assert np.array_equal(random_state[1], ref_state[1])
+        assert random_state[0] == ref_state[0]
+        assert random_state[2:] == ref_state[2:]
+
+        # Reset the seed
+        random.seed(new_seed)
+
+        # Check state updates
+        _ = random.normal()
+
+        # Now, change the seed when there are state updates
+        random.seed(new_seed)
+
+        rng = np.random.RandomState(new_seed)
+        update_seed = rng.randint(2 ** 30)
+        ref_state = np.random.RandomState(update_seed).get_state()
+        random_state = random.state_updates[0][0].get_value(borrow=True).get_state()
+        assert np.array_equal(random_state[1], ref_state[1])
+        assert random_state[0] == ref_state[0]
+        assert random_state[2:] == ref_state[2:]
+
+    def test_uniform(self):
+        # Test that RandomStream.uniform generates the same results as numpy
+        # Check over two calls to see if the random state is correctly updated.
+        random = RandomStream(utt.fetch_seed())
+        fn = function([], random.uniform(-1, 1, size=(2, 2)))
+        fn_val0 = fn()
+        fn_val1 = fn()
+
+        rng_seed = np.random.RandomState(utt.fetch_seed()).randint(2 ** 30)
+        rng = np.random.RandomState(int(rng_seed))  # int() is for 32bit
+        numpy_val0 = rng.uniform(-1, 1, size=(2, 2))
+        numpy_val1 = rng.uniform(-1, 1, size=(2, 2))
+
+        assert np.allclose(fn_val0, numpy_val0)
+        assert np.allclose(fn_val1, numpy_val1)
+
+    def test_default_updates(self):
+        # Basic case: default_updates
+        random_a = RandomStream(utt.fetch_seed())
+        out_a = random_a.uniform(0, 1, size=(2, 2))
+        fn_a = function([], out_a)
+        fn_a_val0 = fn_a()
+        fn_a_val1 = fn_a()
+        assert not np.all(fn_a_val0 == fn_a_val1)
+
+        nearly_zeros = function([], out_a + out_a - 2 * out_a)
+        assert np.all(abs(nearly_zeros()) < 1e-5)
+
+        # Explicit updates #1
+        random_b = RandomStream(utt.fetch_seed())
+        out_b = random_b.uniform(0, 1, size=(2, 2))
+        fn_b = function([], out_b, updates=random_b.updates())
+        fn_b_val0 = fn_b()
+        fn_b_val1 = fn_b()
+        assert np.all(fn_b_val0 == fn_a_val0)
+        assert np.all(fn_b_val1 == fn_a_val1)
+
+        # Explicit updates #2
+        random_c = RandomStream(utt.fetch_seed())
+        out_c = random_c.uniform(0, 1, size=(2, 2))
+        fn_c = function([], out_c, updates=[out_c.update])
+        fn_c_val0 = fn_c()
+        fn_c_val1 = fn_c()
+        assert np.all(fn_c_val0 == fn_a_val0)
+        assert np.all(fn_c_val1 == fn_a_val1)
+
+        # No updates at all
+        random_d = RandomStream(utt.fetch_seed())
+        out_d = random_d.uniform(0, 1, size=(2, 2))
+        fn_d = function([], out_d, no_default_updates=True)
+        fn_d_val0 = fn_d()
+        fn_d_val1 = fn_d()
+        assert np.all(fn_d_val0 == fn_a_val0)
+        assert np.all(fn_d_val1 == fn_d_val0)
+
+        # No updates for out
+        random_e = RandomStream(utt.fetch_seed())
+        out_e = random_e.uniform(0, 1, size=(2, 2))
+        fn_e = function([], out_e, no_default_updates=[out_e.rng])
+        fn_e_val0 = fn_e()
+        fn_e_val1 = fn_e()
+        assert np.all(fn_e_val0 == fn_a_val0)
+        assert np.all(fn_e_val1 == fn_e_val0)
+
+    def test_multiple_rng_aliasing(self):
+        # Test that when we have multiple random number generators, we do not alias
+        # the state_updates member. `state_updates` can be useful when attempting to
+        # copy the (random) state between two similar theano graphs. The test is
+        # meant to detect a previous bug where state_updates was initialized as a
+        # class-attribute, instead of the __init__ function.
+
+        rng1 = RandomStream(1234)
+        rng2 = RandomStream(2392)
+        assert rng1.state_updates is not rng2.state_updates
+        assert rng1.gen_seedgen is not rng2.gen_seedgen
+
+    def test_random_state_transfer(self):
+        # Test that random state can be transferred from one theano graph to another.
+
+        class Graph:
+            def __init__(self, seed=123):
+                self.rng = RandomStream(seed)
+                self.y = self.rng.uniform(0, 1, size=(1,))
+
+        g1 = Graph(seed=123)
+        f1 = function([], g1.y)
+        g2 = Graph(seed=987)
+        f2 = function([], g2.y)
+
+        for (su1, su2) in zip(g1.rng.state_updates, g2.rng.state_updates):
+            su2[0].set_value(su1[0].get_value())
+
+        np.testing.assert_array_almost_equal(f1(), f2(), decimal=6)

tests/tensor/random/test_var.py

+import numpy as np
+
+from theano import shared
+
+
+def test_RandomStateSharedVariable():
+    rng = np.random.RandomState(123)
+    s_rng_default = shared(rng)
+    s_rng_True = shared(rng, borrow=True)
+    s_rng_False = shared(rng, borrow=False)
+
+    # test borrow contract: that False means a copy must have been made
+    assert s_rng_default.container.storage[0] is not rng
+    assert s_rng_False.container.storage[0] is not rng
+
+    # test current implementation: that True means a copy was not made
+    assert s_rng_True.container.storage[0] is rng
+
+    # ensure that all the random number generators are in the same state
+    v = rng.randn()
+    v0 = s_rng_default.container.storage[0].randn()
+    v1 = s_rng_False.container.storage[0].randn()
+    assert v == v0 == v1
+
+
+def test_get_value_borrow():
+
+    rng = np.random.RandomState(123)
+    s_rng = shared(rng)
+
+    r_ = s_rng.container.storage[0]
+    r_T = s_rng.get_value(borrow=True)
+    r_F = s_rng.get_value(borrow=False)
+
+    # the contract requires that borrow=False returns a copy
+    assert r_ is not r_F
+
+    # the current implementation allows for True to return the real thing
+    assert r_ is r_T
+
+    # either way, the rngs should all be in the same state
+    assert r_.rand() == r_F.rand()
+
+
+def test_get_value_internal_type():
+    rng = np.random.RandomState(123)
+    s_rng = shared(rng)
+
+    # there is no special behaviour required of return_internal_type
+    # this test just ensures that the flag doesn't screw anything up
+    # by repeating the get_value_borrow test.
+    r_ = s_rng.container.storage[0]
+    r_T = s_rng.get_value(borrow=True, return_internal_type=True)
+    r_F = s_rng.get_value(borrow=False, return_internal_type=True)
+
+    # the contract requires that borrow=False returns a copy
+    assert r_ is not r_F
+
+    # the current implementation allows for True to return the real thing
+    assert r_ is r_T
+
+    # either way, the rngs should all be in the same state
+    assert r_.rand() == r_F.rand()
+
+
+def test_set_value_borrow():
+    rng = np.random.RandomState(123)
+
+    s_rng = shared(rng)
+
+    new_rng = np.random.RandomState(234234)
+
+    # Test the borrow contract is respected:
+    # assigning with borrow=False makes a copy
+    s_rng.set_value(new_rng, borrow=False)
+    assert new_rng is not s_rng.container.storage[0]
+    assert new_rng.randn() == s_rng.container.storage[0].randn()
+
+    # Test that the current implementation is actually borrowing when it can.
+    rr = np.random.RandomState(33)
+    s_rng.set_value(rr, borrow=True)
+    assert rr is s_rng.container.storage[0]

tests/tensor/test_raw_random.py

-import pickle
-
-import numpy as np
-import pytest
-
-import theano
-from tests import unittest_tools as utt
-from theano import config, tensor
-from theano.compile.function import function
-from theano.compile.io import In
-from theano.tensor import dcol, dvector, ivector, raw_random
-from theano.tensor.raw_random import (
-    RandomFunction,
-    binomial,
-    choice,
-    multinomial,
-    normal,
-    permutation,
-    permutation_helper,
-    poisson,
-    random_integers,
-    random_state_type,
-    uniform,
-)
-
-
-__docformat__ = "restructuredtext en"
-
-
-class TestRandomFunction(utt.InferShapeTester):
-    def setup_method(self):
-        super().setup_method()
-        utt.seed_rng()
-
-    def test_basic_usage(self):
-        rf = RandomFunction(np.random.RandomState.uniform, tensor.dvector)
-        assert not rf.inplace
-        assert getattr(rf, "destroy_map", {}) == {}
-
-        rng_R = random_state_type()
-
-        # If calling RandomFunction directly, all args have to be specified,
-        # because shape will have to be moved to the end
-        post_r, out = rf(rng_R, (4,), 0.0, 1.0)
-
-        assert out.type == tensor.dvector
-
-        f = function([rng_R], out)
-
-        rng_state0 = np.random.RandomState(utt.fetch_seed())
-
-        f_0 = f(rng_state0)
-        f_1 = f(rng_state0)
-
-        assert np.all(f_0 == f_1)
-
-    def test_inplace_norun(self):
-        rf = RandomFunction(np.random.RandomState.uniform, tensor.dvector, inplace=True)
-        assert rf.inplace
-        assert getattr(rf, "destroy_map", {}) != {}
-
-    def test_args(self):
-        # Test that arguments to RandomFunction are honored
-        rf2 = RandomFunction(np.random.RandomState.uniform, tensor.dvector)
-        rf4 = RandomFunction(
-            np.random.RandomState.uniform, tensor.dvector, inplace=True
-        )
-        rng_R = random_state_type()
-
-        # use make_node to override some of the self.args
-        post_r2, out2 = rf2(rng_R, (4,), -2, 2)  # NOT INPLACE
-        post_r4, out4 = rf4(rng_R, (4,), -4, 4)  # INPLACE
-        post_r2_4, out2_4 = rf2(rng_R, (4,), -4.0, 2)  # NOT INPLACE
-        post_r2_4_4, out2_4_4 = rf2(rng_R, (4,), -4.0, 4.0)  # NOT INPLACE
-
-        # configure out4 to be computed inplace
-        # The update expression means that the random state rng_R will
-        # be maintained by post_r4
-        f = function(
-            [
-                In(
-                    rng_R,
-                    value=np.random.RandomState(utt.fetch_seed()),
-                    update=post_r4,
-                    mutable=True,
-                )
-            ],
-            [out2, out4, out2_4, out2_4_4],
-            accept_inplace=True,
-        )
-
-        f2, f4, f2_4, f2_4_4 = f()
-        f2b, f4b, f2_4b, f2_4_4b = f()
-
-        # print f2
-        # print f4
-        # print f2_4
-        # print f2_4_4
-
-        # print f2b
-        # print f4b
-        # print f2_4b
-        # print f2_4_4b
-
-        # setting bounds is same as multiplying by 2
-        assert np.allclose(f2 * 2, f4), (f2, f4)
-
-        # retrieving from non-inplace generator
-        # is same as inplace one for first call
-        assert np.allclose(f2_4_4, f4), (f2_4_4, f4)
-
-        # f4 changes from call to call, that the update has worked
-        assert not np.allclose(f4, f4b), (f4, f4b)
-
-    def test_inplace_optimization(self):
-        # Test that FAST_RUN includes the random_make_inplace optimization
-        # inplace = False
-        rf2 = RandomFunction(np.random.RandomState.uniform, tensor.dvector)
-        rng_R = random_state_type()
-
-        # If calling RandomFunction directly, all args have to be specified,
-        # because shape will have to be moved to the end
-        post_r2, out2 = rf2(rng_R, (4,), 0.0, 1.0)
-
-        f = function(
-            [
-                In(
-                    rng_R,
-                    value=np.random.RandomState(utt.fetch_seed()),
-                    update=post_r2,
-                    mutable=True,
-                )
-            ],
-            out2,
-            mode="FAST_RUN",
-        )  # DEBUG_MODE can't pass the id-based
-        # test below
-
-        # test that the RandomState object stays the same from function call to
-        # function call, but that the values returned change from call to call.
-
-        id0 = id(f[rng_R])
-        val0 = f()
-        assert id0 == id(f[rng_R])
-        val1 = f()
-        assert id0 == id(f[rng_R])
-
-        assert not np.allclose(val0, val1)
-
-    def test_no_inplace(self):
-        # Test that when not running inplace, the RandomState is not updated
-        rf = RandomFunction("uniform", tensor.dvector)
-        rng_R = random_state_type()
-
-        post_r, out = rf(rng_R, (3,), 0.0, 1.0)
-        f = function([rng_R], [post_r, out])
-        rng = np.random.RandomState(utt.fetch_seed())
-
-        rng0, val0 = f(rng)
-        rng_ = np.random.RandomState(utt.fetch_seed())
-        # rng should still be in a fresh state
-        assert rng_R.type.values_eq(rng, rng_)
-        # rng0 should be in an updated state
-        assert not rng_R.type.values_eq(rng, rng0)
-
-        f2 = function(
-            [In(rng_R, value=rng, update=post_r, mutable=False)], [post_r, out]
-        )
-        rng2, val2 = f2()
-        # rng should be in a fresh state
-        assert rng_R.type.values_eq(rng, rng_)
-        # rng2 should be in an updated state
-        assert not rng_R.type.values_eq(rng, rng2)
-        # The updated state should be the same for both functions
-        assert rng_R.type.values_eq(rng2, rng0)
-
-        rng3, val3 = f2()
-        # rng2 should not have changed
-        assert rng_R.type.values_eq(rng2, rng0)
-        # rng3 should be an updated again version of rng2
-        assert not rng_R.type.values_eq(rng3, rng2)
-        assert not rng_R.type.values_eq(rng3, rng)
-
-    def test_random_function_ndim(self):
-        # Test that random_function helper function accepts argument ndim
-        rng_R = random_state_type()
-
-        # ndim is an optional argument indicating the length of the 'shape'
-        # ndim not specified, OK
-        post_out4, out4 = uniform(rng_R, (4,))
-
-        # ndim specified, consistent with shape, OK
-        post_out1_4, out1_4 = uniform(rng_R, (4,), ndim=1)
-        post_out2_4_4, out2_4_4 = uniform(rng_R, (4, 4), ndim=2)
-
-        # ndim specified, but not compatible with shape
-        with pytest.raises(ValueError):
-            uniform(rng_R, (4,), ndim=2)
-
-        f_ok = function(
-            [
-                In(
-                    rng_R,
-                    value=np.random.RandomState(utt.fetch_seed()),
-                    update=post_out2_4_4,
-                    mutable=True,
-                )
-            ],
-            [out4, out1_4, out2_4_4],
-            accept_inplace=True,
-        )
-
-        # The correct cases should execute properly
-        o4, o1_4, o2_4_4 = f_ok()
-
-        # Check the sanity of the answers
-        assert np.allclose(o4, o1_4)
-        assert np.allclose(o4, o2_4_4[0])
-
-    def test_random_function_noshape_args(self):
-        # Test if random_function helper works with args but without shape
-        rng_R = random_state_type()
-
-        # No shape, default args -> OK
-        post_out, out = uniform(rng_R, size=None, ndim=2)
-        f = function(
-            [
-                In(
-                    rng_R,
-                    value=np.random.RandomState(utt.fetch_seed()),
-                    update=post_out,
-                    mutable=True,
-                )
-            ],
-            [out],
-            accept_inplace=True,
-        )
-        (o,) = f()
-
-        # No shape, args that have to be broadcasted -> OK
-        low = tensor.TensorType(dtype="float64", broadcastable=(False, True, True))()
-        high = tensor.TensorType(
-            dtype="float64", broadcastable=(True, True, True, False)
-        )()
-        post_out2, out2 = uniform(rng_R, size=None, ndim=2, low=low, high=high)
-        assert out2.ndim == 4
-        assert out2.broadcastable == (True, False, True, False)
-
-        g = function(
-            [
-                low,
-                high,
-                In(
-                    rng_R,
-                    value=np.random.RandomState(utt.fetch_seed()),
-                    update=post_out2,
-                    mutable=True,
-                ),
-            ],
-            [out2],
-            accept_inplace=True,
-        )
-        low_v = [[[3]], [[4]], [[-5]]]
-        high_v = [[[[5, 8]]]]
-        (o2,) = g(low_v, high_v)
-        assert o2.shape == (1, 3, 1, 2)
-
-    def test_random_function_noshape_noargs(self):
-        # Test if random_function helper works without args or shape
-        rng_R = random_state_type()
-
-        # No shape, no args -> TypeError
-        with pytest.raises(TypeError):
-            poisson(rng_R, size=None, ndim=2)
-
-    def test_random_function_ndim_added(self):
-        # Test that random_function helper function accepts ndim_added as
-        # keyword argument
-        # If using numpy's uniform distribution, ndim_added should be 0,
-        # because the shape provided as argument is the output shape.
-        # Specifying a different ndim_added will change the Op's output ndim,
-        # so np.uniform will produce a result of incorrect shape,
-        # and a ValueError should be raised.
-        def ndim_added_deco(ndim_added):
-            def randomfunction(random_state, size=(), low=0.0, high=0.0, ndim=None):
-                ndim, size, bcast = raw_random._infer_ndim_bcast(ndim, size)
-                if ndim_added < 0:
-                    bcast = bcast[:ndim_added]
-                else:
-                    bcast = bcast + ((False,) * ndim_added)
-                assert len(bcast) == ndim + ndim_added
-                op = RandomFunction(
-                    "uniform",
-                    tensor.TensorType(dtype="float64", broadcastable=bcast),
-                    ndim_added=ndim_added,
-                )
-                return op(random_state, size, low, high)
-
-            return randomfunction
-
-        uni_1 = ndim_added_deco(1)
-        uni_0 = ndim_added_deco(0)
-        uni_m1 = ndim_added_deco(-1)
-
-        rng_R = random_state_type()
-
-        p_uni11, uni11 = uni_1(rng_R, size=(4,))
-        p_uni12, uni12 = uni_1(rng_R, size=(3, 4))
-        p_uni01, uni01 = uni_0(rng_R, size=(4,))
-        p_uni02, uni02 = uni_0(rng_R, size=(3, 4))
-        p_unim11, unim11 = uni_m1(rng_R, size=(4,))
-        p_unim12, unim12 = uni_m1(rng_R, size=(3, 4))
-
-        assert uni11.ndim == 2
-        assert uni12.ndim == 3
-        assert uni01.ndim == 1
-        assert uni02.ndim == 2
-        assert unim11.ndim == 0
-        assert unim12.ndim == 1
-
-        f11 = function(
-            [
-                In(
-                    rng_R,
-                    value=np.random.RandomState(utt.fetch_seed()),
-                    update=p_uni11,
-                    mutable=True,
-                )
-            ],
-            [uni11],
-            accept_inplace=True,
-        )
-        f12 = function(
-            [
-                In(
-                    rng_R,
-                    value=np.random.RandomState(utt.fetch_seed()),
-                    update=p_uni12,
-                    mutable=True,
-                )
-            ],
-            [uni12],
-            accept_inplace=True,
-        )
-        fm11 = function(
-            [
-                In(
-                    rng_R,
-                    value=np.random.RandomState(utt.fetch_seed()),
-                    update=p_unim11,
-                    mutable=True,
-                )
-            ],
-            [unim11],
-            accept_inplace=True,
-        )
-        fm12 = function(
-            [
-                In(
-                    rng_R,
-                    value=np.random.RandomState(utt.fetch_seed()),
-                    update=p_unim12,
-                    mutable=True,
-                )
-            ],
-            [unim12],
-            accept_inplace=True,
-        )
-        f0 = function(
-            [
-                In(
-                    rng_R,
-                    value=np.random.RandomState(utt.fetch_seed()),
-                    update=p_uni02,
-                    mutable=True,
-                )
-            ],
-            [uni01, uni02],
-            accept_inplace=True,
-        )
-        with pytest.raises(ValueError):
-            f11()
-        with pytest.raises(ValueError):
-            f12()
-        with pytest.raises(ValueError):
-            fm11()
-        with pytest.raises(ValueError):
-            fm12()
-        u01, u02 = f0()
-        assert np.allclose(u01, u02[0])
-
-    def test_uniform(self):
-        # Test that raw_random.uniform generates the same results as numpy.
-        # Check over two calls to see if the random state is correctly updated.
-        rng_R = random_state_type()
-        # Use non-default parameters
-        post_r, out = uniform(rng_R, (4,), -2.0, 2.0)
-
-        f = function(
-            [
-                In(
-                    rng_R,
-                    value=np.random.RandomState(utt.fetch_seed()),
-                    update=post_r,
-                    mutable=True,
-                )
-            ],
-            [out],
-            accept_inplace=True,
-        )
-
-        numpy_rng = np.random.RandomState(utt.fetch_seed())
-        val0 = f()
-        val1 = f()
-        numpy_val0 = numpy_rng.uniform(-2.0, 2.0, size=(4,))
-        numpy_val1 = numpy_rng.uniform(-2.0, 2.0, size=(4,))
-        assert np.allclose(val0, numpy_val0)
-        assert np.allclose(val1, numpy_val1)
-
-    def test_binomial(self):
-        # Test that raw_random.binomial generates the same results as numpy.
-        # Check over two calls to see if the random state is correctly updated.
-        rng_R = random_state_type()
-        # Use non-default parameters, and larger dimensions because of
-        # the integer nature of the result
-        post_r, bin = binomial(rng_R, (7, 12), 5, 0.8)
-
-        f = function(
-            [
-                In(
-                    rng_R,
-                    value=np.random.RandomState(utt.fetch_seed()),
-                    update=post_r,
-                    mutable=True,
-                )
-            ],
-            [bin],
-            accept_inplace=True,
-        )
-
-        numpy_rng = np.random.RandomState(utt.fetch_seed())
-        val0 = f()
-        val1 = f()
-        numpy_val0 = numpy_rng.binomial(5, 0.8, size=(7, 12))
-        numpy_val1 = numpy_rng.binomial(5, 0.8, size=(7, 12))
-        assert np.all(val0 == numpy_val0)
-        assert np.all(val1 == numpy_val1)
-
-    def test_normal(self):
-        # Test that raw_random.normal generates the same results as numpy.
-        # Check over two calls to see if the random state is correctly updated.
-        rng_R = random_state_type()
-        # Use non-default parameters
-        post_r, out = normal(rng_R, (2, 3), 4.0, 2.0)
-
-        f = function(
-            [
-                In(
-                    rng_R,
-                    value=np.random.RandomState(utt.fetch_seed()),
-                    update=post_r,
-                    mutable=True,
-                )
-            ],
-            [out],
-            accept_inplace=True,
-        )
-
-        numpy_rng = np.random.RandomState(utt.fetch_seed())
-        val0 = f()
-        val1 = f()
-        numpy_val0 = numpy_rng.normal(4.0, 2.0, size=(2, 3))
-        numpy_val1 = numpy_rng.normal(4.0, 2.0, size=(2, 3))
-        assert np.allclose(val0, numpy_val0)
-        assert np.allclose(val1, numpy_val1)
-
-    def test_random_integers(self):
-        # Test that raw_random.random_integers generates the same
-        # results as numpy.  We use randint() for comparison since
-        # random_integers() is deprecated.
-
-        # Check over two calls to see if the random state is correctly updated.
-        rng_R = random_state_type()
-        # Use non-default parameters, and larger dimensions because of
-        # the integer nature of the result
-        post_r, out = random_integers(rng_R, (11, 8), -3, 16)
-
-        f = function(
-            [
-                In(
-                    rng_R,
-                    value=np.random.RandomState(utt.fetch_seed()),
-                    update=post_r,
-                    mutable=True,
-                )
-            ],
-            [out],
-            accept_inplace=True,
-        )
-
-        numpy_rng = np.random.RandomState(utt.fetch_seed())
-        val0 = f()
-        val1 = f()
-        numpy_val0 = numpy_rng.randint(-3, 17, size=(11, 8))
-        numpy_val1 = numpy_rng.randint(-3, 17, size=(11, 8))
-        assert np.allclose(val0, numpy_val0)
-        assert np.allclose(val1, numpy_val1)
-
-    def test_permutation_helper(self):
-        # Test that raw_random.permutation_helper generates the same
-        # results as numpy,
-        # and that the 'ndim_added' keyword behaves correctly.
-
-        # permutation_helper needs "ndim_added=1", because its output
-        # is one dimension more than its "shape" argument (and there's
-        # no way to determine that automatically).
-        # Check the working case, over two calls to see if the random
-        # state is correctly updated.
-        rf = RandomFunction(permutation_helper, tensor.imatrix, 8, ndim_added=1)
-        rng_R = random_state_type()
-        post_r, out = rf(rng_R, (7,), 8)
-
-        f = function(
-            [
-                In(
-                    rng_R,
-                    value=np.random.RandomState(utt.fetch_seed()),
-                    update=post_r,
-                    mutable=True,
-                )
-            ],
-            [out],
-            accept_inplace=True,
-        )
-
-        numpy_rng = np.random.RandomState(utt.fetch_seed())
-        val0 = f()
-        val1 = f()
-        # numpy_rng.permutation outputs one vector at a time,
-        # so we call it iteratively to generate all the samples.
-        numpy_val0 = np.asarray([numpy_rng.permutation(8) for i in range(7)])
-        numpy_val1 = np.asarray([numpy_rng.permutation(8) for i in range(7)])
-        assert np.all(val0 == numpy_val0)
-        assert np.all(val1 == numpy_val1)
-
-        # This call lacks "ndim_added=1", so ndim_added defaults to 0.
-        # A ValueError should be raised.
-        rf0 = RandomFunction(permutation_helper, tensor.imatrix, 8)
-        post_r0, out0 = rf0(rng_R, (7,), 8)
-        f0 = function(
-            [
-                In(
-                    rng_R,
-                    value=np.random.RandomState(utt.fetch_seed()),
-                    update=post_r0,
-                    mutable=True,
-                )
-            ],
-            [out0],
-            accept_inplace=True,
-        )
-        with pytest.raises(ValueError):
-            f0()
-
-        # Here, ndim_added is 2 instead of 1. A ValueError should be raised.
-        rf2 = RandomFunction(permutation_helper, tensor.imatrix, 8, ndim_added=2)
-        post_r2, out2 = rf2(rng_R, (7,), 8)
-        f2 = function(
-            [
-                In(
-                    rng_R,
-                    value=np.random.RandomState(utt.fetch_seed()),
-                    update=post_r2,
-                    mutable=True,
-                )
-            ],
-            [out2],
-            accept_inplace=True,
-        )
-        with pytest.raises(ValueError):
-            f2()
-
-    def test_choice(self):
-        # Test that raw_random.choice generates the same results as numpy.
-        # Check over two calls to see if the random state is correctly updated.
-        rng_R = random_state_type()
-        # Use non-default parameters, and larger dimensions because of
-        # the integer nature of the result
-        post_r, out = choice(rng_R, (11, 8), 10, 1, 0)
-
-        f = function(
-            [
-                In(
-                    rng_R,
-                    value=np.random.RandomState(utt.fetch_seed()),
-                    update=post_r,
-                    mutable=True,
-                )
-            ],
-            [out],
-            accept_inplace=True,
-        )
-
-        numpy_rng = np.random.RandomState(utt.fetch_seed())
-        val0 = f()
-        val1 = f()
-        numpy_val0 = numpy_rng.choice(10, (11, 8), True, None)
-        numpy_val1 = numpy_rng.choice(10, (11, 8), True, None)
-        assert np.allclose(val0, numpy_val0)
-        assert np.allclose(val1, numpy_val1)
-
-    def test_poisson(self):
-        # Test that raw_random.poisson generates the same results as numpy.
-        # Check over two calls to see if the random state is correctly updated.
-        rng_R = random_state_type()
-        # Use non-default parameters, and larger dimensions because of
-        # the integer nature of the result
-        post_r, out = poisson(rng_R, lam=5, size=(11, 8))
-
-        f = function(
-            [
-                In(
-                    rng_R,
-                    value=np.random.RandomState(utt.fetch_seed()),
-                    update=post_r,
-                    mutable=True,
-                )
-            ],
-            [out],
-            accept_inplace=True,
-        )
-
-        numpy_rng = np.random.RandomState(utt.fetch_seed())
-        val0 = f()
-        val1 = f()
-        numpy_val0 = numpy_rng.poisson(5, size=(11, 8))
-        numpy_val1 = numpy_rng.poisson(5, size=(11, 8))
-        assert np.allclose(val0, numpy_val0)
-        assert np.allclose(val1, numpy_val1)
-
-    def test_permutation(self):
-        # Test that raw_random.permutation generates the same results as numpy.
-        rng_R = random_state_type()
-        post_r, out = permutation(rng_R, size=(9,), n=6)
-        f = function(
-            [
-                In(
-                    rng_R,
-                    value=np.random.RandomState(utt.fetch_seed()),
-                    update=post_r,
-                    mutable=True,
-                )
-            ],
-            [out],
-            accept_inplace=True,
-        )
-
-        numpy_rng = np.random.RandomState(utt.fetch_seed())
-        # Check over two calls to see if the random state is correctly updated.
-        # numpy_rng.permutation outputs one vector at a time,
-        # so we call it iteratively to generate all the samples.
-        val0 = f()
-        val1 = f()
-        numpy_val0 = np.asarray([numpy_rng.permutation(6) for i in range(9)])
-        numpy_val1 = np.asarray([numpy_rng.permutation(6) for i in range(9)])
-        assert np.all(val0 == numpy_val0)
-        assert np.all(val1 == numpy_val1)
-
-        # Test that we can generate a list: have size=None or ().
-        for ndim in [1, None]:
-            post_r, out = permutation(rng_R, n=10, size=None, ndim=ndim)
-            inp = In(
-                rng_R,
-                value=np.random.RandomState(utt.fetch_seed()),
-                update=post_r,
-                mutable=True,
-            )
-            f = theano.function([inp], out)
-            o = f()
-            assert o.shape == (10,)
-            assert (np.sort(o) == np.arange(10)).all()
-        # Wrong number of dimensions asked
-        with pytest.raises(TypeError):
-            permutation(rng_R, size=None, ndim=2)
-
-    def test_multinomial(self):
-        # Test that raw_random.multinomial generates the same results as numpy.
-        # Check over two calls to see if the random state is correctly updated.
-        rng_R = random_state_type()
-        post_r, out = multinomial(rng_R, (7, 3), 6, [0.2] * 5)
-
-        f = function(
-            [
-                In(
-                    rng_R,
-                    value=np.random.RandomState(utt.fetch_seed()),
-                    update=post_r,
-                    mutable=True,
-                )
-            ],
-            [out],
-            accept_inplace=True,
-        )
-
-        numpy_rng = np.random.RandomState(utt.fetch_seed())
-        (val0,) = f()
-        (val1,) = f()
-        numpy_val0 = numpy_rng.multinomial(6, [0.2] * 5, (7, 3))
-        numpy_val1 = numpy_rng.multinomial(6, [0.2] * 5, (7, 3))
-        assert np.all(val0 == numpy_val0)
-        assert np.all(val1 == numpy_val1)
-
-        assert val0.shape == (7, 3, 5)
-        assert val1.shape == (7, 3, 5)
-
-    def test_symbolic_shape(self):
-        rng_R = random_state_type()
-        shape = tensor.lvector()
-        post_r, out = uniform(rng_R, shape, ndim=2)
-        f = function([rng_R, shape], out)
-        rng_state0 = np.random.RandomState(utt.fetch_seed())
-
-        assert f(rng_state0, [2, 3]).shape == (2, 3)
-        assert f(rng_state0, [4, 8]).shape == (4, 8)
-
-        with pytest.raises(ValueError):
-            f(rng_state0, [4])
-        with pytest.raises(ValueError):
-            f(rng_state0, [4, 3, 4, 5])
-
-    def test_mixed_shape(self):
-        # Test when the provided shape is a tuple of ints and scalar vars
-        rng_R = random_state_type()
-        shape0 = tensor.lscalar()
-        shape = (shape0, 3)
-        post_r, u = uniform(rng_R, size=shape, ndim=2)
-        f = function([rng_R, shape0], u)
-        rng_state0 = np.random.RandomState(utt.fetch_seed())
-
-        assert f(rng_state0, 2).shape == (2, 3)
-        assert f(rng_state0, 8).shape == (8, 3)
-
-        post_r, v = uniform(rng_R, size=shape)
-        g = function([rng_R, shape0], v)
-        assert g(rng_state0, 2).shape == (2, 3)
-        assert g(rng_state0, 8).shape == (8, 3)
-
-    def test_mixed_shape_bcastable(self):
-        # Test when the provided shape is a tuple of ints and scalar vars
-        rng_R = random_state_type()
-        shape0 = tensor.lscalar()
-        shape = (shape0, 1)
-        post_r, u = uniform(rng_R, size=shape, ndim=2)
-        assert u.broadcastable == (False, True)
-        f = function([rng_R, shape0], u)
-        rng_state0 = np.random.RandomState(utt.fetch_seed())
-
-        assert f(rng_state0, 2).shape == (2, 1)
-        assert f(rng_state0, 8).shape == (8, 1)
-
-        post_r, v = uniform(rng_R, size=shape)
-        assert v.broadcastable == (False, True)
-        g = function([rng_R, shape0], v)
-        assert g(rng_state0, 2).shape == (2, 1)
-        assert g(rng_state0, 8).shape == (8, 1)
-
-    def test_default_shape(self):
-        rng_R = random_state_type()
-        post_r, out = uniform(rng_R)
-        f = function([rng_R], [post_r, out], accept_inplace=True)
-
-        rng_state0 = np.random.RandomState(utt.fetch_seed())
-        numpy_rng = np.random.RandomState(utt.fetch_seed())
-        post0, val0 = f(rng_state0)
-        post1, val1 = f(post0)
-        numpy_val0 = np.asarray(numpy_rng.uniform(), dtype=theano.config.floatX)
-        numpy_val1 = np.asarray(numpy_rng.uniform(), dtype=theano.config.floatX)
-
-        assert np.all(val0 == numpy_val0)
-        assert np.all(val1 == numpy_val1)
-
-        post_r, out = multinomial(rng_R)
-        g = function([rng_R], [post_r, out], accept_inplace=True)
-        post2, val2 = g(post1)
-        numpy_val2 = np.asarray(
-            numpy_rng.multinomial(n=1, pvals=[0.5, 0.5]), dtype=theano.config.floatX
-        )
-
-        assert np.all(val2 == numpy_val2)
-
-    def test_vector_arguments(self):
-        rng_R = random_state_type()
-        low = tensor.vector()
-        post_r, out = uniform(rng_R, low=low, high=1)
-        assert out.ndim == 1
-        f = function([rng_R, low], [post_r, out], accept_inplace=True)
-
-        def as_floatX(thing):
-            return np.asarray(thing, dtype=theano.config.floatX)
-
-        rng_state0 = np.random.RandomState(utt.fetch_seed())
-        numpy_rng = np.random.RandomState(utt.fetch_seed())
-        post0, val0 = f(rng_state0, [-5, 0.5, 0, 1])
-        post1, val1 = f(post0, as_floatX([0.9]))
-        numpy_val0 = as_floatX(numpy_rng.uniform(low=[-5, 0.5, 0, 1], high=1))
-        numpy_val1 = as_floatX(numpy_rng.uniform(low=as_floatX([0.9]), high=1))
-
-        assert np.all(val0 == numpy_val0)
-        assert np.all(val1 == numpy_val1)
-
-        high = tensor.vector()
-        post_rb, outb = uniform(rng_R, low=low, high=high)
-        assert outb.ndim == 1
-        fb = function([rng_R, low, high], [post_rb, outb], accept_inplace=True)
-
-        post0b, val0b = fb(post1, [-4.0, -2], [-1, 0])
-        post1b, val1b = fb(post0b, [-4.0], [-1])
-        numpy_val0b = as_floatX(numpy_rng.uniform(low=[-4.0, -2], high=[-1, 0]))
-        numpy_val1b = as_floatX(numpy_rng.uniform(low=[-4.0], high=[-1]))
-        assert np.all(val0b == numpy_val0b)
-        assert np.all(val1b == numpy_val1b)
-        with pytest.raises(ValueError):
-            fb(post1b, [-4.0, -2], [-1, 0, 1])
-        # TODO: do we want that?
-        # with pytest.raises(ValueError):
-        #     fb(post1b, [-4., -2], [-1])
-
-        size = tensor.lvector()
-        post_rc, outc = uniform(rng_R, low=low, high=high, size=size, ndim=1)
-        fc = function([rng_R, low, high, size], [post_rc, outc], accept_inplace=True)
-        post0c, val0c = fc(post1b, [-4.0, -2], [-1, 0], [2])
-        post1c, val1c = fc(post0c, [-4.0], [-1], [1])
-        numpy_val0c = as_floatX(numpy_rng.uniform(low=[-4.0, -2], high=[-1, 0]))
-        numpy_val1c = as_floatX(numpy_rng.uniform(low=[-4.0], high=[-1]))
-        assert np.all(val0c == numpy_val0c)
-        assert np.all(val1c == numpy_val1c)
-
-        with pytest.raises(ValueError):
-            fc(post1c, [-4.0, -2], [-1, 0], [1, 2])
-        with pytest.raises(ValueError):
-            fc(post1c, [-4.0, -2], [-1, 0], [2, 1])
-        with pytest.raises(ValueError):
-            fc(post1c, [-4.0, -2], [-1, 0], [1])
-        with pytest.raises(ValueError):
-            fc(post1c, [-4.0, -2], [-1], [1])
-
-        # TODO: do we want that?
-        # with pytest.raises(ValueError):
-        #     fc(post1c, [-4., -2], [-1], [2])
-
-    def test_broadcast_arguments(self):
-        rng_R = random_state_type()
-        low = tensor.dvector()
-        high = tensor.dcol()
-        post_r, out = uniform(rng_R, low=low, high=high)
-        assert out.ndim == 2
-        f = function([rng_R, low, high], [post_r, out], accept_inplace=True)
-
-        rng_state0 = np.random.RandomState(utt.fetch_seed())
-        numpy_rng = np.random.RandomState(utt.fetch_seed())
-        post0, val0 = f(rng_state0, [-5, 0.5, 0, 1], [[1.0]])
-        post1, val1 = f(post0, [0.9], [[1.0], [1.1], [1.5]])
-        post2, val2 = f(post1, [-5, 0.5, 0, 1], [[1.0], [1.1], [1.5]])
-
-        numpy_val0 = numpy_rng.uniform(low=[-5, 0.5, 0, 1], high=[1.0])
-        numpy_val1 = numpy_rng.uniform(low=[0.9], high=[[1.0], [1.1], [1.5]])
-        numpy_val2 = numpy_rng.uniform(low=[-5, 0.5, 0, 1], high=[[1.0], [1.1], [1.5]])
-
-        assert np.all(val0 == numpy_val0), (val0, numpy_val0)
-        assert np.all(val1 == numpy_val1)
-        assert np.all(val2 == numpy_val2)
-
-    def test_uniform_vector(self):
-        rng_R = random_state_type()
-        low = tensor.vector()
-        high = tensor.vector()
-        post_r, out = uniform(rng_R, low=low, high=high)
-        assert out.ndim == 1
-        f = function([rng_R, low, high], [post_r, out], accept_inplace=True)
-
-        def as_floatX(thing):
-            return np.asarray(thing, dtype=theano.config.floatX)
-
-        low_val = as_floatX([0.1, 0.2, 0.3])
-        high_val = as_floatX([1.1, 2.2, 3.3])
-        rng = np.random.RandomState(utt.fetch_seed())
-        numpy_rng = np.random.RandomState(utt.fetch_seed())
-
-        # Arguments of size (3,)
-        rng0, val0 = f(rng, low_val, high_val)
-        numpy_val0 = as_floatX(numpy_rng.uniform(low=low_val, high=high_val))
-        assert np.all(val0 == numpy_val0)
-
-        # arguments of size (2,)
-        rng1, val1 = f(rng0, low_val[:-1], high_val[:-1])
-        numpy_val1 = as_floatX(numpy_rng.uniform(low=low_val[:-1], high=high_val[:-1]))
-        assert np.all(val1 == numpy_val1)
-
-        # Specifying the size explicitly
-        g = function(
-            [rng_R, low, high],
-            uniform(rng_R, low=low, high=high, size=(3,)),
-            accept_inplace=True,
-        )
-        rng2, val2 = g(rng1, low_val, high_val)
-        numpy_val2 = as_floatX(numpy_rng.uniform(low=low_val, high=high_val, size=(3,)))
-        assert np.all(val2 == numpy_val2)
-        with pytest.raises(ValueError):
-            g(rng2, low_val[:-1], high_val[:-1])
-
-    def test_binomial_vector(self):
-        rng_R = random_state_type()
-        n = tensor.lvector()
-        prob = tensor.vector()
-        post_r, out = binomial(rng_R, n=n, p=prob)
-        assert out.ndim == 1
-        f = function([rng_R, n, prob], [post_r, out], accept_inplace=True)
-
-        n_val = [1, 2, 3]
-        prob_val = np.asarray([0.1, 0.2, 0.3], dtype=config.floatX)
-        rng = np.random.RandomState(utt.fetch_seed())
-        numpy_rng = np.random.RandomState(utt.fetch_seed())
-
-        # Arguments of size (3,)
-        rng0, val0 = f(rng, n_val, prob_val)
-        numpy_val0 = numpy_rng.binomial(n=n_val, p=prob_val)
-        assert np.all(val0 == numpy_val0)
-
-        # arguments of size (2,)
-        rng1, val1 = f(rng0, n_val[:-1], prob_val[:-1])
-        numpy_val1 = numpy_rng.binomial(n=n_val[:-1], p=prob_val[:-1])
-        assert np.all(val1 == numpy_val1)
-
-        # Specifying the size explicitly
-        g = function(
-            [rng_R, n, prob],
-            binomial(rng_R, n=n, p=prob, size=(3,)),
-            accept_inplace=True,
-        )
-        rng2, val2 = g(rng1, n_val, prob_val)
-        numpy_val2 = numpy_rng.binomial(n=n_val, p=prob_val, size=(3,))
-        assert np.all(val2 == numpy_val2)
-        with pytest.raises(ValueError):
-            g(rng2, n_val[:-1], prob_val[:-1])
-
-    def test_normal_vector(self):
-        rng_R = random_state_type()
-        avg = tensor.vector()
-        std = tensor.vector()
-        post_r, out = normal(rng_R, avg=avg, std=std)
-        assert out.ndim == 1
-        f = function([rng_R, avg, std], [post_r, out], accept_inplace=True)
-
-        def as_floatX(thing):
-            return np.asarray(thing, dtype=theano.config.floatX)
-
-        avg_val = [1, 2, 3]
-        std_val = as_floatX([0.1, 0.2, 0.3])
-        rng = np.random.RandomState(utt.fetch_seed())
-        numpy_rng = np.random.RandomState(utt.fetch_seed())
-
-        # Arguments of size (3,)
-        rng0, val0 = f(rng, avg_val, std_val)
-        numpy_val0 = as_floatX(
-            numpy_rng.normal(loc=as_floatX(avg_val), scale=as_floatX(std_val))
-        )
-        assert np.all(val0 == numpy_val0)
-
-        # arguments of size (2,)
-        rng1, val1 = f(rng0, avg_val[:-1], std_val[:-1])
-        numpy_val1 = np.asarray(
-            numpy_rng.normal(loc=avg_val[:-1], scale=std_val[:-1]),
-            dtype=theano.config.floatX,
-        )
-        assert np.all(val1 == numpy_val1)
-
-        # Specifying the size explicitly
-        g = function(
-            [rng_R, avg, std],
-            normal(rng_R, avg=avg, std=std, size=(3,)),
-            accept_inplace=True,
-        )
-        rng2, val2 = g(rng1, avg_val, std_val)
-        numpy_val2 = np.asarray(
-            numpy_rng.normal(loc=avg_val, scale=std_val, size=(3,)),
-            dtype=theano.config.floatX,
-        )
-        assert np.all(val2 == numpy_val2)
-        with pytest.raises(ValueError):
-            g(rng2, avg_val[:-1], std_val[:-1])
-
-    def test_random_integers_vector(self):
-        rng_R = random_state_type()
-        low = tensor.lvector()
-        high = tensor.lvector()
-        post_r, out = random_integers(rng_R, low=low, high=high)
-        assert out.ndim == 1
-        f = function([rng_R, low, high], [post_r, out], accept_inplace=True)
-
-        low_val = [100, 200, 300]
-        high_val = [110, 220, 330]
-        rng = np.random.RandomState(utt.fetch_seed())
-        numpy_rng = np.random.RandomState(utt.fetch_seed())
-
-        # Arguments of size (3,)
-        rng0, val0 = f(rng, low_val, high_val)
-        numpy_val0 = np.asarray(
-            [
-                numpy_rng.randint(low=lv, high=hv + 1)
-                for lv, hv in zip(low_val, high_val)
-            ]
-        )
-        assert np.all(val0 == numpy_val0)
-
-        # arguments of size (2,)
-        rng1, val1 = f(rng0, low_val[:-1], high_val[:-1])
-        numpy_val1 = np.asarray(
-            [
-                numpy_rng.randint(low=lv, high=hv + 1)
-                for lv, hv in zip(low_val[:-1], high_val[:-1])
-            ]
-        )
-        assert np.all(val1 == numpy_val1)
-
-        # Specifying the size explicitly
-        g = function(
-            [rng_R, low, high],
-            random_integers(rng_R, low=low, high=high, size=(3,)),
-            accept_inplace=True,
-        )
-        rng2, val2 = g(rng1, low_val, high_val)
-        numpy_val2 = np.asarray(
-            [
-                numpy_rng.randint(low=lv, high=hv + 1)
-                for lv, hv in zip(low_val, high_val)
-            ]
-        )
-        assert np.all(val2 == numpy_val2)
-        with pytest.raises(ValueError):
-            g(rng2, low_val[:-1], high_val[:-1])
-
-    # Vectorized permutation don't make sense: the only parameter, n,
-    # controls one dimension of the returned tensor.
-
-    def test_multinomial_vector(self):
-        rng_R = random_state_type()
-        n = tensor.lvector()
-        pvals = tensor.matrix()
-        post_r, out = multinomial(rng_R, n=n, pvals=pvals)
-        assert out.ndim == 2
-        f = function([rng_R, n, pvals], [post_r, out], accept_inplace=True)
-
-        n_val = [1, 2, 3]
-        pvals_val = [[0.1, 0.9], [0.2, 0.8], [0.3, 0.7]]
-        pvals_val = np.asarray(pvals_val, dtype=config.floatX)
-        rng = np.random.RandomState(utt.fetch_seed())
-        numpy_rng = np.random.RandomState(utt.fetch_seed())
-
-        # Arguments of size (3,)
-        rng0, val0 = f(rng, n_val, pvals_val)
-        numpy_val0 = np.asarray(
-            [numpy_rng.multinomial(n=nv, pvals=pv) for nv, pv in zip(n_val, pvals_val)]
-        )
-        assert np.all(val0 == numpy_val0)
-
-        # arguments of size (2,)
-        rng1, val1 = f(rng0, n_val[:-1], pvals_val[:-1])
-        numpy_val1 = np.asarray(
-            [
-                numpy_rng.multinomial(n=nv, pvals=pv)
-                for nv, pv in zip(n_val[:-1], pvals_val[:-1])
-            ]
-        )
-        assert np.all(val1 == numpy_val1)
-
-        # Specifying the size explicitly
-        g = function(
-            [rng_R, n, pvals],
-            multinomial(rng_R, n=n, pvals=pvals, size=(3,)),
-            accept_inplace=True,
-        )
-        rng2, val2 = g(rng1, n_val, pvals_val)
-        numpy_val2 = np.asarray(
-            [numpy_rng.multinomial(n=nv, pvals=pv) for nv, pv in zip(n_val, pvals_val)]
-        )
-        assert np.all(val2 == numpy_val2)
-        with pytest.raises(ValueError):
-            g(rng2, n_val[:-1], pvals_val[:-1])
-
-    def test_multinomial_tensor3_a(self):
-        # Test the examples given in the multinomial documentation regarding
-        # tensor3 objects
-        rng_R = random_state_type()
-        n = 9
-        pvals = tensor.dtensor3()
-        post_r, out = multinomial(rng_R, n=n, pvals=pvals, size=(1, -1))
-        assert out.ndim == 3
-        assert out.broadcastable == (True, False, False)
-
-        f = function([rng_R, pvals], [post_r, out], accept_inplace=True)
-
-        rng = np.random.RandomState(utt.fetch_seed())
-
-        pvals_val = np.asarray([[[0.1, 0.9], [0.2, 0.8], [0.3, 0.7]]])
-        assert pvals_val.shape == (1, 3, 2)
-
-        new_rng, draw = f(rng, pvals_val)
-        assert draw.shape == (1, 3, 2)
-        assert np.allclose(draw.sum(axis=2), 9)
-
-    def test_multinomial_tensor3_b(self):
-        # Test the examples given in the multinomial documentation regarding
-        # tensor3 objects
-        rng_R = random_state_type()
-        n = 9
-        pvals = tensor.dtensor3()
-        post_r, out = multinomial(rng_R, n=n, pvals=pvals, size=(10, 1, -1))
-        assert out.ndim == 4
-        assert out.broadcastable == (False, True, False, False)
-
-        f = function([rng_R, pvals], [post_r, out], accept_inplace=True)
-
-        rng = np.random.RandomState(utt.fetch_seed())
-
-        pvals_val = np.asarray([[[0.1, 0.9], [0.2, 0.8], [0.3, 0.7]]])
-        assert pvals_val.shape == (1, 3, 2)
-
-        out_rng, draw = f(rng, pvals_val)
-        assert draw.shape == (10, 1, 3, 2)
-        assert np.allclose(draw.sum(axis=3), 9)
-
-    def test_dtype(self):
-        rng_R = random_state_type()
-        low = tensor.lscalar()
-        high = tensor.lscalar()
-        post_r, out = random_integers(
-            rng_R, low=low, high=high, size=(20,), dtype="int8"
-        )
-        assert out.dtype == "int8"
-        f = function([rng_R, low, high], [post_r, out])
-
-        rng = np.random.RandomState(utt.fetch_seed())
-        rng0, val0 = f(rng, 0, 9)
-        assert val0.dtype == "int8"
-
-        rng1, val1 = f(rng0, 255, 257)
-        assert val1.dtype == "int8"
-        assert np.all(abs(val1) <= 1)
-
-    def test_dtype_normal_uniform_687(self):
-        # Regression test for #687.
-        rng_R = random_state_type()
-        assert (
-            uniform(rng_R, low=tensor.constant(0, dtype="float64"), dtype="float32")[
-                1
-            ].dtype
-            == "float32"
-        )
-
-        assert (
-            normal(rng_R, avg=tensor.constant(0, dtype="float64"), dtype="float32")[
-                1
-            ].dtype
-            == "float32"
-        )
-
-    def test_infer_shape(self):
-        rng_R = random_state_type()
-        rng_R_val = np.random.RandomState(utt.fetch_seed())
-
-        # no shape specified, default args
-        post_r, out = uniform(rng_R)
-        self._compile_and_check([rng_R], [out], [rng_R_val], RandomFunction)
-
-        post_r, out = uniform(rng_R, size=None, ndim=2)
-        self._compile_and_check([rng_R], [out], [rng_R_val], RandomFunction)
-
-        """
-        #infer_shape don't work for multinomial.
-        #The parameter ndim_added is set to 1 and in this case, the infer_shape
-        #inplementation don't know how to infer the shape
-        post_r, out = multinomial(rng_R)
-
-        self._compile_and_check([rng_R], [out], [rng_R_val],
-                                RandomFunction)
-        """
-
-        # no shape specified, args have to be broadcasted
-        low = tensor.TensorType(dtype="float64", broadcastable=(False, True, True))()
-        high = tensor.TensorType(
-            dtype="float64", broadcastable=(True, True, True, False)
-        )()
-        post_r, out = uniform(rng_R, size=None, ndim=2, low=low, high=high)
-        low_val = [[[3]], [[4]], [[-5]]]
-        high_val = [[[[5, 8]]]]
-        self._compile_and_check(
-            [rng_R, low, high], [out], [rng_R_val, low_val, high_val], RandomFunction
-        )
-
-        # multinomial, specified shape
-        """
-        #infer_shape don't work for multinomial
-        n = iscalar()
-        pvals = dvector()
-        size_val = (7, 3)
-        n_val = 6
-        pvals_val = [0.2] * 5
-        post_r, out = multinomial(rng_R, size=size_val, n=n, pvals=pvals,
-                                  ndim=2)
-
-        self._compile_and_check([rng_R, n, pvals], [out],
-                                [rng_R_val, n_val, pvals_val],
-                                RandomFunction)
-        """
-
-        # uniform vector low and high
-        low = dvector()
-        high = dvector()
-        post_r, out = uniform(rng_R, low=low, high=1)
-        low_val = [-5, 0.5, 0, 1]
-        self._compile_and_check(
-            [rng_R, low], [out], [rng_R_val, low_val], RandomFunction
-        )
-
-        low_val = [0.9]
-        self._compile_and_check(
-            [rng_R, low], [out], [rng_R_val, low_val], RandomFunction
-        )
-
-        post_r, out = uniform(rng_R, low=low, high=high)
-        low_val = [-4.0, -2]
-        high_val = [-1, 0]
-        self._compile_and_check(
-            [rng_R, low, high], [out], [rng_R_val, low_val, high_val], RandomFunction
-        )
-
-        low_val = [-4.0]
-        high_val = [-1]
-        self._compile_and_check(
-            [rng_R, low, high], [out], [rng_R_val, low_val, high_val], RandomFunction
-        )
-
-        # uniform broadcasting low and high
-        low = dvector()
-        high = dcol()
-        post_r, out = uniform(rng_R, low=low, high=high)
-        low_val = [-5, 0.5, 0, 1]
-        high_val = [[1.0]]
-        self._compile_and_check(
-            [rng_R, low, high], [out], [rng_R_val, low_val, high_val], RandomFunction
-        )
-
-        low_val = [0.9]
-        high_val = [[1.0], [1.1], [1.5]]
-        self._compile_and_check(
-            [rng_R, low, high], [out], [rng_R_val, low_val, high_val], RandomFunction
-        )
-
-        low_val = [-5, 0.5, 0, 1]
-        high_val = [[1.0], [1.1], [1.5]]
-        self._compile_and_check(
-            [rng_R, low, high], [out], [rng_R_val, low_val, high_val], RandomFunction
-        )
-
-        # uniform with vector slice
-        low = dvector()
-        high = dvector()
-        post_r, out = uniform(rng_R, low=low, high=high)
-        low_val = [0.1, 0.2, 0.3]
-        high_val = [1.1, 2.2, 3.3]
-        size_val = (3,)
-        self._compile_and_check(
-            [rng_R, low, high],
-            [out],
-            [rng_R_val, low_val[:-1], high_val[:-1]],
-            RandomFunction,
-        )
-
-        # uniform with explicit size and size implicit in parameters
-        # NOTE 1: Would it be desirable that size could also be supplied
-        # as a Theano variable?
-        post_r, out = uniform(rng_R, size=size_val, low=low, high=high)
-        self._compile_and_check(
-            [rng_R, low, high], [out], [rng_R_val, low_val, high_val], RandomFunction
-        )
-
-        # binomial with vector slice
-        n = ivector()
-        prob = dvector()
-        post_r, out = binomial(rng_R, n=n, p=prob)
-        n_val = [1, 2, 3]
-        prob_val = [0.1, 0.2, 0.3]
-        size_val = (3,)
-        self._compile_and_check(
-            [rng_R, n, prob],
-            [out],
-            [rng_R_val, n_val[:-1], prob_val[:-1]],
-            RandomFunction,
-        )
-
-        # binomial with explicit size and size implicit in parameters
-        # cf. NOTE 1
-        post_r, out = binomial(rng_R, n=n, p=prob, size=size_val)
-        self._compile_and_check(
-            [rng_R, n, prob], [out], [rng_R_val, n_val, prob_val], RandomFunction
-        )
-
-        # normal with vector slice
-        avg = dvector()
-        std = dvector()
-        post_r, out = normal(rng_R, avg=avg, std=std)
-        avg_val = [1, 2, 3]
-        std_val = [0.1, 0.2, 0.3]
-        size_val = (3,)
-        self._compile_and_check(
-            [rng_R, avg, std],
-            [out],
-            [rng_R_val, avg_val[:-1], std_val[:-1]],
-            RandomFunction,
-        )
-
-        # normal with explicit size and size implicit in parameters
-        # cf. NOTE 1
-        post_r, out = normal(rng_R, avg=avg, std=std, size=size_val)
-        self._compile_and_check(
-            [rng_R, avg, std], [out], [rng_R_val, avg_val, std_val], RandomFunction
-        )
-
-        # multinomial with tensor-3 probabilities
-        """
-        #multinomial infer_shape don't work.
-        pvals = dtensor3()
-        n = iscalar()
-        post_r, out = multinomial(rng_R, n=n, pvals=pvals, size=(1, -1))
-        pvals_val = [[[.1, .9], [.2, .8], [.3, .7]]]
-        n_val = 9
-
-        self._compile_and_check([rng_R, n, pvals], [out],
-                                [rng_R_val, n_val,
-                                pvals_val], RandomFunction)
-
-        post_r, out = multinomial(rng_R, n=n, pvals=pvals, size=(10, 1, -1))
-
-        self._compile_and_check([rng_R, n, pvals], [out],
-                                [rng_R_val, n_val,
-                                pvals_val], RandomFunction)
-        """
-
-    def test_pkl(self):
-        # Test pickling of RandomFunction.
-        # binomial was created by calling RandomFunction on a string,
-        # random_integers by calling it on a function.
-        rng_r = random_state_type()
-        mode = None
-        if theano.config.mode in ["DEBUG_MODE", "DebugMode"]:
-            mode = "FAST_COMPILE"
-        post_bin_r, bin_sample = binomial(rng_r, (3, 5), 1, 0.3)
-        f = theano.function([rng_r], [post_bin_r, bin_sample], mode=mode)
-        pickle.dumps(f)
-
-        post_int_r, int_sample = random_integers(rng_r, (3, 5), -1, 8)
-        g = theano.function([rng_r], [post_int_r, int_sample], mode=mode)
-        pkl_g = pickle.dumps(g)
-        pickle.loads(pkl_g)

tests/tensor/test_shared_randomstreams.py

-import numpy as np
-import pytest
-
-from tests import unittest_tools as utt
-from theano import config, function, shared, tensor
-from theano.tensor.shared_randomstreams import RandomStreams
-
-
-class TestSharedRandomStreams:
-    def setup_method(self):
-        utt.seed_rng()
-
-    def test_tutorial(self):
-        srng = RandomStreams(seed=234)
-        rv_u = srng.uniform((2, 2))
-        rv_n = srng.normal((2, 2))
-        f = function([], rv_u)
-        g = function([], rv_n, no_default_updates=True)  # Not updating rv_n.rng
-        nearly_zeros = function([], rv_u + rv_u - 2 * rv_u)
-
-        assert np.all(f() != f())
-        assert np.all(g() == g())
-        assert np.all(abs(nearly_zeros()) < 1e-5)
-        assert isinstance(rv_u.rng.get_value(borrow=True), np.random.RandomState)
-
-    def test_basics(self):
-        random = RandomStreams(utt.fetch_seed())
-        fn = function([], random.uniform((2, 2)), updates=random.updates())
-        # gn = function([], random.normal((2, 2)), updates=random.updates())
-
-        fn_val0 = fn()
-        fn_val1 = fn()
-        # gn_val0 = gn()
-
-        rng_seed = np.random.RandomState(utt.fetch_seed()).randint(2 ** 30)
-        rng = np.random.RandomState(int(rng_seed))  # int() is for 32bit
-
-        numpy_val0 = rng.uniform(size=(2, 2))
-        numpy_val1 = rng.uniform(size=(2, 2))
-
-        assert np.allclose(fn_val0, numpy_val0)
-        assert np.allclose(fn_val1, numpy_val1)
-
-    def test_seed_fn(self):
-        random = RandomStreams(234)
-        fn = function([], random.uniform((2, 2)), updates=random.updates())
-
-        random.seed(utt.fetch_seed())
-
-        fn_val0 = fn()
-        fn_val1 = fn()
-
-        rng_seed = np.random.RandomState(utt.fetch_seed()).randint(2 ** 30)
-        rng = np.random.RandomState(int(rng_seed))  # int() is for 32bit
-
-        numpy_val0 = rng.uniform(size=(2, 2))
-        numpy_val1 = rng.uniform(size=(2, 2))
-
-        assert np.allclose(fn_val0, numpy_val0)
-        assert np.allclose(fn_val1, numpy_val1)
-
-    def test_getitem(self):
-
-        random = RandomStreams(234)
-        out = random.uniform((2, 2))
-        fn = function([], out, updates=random.updates())
-
-        random.seed(utt.fetch_seed())
-
-        rng = np.random.RandomState()
-        rng.set_state(random[out.rng].get_state())  # tests getitem
-
-        fn_val0 = fn()
-        fn_val1 = fn()
-        numpy_val0 = rng.uniform(size=(2, 2))
-        numpy_val1 = rng.uniform(size=(2, 2))
-        assert np.allclose(fn_val0, numpy_val0)
-        assert np.allclose(fn_val1, numpy_val1)
-
-    def test_setitem(self):
-
-        random = RandomStreams(234)
-        out = random.uniform((2, 2))
-        fn = function([], out, updates=random.updates())
-
-        random.seed(888)
-
-        rng = np.random.RandomState(utt.fetch_seed())
-        random[out.rng] = np.random.RandomState(utt.fetch_seed())
-
-        fn_val0 = fn()
-        fn_val1 = fn()
-        numpy_val0 = rng.uniform(size=(2, 2))
-        numpy_val1 = rng.uniform(size=(2, 2))
-        assert np.allclose(fn_val0, numpy_val0)
-        assert np.allclose(fn_val1, numpy_val1)
-
-    def test_ndim(self):
-        # Test that the behaviour of 'ndim' optional parameter
-        # 'ndim' is an optional integer parameter, specifying the length
-        # of the 'shape', passed as a keyword argument.
-
-        # ndim not specified, OK
-        random = RandomStreams(utt.fetch_seed())
-        fn = function([], random.uniform((2, 2)))
-
-        # ndim specified, consistent with shape, OK
-        random2 = RandomStreams(utt.fetch_seed())
-        fn2 = function([], random2.uniform((2, 2), ndim=2))
-
-        val1 = fn()
-        val2 = fn2()
-        assert np.all(val1 == val2)
-
-        # ndim specified, inconsistent with shape, should raise ValueError
-        random3 = RandomStreams(utt.fetch_seed())
-        with pytest.raises(ValueError):
-            random3.uniform((2, 2), ndim=1)
-
-    def test_uniform(self):
-        # Test that RandomStreams.uniform generates the same results as numpy
-        # Check over two calls to see if the random state is correctly updated.
-        random = RandomStreams(utt.fetch_seed())
-        fn = function([], random.uniform((2, 2), -1, 1))
-        fn_val0 = fn()
-        fn_val1 = fn()
-
-        rng_seed = np.random.RandomState(utt.fetch_seed()).randint(2 ** 30)
-        rng = np.random.RandomState(int(rng_seed))  # int() is for 32bit
-        numpy_val0 = rng.uniform(-1, 1, size=(2, 2))
-        numpy_val1 = rng.uniform(-1, 1, size=(2, 2))
-
-        assert np.allclose(fn_val0, numpy_val0)
-        assert np.allclose(fn_val1, numpy_val1)
-
-    def test_normal(self):
-        # Test that RandomStreams.normal generates the same results as numpy
-        # Check over two calls to see if the random state is correctly updated.
-
-        random = RandomStreams(utt.fetch_seed())
-        fn = function([], random.normal((2, 2), -1, 2))
-        fn_val0 = fn()
-        fn_val1 = fn()
-
-        rng_seed = np.random.RandomState(utt.fetch_seed()).randint(2 ** 30)
-        rng = np.random.RandomState(int(rng_seed))  # int() is for 32bit
-        numpy_val0 = rng.normal(-1, 2, size=(2, 2))
-        numpy_val1 = rng.normal(-1, 2, size=(2, 2))
-
-        assert np.allclose(fn_val0, numpy_val0)
-        assert np.allclose(fn_val1, numpy_val1)
-
-    def test_random_integers(self):
-        # Test that RandomStreams.random_integers generates the same
-        # results as numpy.  We use randint() for numpy since
-        # random_integers() is deprecated.
-
-        # Check over two calls to see if the random state is correctly updated.
-        random = RandomStreams(utt.fetch_seed())
-        fn = function([], random.random_integers((20, 20), -5, 5))
-        fn_val0 = fn()
-        fn_val1 = fn()
-
-        rng_seed = np.random.RandomState(utt.fetch_seed()).randint(2 ** 30)
-        rng = np.random.RandomState(int(rng_seed))  # int() is for 32bit
-        numpy_val0 = rng.randint(-5, 6, size=(20, 20))
-        numpy_val1 = rng.randint(-5, 6, size=(20, 20))
-
-        assert np.all(fn_val0 == numpy_val0)
-        assert np.all(fn_val1 == numpy_val1)
-
-    def test_choice(self):
-        # Test that RandomStreams.choice generates the same results as numpy
-        # Check over two calls to see if the random state is correctly updated.
-        random = RandomStreams(utt.fetch_seed())
-        fn = function([], random.choice((11, 8), 10, 1, 0))
-        fn_val0 = fn()
-        fn_val1 = fn()
-
-        rng_seed = np.random.RandomState(utt.fetch_seed()).randint(2 ** 30)
-        rng = np.random.RandomState(int(rng_seed))  # int() is for 32bit
-        numpy_val0 = rng.choice(10, (11, 8), True, None)
-        numpy_val1 = rng.choice(10, (11, 8), True, None)
-
-        assert np.all(fn_val0 == numpy_val0)
-        assert np.all(fn_val1 == numpy_val1)
-
-    def test_poisson(self):
-        # Test that RandomStreams.poisson generates the same results as numpy
-
-        # Check over two calls to see if the random state is correctly updated.
-        random = RandomStreams(utt.fetch_seed())
-        fn = function([], random.poisson(lam=5, size=(11, 8)))
-        fn_val0 = fn()
-        fn_val1 = fn()
-
-        rng_seed = np.random.RandomState(utt.fetch_seed()).randint(2 ** 30)
-        rng = np.random.RandomState(int(rng_seed))  # int() is for 32bit
-        numpy_val0 = rng.poisson(lam=5, size=(11, 8))
-        numpy_val1 = rng.poisson(lam=5, size=(11, 8))
-
-        assert np.all(fn_val0 == numpy_val0)
-        assert np.all(fn_val1 == numpy_val1)
-
-    def test_permutation(self):
-        # Test that RandomStreams.permutation generates the same results as numpy
-        # Check over two calls to see if the random state is correctly updated.
-        random = RandomStreams(utt.fetch_seed())
-        fn = function([], random.permutation((20,), 10), updates=random.updates())
-
-        fn_val0 = fn()
-        fn_val1 = fn()
-
-        rng_seed = np.random.RandomState(utt.fetch_seed()).randint(2 ** 30)
-        rng = np.random.RandomState(int(rng_seed))  # int() is for 32bit
-
-        # rng.permutation outputs one vector at a time, so we iterate.
-        numpy_val0 = np.asarray([rng.permutation(10) for i in range(20)])
-        numpy_val1 = np.asarray([rng.permutation(10) for i in range(20)])
-
-        assert np.all(fn_val0 == numpy_val0)
-        assert np.all(fn_val1 == numpy_val1)
-
-    def test_multinomial(self):
-        # Test that RandomStreams.multinomial generates the same results as numpy
-        # Check over two calls to see if the random state is correctly updated.
-        random = RandomStreams(utt.fetch_seed())
-        fn = function(
-            [], random.multinomial((4, 4), 1, [0.1] * 10), updates=random.updates()
-        )
-
-        fn_val0 = fn()
-        fn_val1 = fn()
-
-        rng_seed = np.random.RandomState(utt.fetch_seed()).randint(2 ** 30)
-        rng = np.random.RandomState(int(rng_seed))  # int() is for 32bit
-        numpy_val0 = rng.multinomial(1, [0.1] * 10, size=(4, 4))
-        numpy_val1 = rng.multinomial(1, [0.1] * 10, size=(4, 4))
-
-        assert np.all(fn_val0 == numpy_val0)
-        assert np.all(fn_val1 == numpy_val1)
-
-    def test_shuffle_row_elements(self):
-        # Test that RandomStreams.shuffle_row_elements generates the right results
-        # Check over two calls to see if the random state is correctly updated.
-
-        # On matrices, for each row, the elements of that row should be shuffled.
-        # Note that this differs from np.random.shuffle, where all the elements
-        # of the matrix are shuffled.
-        random = RandomStreams(utt.fetch_seed())
-        m_input = tensor.dmatrix()
-        f = function(
-            [m_input], random.shuffle_row_elements(m_input), updates=random.updates()
-        )
-
-        # Generate the elements to be shuffled
-        val_rng = np.random.RandomState(utt.fetch_seed() + 42)
-        in_mval = val_rng.uniform(-2, 2, size=(20, 5))
-        fn_mval0 = f(in_mval)
-        fn_mval1 = f(in_mval)
-        assert not np.all(in_mval == fn_mval0)
-        assert not np.all(in_mval == fn_mval1)
-        assert not np.all(fn_mval0 == fn_mval1)
-
-        rng_seed = np.random.RandomState(utt.fetch_seed()).randint(2 ** 30)
-        rng = np.random.RandomState(int(rng_seed))
-        numpy_mval0 = in_mval.copy()
-        numpy_mval1 = in_mval.copy()
-        for row in numpy_mval0:
-            rng.shuffle(row)
-        for row in numpy_mval1:
-            rng.shuffle(row)
-
-        assert np.all(numpy_mval0 == fn_mval0)
-        assert np.all(numpy_mval1 == fn_mval1)
-
-        # On vectors, the behaviour is the same as np.random.shuffle,
-        # except that it does not work in place, but returns a shuffled vector.
-        random1 = RandomStreams(utt.fetch_seed())
-        v_input = tensor.dvector()
-        f1 = function([v_input], random1.shuffle_row_elements(v_input))
-
-        in_vval = val_rng.uniform(-3, 3, size=(12,))
-        fn_vval = f1(in_vval)
-        numpy_vval = in_vval.copy()
-        vrng = np.random.RandomState(int(rng_seed))
-        vrng.shuffle(numpy_vval)
-        assert np.all(numpy_vval == fn_vval)
-
-        # Trying to shuffle a vector with function that should shuffle
-        # matrices, or vice versa, raises a TypeError
-        with pytest.raises(TypeError):
-            f1(in_mval)
-        with pytest.raises(TypeError):
-            f(in_vval)
-
-    def test_default_updates(self):
-        # Basic case: default_updates
-        random_a = RandomStreams(utt.fetch_seed())
-        out_a = random_a.uniform((2, 2))
-        fn_a = function([], out_a)
-        fn_a_val0 = fn_a()
-        fn_a_val1 = fn_a()
-        assert not np.all(fn_a_val0 == fn_a_val1)
-
-        nearly_zeros = function([], out_a + out_a - 2 * out_a)
-        assert np.all(abs(nearly_zeros()) < 1e-5)
-
-        # Explicit updates #1
-        random_b = RandomStreams(utt.fetch_seed())
-        out_b = random_b.uniform((2, 2))
-        fn_b = function([], out_b, updates=random_b.updates())
-        fn_b_val0 = fn_b()
-        fn_b_val1 = fn_b()
-        assert np.all(fn_b_val0 == fn_a_val0)
-        assert np.all(fn_b_val1 == fn_a_val1)
-
-        # Explicit updates #2
-        random_c = RandomStreams(utt.fetch_seed())
-        out_c = random_c.uniform((2, 2))
-        fn_c = function([], out_c, updates=[out_c.update])
-        fn_c_val0 = fn_c()
-        fn_c_val1 = fn_c()
-        assert np.all(fn_c_val0 == fn_a_val0)
-        assert np.all(fn_c_val1 == fn_a_val1)
-
-        # No updates at all
-        random_d = RandomStreams(utt.fetch_seed())
-        out_d = random_d.uniform((2, 2))
-        fn_d = function([], out_d, no_default_updates=True)
-        fn_d_val0 = fn_d()
-        fn_d_val1 = fn_d()
-        assert np.all(fn_d_val0 == fn_a_val0)
-        assert np.all(fn_d_val1 == fn_d_val0)
-
-        # No updates for out
-        random_e = RandomStreams(utt.fetch_seed())
-        out_e = random_e.uniform((2, 2))
-        fn_e = function([], out_e, no_default_updates=[out_e.rng])
-        fn_e_val0 = fn_e()
-        fn_e_val1 = fn_e()
-        assert np.all(fn_e_val0 == fn_a_val0)
-        assert np.all(fn_e_val1 == fn_e_val0)
-
-    def test_symbolic_shape(self):
-        random = RandomStreams(utt.fetch_seed())
-        shape = tensor.lvector()
-        f = function([shape], random.uniform(size=shape, ndim=2))
-
-        assert f([2, 3]).shape == (2, 3)
-        assert f([4, 8]).shape == (4, 8)
-        with pytest.raises(ValueError):
-            f([4])
-        with pytest.raises(ValueError):
-            f([4, 3, 4, 5])
-
-    def test_mixed_shape(self):
-        # Test when the provided shape is a tuple of ints and scalar vars
-        random = RandomStreams(utt.fetch_seed())
-        shape0 = tensor.lscalar()
-        shape = (shape0, 3)
-        f = function([shape0], random.uniform(size=shape, ndim=2))
-        assert f(2).shape == (2, 3)
-        assert f(8).shape == (8, 3)
-
-        g = function([shape0], random.uniform(size=shape))
-        assert g(2).shape == (2, 3)
-        assert g(8).shape == (8, 3)
-
-    def test_mixed_shape_bcastable(self):
-        # Test when the provided shape is a tuple of ints and scalar vars
-        random = RandomStreams(utt.fetch_seed())
-        shape0 = tensor.lscalar()
-        shape = (shape0, 1)
-        u = random.uniform(size=shape, ndim=2)
-        assert u.broadcastable == (False, True)
-        f = function([shape0], u)
-        assert f(2).shape == (2, 1)
-        assert f(8).shape == (8, 1)
-
-        v = random.uniform(size=shape)
-        assert v.broadcastable == (False, True)
-        g = function([shape0], v)
-        assert g(2).shape == (2, 1)
-        assert g(8).shape == (8, 1)
-
-    def test_default_shape(self):
-        random = RandomStreams(utt.fetch_seed())
-        f = function([], random.uniform())
-        g = function([], random.multinomial())
-
-        # seed_rng is generator for generating *seeds* for RandomStates
-        seed_rng = np.random.RandomState(utt.fetch_seed())
-        uniform_rng = np.random.RandomState(int(seed_rng.randint(2 ** 30)))
-        multinomial_rng = np.random.RandomState(int(seed_rng.randint(2 ** 30)))
-
-        val0 = f()
-        val1 = f()
-        numpy_val0 = uniform_rng.uniform()
-        numpy_val1 = uniform_rng.uniform()
-        assert np.allclose(val0, numpy_val0)
-        assert np.allclose(val1, numpy_val1)
-
-        for i in range(
-            10
-        ):  # every test has 50% chance of passing even with non-matching random states
-            val2 = g()
-            numpy_val2 = multinomial_rng.multinomial(n=1, pvals=[0.5, 0.5])
-            assert np.all(val2 == numpy_val2)
-
-    def test_vector_arguments(self):
-        random = RandomStreams(utt.fetch_seed())
-        low = tensor.dvector()
-        out = random.uniform(low=low, high=1)
-        assert out.ndim == 1
-        f = function([low], out)
-
-        seed_gen = np.random.RandomState(utt.fetch_seed())
-        numpy_rng = np.random.RandomState(int(seed_gen.randint(2 ** 30)))
-        val0 = f([-5, 0.5, 0, 1])
-        val1 = f([0.9])
-        numpy_val0 = numpy_rng.uniform(low=[-5, 0.5, 0, 1], high=1)
-        numpy_val1 = numpy_rng.uniform(low=[0.9], high=1)
-        assert np.all(val0 == numpy_val0)
-        assert np.all(val1 == numpy_val1)
-
-        high = tensor.vector()
-        outb = random.uniform(low=low, high=high)
-        assert outb.ndim == 1
-        fb = function([low, high], outb)
-
-        numpy_rng = np.random.RandomState(int(seed_gen.randint(2 ** 30)))
-        val0b = fb([-4.0, -2], [-1, 0])
-        val1b = fb([-4.0], [-1])
-        numpy_val0b = numpy_rng.uniform(low=[-4.0, -2], high=[-1, 0])
-        numpy_val1b = numpy_rng.uniform(low=[-4.0], high=[-1])
-        assert np.all(val0b == numpy_val0b)
-        assert np.all(val1b == numpy_val1b)
-        with pytest.raises(ValueError):
-            fb([-4.0, -2], [-1, 0, 1])
-        # TODO: do we want that?
-        # with pytest.raises(ValueError):
-        #    fb([-4., -2], [-1])
-
-        size = tensor.lvector()
-        outc = random.uniform(low=low, high=high, size=size, ndim=1)
-        fc = function([low, high, size], outc)
-
-        numpy_rng = np.random.RandomState(int(seed_gen.randint(2 ** 30)))
-        val0c = fc([-4.0, -2], [-1, 0], [2])
-        val1c = fc([-4.0], [-1], [1])
-        numpy_val0c = numpy_rng.uniform(low=[-4.0, -2], high=[-1, 0])
-        numpy_val1c = numpy_rng.uniform(low=[-4.0], high=[-1])
-        assert np.all(val0c == numpy_val0c)
-        assert np.all(val1c == numpy_val1c)
-
-        with pytest.raises(ValueError):
-            fc([-4.0, -2], [-1, 0], [1, 2])
-        with pytest.raises(ValueError):
-            fc([-4.0, -2], [-1, 0], [2, 1])
-        with pytest.raises(ValueError):
-            fc([-4.0, -2], [-1, 0], [1])
-        with pytest.raises(ValueError):
-            fc([-4.0, -2], [-1], [1])
-
-    def test_broadcast_arguments(self):
-        random = RandomStreams(utt.fetch_seed())
-        low = tensor.dvector()
-        high = tensor.dcol()
-        out = random.uniform(low=low, high=high)
-        assert out.ndim == 2
-        f = function([low, high], out)
-
-        rng_seed = np.random.RandomState(utt.fetch_seed()).randint(2 ** 30)
-        numpy_rng = np.random.RandomState(int(rng_seed))
-        val0 = f([-5, 0.5, 0, 1], [[1.0]])
-        val1 = f([0.9], [[1.0], [1.1], [1.5]])
-        val2 = f([-5, 0.5, 0, 1], [[1.0], [1.1], [1.5]])
-
-        numpy_val0 = numpy_rng.uniform(low=[-5, 0.5, 0, 1], high=[1.0])
-        numpy_val1 = numpy_rng.uniform(low=[0.9], high=[[1.0], [1.1], [1.5]])
-        numpy_val2 = numpy_rng.uniform(low=[-5, 0.5, 0, 1], high=[[1.0], [1.1], [1.5]])
-
-        assert np.all(val0 == numpy_val0)
-        assert np.all(val1 == numpy_val1)
-        assert np.all(val2 == numpy_val2)
-
-    def test_uniform_vector(self):
-        random = RandomStreams(utt.fetch_seed())
-        low = tensor.dvector()
-        high = tensor.dvector()
-        out = random.uniform(low=low, high=high)
-        assert out.ndim == 1
-        f = function([low, high], out)
-
-        low_val = [0.1, 0.2, 0.3]
-        high_val = [1.1, 2.2, 3.3]
-        seed_gen = np.random.RandomState(utt.fetch_seed())
-        numpy_rng = np.random.RandomState(int(seed_gen.randint(2 ** 30)))
-
-        # Arguments of size (3,)
-        val0 = f(low_val, high_val)
-        numpy_val0 = numpy_rng.uniform(low=low_val, high=high_val)
-        assert np.all(val0 == numpy_val0)
-
-        # arguments of size (2,)
-        val1 = f(low_val[:-1], high_val[:-1])
-        numpy_val1 = numpy_rng.uniform(low=low_val[:-1], high=high_val[:-1])
-        assert np.all(val1 == numpy_val1)
-
-        # Specifying the size explicitly
-        g = function([low, high], random.uniform(low=low, high=high, size=(3,)))
-        val2 = g(low_val, high_val)
-        numpy_rng = np.random.RandomState(int(seed_gen.randint(2 ** 30)))
-        numpy_val2 = numpy_rng.uniform(low=low_val, high=high_val, size=(3,))
-        assert np.all(val2 == numpy_val2)
-        with pytest.raises(ValueError):
-            g(low_val[:-1], high_val[:-1])
-
-    def test_binomial_vector(self):
-        random = RandomStreams(utt.fetch_seed())
-        n = tensor.lvector()
-        prob = tensor.vector()
-        out = random.binomial(n=n, p=prob)
-        assert out.ndim == 1
-        f = function([n, prob], out)
-
-        n_val = [1, 2, 3]
-        prob_val = np.asarray([0.1, 0.2, 0.3], dtype=config.floatX)
-        seed_gen = np.random.RandomState(utt.fetch_seed())
-        numpy_rng = np.random.RandomState(int(seed_gen.randint(2 ** 30)))
-
-        # Arguments of size (3,)
-        val0 = f(n_val, prob_val)
-        numpy_val0 = numpy_rng.binomial(n=n_val, p=prob_val)
-        assert np.all(val0 == numpy_val0)
-
-        # arguments of size (2,)
-        val1 = f(n_val[:-1], prob_val[:-1])
-        numpy_val1 = numpy_rng.binomial(n=n_val[:-1], p=prob_val[:-1])
-        assert np.all(val1 == numpy_val1)
-
-        # Specifying the size explicitly
-        g = function([n, prob], random.binomial(n=n, p=prob, size=(3,)))
-        val2 = g(n_val, prob_val)
-        numpy_rng = np.random.RandomState(int(seed_gen.randint(2 ** 30)))
-        numpy_val2 = numpy_rng.binomial(n=n_val, p=prob_val, size=(3,))
-        assert np.all(val2 == numpy_val2)
-        with pytest.raises(ValueError):
-            g(n_val[:-1], prob_val[:-1])
-
-    def test_normal_vector(self):
-        random = RandomStreams(utt.fetch_seed())
-        avg = tensor.dvector()
-        std = tensor.dvector()
-        out = random.normal(avg=avg, std=std)
-        assert out.ndim == 1
-        f = function([avg, std], out)
-
-        avg_val = [1, 2, 3]
-        std_val = [0.1, 0.2, 0.3]
-        seed_gen = np.random.RandomState(utt.fetch_seed())
-        numpy_rng = np.random.RandomState(int(seed_gen.randint(2 ** 30)))
-
-        # Arguments of size (3,)
-        val0 = f(avg_val, std_val)
-        numpy_val0 = numpy_rng.normal(loc=avg_val, scale=std_val)
-        assert np.allclose(val0, numpy_val0)
-
-        # arguments of size (2,)
-        val1 = f(avg_val[:-1], std_val[:-1])
-        numpy_val1 = numpy_rng.normal(loc=avg_val[:-1], scale=std_val[:-1])
-        assert np.allclose(val1, numpy_val1)
-
-        # Specifying the size explicitly
-        g = function([avg, std], random.normal(avg=avg, std=std, size=(3,)))
-        val2 = g(avg_val, std_val)
-        numpy_rng = np.random.RandomState(int(seed_gen.randint(2 ** 30)))
-        numpy_val2 = numpy_rng.normal(loc=avg_val, scale=std_val, size=(3,))
-        assert np.allclose(val2, numpy_val2)
-        with pytest.raises(ValueError):
-            g(avg_val[:-1], std_val[:-1])
-
-    def test_random_integers_vector(self):
-        random = RandomStreams(utt.fetch_seed())
-        low = tensor.lvector()
-        high = tensor.lvector()
-        out = random.random_integers(low=low, high=high)
-        assert out.ndim == 1
-        f = function([low, high], out)
-
-        low_val = [100, 200, 300]
-        high_val = [110, 220, 330]
-        seed_gen = np.random.RandomState(utt.fetch_seed())
-        numpy_rng = np.random.RandomState(int(seed_gen.randint(2 ** 30)))
-
-        # Arguments of size (3,)
-        val0 = f(low_val, high_val)
-        numpy_val0 = np.asarray(
-            [
-                numpy_rng.randint(low=lv, high=hv + 1)
-                for lv, hv in zip(low_val, high_val)
-            ]
-        )
-        assert np.all(val0 == numpy_val0)
-
-        # arguments of size (2,)
-        val1 = f(low_val[:-1], high_val[:-1])
-        numpy_val1 = np.asarray(
-            [
-                numpy_rng.randint(low=lv, high=hv + 1)
-                for lv, hv in zip(low_val[:-1], high_val[:-1])
-            ]
-        )
-        assert np.all(val1 == numpy_val1)
-
-        # Specifying the size explicitly
-        g = function([low, high], random.random_integers(low=low, high=high, size=(3,)))
-        val2 = g(low_val, high_val)
-        numpy_rng = np.random.RandomState(int(seed_gen.randint(2 ** 30)))
-        numpy_val2 = np.asarray(
-            [
-                numpy_rng.randint(low=lv, high=hv + 1)
-                for lv, hv in zip(low_val, high_val)
-            ]
-        )
-        assert np.all(val2 == numpy_val2)
-        with pytest.raises(ValueError):
-            g(low_val[:-1], high_val[:-1])
-
-    # Vectorized permutation don't make sense: the only parameter, n,
-    # controls one dimension of the returned tensor.
-
-    def test_multinomial_vector(self):
-        random = RandomStreams(utt.fetch_seed())
-        n = tensor.lvector()
-        pvals = tensor.matrix()
-        out = random.multinomial(n=n, pvals=pvals)
-        assert out.ndim == 2
-        f = function([n, pvals], out)
-
-        n_val = [1, 2, 3]
-        pvals_val = [[0.1, 0.9], [0.2, 0.8], [0.3, 0.7]]
-        pvals_val = np.asarray(pvals_val, dtype=config.floatX)
-        seed_gen = np.random.RandomState(utt.fetch_seed())
-        numpy_rng = np.random.RandomState(int(seed_gen.randint(2 ** 30)))
-
-        # Arguments of size (3,)
-        val0 = f(n_val, pvals_val)
-        numpy_val0 = np.asarray(
-            [numpy_rng.multinomial(n=nv, pvals=pv) for nv, pv in zip(n_val, pvals_val)]
-        )
-        assert np.all(val0 == numpy_val0)
-
-        # arguments of size (2,)
-        val1 = f(n_val[:-1], pvals_val[:-1])
-        numpy_val1 = np.asarray(
-            [
-                numpy_rng.multinomial(n=nv, pvals=pv)
-                for nv, pv in zip(n_val[:-1], pvals_val[:-1])
-            ]
-        )
-        assert np.all(val1 == numpy_val1)
-
-        # Specifying the size explicitly
-        g = function([n, pvals], random.multinomial(n=n, pvals=pvals, size=(3,)))
-        val2 = g(n_val, pvals_val)
-        numpy_rng = np.random.RandomState(int(seed_gen.randint(2 ** 30)))
-        numpy_val2 = np.asarray(
-            [numpy_rng.multinomial(n=nv, pvals=pv) for nv, pv in zip(n_val, pvals_val)]
-        )
-        assert np.all(val2 == numpy_val2)
-        with pytest.raises(ValueError):
-            g(n_val[:-1], pvals_val[:-1])
-
-    def test_dtype(self):
-        random = RandomStreams(utt.fetch_seed())
-        low = tensor.lscalar()
-        high = tensor.lscalar()
-        out = random.random_integers(low=low, high=high, size=(20,), dtype="int8")
-        assert out.dtype == "int8"
-        f = function([low, high], out)
-
-        val0 = f(0, 9)
-        assert val0.dtype == "int8"
-
-        val1 = f(255, 257)
-        assert val1.dtype == "int8"
-        assert np.all(abs(val1) <= 1)
-
-    def test_default_dtype(self):
-        random = RandomStreams(utt.fetch_seed())
-        low = tensor.dscalar()
-        high = tensor.dscalar()
-
-        # Should not silently downcast from low and high
-        out0 = random.uniform(low=low, high=high, size=(42,))
-        assert out0.dtype == "float64"
-        f0 = function([low, high], out0)
-        val0 = f0(-2.1, 3.1)
-        assert val0.dtype == "float64"
-
-        # Should downcast, since asked explicitly
-        out1 = random.uniform(low=low, high=high, size=(42,), dtype="float32")
-        assert out1.dtype == "float32"
-        f1 = function([low, high], out1)
-        val1 = f1(-1.1, 1.1)
-        assert val1.dtype == "float32"
-
-        # Should use floatX
-        lowf = tensor.fscalar()
-        highf = tensor.fscalar()
-        outf = random.uniform(low=lowf, high=highf, size=(42,))
-        assert outf.dtype == config.floatX
-        ff = function([lowf, highf], outf)
-        valf = ff(np.float32(-0.1), np.float32(0.3))
-        assert valf.dtype == config.floatX
-
-    def test_shared_constructor_borrow(self):
-        rng = np.random.RandomState(123)
-        s_rng_default = shared(rng)
-        s_rng_True = shared(rng, borrow=True)
-        s_rng_False = shared(rng, borrow=False)
-
-        # test borrow contract: that False means a copy must have been made
-        assert s_rng_default.container.storage[0] is not rng
-        assert s_rng_False.container.storage[0] is not rng
-
-        # test current implementation: that True means a copy was not made
-        assert s_rng_True.container.storage[0] is rng
-
-        # ensure that all the random number generators are in the same state
-        v = rng.randn()
-        v0 = s_rng_default.container.storage[0].randn()
-        v1 = s_rng_False.container.storage[0].randn()
-        assert v == v0 == v1
-
-    def test_get_value_borrow(self):
-
-        rng = np.random.RandomState(123)
-        s_rng = shared(rng)
-
-        r_ = s_rng.container.storage[0]
-        r_T = s_rng.get_value(borrow=True)
-        r_F = s_rng.get_value(borrow=False)
-
-        # the contract requires that borrow=False returns a copy
-        assert r_ is not r_F
-
-        # the current implementation allows for True to return the real thing
-        assert r_ is r_T
-
-        # either way, the rngs should all be in the same state
-        assert r_.rand() == r_F.rand()
-
-    def test_get_value_internal_type(self):
-        rng = np.random.RandomState(123)
-        s_rng = shared(rng)
-
-        # there is no special behaviour required of return_internal_type
-        # this test just ensures that the flag doesn't screw anything up
-        # by repeating the get_value_borrow test.
-        r_ = s_rng.container.storage[0]
-        r_T = s_rng.get_value(borrow=True, return_internal_type=True)
-        r_F = s_rng.get_value(borrow=False, return_internal_type=True)
-
-        # the contract requires that borrow=False returns a copy
-        assert r_ is not r_F
-
-        # the current implementation allows for True to return the real thing
-        assert r_ is r_T
-
-        # either way, the rngs should all be in the same state
-        assert r_.rand() == r_F.rand()
-
-    def test_set_value_borrow(self):
-        rng = np.random.RandomState(123)
-
-        s_rng = shared(rng)
-
-        new_rng = np.random.RandomState(234234)
-
-        # Test the borrow contract is respected:
-        # assigning with borrow=False makes a copy
-        s_rng.set_value(new_rng, borrow=False)
-        assert new_rng is not s_rng.container.storage[0]
-        assert new_rng.randn() == s_rng.container.storage[0].randn()
-
-        # Test that the current implementation is actually borrowing when it can.
-        rr = np.random.RandomState(33)
-        s_rng.set_value(rr, borrow=True)
-        assert rr is s_rng.container.storage[0]
-
-    def test_multiple_rng_aliasing(self):
-        # Test that when we have multiple random number generators, we do not alias
-        # the state_updates member. `state_updates` can be useful when attempting to
-        # copy the (random) state between two similar theano graphs. The test is
-        # meant to detect a previous bug where state_updates was initialized as a
-        # class-attribute, instead of the __init__ function.
-
-        rng1 = RandomStreams(1234)
-        rng2 = RandomStreams(2392)
-        assert rng1.state_updates is not rng2.state_updates
-        assert rng1.gen_seedgen is not rng2.gen_seedgen
-
-    def test_random_state_transfer(self):
-        # Test that random state can be transferred from one theano graph to another.
-
-        class Graph:
-            def __init__(self, seed=123):
-                self.rng = RandomStreams(seed)
-                self.y = self.rng.uniform(size=(1,))
-
-        g1 = Graph(seed=123)
-        f1 = function([], g1.y)
-        g2 = Graph(seed=987)
-        f2 = function([], g2.y)
-
-        for (su1, su2) in zip(g1.rng.state_updates, g2.rng.state_updates):
-            su2[0].set_value(su1[0].get_value())
-
-        np.testing.assert_array_almost_equal(f1(), f2(), decimal=6)

tests/test_gradient.py

@@ -7,7 +7,7 @@ import theano
 from tests import unittest_tools as utt
 from theano import config, gof, gradient
 from theano.gof.null_type import NullType
-from theano.sandbox.rng_mrg import MRG_RandomStreams as RandomStreams
+from theano.sandbox.rng_mrg import MRG_RandomStream
 
 
 one = theano.tensor.as_tensor_variable(1.0)
@@ -811,17 +811,20 @@ def test_grad_scale():
 @config.change_flags(compute_test_value="off")
 def test_undefined_grad_opt():
     # Make sure that undefined grad get removed in optimized graph.
-    random = RandomStreams(np.random.randint(1, 2147462579))
+    random = MRG_RandomStream(np.random.randint(1, 2147462579))
+
     pvals = theano.shared(np.random.rand(10, 20).astype(config.floatX))
     pvals = pvals / pvals.sum(axis=1)
     pvals = gradient.zero_grad(pvals)
+
     samples = random.multinomial(pvals=pvals, n=1)
     samples = theano.tensor.cast(samples, pvals.dtype)
     samples = gradient.zero_grad(samples)
+
     cost = theano.tensor.sum(samples + pvals)
     grad = theano.tensor.grad(cost, samples)
+
     f = theano.function([], grad)
-    theano.printing.debugprint(f)
     assert not any(
         [
             isinstance(node.op, gradient.UndefinedGrad)

theano/__init__.py

@@ -197,5 +197,5 @@ def sparse_grad(var):
     return ret
 
 
-import theano.tensor.shared_randomstreams
+import theano.tensor.random.var
 from theano.scan import checkpoints, clone, foldl, foldr, map, reduce, scan

theano/compile/profiling.py

@@ -1516,7 +1516,7 @@ class ProfileStats:
 
         import theano
 
-        RandomFunction = theano.tensor.raw_random.RandomFunction
+        RandomVariable = theano.tensor.random.op.RandomVariable
         scal = theano.scalar
         T = theano.tensor
         scalar_op_amdlibm_no_speed_up = [
@@ -1664,18 +1664,18 @@ class ProfileStats:
         # tip 5
         for (fgraph, a) in self.apply_time:
             node = a
-            if isinstance(node.op, RandomFunction):
+            if isinstance(node.op, RandomVariable):
                 printed_tip = True
                 print(
                     "  - Replace the default random number generator by "
-                    "'from theano.sandbox.rng_mrg import MRG_RandomStreams "
-                    "as RandomStreams', as this is is faster. It is still "
+                    "'from theano.sandbox.rng_mrg import MRG_RandomStream "
+                    "as RandomStream', as this is is faster. It is still "
                     "experimental, but seems to work correctly.",
                     file=file,
                 )
                 if config.device.startswith("gpu"):
                     print(
-                        "     - MRG_RandomStreams is the only random number"
+                        "     - MRG_RandomStream is the only random number"
                         " generator supported on the GPU.",
                         file=file,
                     )

theano/gpuarray/rng_mrg.py

@@ -39,7 +39,7 @@ class GPUA_mrg_uniform(GpuKernelBase, mrg_uniform_base):
         # error checking slightly redundant here, since
         # this op should not be called directly.
         #
-        # call through MRG_RandomStreams instead.
+        # call through MRG_RandomStream instead.
         broad = []
         for i in range(self.output_type.ndim):
             broad.append(tensor.extract_constant(size[i]) == 1)

theano/sandbox/rng_mrg.py

@@ -369,7 +369,7 @@ class mrg_uniform(mrg_uniform_base):
         # error checking slightly redundant here, since
         # this op should not be called directly.
         #
-        # call through MRG_RandomStreams instead.
+        # call through MRG_RandomStream instead.
         broad = []
         for i in range(self.output_type.ndim):
             broad.append(tensor.extract_constant(size[i]) == 1)
@@ -669,7 +669,7 @@ def guess_n_streams(size, warn=False):
 
     """
     # TODO: a smart way of choosing the number of streams, see #612.
-    # Note that this code was moved out of `MRG_RandomStreams` so that it can
+    # Note that this code was moved out of `MRG_RandomStream` so that it can
     # be easily accessed from tests, where we want to disable the warning.
     if isinstance(size, (tuple, list)) and all([isinstance(i, int) for i in size]):
         # We can make a guess.
@@ -692,7 +692,7 @@ def guess_n_streams(size, warn=False):
         if warn:
             warnings.warn(
                 (
-                    "MRG_RandomStreams Can't determine #streams "
+                    "MRG_RandomStream Can't determine #streams "
                     f"from size ({size}), guessing 60*256"
                 ),
                 stacklevel=3,
@@ -700,7 +700,7 @@ def guess_n_streams(size, warn=False):
         return 60 * 256
 
 
-class MRG_RandomStreams:
+class MRG_RandomStream:
     """
     Module component with similar interface to numpy.random
     (numpy.random.RandomState).
@@ -723,7 +723,7 @@ class MRG_RandomStreams:
     def __init__(self, seed=12345):
         # A list of pairs of the form (input_r, output_r), representing the
         # update rules of all the random states generated
-        # by this RandomStreams.
+        # by this RandomStream.
         self.state_updates = []
 
         super().__init__()
@@ -948,7 +948,7 @@ class MRG_RandomStreams:
             x = self.uniform(size=size, nstreams=nstreams, **kwargs)
             return cast(x < p, dtype)
         else:
-            raise NotImplementedError("MRG_RandomStreams.binomial with n > 1")
+            raise NotImplementedError("MRG_RandomStream.binomial with n > 1")
 
     def multinomial(
         self,
@@ -993,12 +993,12 @@ class MRG_RandomStreams:
 
         if size is not None:
             raise ValueError(
-                "Provided a size argument to MRG_RandomStreams.multinomial, "
+                "Provided a size argument to MRG_RandomStream.multinomial, "
                 "which does not use the size argument."
             )
         if ndim is not None:
             raise ValueError(
-                "Provided an ndim argument to MRG_RandomStreams.multinomial, "
+                "Provided an ndim argument to MRG_RandomStream.multinomial, "
                 "which does not use the ndim argument."
             )
         if pvals.ndim == 2:
@@ -1009,7 +1009,7 @@ class MRG_RandomStreams:
             return op(pvals, unis, n_samples)
         else:
             raise NotImplementedError(
-                "MRG_RandomStreams.multinomial only" " implemented for pvals.ndim = 2"
+                "MRG_RandomStream.multinomial only" " implemented for pvals.ndim = 2"
             )
 
     def choice(
@@ -1065,31 +1065,31 @@ class MRG_RandomStreams:
         """
         if replace:
             raise NotImplementedError(
-                "MRG_RandomStreams.choice only works without replacement " "for now."
+                "MRG_RandomStream.choice only works without replacement " "for now."
             )
 
         if a is not None:
             raise TypeError(
                 "For now, a has to be None in "
-                "MRG_RandomStreams.choice. Sampled values are "
+                "MRG_RandomStream.choice. Sampled values are "
                 "between 0 and p.shape[1]-1"
             )
 
         if p is None:
             raise TypeError(
-                "For now, p has to be specified in " "MRG_RandomStreams.choice."
+                "For now, p has to be specified in " "MRG_RandomStream.choice."
             )
         p = as_tensor_variable(p)
         p = undefined_grad(p)
 
         if ndim is not None:
             raise ValueError(
-                "ndim argument to " "MRG_RandomStreams.choice " "is not used."
+                "ndim argument to " "MRG_RandomStream.choice " "is not used."
             )
 
         if p.ndim != 2:
             raise NotImplementedError(
-                "MRG_RandomStreams.choice is only implemented for p.ndim = 2"
+                "MRG_RandomStream.choice is only implemented for p.ndim = 2"
             )
 
         shape = p[:, 0].shape * size
@@ -1108,9 +1108,9 @@ class MRG_RandomStreams:
         **kwargs,
     ):
         warnings.warn(
-            "MRG_RandomStreams.multinomial_wo_replacement() is "
+            "MRG_RandomStream.multinomial_wo_replacement() is "
             "deprecated and will be removed in the next release of "
-            "Theano. Please use MRG_RandomStreams.choice() instead."
+            "Theano. Please use MRG_RandomStream.choice() instead."
         )
         assert size is None
         return self.choice(

theano/tensor/__init__.py

@@ -57,9 +57,3 @@ from theano.tensor.var import (
     TensorVariable,
     _tensor_py_operators,
 )
-
-
-# These imports cannot be performed here because the modules depend on tensor.  This is done at the
-# end of theano.__init__.py instead.
-# from theano.tensor import raw_random
-# from theano.tensor import shared_randomstreams

theano/tensor/random/__init__.py

+# Initialize `RandomVariable` optimizations
+import theano.tensor.random.opt
+import theano.tensor.random.utils

theano/tensor/random/basic.py

+import numpy as np
+import scipy.stats as stats
+
+import theano
+from theano.tensor.basic import as_tensor_variable
+from theano.tensor.random.op import RandomVariable, default_shape_from_params
+from theano.tensor.random.utils import broadcast_params
+
+
+try:
+    from pypolyagamma import PyPolyaGamma
+except ImportError:  # pragma: no cover
+
+    def PyPolyaGamma(*args, **kwargs):
+        raise RuntimeError("pypolygamma not installed!")
+
+
+class UniformRV(RandomVariable):
+    name = "uniform"
+    ndim_supp = 0
+    ndims_params = [0, 0]
+    dtype = theano.config.floatX
+    _print_name = ("U", "\\operatorname{U}")
+
+    def __call__(self, low=0.0, high=1.0, size=None, **kwargs):
+        return super().__call__(low, high, size=size, **kwargs)
+
+
+uniform = UniformRV()
+
+
+class BetaRV(RandomVariable):
+    name = "beta"
+    ndim_supp = 0
+    ndims_params = [0, 0]
+    dtype = theano.config.floatX
+    _print_name = ("Beta", "\\operatorname{Beta}")
+
+
+beta = BetaRV()
+
+
+class NormalRV(RandomVariable):
+    name = "normal"
+    ndim_supp = 0
+    ndims_params = [0, 0]
+    dtype = theano.config.floatX
+    _print_name = ("N", "\\operatorname{N}")
+
+    def __call__(self, loc=0.0, scale=1.1, size=None, **kwargs):
+        return super().__call__(loc, scale, size=size, **kwargs)
+
+
+normal = NormalRV()
+
+
+class HalfNormalRV(RandomVariable):
+    name = "halfnormal"
+    ndim_supp = 0
+    ndims_params = [0, 0]
+    dtype = theano.config.floatX
+    _print_name = ("N**+", "\\operatorname{N^{+}}")
+
+    def __call__(self, loc=0.0, scale=1.0, size=None, **kwargs):
+        return super().__call__(loc, scale, size=size, **kwargs)
+
+    @classmethod
+    def rng_fn(cls, rng, loc, scale, size):
+        return stats.halfnorm.rvs(loc, scale, random_state=rng, size=size)
+
+
+halfnormal = HalfNormalRV()
+
+
+class GammaRV(RandomVariable):
+    name = "halfnormal"
+    ndim_supp = 0
+    ndims_params = [0, 0]
+    dtype = theano.config.floatX
+    _print_name = ("Gamma", "\\operatorname{Gamma}")
+
+    def __call__(self, shape, rate, size=None, **kwargs):
+        return super().__call__(shape, 1.0 / rate, size=size, **kwargs)
+
+    @classmethod
+    def rng_fn(cls, rng, shape, scale, size):
+        return stats.gamma.rvs(shape, scale=scale, size=size, random_state=rng)
+
+
+gamma = GammaRV()
+
+
+class ExponentialRV(RandomVariable):
+    name = "exponential"
+    ndim_supp = 0
+    ndims_params = [0]
+    dtype = theano.config.floatX
+    _print_name = ("Exp", "\\operatorname{Exp}")
+
+    def __call__(self, scale=1.0, size=None, **kwargs):
+        return super().__call__(scale, size=size, **kwargs)
+
+
+exponential = ExponentialRV()
+
+
+def safe_multivariate_normal(mean, cov, size=None, rng=None):
+    """A shape consistent multivariate normal sampler.
+
+    What we mean by "shape consistent": SciPy will return scalars when the
+    arguments are vectors with dimension of size 1.  We require that the output
+    be at least 1D, so that it's consistent with the underlying random
+    variable.
+
+    """
+    res = np.atleast_1d(
+        stats.multivariate_normal(mean=mean, cov=cov, allow_singular=True).rvs(
+            size=size, random_state=rng
+        )
+    )
+
+    if size is not None:
+        res = res.reshape(list(size) + [-1])
+
+    return res
+
+
+class MvNormalRV(RandomVariable):
+    name = "multivariate_normal"
+    ndim_supp = 1
+    ndims_params = [1, 2]
+    dtype = theano.config.floatX
+    _print_name = ("N", "\\operatorname{N}")
+
+    def __call__(self, mean=None, cov=None, size=None, **kwargs):
+        if mean is None:
+            mean = np.array([0.0], dtype=self.dtype)
+        if cov is None:
+            cov = np.array([[1.0]], dtype=self.dtype)
+        return super().__call__(mean, cov, size=size, **kwargs)
+
+    @classmethod
+    def rng_fn(cls, rng, mean, cov, size):
+
+        if mean.ndim > 1 or cov.ndim > 2:
+            # Neither SciPy nor NumPy implement parameter broadcasting for
+            # multivariate normals (or many other multivariate distributions),
+            # so we have implement a quick and dirty one here
+            mean, cov = broadcast_params([mean, cov], cls.ndims_params)
+            size = tuple(size or ())
+
+            if size:
+                mean = np.broadcast_to(mean, size + mean.shape)
+                cov = np.broadcast_to(cov, size + cov.shape)
+
+            res = np.empty(mean.shape)
+            for idx in np.ndindex(mean.shape[:-1]):
+                m = mean[idx]
+                c = cov[idx]
+                res[idx] = safe_multivariate_normal(m, c, rng=rng)
+            return res
+        else:
+            return safe_multivariate_normal(mean, cov, size=size, rng=rng)
+
+
+multivariate_normal = MvNormalRV()
+
+
+class DirichletRV(RandomVariable):
+    name = "dirichlet"
+    ndim_supp = 1
+    ndims_params = [1]
+    dtype = theano.config.floatX
+    _print_name = ("Dir", "\\operatorname{Dir}")
+
+    @classmethod
+    def rng_fn(cls, rng, alphas, size):
+        if size is None:
+            size = ()
+        samples_shape = tuple(np.atleast_1d(size)) + alphas.shape
+        samples = np.empty(samples_shape)
+        alphas_bcast = np.broadcast_to(alphas, samples_shape)
+
+        for index in np.ndindex(*samples_shape[:-1]):
+            samples[index] = rng.dirichlet(alphas_bcast[index])
+
+        return samples
+
+
+dirichlet = DirichletRV()
+
+
+class PoissonRV(RandomVariable):
+    name = "poisson"
+    ndim_supp = 0
+    ndims_params = [0]
+    dtype = "int64"
+    _print_name = ("Pois", "\\operatorname{Pois}")
+
+    def __call__(self, lam=1.0, size=None, **kwargs):
+        return super().__call__(lam, size=size, **kwargs)
+
+
+poisson = PoissonRV()
+
+
+class CauchyRV(RandomVariable):
+    name = "cauchy"
+    ndim_supp = 0
+    ndims_params = [0, 0]
+    dtype = theano.config.floatX
+    _print_name = ("C", "\\operatorname{C}")
+
+    def __call__(self, loc=0.0, scale=1.0, size=None, **kwargs):
+        return super().__call__(loc, scale, size=size, **kwargs)
+
+    @classmethod
+    def rng_fn(cls, rng, loc, scale, size):
+        return stats.cauchy.rvs(loc=loc, scale=scale, random_state=rng, size=size)
+
+
+cauchy = CauchyRV()
+
+
+class HalfCauchyRV(RandomVariable):
+    name = "cauchy"
+    ndim_supp = 0
+    ndims_params = [0, 0]
+    dtype = theano.config.floatX
+    _print_name = ("C**+", "\\operatorname{C^{+}}")
+
+    def __call__(self, loc=0.0, scale=1.0, size=None, **kwargs):
+        return super().__call__(loc, scale, size=size, **kwargs)
+
+    @classmethod
+    def rng_fn(cls, rng, loc, scale, size):
+        return stats.halfcauchy.rvs(loc=loc, scale=scale, random_state=rng, size=size)
+
+
+halfcauchy = HalfCauchyRV()
+
+
+class InvGammaRV(RandomVariable):
+    name = "invgamma"
+    ndim_supp = 0
+    ndims_params = [0, 0]
+    dtype = theano.config.floatX
+    _print_name = ("InvGamma", "\\operatorname{Gamma^{-1}}")
+
+    @classmethod
+    def rng_fn(cls, rng, shape, rate, size=None):
+        return stats.invgamma.rvs(shape, scale=rate, size=size, random_state=rng)
+
+
+invgamma = InvGammaRV()
+
+
+class TruncExponentialRV(RandomVariable):
+    name = "truncexpon"
+    ndim_supp = 0
+    ndims_params = [0, 0, 0]
+    dtype = theano.config.floatX
+    _print_name = ("TruncExp", "\\operatorname{TruncExp}")
+
+    @classmethod
+    def rng_fn(cls, rng, b, loc, scale, size=None):
+        return stats.truncexpon.rvs(
+            b, loc=loc, scale=scale, size=size, random_state=rng
+        )
+
+
+truncexpon = TruncExponentialRV()
+
+
+class BernoulliRV(RandomVariable):
+    name = "bernoulli"
+    ndim_supp = 0
+    ndims_params = [0]
+    dtype = "int64"
+    _print_name = ("Bern", "\\operatorname{Bern}")
+
+    @classmethod
+    def rng_fn(cls, rng, p, size=None):
+        return stats.bernoulli.rvs(p, size=size, random_state=rng)
+
+
+bernoulli = BernoulliRV()
+
+
+class BinomialRV(RandomVariable):
+    name = "binomial"
+    ndim_supp = 0
+    ndims_params = [0, 0]
+    dtype = "int64"
+    _print_name = ("Binom", "\\operatorname{Binom}")
+
+
+binomial = BinomialRV()
+
+
+class NegBinomialRV(RandomVariable):
+    name = "nbinom"
+    ndim_supp = 0
+    ndims_params = [0, 0]
+    dtype = "int64"
+    _print_name = ("NB", "\\operatorname{NB}")
+
+    @classmethod
+    def rng_fn(cls, rng, n, p, size=None):
+        return stats.nbinom.rvs(n, p, size=size, random_state=rng)
+
+
+nbinom = NegBinomialRV()
+
+
+class BetaBinomialRV(RandomVariable):
+    name = "beta_binomial"
+    ndim_supp = 0
+    ndims_params = [0, 0, 0]
+    dtype = "int64"
+    _print_name = ("BetaBinom", "\\operatorname{BetaBinom}")
+
+    @classmethod
+    def rng_fn(cls, rng, n, a, b, size=None):
+        return stats.betabinom.rvs(n, a, b, size=size, random_state=rng)
+
+
+betabinom = BetaBinomialRV()
+
+
+class MultinomialRV(RandomVariable):
+    """A Multinomial random variable type.
+
+    FYI: Support shape is determined by the first dimension in the *second*
+    parameter (i.e.  the probabilities vector).
+
+    """
+
+    name = "multinomial"
+    ndim_supp = 1
+    ndims_params = [0, 1]
+    dtype = "int64"
+    _print_name = ("MN", "\\operatorname{MN}")
+
+    def _shape_from_params(self, dist_params, rep_param_idx=1, param_shapes=None):
+        return default_shape_from_params(
+            self.ndim_supp, dist_params, rep_param_idx, param_shapes
+        )
+
+
+multinomial = MultinomialRV()
+
+vsearchsorted = np.vectorize(np.searchsorted, otypes=[np.int], signature="(n),()->()")
+
+
+class CategoricalRV(RandomVariable):
+    name = "categorical"
+    ndim_supp = 0
+    ndims_params = [1]
+    dtype = "int64"
+    _print_name = ("Cat", "\\operatorname{Cat}")
+
+    @classmethod
+    def rng_fn(cls, rng, p, size):
+        if size is None:
+            size = ()
+
+        size = tuple(np.atleast_1d(size))
+        ind_shape = p.shape[:-1]
+
+        if len(size) > 0 and size[-len(ind_shape) :] != ind_shape:
+            raise ValueError("Parameters shape and size do not match.")
+
+        samples_shape = size[: -len(ind_shape)] + ind_shape
+        unif_samples = rng.uniform(size=samples_shape)
+        samples = vsearchsorted(p.cumsum(axis=-1), unif_samples)
+
+        return samples
+
+
+categorical = CategoricalRV()
+
+
+class PolyaGammaRV(RandomVariable):
+    """Polya-Gamma random variable.
+
+    XXX: This doesn't really use the given RNG, due to the narrowness of the
+    sampler package's implementation.
+    """
+
+    name = "polya-gamma"
+    ndim_supp = 0
+    ndims_params = [0, 0]
+    dtype = theano.config.floatX
+    _print_name = ("PG", "\\operatorname{PG}")
+
+    @classmethod
+    def rng_fn(cls, rng, b, c, size):
+        pg = PyPolyaGamma(rng.randint(2 ** 16))
+
+        if not size and b.shape == c.shape == ():
+            return pg.pgdraw(b, c)
+        else:
+            b, c = np.broadcast_arrays(b, c)
+            size = tuple(size or ())
+
+            if len(size) > 0:
+                b = np.broadcast_to(b, size)
+                c = np.broadcast_to(c, size)
+
+            smpl_val = np.empty(b.shape, dtype="double")
+
+            pg.pgdrawv(
+                np.asarray(b.flat).astype("double", copy=True),
+                np.asarray(c.flat).astype("double", copy=True),
+                np.asarray(smpl_val.flat),
+            )
+            return smpl_val
+
+
+polyagamma = PolyaGammaRV()
+
+
+class RandIntRV(RandomVariable):
+    name = "randint"
+    ndim_supp = 0
+    ndims_params = [0, 0]
+    dtype = "int64"
+    _print_name = ("randint", "\\operatorname{randint}")
+
+    def __call__(self, low, high=None, size=None, **kwargs):
+        if high is None:
+            low, high = 0, low
+        return super().__call__(low, high, size=size, **kwargs)
+
+
+randint = RandIntRV()
+
+
+class ChoiceRV(RandomVariable):
+    name = "choice"
+    ndim_supp = 0
+    ndims_params = [1, 1, 0]
+    dtype = None
+    _print_name = ("choice", "\\operatorname{choice}")
+
+    @classmethod
+    def rng_fn(cls, rng, a, p, replace, size):
+        return rng.choice(a, size, replace, p)
+
+    def _shape_from_params(self, *args, **kwargs):
+        raise NotImplementedError()
+
+    def _infer_shape(self, size, dist_params, param_shapes=None):
+        return size
+
+    def __call__(self, a, size=None, replace=True, p=None, **kwargs):
+
+        a = as_tensor_variable(a, ndim=1)
+
+        if p is None:
+            p = theano.tensor.type_other.NoneConst.clone()
+
+        if isinstance(replace, bool):
+            replace = theano.tensor.constant(np.array(replace))
+
+        return super().__call__(a, p, replace, size=size, dtype=a.dtype, **kwargs)
+
+
+choice = ChoiceRV()
+
+
+class PermutationRV(RandomVariable):
+    name = "permutation"
+    ndim_supp = 1
+    ndims_params = [1]
+    dtype = None
+    _print_name = ("permutation", "\\operatorname{permutation}")
+
+    @classmethod
+    def rng_fn(cls, rng, x, size):
+        return rng.permutation(x if x.ndim > 0 else x.item())
+
+    def _infer_shape(self, size, dist_params, param_shapes=None):
+
+        param_shapes = param_shapes or [p.shape for p in dist_params]
+
+        (x,) = dist_params
+        (x_shape,) = param_shapes
+
+        if x.ndim == 0:
+            return (x,)
+        else:
+            return x_shape
+
+    def __call__(self, x, **kwargs):
+        x = as_tensor_variable(x)
+        return super().__call__(x, dtype=x.dtype, **kwargs)
+
+
+permutation = PermutationRV()

theano/tensor/random/op.py

+from collections.abc import Sequence
+from copy import copy
+
+import numpy as np
+
+import theano
+from theano.gof.graph import Apply, Variable
+from theano.gof.op import Op
+from theano.tensor.basic import (
+    NotScalarConstantError,
+    all_dtypes,
+    as_tensor_variable,
+    cast,
+    constant,
+    get_scalar_constant_value,
+    get_vector_length,
+    int_dtypes,
+)
+from theano.tensor.random.type import RandomStateType
+from theano.tensor.random.utils import params_broadcast_shapes
+from theano.tensor.type import TensorType
+from theano.tensor.type_other import NoneConst
+
+
+def default_shape_from_params(
+    ndim_supp, dist_params, rep_param_idx=0, param_shapes=None
+):
+    """Infer the dimensions for the output of a `RandomVariable`.
+
+    This is a function that derives a random variable's support
+    shape/dimensions from one of its parameters.
+
+    XXX: It's not always possible to determine a random variable's support
+    shape from its parameters, so this function has fundamentally limited
+    applicability and must be replaced by custom logic in such cases.
+
+    XXX: This function is not expected to handle `ndim_supp = 0` (i.e.
+    scalars), since that is already definitively handled in the `Op` that
+    calls this.
+
+    TODO: Consider using `theano.compile.ops.shape_i` alongside `ShapeFeature`.
+
+    Parameters
+    ----------
+    ndim_supp: int
+        Total number of dimensions for a single draw of the random variable
+        (e.g. a multivariate normal draw is 1D, so `ndim_supp = 1`).
+    dist_params: list of `theano.gof.graph.Variable`
+        The distribution parameters.
+    param_shapes: list of tuple of `ScalarVariable` (optional)
+        Symbolic shapes for each distribution parameter.  These will
+        be used in place of distribution parameter-generated shapes.
+    rep_param_idx: int (optional)
+        The index of the distribution parameter to use as a reference
+        In other words, a parameter in `dist_param` with a shape corresponding
+        to the support's shape.
+        The default is the first parameter (i.e. the value 0).
+
+    Results
+    -------
+    out: a tuple representing the support shape for a distribution with the
+    given `dist_params`.
+
+    """
+    if ndim_supp <= 0:
+        raise ValueError("ndim_supp must be greater than 0")
+    if param_shapes is not None:
+        ref_param = param_shapes[rep_param_idx]
+        return (ref_param[-ndim_supp],)
+    else:
+        ref_param = dist_params[rep_param_idx]
+        if ref_param.ndim < ndim_supp:
+            raise ValueError(
+                (
+                    "Reference parameter does not match the "
+                    f"expected dimensions; {ref_param} has less than {ndim_supp} dim(s)."
+                )
+            )
+        return ref_param.shape[-ndim_supp:]
+
+
+class RandomVariable(Op):
+    """An `Op` that produces a sample from a random variable.
+
+    This is essentially `RandomFunction`, except that it removes the
+    `outtype` dependency and handles shape dimension information more
+    directly.
+
+    """
+
+    __props__ = ("name", "ndim_supp", "ndims_params", "dtype", "inplace")
+    default_output = 1
+    nondeterministic = True
+
+    def __init__(
+        self,
+        name=None,
+        ndim_supp=None,
+        ndims_params=None,
+        dtype=None,
+        inplace=None,
+    ):
+        """Create a random variable `Op`.
+
+        Parameters
+        ----------
+        name: str
+            The `Op`'s display name.
+        ndim_supp: int
+            Total number of dimensions for a single draw of the random variable
+            (e.g. a multivariate normal draw is 1D, so `ndim_supp = 1`).
+        ndims_params: list of int
+            Number of dimensions for each distribution parameter when the
+            parameters only specify a single drawn of the random variable (e.g. a
+            multivariate normal's mean is 1D and covariance is 2D, so `ndims_params
+            = [1, 2]`).
+        dtype: Theano dtype (optional)
+            The dtype of the sampled output(s).  If `None` (the default), the
+            `dtype` keyword must be set when `RandomVariable.make_node` is
+            called.
+        inplace: boolean (optional)
+            Determine whether or not the underlying rng state is updated
+            in-place or not (i.e. copied).
+
+        """
+        super().__init__()
+
+        self.name = name or getattr(self, "name")
+        self.ndim_supp = (
+            ndim_supp if ndim_supp is not None else getattr(self, "ndim_supp")
+        )
+        self.ndims_params = (
+            ndims_params if ndims_params is not None else getattr(self, "ndims_params")
+        )
+        self.dtype = dtype or getattr(self, "dtype", None)
+        self.inplace = (
+            inplace if inplace is not None else getattr(self, "inplace", False)
+        )
+
+        if not isinstance(self.ndims_params, Sequence):
+            raise TypeError("Parameter ndims_params must be sequence type.")
+
+        self.ndims_params = tuple(self.ndims_params)
+
+        if self.inplace:
+            self.destroy_map = {0: [len(self.ndims_params) + 1]}
+
+    def _shape_from_params(self, dist_params, **kwargs):
+        """Determine the shape of a `RandomVariable`'s output given its parameters.
+
+        This does *not* consider the extra dimensions added by the `size` parameter.
+
+        Defaults to `param_supp_shape_fn`.
+        """
+        return default_shape_from_params(self.ndim_supp, dist_params, **kwargs)
+
+    def rng_fn(self, rng, *args, **kwargs):
+        """Sample a numeric random variate."""
+        return getattr(np.random.RandomState, self.name)(rng, *args, **kwargs)
+
+    def __str__(self):
+        return "{}_rv".format(self.name)
+
+    def _infer_shape(self, size, dist_params, param_shapes=None):
+        """Compute the output shape given the size and distribution parameters.
+
+        Parameters
+        ----------
+        size : TensorVariable
+            The size parameter specified for this `RandomVariable`.
+        dist_params : list of TensorVariable
+            The symbolic parameter for this `RandomVariable`'s distribution.
+        param_shapes : list of tuples of TensorVariable (optional)
+            The shapes of the `dist_params` as given by `ShapeFeature`'s
+            via `Op.infer_shape`'s `input_shapes` argument.  This parameter's
+            values are essentially more accurate versions of ``[d.shape for d
+            in dist_params]``.
+
+        Outputs
+        -------
+        shape : tuple of `ScalarVariable`
+
+        """
+
+        size_len = get_vector_length(size)
+
+        if self.ndim_supp == 0 and size_len > 0:
+            # In this case, we have a univariate distribution with a non-empty
+            # `size` parameter, which means that the `size` parameter
+            # completely determines the shape of the random variable.  More
+            # importantly, the `size` parameter may be the only correct source
+            # of information for the output shape, in that we would be misled
+            # by the `dist_params` if we tried to infer the relevant parts of
+            # the output shape from those.
+            return size
+
+        # Broadcast the parameters
+        param_shapes = params_broadcast_shapes(
+            param_shapes or [p.shape for p in dist_params], self.ndims_params
+        )
+
+        def slice_ind_dims(p, ps, n):
+            shape = tuple(ps)
+
+            if n == 0:
+                return (p, shape)
+
+            ind_slice = (slice(None),) * (p.ndim - n) + (0,) * n
+            ind_shape = [
+                s if b is False else constant(1, "int64")
+                for s, b in zip(shape[:-n], p.broadcastable[:-n])
+            ]
+            return (
+                p[ind_slice],
+                ind_shape,
+            )
+
+        # These are versions of our actual parameters with the anticipated
+        # dimensions (i.e. support dimensions) removed so that only the
+        # independent variate dimensions are left.
+        params_ind_slice = tuple(
+            slice_ind_dims(p, ps, n)
+            for p, ps, n in zip(dist_params, param_shapes, self.ndims_params)
+        )
+
+        if len(params_ind_slice) == 1:
+            ind_param, ind_shape = params_ind_slice[0]
+            ndim_ind = len(ind_shape)
+            shape_ind = ind_shape
+        elif len(params_ind_slice) > 1:
+            # If there are multiple parameters, the dimensions of their
+            # independent variates should broadcast together.
+            p_slices, p_shapes = zip(*params_ind_slice)
+
+            shape_ind = theano.tensor.extra_ops.broadcast_shape_iter(
+                p_shapes, arrays_are_shapes=True
+            )
+
+            ndim_ind = len(shape_ind)
+        else:
+            ndim_ind = 0
+
+        if self.ndim_supp == 0:
+            shape_supp = tuple()
+            shape_reps = tuple(size)
+
+            if ndim_ind > 0:
+                shape_reps = shape_reps[:-ndim_ind]
+
+            ndim_reps = len(shape_reps)
+        else:
+            shape_supp = self._shape_from_params(
+                dist_params,
+                param_shapes=param_shapes,
+            )
+
+            ndim_reps = size_len
+            shape_reps = size
+
+        ndim_shape = self.ndim_supp + ndim_ind + ndim_reps
+
+        if ndim_shape == 0:
+            shape = constant([], dtype="int64")
+        else:
+            shape = tuple(shape_reps) + tuple(shape_ind) + tuple(shape_supp)
+
+        # if shape is None:
+        #     raise ShapeError()
+
+        return shape
+
+    @theano.change_flags(compute_test_value="off")
+    def compute_bcast(self, dist_params, size):
+        """Compute the broadcast array for this distribution's `TensorType`.
+
+        Parameters
+        ----------
+        dist_params: list
+            Distribution parameters.
+        size: int or Sequence (optional)
+            Numpy-like size of the output (i.e. replications).
+
+        """
+        shape = self._infer_shape(size, dist_params)
+
+        # Let's try to do a better job than `_infer_ndim_bcast` when
+        # dimension sizes are symbolic.
+        bcast = []
+        for s in shape:
+            s_owner = getattr(s, "owner", None)
+
+            # Get rid of the `Assert`s added by `broadcast_shape`
+            if s_owner and isinstance(s_owner.op, theano.tensor.opt.Assert):
+                s = s_owner.inputs[0]
+
+            try:
+                s_val = get_scalar_constant_value(s)
+            except NotScalarConstantError:
+                s_val = False
+
+            bcast += [s_val == 1]
+        return bcast
+
+    def infer_shape(self, fgraph, node, input_shapes):
+        _, size, _, *dist_params = node.inputs
+        _, _, _, *param_shapes = input_shapes
+
+        shape = self._infer_shape(size, dist_params, param_shapes=param_shapes)
+
+        return [None, [s for s in shape]]
+
+    def __call__(self, *args, size=None, name=None, rng=None, dtype=None, **kwargs):
+        res = super().__call__(rng, size, dtype, *args, **kwargs)
+
+        if name is not None:
+            res.name = name
+
+        return res
+
+    def make_node(self, rng, size, dtype, *dist_params):
+        """Create a random variable node.
+
+        XXX: Unnamed/non-keyword arguments are considered distribution
+        parameters!  If you want to set `size`, `rng`, and/or `name`, use their
+        keywords.
+
+        Parameters
+        ----------
+        rng: RandomStateType
+            Existing Theano `RandomState` object to be used.  Creates a
+            new one, if `None`.
+        size: int or Sequence
+            Numpy-like size of the output (i.e. replications).
+        dtype: Theano dtype
+            The dtype of the sampled output.  This value is only used when
+            `self.dtype` isn't set.
+        dist_params: list
+            Distribution parameters.
+
+        Results
+        -------
+        out: `Apply`
+            A node with inputs `(rng, size, dtype) + dist_args` and outputs
+            `(rng_var, out_var)`.
+
+        """
+        if size is None:
+            size = constant([], dtype="int64")
+        elif isinstance(size, int):
+            size = as_tensor_variable([size], ndim=1)
+        elif not isinstance(size, (np.ndarray, Variable, Sequence)):
+            raise TypeError(
+                "Parameter size must be None, an integer, or a sequence with integers."
+            )
+        else:
+            size = cast(as_tensor_variable(size, ndim=1), "int64")
+
+        assert size.dtype in int_dtypes
+
+        dist_params = tuple(
+            as_tensor_variable(p) if not isinstance(p, Variable) else p
+            for p in dist_params
+        )
+
+        if rng is None:
+            rng = theano.shared(np.random.RandomState())
+        elif not isinstance(rng.type, RandomStateType):
+            raise TypeError("The type of rng should be an instance of RandomStateType")
+
+        bcast = self.compute_bcast(dist_params, size)
+        dtype = self.dtype or dtype
+
+        if dtype is None or (isinstance(dtype, str) and dtype not in all_dtypes):
+            # dtype = tt.scal.upcast(self.dtype, *[p.dtype for p in dist_params])
+            raise TypeError("dtype is unspecified")
+
+        if isinstance(dtype, str):
+            dtype_idx = constant(all_dtypes.index(dtype), dtype="int64")
+        else:
+            dtype_idx = constant(dtype, dtype="int64")
+            dtype = all_dtypes[dtype_idx.data]
+
+        outtype = TensorType(dtype=dtype, broadcastable=bcast)
+        out_var = outtype()
+        inputs = (rng, size, dtype_idx) + dist_params
+        outputs = (rng.type(), out_var)
+
+        return Apply(self, inputs, outputs)
+
+    def perform(self, node, inputs, outputs):
+        rng_var_out, smpl_out = outputs
+
+        rng, size, dtype, *args = inputs
+
+        out_var = node.outputs[1]
+
+        # If `size == []`, that means no size is enforced, and NumPy is trusted
+        # to draw the appropriate number of samples, NumPy uses `size=None` to
+        # represent that.  Otherwise, NumPy expects a tuple.
+        if np.size(size) == 0:
+            size = None
+        else:
+            size = tuple(size)
+
+        # Draw from `rng` if `self.inplace` is `True`, and from a copy of `rng`
+        # otherwise.
+        if not self.inplace:
+            rng = copy(rng)
+
+        rng_var_out[0] = rng
+
+        smpl_val = self.rng_fn(rng, *(args + [size]))
+
+        if (
+            not isinstance(smpl_val, np.ndarray)
+            or str(smpl_val.dtype) != out_var.type.dtype
+        ):
+            smpl_val = theano._asarray(smpl_val, dtype=out_var.type.dtype)
+
+        smpl_out[0] = smpl_val
+
+    def grad(self, inputs, outputs):
+        return [
+            theano.gradient.grad_undefined(
+                self, k, inp, "No gradient defined for random variables"
+            )
+            for k, inp in enumerate(inputs)
+        ]
+
+    def R_op(self, inputs, eval_points):
+        return [None for i in eval_points]
+
+
+class Observed(Op):
+    """An `Op` that represents an observed random variable.
+
+    This `Op` establishes an observation relationship between a random
+    variable and a specific value.
+    """
+
+    default_output = 0
+    view_map = {0: [1]}
+
+    def make_node(self, rv, val):
+        """Make an `Observed` random variable.
+
+        Parameters
+        ----------
+        rv: RandomVariable
+            The distribution from which `val` is assumed to be a sample value.
+        val: Variable
+            The observed value.
+        """
+        val = as_tensor_variable(val)
+
+        if rv is not None:
+            if not hasattr(rv, "type") or rv.type.convert_variable(val) is None:
+                raise TypeError(
+                    (
+                        "`rv` and `val` do not have compatible types:"
+                        f" rv={rv}, val={val}"
+                    )
+                )
+        else:
+            rv = NoneConst.clone()
+
+        inputs = [rv, val]
+
+        return Apply(self, inputs, [val.type()])
+
+    def perform(self, node, inputs, out):
+        out[0][0] = inputs[1]
+
+    def grad(self, inputs, outputs):
+        return [
+            theano.gradient.grad_undefined(
+                self, k, inp, "No gradient defined for random variables"
+            )
+            for k, inp in enumerate(inputs)
+        ]
+
+
+observed = Observed()

theano/tensor/random/opt.py

+from theano.compile import optdb
+from theano.gof.opt import local_optimizer
+from theano.tensor.opt import in2out
+from theano.tensor.random.op import RandomVariable
+
+
+@local_optimizer([RandomVariable])
+def random_make_inplace(fgraph, node):
+    op = node.op
+
+    if isinstance(op, RandomVariable) and not op.inplace:
+
+        name, ndim_supp, ndims_params, dtype, _ = op._props()
+        new_op = type(op)(name, ndim_supp, ndims_params, dtype, True)
+        # rng, size, dtype, *dist_params = node.inputs
+
+        return new_op.make_node(*node.inputs).outputs
+
+    return False
+
+
+optdb.register(
+    "random_make_inplace",
+    in2out(random_make_inplace, ignore_newtrees=True),
+    99,
+    "fast_run",
+    "inplace",
+)

theano/tensor/random/type.py

+import sys
+
+import numpy as np
+
+import theano
+from theano.gof.type import Type
+
+
+class RandomStateType(Type):
+    """A Type wrapper for `numpy.random.RandomState`.
+
+    The reason this exists (and `Generic` doesn't suffice) is that
+    `RandomState` objects that would appear to be equal do not compare equal
+    with the `==` operator.  This `Type` exists to provide an equals function
+    that is used by `DebugMode`.
+
+    """
+
+    def __repr__(self):
+        return "RandomStateType"
+
+    @classmethod
+    def filter(cls, data, strict=False, allow_downcast=None):
+        if cls.is_valid_value(data):
+            return data
+        else:
+            raise TypeError()
+
+    @staticmethod
+    def is_valid_value(a):
+        return isinstance(a, np.random.RandomState)
+
+    @staticmethod
+    def values_eq(a, b):
+        sa = a.get_state(legacy=False)
+        sb = b.get_state(legacy=False)
+
+        def _eq(sa, sb):
+            for key in sa:
+                if isinstance(sa[key], dict):
+                    if not _eq(sa[key], sb[key]):
+                        return False
+                elif isinstance(sa[key], np.ndarray):
+                    if not np.array_equal(sa[key], sb[key]):
+                        return False
+                else:
+                    if sa[key] != sb[key]:
+                        return False
+
+            return True
+
+        return _eq(sa, sb)
+
+    @staticmethod
+    def get_shape_info(obj):
+        return obj.get_value(borrow=True)
+
+    @staticmethod
+    def get_size(shape_info):
+        return sys.getsizeof(shape_info.get_state(legacy=False))
+
+    @staticmethod
+    def may_share_memory(a, b):
+        return a._bit_generator is b._bit_generator
+
+
+# Register `RandomStateType`'s C code for `ViewOp`.
+theano.compile.register_view_op_c_code(
+    RandomStateType,
+    """
+    Py_XDECREF(%(oname)s);
+    %(oname)s = %(iname)s;
+    Py_XINCREF(%(oname)s);
+    """,
+    1,
+)
+
+random_state_type = RandomStateType()

theano/tensor/random/utils.py

+from functools import wraps
+from itertools import zip_longest
+
+import numpy as np
+
+from theano.compile.sharedvalue import shared
+from theano.gof.graph import Variable
+from theano.tensor.basic import maximum
+from theano.tensor.extra_ops import broadcast_to
+
+
+def params_broadcast_shapes(param_shapes, ndims_params, use_theano=True):
+    """Broadcast parameters that have different dimensions.
+
+    Parameters
+    ==========
+    param_shapes : list of ndarray or Variable
+        The shapes of each parameters to broadcast.
+    ndims_params : list of int
+        The expected number of dimensions for each element in `params`.
+    use_theano : bool
+        If ``True``, use Theano `Op`; otherwise, use NumPy.
+
+    Returns
+    =======
+    bcast_shapes : list of ndarray
+        The broadcasted values of `params`.
+    """
+    max_fn = maximum if use_theano else max
+
+    rev_extra_dims = []
+    for ndim_param, param_shape in zip(ndims_params, param_shapes):
+        # We need this in order to use `len`
+        param_shape = tuple(param_shape)
+        extras = tuple(param_shape[: (len(param_shape) - ndim_param)])
+        rev_extra_dims = [
+            max_fn(a, b)
+            for a, b in zip_longest(reversed(extras), rev_extra_dims, fillvalue=1)
+        ]
+
+    extra_dims = tuple(reversed(rev_extra_dims))
+
+    bcast_shapes = [
+        (extra_dims + tuple(param_shape)[-ndim_param:])
+        if ndim_param > 0
+        else extra_dims
+        for ndim_param, param_shape in zip(ndims_params, param_shapes)
+    ]
+
+    return bcast_shapes
+
+
+def broadcast_params(params, ndims_params):
+    """Broadcast parameters that have different dimensions.
+
+    >>> ndims_params = [1, 2]
+    >>> mean = np.array([1, 2, 3])
+    >>> cov = np.stack([np.eye(3), np.eye(3)])
+    >>> params = [mean, cov]
+    >>> res = broadcast_params(params, ndims_params)
+    [array([[1, 2, 3]]),
+    array([[[1., 0., 0.],
+             [0., 1., 0.],
+             [0., 0., 1.]],
+            [[1., 0., 0.],
+             [0., 1., 0.],
+             [0., 0., 1.]]])]
+
+    Parameters
+    ==========
+    params : list of ndarray
+        The parameters to broadcast.
+    ndims_params : list of int
+        The expected number of dimensions for each element in `params`.
+
+    Returns
+    =======
+    bcast_params : list of ndarray
+        The broadcasted values of `params`.
+    """
+    use_theano = False
+    param_shapes = []
+    for p in params:
+        param_shape = p.shape
+        use_theano |= isinstance(p, Variable)
+        param_shapes.append(param_shape)
+
+    shapes = params_broadcast_shapes(param_shapes, ndims_params, use_theano=use_theano)
+    broadcast_to_fn = broadcast_to if use_theano else np.broadcast_to
+
+    bcast_params = [
+        broadcast_to_fn(param, shape) for shape, param in zip(shapes, params)
+    ]
+
+    return bcast_params
+
+
+class RandomStream:
+    """Module component with similar interface to `numpy.random.RandomState`.
+
+    Attributes
+    ----------
+    seed: None or int
+        A default seed to initialize the RandomState instances after build.
+    state_updates: list
+        A list of pairs of the form `(input_r, output_r)`.  This will be
+        over-ridden by the module instance to contain stream generators.
+    default_instance_seed: int
+        Instance variable should take None or integer value. Used to seed the
+        random number generator that provides seeds for member streams.
+    gen_seedgen: numpy.random.RandomState
+        `RandomState` instance that `RandomStream.gen` uses to seed new
+        streams.
+
+    """
+
+    def __init__(self, seed=None, namespace=None):
+        if namespace is None:
+            from theano.tensor.random import basic  # pylint: disable=import-self
+
+            self.namespaces = [basic]
+        else:
+            self.namespaces = [namespace]
+
+        self.default_instance_seed = seed
+        self.state_updates = []
+        self.gen_seedgen = np.random.RandomState(seed)
+
+    def __getattr__(self, obj):
+
+        ns_obj = next(
+            (getattr(ns, obj) for ns in self.namespaces if hasattr(ns, obj)), None
+        )
+
+        if ns_obj is None:
+            raise AttributeError("No attribute {}.".format(obj))
+
+        from theano.tensor.random.op import RandomVariable
+
+        if isinstance(ns_obj, RandomVariable):
+
+            @wraps(ns_obj)
+            def meta_obj(*args, **kwargs):
+                return self.gen(ns_obj, *args, **kwargs)
+
+        else:
+            raise AttributeError("No attribute {}.".format(obj))
+
+        setattr(self, obj, meta_obj)
+        return getattr(self, obj)
+
+    def updates(self):
+        return list(self.state_updates)
+
+    def seed(self, seed=None):
+        """
+        Re-initialize each random stream.
+
+        Parameters
+        ----------
+        seed : None or integer in range 0 to 2**30
+            Each random stream will be assigned a unique state that depends
+            deterministically on this value.
+
+        Returns
+        -------
+        None
+
+        """
+        if seed is None:
+            seed = self.default_instance_seed
+
+        self.gen_seedgen.seed(seed)
+
+        for old_r, new_r in self.state_updates:
+            old_r_seed = self.gen_seedgen.randint(2 ** 30)
+            old_r.set_value(np.random.RandomState(int(old_r_seed)), borrow=True)
+
+    def gen(self, op, *args, **kwargs):
+        """Create a new random stream in this container.
+
+        Parameters
+        ----------
+        op : RandomVariable
+            A `RandomVariable` instance
+        args
+            Positional arguments passed to `op`.
+        kwargs
+            Keyword arguments passed to `op`.
+
+        Returns
+        -------
+        TensorVariable
+            The symbolic random draw part of op()'s return value.
+            This function stores the updated `RandomStateType` variable
+            for use at `build` time.
+
+        """
+        if "rng" in kwargs:
+            raise TypeError(
+                "The rng option cannot be used with a variate in a RandomStream"
+            )
+
+        # Generate a new random state
+        seed = int(self.gen_seedgen.randint(2 ** 30))
+        random_state_variable = shared(np.random.RandomState(seed))
+
+        # Distinguish it from other shared variables (why?)
+        random_state_variable.tag.is_rng = True
+
+        # Generate the sample
+        out = op(*args, **kwargs, rng=random_state_variable)
+        out.rng = random_state_variable
+
+        # Update the tracked states
+        new_r = out.owner.outputs[0]
+        out.update = (random_state_variable, new_r)
+        self.state_updates.append(out.update)
+
+        random_state_variable.default_update = new_r
+
+        return out

theano/tensor/random/var.py

+import copy
+
+import numpy as np
+
+from theano.compile.sharedvalue import SharedVariable, shared_constructor
+from theano.tensor.random.type import random_state_type
+
+
+class RandomStateSharedVariable(SharedVariable):
+    def __str__(self):
+        return "RandomStateSharedVariable({})".format(repr(self.container))
+
+
+@shared_constructor
+def randomstate_constructor(
+    value, name=None, strict=False, allow_downcast=None, borrow=False
+):
+    """
+    SharedVariable Constructor for RandomState.
+
+    """
+    if not isinstance(value, np.random.RandomState):
+        raise TypeError
+    if not borrow:
+        value = copy.deepcopy(value)
+    return RandomStateSharedVariable(
+        type=random_state_type,
+        value=value,
+        name=name,
+        strict=strict,
+        allow_downcast=allow_downcast,
+    )

theano/tensor/raw_random.py

-"""Define random number Type (`RandomStateType`) and Op (`RandomFunction`)."""
-
-from copy import copy
-from functools import reduce
-from warnings import warn
-
-import numpy as np
-
-import theano
-from theano import gof, tensor
-from theano.compile import optdb
-from theano.tensor import opt
-
-
-__docformat__ = "restructuredtext en"
-
-
-class RandomStateType(gof.Type):
-    """
-    A Type wrapper for numpy.random.RandomState.
-
-    The reason this exists (and `Generic` doesn't suffice) is that
-    RandomState objects that would appear to be equal do not compare
-    equal with the '==' operator.  This Type exists to provide an equals
-    function that is used by DebugMode.
-
-    """
-
-    def __str__(self):
-        return "RandomStateType"
-
-    def filter(self, data, strict=False, allow_downcast=None):
-        if self.is_valid_value(data):
-            return data
-        else:
-            raise TypeError()
-
-    def is_valid_value(self, a):
-        return type(a) == np.random.RandomState
-
-    def values_eq(self, a, b):
-        sa = a.get_state()
-        sb = b.get_state()
-        # Should always be the string 'MT19937'
-        if sa[0] != sb[0]:
-            return False
-        # 1-D array of 624 unsigned integer keys
-        if not np.all(sa[1] == sb[1]):
-            return False
-        # integer "pos" representing the position in the array
-        if sa[2] != sb[2]:
-            return False
-        # integer "has_gauss"
-        if sa[3] != sb[3]:
-            return False
-        # float "cached_gaussian".
-        # /!\ It is not initialized if has_gauss == 0
-        if sa[3] != 0:
-            if sa[4] != sb[4]:
-                return False
-        return True
-
-    def get_shape_info(self, obj):
-        return None
-
-    def get_size(self, shape_info):
-        # The size is the data, that have constant size.
-        state = np.random.RandomState().get_state()
-        size = 0
-        for elem in state:
-            if isinstance(elem, str):
-                size += len(elem)
-            elif isinstance(elem, np.ndarray):
-                size += elem.size * elem.itemsize
-            elif isinstance(elem, int):
-                size += np.dtype("int").itemsize
-            elif isinstance(elem, float):
-                size += np.dtype("float").itemsize
-            else:
-                raise NotImplementedError()
-        return size
-
-    @staticmethod
-    def may_share_memory(a, b):
-        return a is b
-
-
-# Register RandomStateType's C code for ViewOp.
-theano.compile.register_view_op_c_code(
-    RandomStateType,
-    """
-    Py_XDECREF(%(oname)s);
-    %(oname)s = %(iname)s;
-    Py_XINCREF(%(oname)s);
-    """,
-    1,
-)
-
-random_state_type = RandomStateType()
-
-
-class RandomFunction(gof.Op):
-    """
-    Op that draws random numbers from a numpy.random.RandomState object.
-
-    Parameters
-    ----------
-    fn : string or function reference
-        A member function of numpy.random.RandomState. A string will
-        be interpreted as the name of a member function of
-        numpy.random.RandomState.
-        Technically, any function with a signature like the ones in
-        numpy.random.RandomState will do. This function must accept
-        the shape (sometimes called size) of the output as the last
-        positional argument.
-    outtype
-        The theano Type of the output.
-    args
-        A list of default arguments for the function
-        kwargs
-        If the 'inplace' key is there, its value will be used to
-        determine if the op operates inplace or not.
-        If the 'ndim_added' key is there, its value indicates how
-        many more dimensions this op will add to the output, in
-        addition to the shape's dimensions (used in multinomial and
-        permutation).
-
-    """
-
-    __props__ = ("fn", "outtype", "inplace", "ndim_added")
-
-    def __init__(self, fn, outtype, inplace=False, ndim_added=0):
-        self.__setstate__([fn, outtype, inplace, ndim_added])
-
-    def __getstate__(self):
-        d = dict(self.__dict__)
-        del d["exec_fn"]
-        if "destroy_map" in d:
-            del d["destroy_map"]
-        return d
-
-    def __setstate__(self, dct):
-        if isinstance(dct, dict):
-            state = [dct["fn"], dct["outtype"], dct["inplace"], dct["ndim_added"]]
-            self.__dict__.update(dct)
-        else:
-            state = dct
-        fn, outtype, inplace, ndim_added = state
-        self.fn = fn
-        if isinstance(fn, str):
-            self.exec_fn = getattr(np.random.RandomState, fn)
-        else:
-            self.exec_fn = fn
-        self.outtype = outtype
-        self.inplace = inplace
-        if self.inplace:
-            self.destroy_map = {0: [0]}
-        self.ndim_added = ndim_added
-
-    def __str__(self):
-        return "RandomFunction{%s}" % self.exec_fn.__name__
-
-    def make_node(self, r, shape, *args):
-        """
-        Parameters
-        ----------
-        r
-            A numpy.random.RandomState instance, or a Variable of Type
-            RandomStateType that will contain a RandomState instance.
-        shape
-            An lvector with a shape defining how many samples
-            to draw.  In the case of scalar distributions, it is the shape
-            of the tensor output by this Op.  In that case, at runtime, the
-            value associated with this lvector must have a length equal to
-            the number of dimensions promised by `self.outtype`.
-            In a more general case, the number of output dimensions,
-            len(self.outtype), is equal to len(shape)+self.ndim_added.
-            The special case where len(shape) == 0 means that the smallest
-            shape compatible with the argument's shape will be used.
-        args
-            The values associated with these variables will be passed to the
-            RandomState function during perform as extra "*args"-style
-            arguments. These should be castable to variables of Type TensorType.
-
-        Returns
-        -------
-        Apply
-            Apply with two outputs. The first output is a gof.generic Variable
-            from which to draw further random numbers.
-            The second output is the outtype() instance holding the random
-            draw.
-
-        """
-        shape_ = tensor.as_tensor_variable(shape, ndim=1)
-        if shape == ():
-            shape = shape_.astype("int64")
-        else:
-            shape = shape_
-        assert shape.type.ndim == 1
-        assert (shape.type.dtype == "int64") or (shape.type.dtype == "int32")
-        if not isinstance(r.type, RandomStateType):
-            warn("RandomState instances should be in RandomStateType")
-
-        # the following doesn't work because we want to ignore the
-        # broadcastable flags in shape.type
-        # assert shape.type == tensor.lvector
-
-        # convert args to TensorType instances
-        # and append enough None's to match the length of self.args
-        args = list(map(tensor.as_tensor_variable, args))
-
-        return gof.Apply(self, [r, shape] + args, [r.type(), self.outtype()])
-
-    def infer_shape(self, fgraph, node, i_shapes):
-        r, shp = node.inputs[0:2]
-
-        # if shp is a constant array of len 0, then it means 'automatic shape'
-        unknown_shape = len(getattr(shp, "data", [0, 1, 2])) == 0
-
-        # if ndim_added == 0 and shape != () then shape
-        if self.ndim_added == 0 and not unknown_shape:
-            sample_shp = shp
-        else:
-            # if shape == () then it will depend on args
-            # if ndim_added != 0 and shape != () then it will depend on args
-            # Use the default infer_shape implementation.
-            raise tensor.ShapeError()
-
-        return [None, [sample_shp[i] for i in range(node.outputs[1].ndim)]]
-
-    def perform(self, node, inputs, out_):
-        rout, out = out_
-        # Use self.fn to draw shape worth of random numbers.
-        # Numbers are drawn from r if self.inplace is True, and from a
-        # copy of r if self.inplace is False
-        r, shape, args = inputs[0], inputs[1], inputs[2:]
-        assert type(r) == np.random.RandomState, (type(r), r)
-
-        # If shape == [], that means no shape is enforced, and numpy is
-        # trusted to draw the appropriate number of samples, numpy uses
-        # shape "None" to represent that. Else, numpy expects a tuple.
-        # TODO: compute the appropriate shape, and pass it to numpy.
-        if len(shape) == 0:
-            shape = None
-        else:
-            shape = tuple(shape)
-
-        if shape is not None and self.outtype.ndim != len(shape) + self.ndim_added:
-            raise ValueError(
-                f"Shape mismatch: self.outtype.ndim ({int(self.outtype.ndim)}) !="
-                f" len(shape) ({len(shape)}) + self.ndim_added ({self.ndim_added})"
-            )
-        if not self.inplace:
-            r = copy(r)
-        rout[0] = r
-        rval = self.exec_fn(r, *(args + [shape]))
-        if (
-            not isinstance(rval, np.ndarray)
-            or str(rval.dtype) != node.outputs[1].type.dtype
-        ):
-            rval = theano._asarray(rval, dtype=node.outputs[1].type.dtype)
-
-        # When shape is None, numpy has a tendency to unexpectedly
-        # return a scalar instead of a higher-dimension array containing
-        # only one element. This value should be reshaped
-        if shape is None and rval.ndim == 0 and self.outtype.ndim > 0:
-            rval = rval.reshape([1] * self.outtype.ndim)
-
-        if len(rval.shape) != self.outtype.ndim:
-            raise ValueError(
-                'Shape mismatch: "out" should have dimension %i,'
-                ' but the value produced by "perform" has'
-                " dimension %i" % (self.outtype.ndim, len(rval.shape))
-            )
-
-        # Check the output has the right shape
-        if shape is not None:
-            if self.ndim_added == 0 and shape != rval.shape:
-                raise ValueError(
-                    'Shape mismatch: "out" should have shape %s, but the'
-                    ' value produced by "perform" has shape %s' % (shape, rval.shape)
-                )
-            elif self.ndim_added > 0 and shape != rval.shape[: -self.ndim_added]:
-                raise ValueError(
-                    'Shape mismatch: "out" should have shape starting with'
-                    " %s (plus %i extra dimensions), but the value produced"
-                    ' by "perform" has shape %s' % (shape, self.ndim_added, rval.shape)
-                )
-
-        out[0] = rval
-
-    def grad(self, inputs, outputs):
-        return [
-            theano.gradient.grad_undefined(
-                self, k, inp, "No gradient defined through raw random numbers op"
-            )
-            for k, inp in enumerate(inputs)
-        ]
-
-    def R_op(self, inputs, eval_points):
-        return [None for i in eval_points]
-
-
-def _infer_ndim_bcast(ndim, shape, *args):
-    """
-    Infer the number of dimensions from the shape or the other arguments.
-
-    Returns
-    -------
-    (int, variable, tuple) triple, where the variable is an integer vector,
-    and the tuple contains Booleans
-        The first element returned is the inferred number of dimensions.
-        The second element is the shape inferred (combining symbolic and
-        constant informations from shape and args).
-        The third element is a broadcasting pattern corresponding to that shape.
-
-    """
-
-    # Find the minimum value of ndim required by the *args
-    if args:
-        args_ndim = max(arg.ndim for arg in args)
-    else:
-        args_ndim = 0
-
-    if isinstance(shape, (tuple, list)):
-        # there is a convention that -1 means the corresponding shape of a
-        # potentially-broadcasted symbolic arg
-        #
-        # This case combines together symbolic and non-symbolic shape
-        # information
-        shape_ndim = len(shape)
-        if ndim is None:
-            ndim = shape_ndim
-        else:
-            if shape_ndim != ndim:
-                raise ValueError(
-                    "ndim should be equal to len(shape), but\n",
-                    f"ndim = {ndim}, len(shape) = {shape_ndim}, shape = {shape}",
-                )
-
-        bcast = []
-        pre_v_shape = []
-        for i, s in enumerate(shape):
-            if hasattr(s, "type"):  # s is symbolic
-                bcast.append(False)  # todo - introspect further
-                pre_v_shape.append(s)
-            else:
-                if s >= 0:
-                    pre_v_shape.append(tensor.as_tensor_variable(s))
-                    bcast.append(s == 1)
-                elif s == -1:
-                    n_a_i = 0
-                    for a in args:
-                        # ndim: _   _   _   _   _   _
-                        # ashp:         s0  s1  s2  s3
-                        #           i
-                        if i >= ndim - a.ndim:
-                            n_a_i += 1
-                            a_i = i + a.ndim - ndim
-                            if not a.broadcastable[a_i]:
-                                pre_v_shape.append(a.shape[a_i])
-                                bcast.append(False)
-                                break
-                    else:
-                        if n_a_i == 0:
-                            raise ValueError(
-                                "Auto-shape of -1 must overlap"
-                                "with the shape of one of the broadcastable"
-                                "inputs"
-                            )
-                        else:
-                            pre_v_shape.append(tensor.as_tensor_variable(1))
-                            bcast.append(True)
-                else:
-                    ValueError("negative shape", s)
-        # post-condition: shape may still contain both symbolic and
-        # non-symbolic things
-        if len(pre_v_shape) == 0:
-            v_shape = tensor.constant([], dtype="int64")
-        else:
-            v_shape = tensor.stack(pre_v_shape)
-
-    elif shape is None:
-        # The number of drawn samples will be determined automatically,
-        # but we need to know ndim
-        if not args:
-            raise TypeError(
-                "_infer_ndim_bcast cannot infer shape without" " either shape or args"
-            )
-        template = reduce(lambda a, b: a + b, args)
-        v_shape = template.shape
-        bcast = template.broadcastable
-        ndim = template.ndim
-    else:
-        v_shape = tensor.as_tensor_variable(shape)
-        if v_shape.ndim != 1:
-            raise TypeError(
-                f"shape must be a vector or list of scalar, got '{v_shape}'"
-            )
-
-        if ndim is None:
-            ndim = tensor.get_vector_length(v_shape)
-        bcast = [False] * ndim
-
-    if v_shape.ndim != 1:
-        raise TypeError(f"shape must be a vector or list of scalar, got '{v_shape}'")
-
-    if v_shape.dtype not in theano.tensor.integer_dtypes:
-        raise TypeError("shape must be an integer vector or list", v_shape.dtype)
-
-    if args_ndim > ndim:
-        raise ValueError(
-            "ndim should be at least as big as required by args value",
-            (ndim, args_ndim),
-            args,
-        )
-
-    assert ndim == len(bcast)
-    return ndim, tensor.cast(v_shape, "int64"), tuple(bcast)
-
-
-def _generate_broadcasting_indices(out_shape, *shapes):
-    """
-    Return indices over each shape that broadcast them to match out_shape.
-
-    The first returned list is equivalent to numpy.ndindex(out_shape),
-    the other returned lists are indices corresponding to the other shapes,
-    such that looping over these indices produce tensors of shape out_shape.
-    In particular, the indices over broadcasted dimensions should all be 0.
-
-    The shapes should have the same length as out_shape. If they are longer,
-    the right-most dimensions are ignored.
-
-    """
-    all_shapes = (out_shape,) + shapes
-    # Will contain the return value: a list of indices for each argument
-    ret_indices = [[()] for shape in all_shapes]
-
-    for dim in range(len(out_shape)):
-        # Temporary list to generate the indices
-        _ret_indices = [[] for shape in all_shapes]
-
-        out_range = list(range(out_shape[dim]))
-
-        # Verify the shapes are compatible along that dimension
-        # and generate the appropriate range: out_range, or [0, ..., 0]
-        ranges = [out_range]
-        for shape in shapes:
-            if shape[dim] == out_shape[dim]:
-                ranges.append(out_range)
-            elif shape[dim] == 1:  # broadcast
-                ranges.append([0] * out_shape[dim])
-            else:
-                raise ValueError(
-                    "shape[%i] (%i) should be equal to out_shape[%i] (%i) or"
-                    " to 1" % (dim, shape[dim], dim, out_shape[dim]),
-                    shape,
-                    out_shape,
-                    shapes,
-                )
-
-        for prev_index in zip(*ret_indices):
-            for dim_index in zip(*ranges):
-                for i in range(len(all_shapes)):
-                    _ret_indices[i].append(prev_index[i] + (dim_index[i],))
-        ret_indices = _ret_indices
-
-    return ret_indices
-
-
-def uniform(random_state, size=None, low=0.0, high=1.0, ndim=None, dtype=None):
-    """
-    Sample from a uniform distribution between low and high.
-
-    If the size argument is ambiguous on the number of dimensions, ndim
-    may be a plain integer to supplement the missing information.
-
-    If size is None, the output shape will be determined by the shapes
-    of low and high.
-
-    If dtype is not specified, it will be inferred from the dtype of
-    low and high, but will be at least as precise as floatX.
-
-    """
-    low = tensor.as_tensor_variable(low)
-    high = tensor.as_tensor_variable(high)
-    if dtype is None:
-        dtype = tensor.scal.upcast(theano.config.floatX, low.dtype, high.dtype)
-    ndim, size, bcast = _infer_ndim_bcast(ndim, size, low, high)
-    op = RandomFunction("uniform", tensor.TensorType(dtype=dtype, broadcastable=bcast))
-    return op(random_state, size, low, high)
-
-
-def normal(random_state, size=None, avg=0.0, std=1.0, ndim=None, dtype=None):
-    """
-    Sample from a normal distribution centered on avg with
-    the specified standard deviation (std).
-
-    If the size argument is ambiguous on the number of dimensions, ndim
-    may be a plain integer to supplement the missing information.
-
-    If size is None, the output shape will be determined by the shapes
-    of avg and std.
-
-    If dtype is not specified, it will be inferred from the dtype of
-    avg and std, but will be at least as precise as floatX.
-
-    """
-    avg = tensor.as_tensor_variable(avg)
-    std = tensor.as_tensor_variable(std)
-    if dtype is None:
-        dtype = tensor.scal.upcast(theano.config.floatX, avg.dtype, std.dtype)
-    ndim, size, bcast = _infer_ndim_bcast(ndim, size, avg, std)
-    op = RandomFunction("normal", tensor.TensorType(dtype=dtype, broadcastable=bcast))
-    return op(random_state, size, avg, std)
-
-
-def binomial(random_state, size=None, n=1, p=0.5, ndim=None, dtype="int64", prob=None):
-    """
-    Sample n times with probability of success prob for each trial,
-    return the number of successes.
-
-    If the size argument is ambiguous on the number of dimensions, ndim
-    may be a plain integer to supplement the missing information.
-
-    If size is None, the output shape will be determined by the shapes
-    of n and prob.
-
-    """
-    if prob is not None:
-        p = prob
-        warn(
-            "The parameter prob to the binomal fct have been renamed to p to have the same name as np.",
-            category=DeprecationWarning,
-        )
-    n = tensor.as_tensor_variable(n)
-    p = tensor.as_tensor_variable(p)
-    ndim, size, bcast = _infer_ndim_bcast(ndim, size, n, p)
-    if n.dtype == "int64":
-        try:
-            np.random.binomial(
-                n=np.asarray([2, 3, 4], dtype="int64"),
-                p=np.asarray([0.1, 0.2, 0.3], dtype="float64"),
-            )
-        except TypeError:
-            # THIS WORKS AROUND A NUMPY BUG on 32bit machine
-            n = tensor.cast(n, "int32")
-    op = RandomFunction(
-        "binomial", tensor.TensorType(dtype=dtype, broadcastable=(False,) * ndim)
-    )
-    return op(random_state, size, n, p)
-
-
-def random_integers_helper(random_state, low, high, size):
-    """
-    Helper function to draw random integers.
-
-    This is a generalization of numpy.random.random_integers to the case where
-    low and high are tensors.
-
-    Since random_integers is deprecated it calls randint() instead.
-
-    """
-    # Figure out the output shape
-    if size is not None:
-        out_ndim = len(size)
-    else:
-        out_ndim = max(low.ndim, high.ndim)
-    # broadcast low and high to out_ndim dimensions
-    if low.ndim > out_ndim:
-        raise ValueError(
-            "low.ndim (%i) should not be larger than len(size) (%i)"
-            % (low.ndim, out_ndim),
-            low,
-            size,
-        )
-    if low.ndim < out_ndim:
-        low = low.reshape((1,) * (out_ndim - low.ndim) + low.shape)
-
-    if high.ndim > out_ndim:
-        raise ValueError(
-            "high.ndim (%i) should not be larger than len(size) (%i)"
-            % (high.ndim, out_ndim),
-            high,
-            size,
-        )
-    if high.ndim < out_ndim:
-        high = high.reshape((1,) * (out_ndim - high.ndim) + high.shape)
-
-    if size is not None:
-        out_size = tuple(size)
-    else:
-        out_size = ()
-        for dim in range(out_ndim):
-            dim_len = max(low.shape[dim], high.shape[dim])
-            out_size = out_size + (dim_len,)
-
-    # Build the indices over which to loop
-    out = np.ndarray(out_size)
-    broadcast_ind = _generate_broadcasting_indices(out_size, low.shape, high.shape)
-    # Iterate over these indices, drawing one sample at a time from numpy
-    for oi, li, hi in zip(*broadcast_ind):
-        out[oi] = random_state.randint(low=low[li], high=high[hi] + 1)
-
-    return out
-
-
-def random_integers(random_state, size=None, low=0, high=1, ndim=None, dtype="int64"):
-    """
-    Sample a random integer between low and high, both inclusive.
-
-    If the size argument is ambiguous on the number of dimensions, ndim
-    may be a plain integer to supplement the missing information.
-
-    If size is None, the output shape will be determined by the shapes
-    of low and high.
-
-    """
-    low = tensor.as_tensor_variable(low)
-    high = tensor.as_tensor_variable(high)
-    ndim, size, bcast = _infer_ndim_bcast(ndim, size, low, high)
-    op = RandomFunction(
-        random_integers_helper, tensor.TensorType(dtype=dtype, broadcastable=bcast)
-    )
-    return op(random_state, size, low, high)
-
-
-def choice_helper(random_state, a, replace, p, size):
-    """
-    Helper function to draw random numbers using numpy's choice function.
-
-    This is a generalization of numpy.random.choice that coerces
-    `replace` to a bool and replaces `p` with None when p is a vector
-    of 0 elements.
-
-    """
-    if a.ndim > 1:
-        raise ValueError("a.ndim (%i) must be 0 or 1" % a.ndim)
-    if p.ndim == 1:
-        if p.size == 0:
-            p = None
-    else:
-        raise ValueError("p.ndim (%i) must be 1" % p.ndim)
-    replace = bool(replace)
-    return random_state.choice(a, size, replace, p)
-
-
-def choice(
-    random_state, size=None, a=2, replace=True, p=None, ndim=None, dtype="int64"
-):
-    """
-    Choose values from `a` with or without replacement. `a` can be a 1-D array
-    or a positive scalar. If `a` is a scalar, the samples are drawn from the
-    range 0,...,a-1.
-
-    If the size argument is ambiguous on the number of dimensions, ndim
-    may be a plain integer to supplement the missing information.
-
-    If size is None, a scalar will be returned.
-
-    """
-    a = tensor.as_tensor_variable(a)
-    if isinstance(replace, bool):
-        replace = tensor.constant(replace, dtype="int8")
-    else:
-        replace = tensor.as_tensor_variable(replace)
-    # encode p=None as an empty vector
-    p = tensor.as_tensor_variable(p or [])
-    ndim, size, bcast = _infer_ndim_bcast(ndim, size)
-    op = RandomFunction(
-        choice_helper, tensor.TensorType(dtype=dtype, broadcastable=bcast)
-    )
-    return op(random_state, size, a, replace, p)
-
-
-def poisson(random_state, size=None, lam=1.0, ndim=None, dtype="int64"):
-    """
-    Draw samples from a Poisson distribution.
-
-    The Poisson distribution is the limit of the Binomial distribution for
-    large N.
-
-    Parameters
-    ----------
-    lam : float or ndarray-like of the same shape as size parameter
-        Expectation of interval, should be >= 0.
-    size: int or tuple of ints, optional
-        Output shape. If the given shape is, e.g., (m, n, k), then m * n * k
-        samples are drawn.
-    dtype
-        The dtype of the return value (which will represent counts).
-
-    size or ndim must be given.
-
-    """
-    lam = tensor.as_tensor_variable(lam)
-
-    ndim, size, bcast = _infer_ndim_bcast(ndim, size)
-
-    op = RandomFunction("poisson", tensor.TensorType(dtype=dtype, broadcastable=bcast))
-    return op(random_state, size, lam)
-
-
-def permutation_helper(random_state, n, shape):
-    """
-    Helper function to generate permutations from integers.
-
-    permutation_helper(random_state, n, (1,)) will generate a permutation of
-    integers 0..n-1.
-    In general, it will generate as many such permutation as required by shape.
-    For instance, if shape=(p,q), p*q permutations will be generated, and the
-    output shape will be (p,q,n), because each permutation is of size n.
-
-    If you wish to perform a permutation of the elements of an existing vector,
-    see shuffle_row_elements.
-
-    This is a generalization of numpy.random.permutation to tensors.
-    Otherwise it behaves the same.
-
-    """
-    # n should be a 0-dimension array
-    assert n.shape == ()
-    # Note that it is important to convert `n` into an integer, because if it
-    # is a long, the numpy permutation function will crash on Windows.
-    n = int(n.item())
-
-    if shape is None:
-        # Draw only one permutation, equivalent to shape = ()
-        shape = ()
-    out_shape = list(shape)
-    out_shape.append(n)
-    out = np.empty(out_shape, int)
-    for i in np.ndindex(*shape):
-        out[i] = random_state.permutation(n)
-
-    # print 'RETURNING', out.shape
-    return out
-
-
-def permutation(random_state, size=None, n=1, ndim=None, dtype="int64"):
-    """
-    Return permutations of the integers between 0 and n-1.
-
-    Returns them as many times as required by size. For instance, if size=(p,q),
-    p*q permutations will be generated, and the output shape will be (p,q,n),
-    because each permutation is of size n.
-
-    Theano tries to infer the number of dimensions from the length of
-    the size argument and the shape of n, but you may always specify it
-    with the `ndim` parameter.
-
-    Notes
-    -----
-    Note that the output will then be of dimension ndim+1.
-
-    """
-    if size is None or size == ():
-        if not (ndim is None or ndim == 1):
-            raise TypeError(
-                "You asked for just one permutation but asked for more then 1 dimensions."
-            )
-        ndim = 1
-        size = ()
-        bcast = ()
-    else:
-        ndim, size, bcast = _infer_ndim_bcast(ndim, size)
-    # print "NDIM", ndim, size
-    op = RandomFunction(
-        permutation_helper,
-        tensor.TensorType(dtype=dtype, broadcastable=bcast + (False,)),
-        ndim_added=1,
-    )
-    return op(random_state, size, n)
-
-
-def multinomial_helper(random_state, n, pvals, size):
-    """
-    Helper function drawing from multinomial distributions.
-
-    This is a generalization of numpy.random.multinomial to the case where
-    n and pvals are tensors.
-
-    """
-    # Figure out the shape if it's None
-    # Note: the output ndim will be ndim+1, because the multinomial
-    # adds a dimension. The length of that dimension is pvals.shape[-1].
-    if size is not None:
-        ndim = len(size)
-    else:
-        ndim = max(n.ndim, pvals.ndim - 1)
-
-    # broadcast n to ndim dimensions and pvals to ndim+1
-    if n.ndim > ndim:
-        raise ValueError(
-            "n.ndim (%i) should not be larger than len(size) (%i)" % (n.ndim, ndim),
-            n,
-            size,
-        )
-    if n.ndim < ndim:
-        n = n.reshape((1,) * (ndim - n.ndim) + n.shape)
-
-    if pvals.ndim - 1 > ndim:
-        raise ValueError(
-            "pvals.ndim-1 (%i) should not be larger than len(size) (%i)"
-            % (pvals.ndim - 1, ndim),
-            pvals,
-            size,
-        )
-    if pvals.ndim - 1 < ndim:
-        pvals = pvals.reshape((1,) * (ndim - pvals.ndim + 1) + pvals.shape)
-
-    if size is not None:
-        size = tuple(size)
-    else:
-        size = ()
-        for dim in range(ndim):
-            dim_len = max(n.shape[dim], pvals.shape[dim])
-            size = size + (dim_len,)
-    out_size = size + (pvals.shape[-1],)
-
-    # Build the indices over which to loop
-    # Note that here, the rows (inner-most 1D subtensors) of pvals and out
-    # are indexed, not their individual elements
-    out = np.ndarray(out_size)
-    broadcast_ind = _generate_broadcasting_indices(size, n.shape, pvals.shape[:-1])
-    # Iterate over these indices, drawing from one multinomial at a
-    # time from numpy
-    assert pvals.min() >= 0
-    for mi, ni, pi in zip(*broadcast_ind):
-        pvi = pvals[pi]
-
-        # This might someday be fixed upstream
-        # Currently numpy raises an exception in this method if the sum
-        # of probabilities meets or exceeds 1.0.
-        # In  perfect arithmetic this would be correct, but in float32 or
-        # float64 it is too strict.
-        pisum = np.sum(pvi)
-        if 1.0 < pisum < 1.0 + 1e-5:  # correct if we went a little over
-            # because mtrand.pyx has a ValueError that will trigger if
-            # sum(pvals[:-1]) > 1.0
-            pvi = pvi * (1.0 - 5e-5)
-            # pvi = pvi * .9
-            pisum = np.sum(pvi)
-        elif pvi[-1] < 5e-5:  # will this even work?
-            pvi = pvi * (1.0 - 5e-5)
-            pisum = np.sum(pvi)
-        assert pisum <= 1.0, pisum
-        out[mi] = random_state.multinomial(n=n[ni], pvals=pvi.astype("float64"))
-    return out
-
-
-def multinomial(random_state, size=None, n=1, pvals=None, ndim=None, dtype="int64"):
-    """
-    Sample from one or more multinomial distributions defined by
-    one-dimensional slices in pvals.
-
-    Parameters
-    ----------
-    pvals
-        A tensor of shape "nmulti+(L,)" describing each multinomial
-        distribution.  This tensor must have the property that
-        numpy.allclose(pvals.sum(axis=-1), 1) is true.
-    size
-        A vector of shape information for the output; this can also
-        specify the "nmulti" part of pvals' shape.  A -1 in the k'th position
-        from the right means to borrow the k'th position from the
-        right in nmulti. (See examples below.)
-        Default ``None`` means size=nmulti.
-    n
-        The number of experiments to simulate for each
-        multinomial. This can be a scalar, or tensor, it will be
-        broadcasted to have shape "nmulti".
-    dtype
-        The dtype of the return value (which will represent counts)
-
-    Returns
-    -------
-    tensor
-        Tensor of len(size)+1 dimensions, and shape[-1]==L, with
-        the specified ``dtype``, with the experiment counts. See
-        examples to understand the shape of the return value, which is
-        derived from both size and pvals.shape. In return value rval,
-        "numpy.allclose(rval.sum(axis=-1), n)" will be true.
-
-    Extended Summary
-    ----------------
-    For example, to simulate n experiments from each multinomial in a batch of
-    size B:
-
-        size=None, pvals.shape=(B,L) --> rval.shape=[B,L]
-
-        rval[i,j] is the count of possibility j in the i'th distribution (row)
-        in pvals.
-
-    Using size:
-
-        size=(1,-1), pvals.shape=(A,B,L)
-        --> rval.shape=[1,B,L], and requires that A==1.
-
-        rval[k,i,j] is the count of possibility j in the distribution specified
-        by pvals[k,i].
-
-    Using size for broadcasting of pvals:
-
-        size=(10, 1, -1), pvals.shape=(A, B, L)
-        --> rval.shape=[10,1,B,L], and requires that A==1.
-
-        rval[l,k,i,j] is the count of possibility j in the
-        distribution specified by pvals[k,i], in the l'th of 10
-        draws.
-
-    """
-    if pvals is None:
-        pvals = [0.5, 0.5]
-    n = tensor.as_tensor_variable(n)
-    pvals = tensor.as_tensor_variable(pvals)
-    # until ellipsis is implemented (argh)
-    tmp = pvals.T[0].T
-    ndim, size, bcast = _infer_ndim_bcast(ndim, size, n, tmp)
-    bcast = bcast + (pvals.type.broadcastable[-1],)
-    op = RandomFunction(
-        multinomial_helper,
-        tensor.TensorType(dtype=dtype, broadcastable=bcast),
-        ndim_added=1,
-    )
-    return op(random_state, size, n, pvals)
-
-
-@gof.local_optimizer([RandomFunction])
-def random_make_inplace(fgraph, node):
-    op = node.op
-    if isinstance(op, RandomFunction) and not op.inplace:
-        # Read op_fn from op.state, not from op.fn, since op.fn
-        # may not be picklable.
-        op_fn, op_outtype, op_inplace, op_ndim_added = op._props()
-        new_op = RandomFunction(
-            op_fn, op_outtype, inplace=True, ndim_added=op_ndim_added
-        )
-        return new_op.make_node(*node.inputs).outputs
-    return False
-
-
-optdb.register(
-    "random_make_inplace",
-    opt.in2out(random_make_inplace, ignore_newtrees=True),
-    99,
-    "fast_run",
-    "inplace",
-)
-
-
-class RandomStreamsBase:
-    def binomial(self, size=None, n=1, p=0.5, ndim=None, dtype="int64", prob=None):
-        """
-        Sample n times with probability of success p for each trial and
-        return the number of successes.
-
-        If the size argument is ambiguous on the number of dimensions,
-        ndim may be a plain integer to supplement the missing information.
-
-        """
-        if prob is not None:
-            p = prob
-            warn(
-                "The parameter prob to the binomal fct have been renamed to p to have the same name as numpy.",
-                category=DeprecationWarning,
-            )
-        return self.gen(binomial, size, n, p, ndim=ndim, dtype=dtype)
-
-    def uniform(self, size=None, low=0.0, high=1.0, ndim=None, dtype=None):
-        """
-        Sample a tensor of given size whose element from a uniform
-        distribution between low and high.
-
-        If the size argument is ambiguous on the number of dimensions,
-        ndim may be a plain integer to supplement the missing information.
-
-        """
-        return self.gen(uniform, size, low, high, ndim=ndim, dtype=dtype)
-
-    def normal(self, size=None, avg=0.0, std=1.0, ndim=None, dtype=None):
-        """
-        Sample from a normal distribution centered on avg with
-        the specified standard deviation (std).
-
-        If the size argument is ambiguous on the number of dimensions,
-        ndim may be a plain integer to supplement the missing information.
-
-        """
-        return self.gen(normal, size, avg, std, ndim=ndim, dtype=dtype)
-
-    def random_integers(self, size=None, low=0, high=1, ndim=None, dtype="int64"):
-        """
-        Sample a random integer between low and high, both inclusive.
-
-        If the size argument is ambiguous on the number of dimensions,
-        ndim may be a plain integer to supplement the missing information.
-
-        """
-        return self.gen(random_integers, size, low, high, ndim=ndim, dtype=dtype)
-
-    def choice(self, size=None, a=2, replace=True, p=None, ndim=None, dtype="int64"):
-        """
-        Choose values from `a` with or without replacement.
-
-        `a` can be a 1-D array or a positive scalar.
-        If `a` is a scalar, the samples are drawn from the range 0,...,a-1.
-
-        If the size argument is ambiguous on the number of dimensions,
-        ndim may be a plain integer to supplement the missing information.
-
-        """
-        return self.gen(choice, size, a, replace, p, ndim=ndim, dtype=dtype)
-
-    def poisson(self, size=None, lam=None, ndim=None, dtype="int64"):
-        """
-        Draw samples from a Poisson distribution.
-
-        The Poisson distribution is the limit of the Binomial distribution for
-        large N.
-
-        If the size argument is ambiguous on the number of dimensions,
-        ndim may be a plain integer to supplement the missing information.
-
-        """
-        return self.gen(poisson, size, lam, ndim=ndim, dtype=dtype)
-
-    def permutation(self, size=None, n=1, ndim=None, dtype="int64"):
-        """
-        Return permutations of the integers between 0 and n-1.
-
-        Returns them as many times as required by size. For instance,
-        if size=(p,q), p*q permutations will be generated,
-        and the output shape will be (p,q,n), because each
-        permutation is of size n.
-
-        Theano tries to infer the number of dimensions from the length
-        of the size argument and the shape of n, but you may always
-        specify it with the `ndim` parameter.
-
-        Notes
-        -----
-        Note that the output will then be of dimension ndim+1.
-
-        """
-        return self.gen(permutation, size, n, ndim=ndim, dtype=dtype)
-
-    def multinomial(self, size=None, n=1, pvals=None, ndim=None, dtype="int64"):
-        """
-        Sample n times from a multinomial distribution defined by
-        probabilities pvals, as many times as required by size. For
-        instance, if size=(p,q), p*q samples will be drawn, and the
-        output shape will be (p,q,len(pvals)).
-
-        Theano tries to infer the number of dimensions from the length
-        of the size argument and the shapes of n and pvals, but you may
-        always specify it with the `ndim` parameter.
-
-        Notes
-        -----
-        Note that the output will then be of dimension ndim+1.
-
-        """
-        if pvals is None:
-            pvals = [0.5, 0.5]
-        return self.gen(multinomial, size, n, pvals, ndim=ndim, dtype=dtype)
-
-    def shuffle_row_elements(self, input):
-        """
-        Return a variable with every row (rightmost index) shuffled.
-
-        This uses permutation random variable internally, available via
-        the ``.permutation`` attribute of the return value.
-
-        """
-        perm = self.permutation(
-            size=input.shape[:-1], n=input.shape[-1], ndim=input.ndim - 1
-        )
-        shuffled = tensor.permute_row_elements(input, perm)
-        shuffled.permutation = perm
-        return shuffled

theano/tensor/shared_randomstreams.py

-"""
-Define RandomStreams, providing random number variables for Theano
-graphs.
-
-"""
-
-
-import copy
-
-import numpy as np
-
-from theano.compile.sharedvalue import SharedVariable, shared, shared_constructor
-from theano.tensor import raw_random
-
-
-__docformat__ = "restructuredtext en"
-
-
-class RandomStateSharedVariable(SharedVariable):
-    pass
-
-
-@shared_constructor
-def randomstate_constructor(
-    value, name=None, strict=False, allow_downcast=None, borrow=False
-):
-    """
-    SharedVariable Constructor for RandomState.
-
-    """
-    if not isinstance(value, np.random.RandomState):
-        raise TypeError
-    if not borrow:
-        value = copy.deepcopy(value)
-    return RandomStateSharedVariable(
-        type=raw_random.random_state_type,
-        value=value,
-        name=name,
-        strict=strict,
-        allow_downcast=allow_downcast,
-    )
-
-
-class RandomStreams(raw_random.RandomStreamsBase):
-    """
-    Module component with similar interface to numpy.random
-    (numpy.random.RandomState)
-
-    Parameters
-    ----------
-    seed: None or int
-        A default seed to initialize the RandomState
-        instances after build.  See `RandomStreamsInstance.__init__`
-        for more details.
-
-    """
-
-    def updates(self):
-        return list(self.state_updates)
-
-    def __init__(self, seed=None):
-        super().__init__()
-        # A list of pairs of the form (input_r, output_r).  This will be
-        # over-ridden by the module instance to contain stream generators.
-        self.state_updates = []
-        # Instance variable should take None or integer value. Used to seed the
-        # random number generator that provides seeds for member streams.
-        self.default_instance_seed = seed
-        # numpy.RandomState instance that gen() uses to seed new streams.
-        self.gen_seedgen = np.random.RandomState(seed)
-
-    def seed(self, seed=None):
-        """
-        Re-initialize each random stream.
-
-        Parameters
-        ----------
-        seed : None or integer in range 0 to 2**30
-            Each random stream will be assigned a unique state that depends
-            deterministically on this value.
-
-        Returns
-        -------
-        None
-
-        """
-        if seed is None:
-            seed = self.default_instance_seed
-
-        seedgen = np.random.RandomState(seed)
-        for old_r, new_r in self.state_updates:
-            old_r_seed = seedgen.randint(2 ** 30)
-            old_r.set_value(np.random.RandomState(int(old_r_seed)), borrow=True)
-
-    def __getitem__(self, item):
-        """
-        Retrieve the numpy RandomState instance associated with a particular
-        stream.
-
-        Parameters
-        ----------
-        item
-            A variable of type RandomStateType, associated
-            with this RandomStream.
-
-        Returns
-        -------
-        numpy RandomState (or None, before initialize)
-
-        Notes
-        -----
-        This is kept for compatibility with `tensor.randomstreams.RandomStreams`.
-        The simpler syntax ``item.rng.get_value()`` is also valid.
-
-        """
-        return item.get_value(borrow=True)
-
-    def __setitem__(self, item, val):
-        """
-        Set the numpy RandomState instance associated with a particular stream.
-
-        Parameters
-        ----------
-        item
-            A variable of type RandomStateType, associated with this
-            RandomStream.
-
-        val : numpy RandomState
-            The new value.
-
-        Returns
-        -------
-        None
-
-        Notes
-        -----
-        This is kept for compatibility with `tensor.randomstreams.RandomStreams`.
-        The simpler syntax ``item.rng.set_value(val)`` is also valid.
-
-        """
-        item.set_value(val, borrow=True)
-
-    def gen(self, op, *args, **kwargs):
-        """
-        Create a new random stream in this container.
-
-        Parameters
-        ----------
-        op
-            A RandomFunction instance to
-        args
-            Interpreted by `op`.
-        kwargs
-            Interpreted by `op`.
-
-        Returns
-        -------
-        Tensor Variable
-            The symbolic random draw part of op()'s return value.
-            This function stores the updated RandomStateType Variable
-            for use at `build` time.
-
-        """
-        seed = int(self.gen_seedgen.randint(2 ** 30))
-        random_state_variable = shared(np.random.RandomState(seed))
-        # Add a reference to distinguish from other shared variables
-        random_state_variable.tag.is_rng = True
-        new_r, out = op(random_state_variable, *args, **kwargs)
-        out.rng = random_state_variable
-        out.update = (random_state_variable, new_r)
-        self.state_updates.append(out.update)
-        random_state_variable.default_update = new_r
-        return out