Add even more documentation for RNNBlock usage.

doc/library/gpuarray/dnn.txt

@@ -149,6 +149,94 @@ To get an error if Theano can not use cuDNN, use this Theano flag:
 - Spatial Transformer:
     - :func:`theano.gpuarray.dnn.dnn_spatialtf`.
 
+
+cuDNN RNN Example
+=================
+
+This is a code example of using the cuDNN RNN functionality.  We
+present the code with some commentary in between to explain some
+peculiarities.
+
+The terminology here assumes that you are familiar with RNN structure.
+
+.. code-block:: python
+
+    dtype = 'float32'
+    input_dim = 32
+    hidden_dim = 16
+    batch_size = 2
+    depth = 3
+    timesteps = 5
+
+To clarify the rest of the code we define some variables to hold sizes.
+
+.. code-block:: python
+
+    X = T.tensor3('X')
+    Y = T.tensor3('Y')
+    h0 = T.tensor3('h0')
+
+We also define some Theano variables to work with.  Here `X` is input,
+`Y` is output (as in expected output) and `h0` is the initial state
+for the recurrent inputs.
+
+.. code-block:: python
+
+    rnnb = dnn.RNNBlock(dtype, hidden_dim, depth, 'gru')
+
+This defines an RNNBlock.  This is a departure from usual Theano
+operations in that it has the structure of a layer more than a
+separate operation.  This is constrained by the underlying API.
+
+.. code-block:: python
+
+    psize = rnnb.get_param_size([batch_size, input_dim])
+    params_cudnn = gpuarray_shared_constructor(
+        np.zeros((psize,), dtype=theano.config.floatX))
+
+Here we allocate space for the trainable parameters of the RNN.  The
+first function tells us how many elements we will need to store the
+parameters.  This space if for all the parameters of all the layers
+inside the RNN and the layout is opaque.
+
+.. code-block:: python
+
+   layer = 0
+    = rnnb.split_params(params_cudnn, layer,
+                                  [batch_size, input_dim])
+
+If you need to access the parameters individually, you can call
+split_params on your shared variable to get all the parameters for a
+single layer. The order and number of returned items depends on the
+type of RNN.
+
+rnn_relu, rnn_tanh
+  input, recurrent
+
+gru
+  input reset, input update, input newmem, recurrent reset, recurrent
+  update, recurrent newmem
+
+lstm
+  input input gate, input forget gate, input newmem gate, input output
+  gate, recurrent input gate, recurrent update gate, recurrent newmem
+  gate, recurrent output gate
+
+All of these elements are composed of a weights and bias (matrix and
+vector).
+
+.. code-block:: python
+
+    y, hy = rnnb.apply(params_cudnn, X, h0)
+
+This is more akin to an op in Theano in that it will apply the RNN
+operation to a set of symbolic inputs and return symbolic outputs.
+`y` is the output, `hy` is the final state for the recurrent inputs.
+
+After this, the gradient works as usual so you can treat the returned
+symbolic outputs as normal Theano symbolic variables.
+
+
 List of Implemented Operations
 ==============================