added incomplete proposal: graphical_models extension

doc/proposals/graphical_models.txt

+==================================================================
+Random Numbers, Random Variables and Compiling Graphical Models
+==================================================================
+
+.. note:
+   Proposed 2010 02 06
+
+
+Objective
+=========
+
+It might be nice to use Theano as a language and compiler for questions about
+graphical models.
+
+In this way, we could express something like Logistic Regression like this:
+
+.. code-block:: python
+
+    from theano import random_variable as RV
+
+    X, Y, s_idx = RV.empirical(my_dataset)
+
+    # model parameters
+    v = shared(numpy.zeros(()))
+    b = shared(numpy.zeros(()))
+
+    Y_hat = RV.multinomial(n=1, p=softmax(dot(X,v)+b))
+
+    cost = sum(-log(Y_hat.density(Y)))
+
+    train_fn = function([s_idx], cost, updates=[[v,b], grad(cost, [v,b]]))
+
+.. code-block:: python
+
+    RandomVariable(Variable)
+
+        def sample(self, n):
+            """[Symbolically] draw a sample of size n"""
+
+        def density(self, pt, givens={}):
+            """Conditional Density/Probability of P(self=pt)
+
+            Implicitly conditioned on knowing the values of all variables
+            on which this one depends.  Optionally override ancestor variables
+            using givens.
+            """
+
+        def mode(self):
+            """Return expression of the most likely value of this distribution"""
+
+We would really like to integrate out certain variables sometimes... 
+
+
+An RBM could be expressed like this:
+
+.. code-block:: python
+
+    w = shared(initial_weights)
+    v = shared(initial_visible_biases)
+    u = shared(initial_hidden_biases)
+    visible = RV.binomial(n=1, p=None) # p filled in by EnergyModel
+    hidden =  RV.binomial(n=1, p=None) # p filled in by EnergyModel
+
+    energy = dot(visible,v) + dot(hidden, u) + dot(dot(visible, w), hidden)
+
+    RBM = EnergyModel(energy, variables={'visible':visible, 'hidden':hidden], params=[w,v,u])
+
+    RBM.energy(v,h) # an expression for the energy at point (v,h)
+
+    RBM.visible.energy(h) # an expression for the free energy
+    RBM.hidden.energy(h) # an expression for the free energy
+    v_given_h = RBM.visible.conditional(h) # a random variable
+
+Rather than program all the training algorithms into an RBM module, 
+the idea would be to express the relationship between RBM variables so that we
+could automatically recognize how to do Gibbs sampling, gradient descent on Free
+Energy, etc.
+