Merged

ChangeLog

-2010-10-22 Theano 0.3
----------------------
-
-This is the first major release of Theano since 0.1. Version 0.2 development
-started internally but it was never advertised as a release.
-
-There have been so many changes since 0.1 that we have lost track of many of
-them. Below is a *partial* list of changes since 0.1.
-
- * GPU code using NVIDIA's CUDA framework is now generated for many Ops.
- * Some interface changes since 0.1:
-     * A new "shared variable" system to allow reusing memory space between
-       Theano functions.
-         * A new memory contract has been formally written for Theano,
-           for people who want to minimize memory copies.
-     * The old module system has been deprecated.
-     * By default, inputs to a Theano function will not be silently
-       downcasted (e.g. from float64 to float32).
-     * An error is now raised when using the result of logical operation on
-       Theano variable in an 'if' (i.e. an implicit call to __nonzeros__).
-     * An error is now raised when we receive a non-aligned ndarray as
-       input to a function (this is not supported).
-     * An error is raised when the list of dimensions passed to
-       dimshuffle() contains duplicates or is otherwise not sensible.
-     * Call NumPy BLAS bindings for gemv operations in addition to the
-       already supported gemm.
-     * If gcc is unavailable at import time, Theano now falls back to a
-       Python-based emulation mode after raising a warning.
-     * An error is now raised when tensor.grad is called on a non-scalar
-       Theano variable (in the past we would implicitly do a sum on the
-       tensor to make it a scalar).
-     * Added support for "erf" and "erfc" functions.
- * The current default value of the parameter axis of theano.{max,min,
-   argmax,argmin,max_and_argmax} is deprecated. We now use the default NumPy
-   behavior of operating on the entire tensor.
- * Theano is now available from PyPI and installable through "easy_install" or
-   "pip".

HISTORY.txt

+
+.. _NEWS:
+
+=============
+Release Notes
+=============
+
+Theano 0.1
+==========
+
+*Release date: 2009-04-02*
+
+What works
+----------
+
+- building symbolic expression.
+- arranging symbolic expressions into Modules so that multiple functions
+  can work on the same data.
+- symbolic gradient descent.
+- graph optimization.
+- compilation to C for many kinds of expression.
+- a debugging mode that checks that your expression results are correct,
+  using a variety of sanity checks.
+
+What's missing?
+---------------
+
+- An algorithm library. We're missing a library of examples and standard
+  component implementations.  Some examples will find their way into
+  the Theano repo, but standard algorithms will go into the 'pylearn'
+  project (toolbox style). Now that we have a stable foundation, we
+  can reach a consensus on style for algorithms.

NEWS.txt

-doc/NEWS.txt
\ No newline at end of file

NEWS.txt

+Theano 0.3 (2010-11-23)
+-----------------------
+
+This is the first major release of Theano since 0.1. Version 0.2 development started internally but it was never advertised as a release.
+
+There have been so many changes since 0.1 that we have lost track of many of them. Below is a *partial* list of changes since 0.1.
+
+ * GPU code using NVIDIA's CUDA framework is now generated for many Ops.
+ * Some interface changes since 0.1:
+     * A new "shared variable" system to allow reusing memory space between Theano functions.
+         * A new memory contract has been formally written for Theano, for people who want to minimize memory copies.
+     * The old module system has been deprecated.
+     * By default, inputs to a Theano function will not be silently downcasted (e.g. from float64 to float32).
+     * An error is now raised when using the result of logical operation on Theano variable in an 'if' (i.e. an implicit call to __nonzeros__).
+     * An error is now raised when we receive a non-aligned ndarray as input to a function (this is not supported).
+     * An error is raised when the list of dimensions passed to dimshuffle() contains duplicates or is otherwise not sensible.
+     * Call NumPy BLAS bindings for gemv operations in addition to the already supported gemm.
+     * If gcc is unavailable at import time, Theano now falls back to a Python-based emulation mode after raising a warning.
+     * An error is now raised when tensor.grad is called on a non-scalar Theano variable (in the past we would implicitly do a sum on the tensor to make it a scalar).
+     * Added support for "erf" and "erfc" functions.
+ * The current default value of the parameter axis of theano.{max,min,argmax,argmin,max_and_argmax} is deprecated. We now use the default NumPy behavior of operating on the entire tensor.
+ * Theano is now available from PyPI and installable through "easy_install" or "pip".

setup.py

@@ -36,7 +36,8 @@ MAINTAINER          = "LISA laboratory, University of Montreal"
 MAINTAINER_EMAIL    = "theano-dev@googlegroups.com"
 DESCRIPTION         = ('Optimizing compiler for evaluating mathematical ' +
                        'expressions on CPUs and GPUs.')
-LONG_DESCRIPTION    = open("DESCRIPTION.txt").read()
+LONG_DESCRIPTION    = (open("DESCRIPTION.txt").read() + "\n\n"
+                       open("NEWS.txt").read())
 URL                 = "http://deeplearning.net/software/theano/"
 DOWNLOAD_URL        = ""
 LICENSE             = 'BSD'

theano/sandbox/cuda/tests/test_tensor_op.py

@@ -40,20 +40,32 @@ def test_shape_i():
         assert len(topo)==1
         assert isinstance(topo[0].op,T.opt.Shape_i)
 
+def test_shape():
+    x = cuda.ftensor3()
+    v = cuda.CudaNdarray(numpy.zeros((3,4,5),dtype='float32'))
+    f = theano.function([x],x.shape)
+    topo = f.maker.env.toposort()
+    assert numpy.all(f(v)==(3,4,5))
+    if theano.config.mode!='FAST_COMPILE':
+        assert len(topo)==4
+        assert isinstance(topo[0].op,T.opt.Shape_i)
+        assert isinstance(topo[1].op,T.opt.Shape_i)
+        assert isinstance(topo[2].op,T.opt.Shape_i)
+        assert isinstance(topo[3].op,T.opt.MakeVector)
 
 def test_softmax_optimizations():
     from theano.tensor.nnet.nnet import softmax, crossentropy_categorical_1hot, crossentropy_softmax_argmax_1hot_with_bias
     x = tensor.fmatrix('x')
     one_of_n = tensor.lvector('one_of_n')
     op = crossentropy_categorical_1hot
-    
+
     xe = op(x, one_of_n)
-    
+
     env = theano.gof.Env(
         [x, one_of_n],
         [op(softmax(x), one_of_n)])
     assert env.outputs[0].owner.op == op
-    
+
     mode_with_gpu.optimizer.optimize(env)
 
     assert str(env.outputs[0].owner.op) == 'OutputGuard'

theano/sparse/basic.py

@@ -248,6 +248,12 @@ class _sparse_py_operators:
     def __dot__(left, right): return structured_dot(left, right)
     def __rdot__(right, left): return structured_dot(left, right)
 
+    #def _as_TensorVariable(self):
+    #    return dense_from_sparse(self)
+    shape = property(lambda self: tensor.shape(self))
+    ndim = property(lambda self: self.type.ndim)
+    dtype = property(lambda self: self.type.dtype)
+
 
 class SparseVariable(gof.Variable, _sparse_py_operators):
     dtype = property(lambda self: self.type.dtype)
@@ -1148,16 +1154,16 @@ def structured_dot_grad(sparse_A, dense_B, ga):
     if sparse_A.type.format in ('csc','csr'):
 
         if sparse_A.type.format == 'csc':
-          sdgcsx = sdg_csc
+            sdgcsx = sdg_csc
         else:
-          sdgcsx = sdg_csr
+            sdgcsx = sdg_csr
         #backport
         #sdgcsx = sdg_csc if sparse_A.type.format == 'csc' else sdg_csr
 
         if sparse_A.type.format == 'csc':
-          CSx = CSC
+            CSx = CSC
         else:
-          CSx = CSR
+            CSx = CSR
         #backport
         #CSx = CSC if sparse_A.type.format == 'csc' else CSR
 
@@ -1380,4 +1386,3 @@ class StructuredDotGradCSR(gof.Op):
 
         """% dict(locals(), **sub)
 sdg_csr = StructuredDotGradCSR()
-

theano/sparse/tests/test_basic.py

@@ -155,7 +155,7 @@ class T_AddMul(unittest.TestCase):
             elif op is mul:
                 self.failUnless(_is_sparse_variable(apb))
                 self.failUnless(numpy.all(val.todense() == (b.multiply(a))))
-                self.failUnless(numpy.all(val.todense() == numpy.array([[1, 0], 
+                self.failUnless(numpy.all(val.todense() == numpy.array([[1, 0],
 [9, 0], [0, 36]])))
 
     def _testDS(self, op, array1 = numpy.array([[1., 0], [3, 0], [0, 6]]),
@@ -187,7 +187,7 @@ class T_AddMul(unittest.TestCase):
             elif op is mul:
                 self.failUnless(_is_sparse_variable(apb))
                 self.failUnless(numpy.all(val.todense() == (a.multiply(b))))
-                self.failUnless(numpy.all(val.todense() == numpy.array([[1, 0], 
+                self.failUnless(numpy.all(val.todense() == numpy.array([[1, 0],
 [9, 0], [0, 36]])))
 
 
@@ -244,7 +244,7 @@ class test_structureddot(unittest.TestCase):
             assert rval.type.dtype == 'float32'
             return rval
 
-        utt.verify_grad(buildgraphCSC, 
+        utt.verify_grad(buildgraphCSC,
                     [spmat.data, mat])
 
     def test_structureddot_csr_grad(self):
@@ -264,7 +264,7 @@ class test_structureddot(unittest.TestCase):
             assert rval.type.dtype == 'float64'
             return rval
 
-        utt.verify_grad(buildgraph, 
+        utt.verify_grad(buildgraph,
                     [spmat.data, mat])
 
     def test_upcast(self):
@@ -307,7 +307,7 @@ class test_structureddot(unittest.TestCase):
         # Test that a graph involving structured_dot(assembled_csc_matrix) is optimized to be
         # just a structured_dot_csc Op and no assembly of a csc_matrix.
         #
-        # The optimization from structured_dot -> structured_dot_csc is currently disabled, 
+        # The optimization from structured_dot -> structured_dot_csc is currently disabled,
         # So this test is not expected to pass
 
         return
@@ -320,7 +320,7 @@ class test_structureddot(unittest.TestCase):
             y = numpy.floor(numpy.random.rand()*spmat.shape[1])
             spmat[x,y] = numpy.random.rand()*10
         spmat = sp.csc_matrix(spmat)
-               
+
         images = tensor.Tensor(dtype='float32', broadcastable=[False, False])('images')
 
         cscmat = CSC(kerns, spmat.indices[:spmat.size], spmat.indptr, spmat.shape)
@@ -364,7 +364,7 @@ class test_structureddot(unittest.TestCase):
 
         #print f.maker.env.toposort()
 
-        for M,N,K,nnz in [(4,3,2,3), 
+        for M,N,K,nnz in [(4,3,2,3),
                 (40,30,20,3),
                 (40,30,20,30),
                 (400,3000,200,6000),
@@ -417,7 +417,7 @@ class test_structureddot(unittest.TestCase):
 
         print f.maker.env.toposort()
 
-        for M,N,K,nnz in [(4,3,2,3), 
+        for M,N,K,nnz in [(4,3,2,3),
                 (40,30,20,3),
                 (40,30,20,30),
                 (400,3000,200,6000),

theano/tensor/opt.py

@@ -1147,18 +1147,18 @@ def apply_rebroadcast_opt(rval):
 
     changed = True
     while changed and rval.owner:
-      changed = False
-      rval2 = theano.tensor.opt.local_useless_rebroadcast.transform(rval.owner)
-      if rval2:
-        assert len(rval2)==1
-        rval = rval2[0]
-        changed = True
-      if rval.owner:
-        rval2 = theano.tensor.opt.local_rebroadcast_lift.transform(rval.owner)
+        changed = False
+        rval2 = theano.tensor.opt.local_useless_rebroadcast.transform(rval.owner)
         if rval2:
-          assert len(rval2)==1
-          rval = rval2[0]
-          changed = True
+            assert len(rval2)==1
+            rval = rval2[0]
+            changed = True
+        if rval.owner:
+            rval2 = theano.tensor.opt.local_rebroadcast_lift.transform(rval.owner)
+            if rval2:
+                assert len(rval2)==1
+                rval = rval2[0]
+                changed = True
     return rval
 
 
@@ -1216,7 +1216,7 @@ def local_mul_switch_sink(node):
                     fct[0].values_eq_approx = fct[0].type.values_eq_approx_remove_nan
                     return fct
             except TypeError:
-               pass
+                pass
             try:
                 if get_constant_value(switch.inputs[2]) == 0.:
                     listmul = node.inputs[:idx] + node.inputs[idx+1:]
@@ -2398,9 +2398,9 @@ def local_log_add(node):
 def add_calculate(num, denum, aslist = False, out_type=None):
     #TODO: make sure that this function and mul_calculate are similar
     if out_type is None:
-      zero = 0.0
+        zero = 0.0
     else:
-      zero = theano._asarray(0, dtype=out_type.dtype)
+        zero = theano._asarray(0, dtype=out_type.dtype)
     #zero = 0.0 if out_type is None else theano._asarray(0, dtype=out_type.dtype)
     v = reduce(N.add, num, zero) - reduce(N.add, denum, zero)
     if aslist:
@@ -2856,7 +2856,7 @@ def local_grad_log_erfc_neg(node):
 
             #The constant is valid. Must check that the
         elif erfc_x is not x:
-                return False
+            return False
 
     else:
         return False
@@ -3098,5 +3098,3 @@ if config.tensor.local_elemwise_fusion:
 else:
     _logger.debug("not enabling optimization fusion elemwise in fast_run")
     compile.optdb.register('elemwise_fusion', FusionOptimizer(local_elemwise_fusion), 71.00, 'fusion', 'local_elemwise_fusion')
-
-