Our transfer call are synchronized version. So no problem there. The
problem that we need to work around is that the Theano gc could free
the output var before we finish with it. cudaFree is instananeous, it
don't get in the stream of command to execute.