This was probably causing a different order of operation
during gradient computation in scan for each run. With this
fix I'm able to finally reproduce results on my RNN system.