It happen in a particular execution order of this optimization and the
element-wise fusion optimization when upcasting some inputs to
float32 (to compute them on the gpu).
(Frederic B., reported by Sander Dieleman)