Since we do constant folding earlier, this make a GpuAlloc test fail.
This could also speed some code as this allow more op to work inplace.