Do not force float64 in the graph when it isn't needed. Do the same dtype requirement as on the CPU.