We are not calling `as_tensor_variable` to avoid transfers when the
inputs are on GPU, but the the cuda/numpy arrays should still be
converted to Constant variables.