Remove the assert in PyGpuArray_DEV_DATA since it might cost a bit too much to verify that condition everytime.