The initial bug that motivated this fix was a crash when trying to
perform x[y] = z with x a matrix and z a vector.