Track whether there is a gpu context active; if not, don't try to free memory. Note that trying to free the memory and then just not displaying the error doesn't work, since the act of trying to free the memory creates a new gpu context.