Calling python method from C++ (or C) callback

up vote26down voteaccepted

There are a few things you need to do if you are invoking a Python function from a C/C++ callback. First when you save off your python function object, you need to increment the reference count with:

Py_INCREF(pFunc)

Otherwise Python has no idea you are holding onto an object reference, and it may garbage collect it, resulting in a segmentation fault when you try to use it from your callback.

Then next thing you need to be concerned about is what thread is running when your C/C++ callback is invoked. If you are getting called back from another non-Python created thread (i.e. a C/C++ thread receiving data on a socket), then you MUST acquire Python's Global Interpreter Lock (GIL) before calling any Python API functions. Otherwise your program's behavior is undefined. To acquire the GIL you do:

void callback() {    PyGILState_STATE gstate;    gstate = PyGILState_Ensure();    // Get args, etc.    // Call your Python function object    PyObject * pInstance = PyObject_CallObject(pFunc, args);    // Do any other needed Python API operations    // Release the thread. No Python API allowed beyond this point.    PyGILState_Release(gstate);}

Also, in your extension module's init function, you should do the following to ensure that threading is properly initialized:

// Make sure the GIL has been created since we need to acquire it in our// callback to safely call into the python application.if (! PyEval_ThreadsInitialized()) {    PyEval_InitThreads();}

Otherwise, crashes and strange behavior may ensue when you attempt to acquire the GIL from a non-Python thread.

See Non-Python Created Threads for more detail on this.