Python stable ABI¶
Links¶
- Petr Viktorin’s Python Stable ABI improvement notes
- PEP 489 – Multi-phase extension module initialization
- bpo-1635741: Py_Finalize() doesn’t clear all Python objects at exit (convert extensions to multi-phase init PEP 489)
- PEP 620 – Hide implementation details from the C API (Victor Stinner)
- Move the default C API towards the limited C API
- Make structures opaque:
- PyObject: https://bugs.python.org/issue39573
- PyTypeObject: https://bugs.python.org/issue40170
- PyFrameObject: https://bugs.python.org/issue40421
- PyThreadState: https://bugs.python.org/issue39947
- PyInterpreterState: DONE in Python 3.8!
- PyGC_Head: DONE in Python 3.9!
- bpo-40989: Remove _Py_NewReference() and _Py_ForgetReference() from the public C API
- bpo-41078: Convert PyTuple_GET_ITEM() macro to a static inline function
- bpo-40601: Hide static types from the limited C API
- PEP 573 – Module State Access from C Extension Methods
- Python 3.10 has a new
_PyType_GetModuleByDef()
function - Python 3.9 added:
- New PyType_FromModuleAndSpec()
- New PyType_GetModuleState()
- New METH_METHOD calling convention flag
- New PyCMethod function signature
- New
defining_class
type in Argument Clinic
- Python 3.10 has a new
- PEP 630 – Isolating Extension Modules (Petr Viktorin)
- bpo-41111: Convert a few stdlib extensions to the limited C API
- HPy project
Relationship between the C API and the ABI¶
Here is a short explanation. For a longer explanation, read A New C API for CPython (September 2017) by Victor Stinner.
Given the following code:
typedef struct {
PyVarObject ob_base;
PyObject **ob_item; // <-- pointer to the array of list items
Py_ssize_t allocated;
} PyListObject;
#define PyList_GET_ITEM(op, i) ((PyListObject *)op)->ob_item[i]
And the following C code:
PyObject *item = PyList_GET_ITEM(list, 0);
On a 64-bit machine, the machine code of a release build becomes something like:
PyObject **items = (PyObject **)(((char*)op) + 24);
PyObject *item = items[0];
whereas a debug build uses an offset of 40 instead of 24, because
PyVarObject
contains additional fields for debugging purpose:
PyObject **items = (PyObject **)(((char*)op) + 40);
PyObject *item = items[0];
As a consequence, the compiled C extension is incompatible at the ABI level: a C extension has to be build twice, once in release mode and once in debug mode.
To reduce the maintenance burden, Linux vendors only provide C extensions compiled in release mode, making the debug mode mostly unusable on Linux in practice.
CPython Py_LIMITED_API¶
- CPython documentation: Stable Application Binary Interface
- Who uses it?
- PEP 384 – Defining a Stable ABI by Martin v. Löwis: implemented in CPython 3.2 (2011)