Roadmap for a new Python C API¶
- Step 1: Identify Bad C API and list functions that should be modified or even removed
- Step 2: Add an opt-in new C API with these cleanups. Test popular C extensions to measure how much code is broken. Start to fix these C extensions by making them forward compatible. Slowly involve more and more players into the game.
- Step 3: Remove more functions. Maybe replace
Py_INCREF() macro with a function call. Finish to hide all C
PyObject.ob_refcnt. Measure the performance. Decide what to do.
- Step 4: if step 3 gone fine and most people are still ok to continue, make the new C API as the default in CPython and add an option for opt-out to stick with the old C API.
- Remove or deprecate APIs using borrowed references? If
PyTuple_GetItem()must be replaced with
PyTuple_GetItemRef(), how do we provide
PyTuple_GetItemRef()for Python 3.7 and older? See Backward compatibility.
- 2020-04-10: PEP: Modify the C API to hide implementation details sent to python-dev.
- 2019-05-01: Status of the stable API and ABI in Python 3.8, slides of Victor Stinner’s lightning talk at the Language Summit (during Pycon US 2019)
- 2019-02-22: [capi-sig] Update on CPython header files reorganization
- 2018-09-04: Creation of CPython fork to experiment a new incompatible C API excluding borrowed references and not access directly structure members.
- 2018-07-29: pythoncapi project created on GitHub
- 2018-06: capi-sig mailing list migrated to Mailman 3
- 2017-12-21: It’s an idea. There is an old PEP draft, but no implementation, the PEP has no number and was not accepted yet (nor really proposed).
- 2017-11: Idea proposed on python-dev, [Python-Dev] Make the stable API-ABI usable
- 2017-09: Blog post: A New C API for CPython
- 2017-09: Idea discussed at the CPython sprint at Instagram (California). Liked by all core developers. The expected performance slowdown is likely to be accepted.
- 2017-07-11: [Python-ideas] PEP: Hide implementation details in the C API
- 2017-07: Idea proposed on python-ideas. [Python-ideas] PEP: Hide implementation details in the C API
- 2017-05: Idea proposed at the Python Language Summit, during PyCon US 2017. My “Python performance” slides (PDF). LWN article: Keeping Python competitive.