Roadmap for a new Python C API

Roadmap

  • Step 1: Identify Bad C API and list functions that should be modified or even removed
  • Step 2: Add an opt-in new C API with these cleanups. Test popular C extensions to measure how much code is broken. Start to fix these C extensions by making them forward compatible. Slowly involve more and more players into the game.
  • Step 3: Remove more functions. Maybe replace Py_INCREF() macro with a function call. Finish to hide all C structures especially PyObject.ob_refcnt. Measure the performance. Decide what to do.
  • Step 4: if step 3 gone fine and most people are still ok to continue, make the new C API as the default in CPython and add an option for opt-out to stick with the old C API.

Open questions

  • Remove or deprecate APIs using borrowed references? If PyTuple_GetItem() must be replaced with PyTuple_GetItemRef(), how do we provide PyTuple_GetItemRef() for Python 3.7 and older? See Backward compatibility.

Status

  • 2018-07-29: pythoncapi project created on GitHub
  • 2017-12-21: It’s an idea. There is an old PEP draft, but no implementation, the PEP has no number and was not accepted yet (nor really proposed).

Players

  • CPython: Victor Stinner
  • PyPy cpyext: Ronan Lamy
  • Cython: Stefan Behnel

Unknown status:

  • RustPython
  • MicroPython?
  • IronPython?
  • Jython?
  • Pyjion
  • Pyston
  • any other?

Timeline