New submission from STINNER Victor <[email protected]>:
I propose to move the PyFrameObject structure to the internal C API. -- Between Python 3.10 and Python 3.11, the work on optimizing ceval.c modified deeply the PyFrameObject structure. Examples: * The f_code member has been removed by in bpo-44032 by the commit b11a951f16f0603d98de24fee5c023df83ea552c. * The f_frame member has been added in bpo-44590 by the commit ae0a2b756255629140efcbe57fc2e714f0267aa3. Most members have been moved to a new PyFrameObject.f_frame member which has the type "struct _interpreter_frame*". Problem: this type is only part of the *internal* C API. Moreover, accessing the few remaining members which "didn't change" became dangerous. For example, f_back can be NULL even if the frame has a previous frame: the PyFrame_GetBack() function *must* now be called. See bpo-46356 "[C API] Enforce usage of PyFrame_GetBack()". Reading directly f_lineno was already dangerous since Python 2.3: the value is only valid if the value is greater than 0. It's way safer to use the clean PyFrame_GetLineNumber() API instead. PyFrame_GetBack() was added to Python 3.9. You can use the pythoncapi_compat project to get this function on Python 3.8 and older: => https://pythoncapi-compat.readthedocs.io/ PyFrame_GetLineNumber() was added to the limited API in Python 3.10. => Documentation: https://docs.python.org/dev/c-api/reflection.html#c.PyFrame_GetBack -- There *are* projects accessing directly PyFrameObject like the gevent project which sets the f_code member (moved to f_frame.f_code in Python 3.11). It's broken on Python 3.11: https://bugs.python.org/issue40421#msg413719 Debuggers and profilers also want to read PyFrameObject directly. IMO for these *specific* use cases, using the *internal* C API is a legit use case and it's fine. Moving PyFrameObject to the internal C API would clarify the situation. Currently, What's New in Python 3.11 documents the change this with warning: "While the documentation notes that the fields of PyFrameObject are subject to change at any time, they have been stable for a long time and were used in several popular extensions. " -- I'm mostly worried about Cython which still get and set many PyFrameObject members directly (ex: f_lasti, f_lineno, f_localsplus, f_trace), since there are no public functions for that. => https://bugs.python.org/issue40421#msg367550 Right now, I would suggest Cython to use the internal C API, and *later* consider adding new getter and setter functions. I don't think that we can solve all problems at once: it takes take to design clean API and use them in Cython. Python 3.11 already broke Cython since most PyFrameObject members moved into the new "internal" PyFrameObject.f_frame API which requires using the internal C API to get "struct _interpreter_frame". => https://github.com/cython/cython/issues/4500 -- Using a frame using the *public* C API was and remains supported. Short example: -- PyThreadState *tstate = PyThreadState_Get(); PyFrameObject* frame = PyThreadState_GetFrame(tstate); int lineno = PyFrame_GetLineNumber(frame); --- The PyFrameObject structure is opaque and members are not accessed directly: it's fine. ---------- components: C API messages: 413795 nosy: vstinner priority: normal severity: normal status: open title: [C API] Move PyFrameObject to the internal C API versions: Python 3.11 _______________________________________ Python tracker <[email protected]> <https://bugs.python.org/issue46836> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
