[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

I created bpo-47092: [C API] Add PyFrame_GetVar(frame, name) function.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[Brandt Bucher]

Change by Brandt Bucher :


--
nosy: +brandtbucher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[Mark Shannon]

Mark Shannon  added the comment:

I've outlined the requirements for a frame stack API at 
https://github.com/faster-cpython/ideas/issues/309.

The problem with adding an API for PyFrameObject (beyond simple getters) is 
that it makes assumptions about the frame stack that aren't valid.

A stack of frames is not just a linked list of frames. It never was, and it 
certainly isn't now.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

I searched for "\<_f" regex in Cython (0.29.x) branch. I found the following 
code getting or setting PyFrameObject fields.

== Set f_back ==

__Pyx_Coroutine_SendEx() at Utility/Coroutine.c:721:

f->f_back = PyThreadState_GetFrame(tstate);

__Pyx_Coroutine_ResetFrameBackpointer() at Utility/Coroutine.c:775:

Py_CLEAR(f->f_back);


== Set f_lineno ==

__Pyx_PyFrame_SetLineNumber() at Utility/ModuleSetupCode.c:543:

#define __Pyx_PyFrame_SetLineNumber(frame, lineno)  (frame)->f_lineno = 
(lineno)


__Pyx_PyFrame_SetLineNumber() is called by 3 functions:

* __Pyx_AddTraceback()
* __Pyx_call_line_trace_func()
* __Pyx_TraceSetupAndCall()

__Pyx_AddTraceback() pseudo-code:

static void __Pyx_AddTraceback(const char *funcname, int c_line,
   int py_line, const char *filename)
{
py_frame = PyFrame_New(..., py_code, ...);
__Pyx_PyFrame_SetLineNumber(py_frame, py_line);
...
}



== f_localsplus ==

If the CYTHON_FAST_PYCALL macro is defined, sizeof(PyFrameObject) is used to 
get the f_localsplus member.

__Pyx_PyFrame_GetLocalsplus() at Utility/ObjectHandling.c:1996:
---
// Initialised by module init code.
static size_t __pyx_pyframe_localsplus_offset = 0;

#include "frameobject.h"

#define __Pxy_PyFrame_Initialize_Offsets()  \
  ((void)__Pyx_BUILD_ASSERT_EXPR(sizeof(PyFrameObject) == 
offsetof(PyFrameObject, f_localsplus) + Py_MEMBER_SIZE(PyFrameObject, 
f_localsplus)), \
   (void)(__pyx_pyframe_localsplus_offset = ((size_t)PyFrame_Type.tp_basicsize) 
- Py_MEMBER_SIZE(PyFrameObject, f_localsplus)))
#define __Pyx_PyFrame_GetLocalsplus(frame)  \
  (assert(__pyx_pyframe_localsplus_offset), (PyObject **)(((char *)(frame)) + 
__pyx_pyframe_localsplus_offset))
---

== Get f_trace ==

__Pyx_TraceLine() at Utility/Profile.c:238:

if (__Pyx_IsTracing(tstate, 0, 0) && tstate->c_tracefunc && 
$frame_cname->f_trace)


== Set f_frame ==

__Pyx_TraceSetupAndCall() at Utility/Profile.c:299:

(*frame)->f_tstate = tstate;

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 8a716bc62c8205bb9deeda17422b7e69204b6897 by Victor Stinner in 
branch 'main':
bpo-40421: What's New in Python 3.11: PyFrameObject.f_lasti (GH-31536)
https://github.com/python/cpython/commit/8a716bc62c8205bb9deeda17422b7e69204b6897


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[miss-islington]

miss-islington  added the comment:


New changeset 763e23e11e785162f9d68f83539404af8e863748 by Miss Islington (bot) 
in branch '3.9':
bpo-40421: Fix PyFrame_GetCode() documentation (GH-31535)
https://github.com/python/cpython/commit/763e23e11e785162f9d68f83539404af8e863748


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[miss-islington]

miss-islington  added the comment:


New changeset b0de6299a840a397d4fe3e6c98159d9f258d3295 by Miss Islington (bot) 
in branch '3.10':
bpo-40421: Fix PyFrame_GetCode() documentation (GH-31535)
https://github.com/python/cpython/commit/b0de6299a840a397d4fe3e6c98159d9f258d3295


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[miss-islington]

Change by miss-islington :


--
pull_requests: +29662
pull_request: https://github.com/python/cpython/pull/31538

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 78859e58e4e016286e648d1dc155e0f6cebfa6ff by Victor Stinner in 
branch 'main':
bpo-40421: Fix PyFrame_GetCode() documentation (GH-31535)
https://github.com/python/cpython/commit/78859e58e4e016286e648d1dc155e0f6cebfa6ff


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 5.0 -> 6.0
pull_requests: +29661
pull_request: https://github.com/python/cpython/pull/31537

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

> I found the uwsgi project on PyPI which uses f_lasti with PyCode_Addr2Line().

Oh, plugins/python/profiler.c uses that to define PyFrame_GetLineNumber() on 
Python older than 2.7, Python 3.0 and Python 3.1. In 2022, it's no longer 
relevant.

But well, there might be other code in the wild using PyCode_Addr2Line() with 
f_lasti, so IMO it's worth it to document the suggestion ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

Mark Shannon:
> The only purpose of `f_lasti` is to get the line number and that can be done 
> directly via `PyFrame_GetLineNumber(PyFrameObject *frame)`

I found the uwsgi project on PyPI which uses f_lasti with PyCode_Addr2Line(). I 
wrote GH-31536 to suggest using PyFrame_GetLineNumber() in What's New in Python 
3.11.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +29660
pull_request: https://github.com/python/cpython/pull/31536

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

Ned Batchelder:
> I went ahead and changed the coverage.py code to this: (...)

I proposed a coverage PR using PyObject_GetAttrString(frame, "f_lasti") which 
should works on all Python versions:
https://github.com/nedbat/coveragepy/pull/1331

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

> The docs for PyFrame_GetCode say it's returning an "int". 

Oh. I  missed your comment. I created GH-31535 to fix the return type in the 
documentation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +29659
pull_request: https://github.com/python/cpython/pull/31535

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

I created bpo-46836: "[C API] Move PyFrameObject to the internal C API".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

The f_code member has been removed by in bpo-44032 by the commit 
b11a951f16f0603d98de24fee5c023df83ea552c.

The f_frame member has been added in bpo-44590 by the commit 
ae0a2b756255629140efcbe57fc2e714f0267aa3.

See also bpo-46355 [C API] Document PyFrameObject and PyThreadState changes and 
explain how to port code to Python 3.11.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

The gevent project is not compatible with Python 3.11: it gets and sets 
directly PyFrameObject.f_code member which has been removed in Python 3.11 
(moved to PyFrameObject.f_frame.f_code internal C API).

gevent issue: https://github.com/gevent/gevent/issues/1867

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[Petr Viktorin]

Petr Viktorin  added the comment:

The docs for PyFrame_GetCode say it's returning an "int". 
https://docs.python.org/3/c-api/reflection.html#c.PyFrame_GetCode
Same for PyFrame_GetBack. Are there more?

"Get the frame code." is not very clear. Could this link to 
https://docs.python.org/3/c-api/code.html ?

--
nosy: +petr.viktorin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[Ned Batchelder]

Ned Batchelder  added the comment:

I went ahead and changed the coverage.py code to this:

#if PY_VERSION_HEX >= 0x030B00A0
// 3.11 moved f_lasti into an internal structure. This is totally the wrong way
// to make this work, but it's all I've got until 
https://bugs.python.org/issue40421
// is resolved.
#include 
#define MyFrame_lasti(f)((f)->f_frame->f_lasti * 2)
#elif PY_VERSION_HEX >= 0x030A00A7
// The f_lasti field changed meaning in 3.10.0a7. It had been bytes, but
// now is instructions, so we need to adjust it to use it as a byte index.
#define MyFrame_lasti(f)((f)->f_lasti * 2)
#else
#define MyFrame_lasti(f)((f)->f_lasti)
#endif


If we had an API that let me distinguish between call and resume and between 
return and yield, I could get rid of this.

--
nosy: +nedbat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-45247: [C API] Add explicit support for Cython to the C API.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

The coverage project has a ctrace C extension which access 
PyFrameObject.f_lasti which is gone in Python 3.11. It uses MyFrame_lasti() 
helper to handle Python 3.10 lasti change:
---
// The f_lasti field changed meaning in 3.10.0a7. It had been bytes, but
// now is instructions, so we need to adjust it to use it as a byte index.
#if PY_VERSION_HEX >= 0x030A00A7
#define MyFrame_lasti(f)(f->f_lasti * 2)
#else
#define MyFrame_lasti(f)f->f_lasti
#endif // 3.10.0a7
---

f_lasti is used for two things in coverage/ctracer/tracer.c:

(1) get the last opcode:

/* Need to distinguish between RETURN_VALUE and YIELD_VALUE. Read
 * the current bytecode to see what it is.  In unusual circumstances
 * (Cython code), co_code can be the empty string, so range-check
 * f_lasti before reading the byte.
 */
int bytecode = RETURN_VALUE;
PyObject * pCode = MyFrame_GetCode(frame)->co_code;
int lasti = MyFrame_lasti(frame);

if (lasti < PyBytes_GET_SIZE(pCode)) {
bytecode = PyBytes_AS_STRING(pCode)[lasti];
}
if (bytecode != YIELD_VALUE) {
int first = MyFrame_GetCode(frame)->co_firstlineno;
if (CTracer_record_pair(self, self->pcur_entry->last_line, 
-first) < 0) {
goto error;
}
}


(2) get the line number, with a special case for generator which is not started 
yet (lasti < 0)
---
/* A call event is really a "start frame" event, and can happen for
 * re-entering a generator also.  f_lasti is -1 for a true call, and a
 * real byte offset for a generator re-entry.
 */
if (frame->f_lasti < 0) {
self->pcur_entry->last_line = -MyFrame_GetCode(frame)->co_firstlineno;
}
else {
self->pcur_entry->last_line = PyFrame_GetLineNumber(frame);
}
---

Since Python 3.10.0a3, PyFrame_GetLineNumber() handles the case of negative 
f_lasti, thanks to the commit 877df851c3ecdb55306840e247596e7b7805a60a related 
to the PEP 626 implementation:
---
int
PyCode_Addr2Line(PyCodeObject *co, int addrq)
{
if (addrq < 0) {
return co->co_firstlineno;
}
...
}
---


=> coverage would need an abstraction to get the last opcode: use case (1).


I recall that an old version of asyncio also had to get latest opcode, in pure 
Python, to workaround the CPython bpo-21209 bug:

+# Check for CPython issue #21209
+def has_yield_from_bug():
+class MyGen:
+def __init__(self):
+self.send_args = None
+def __iter__(self):
+return self
+def __next__(self):
+return 42
+def send(self, *what):
+self.send_args = what
+return None
+def yield_from_gen(gen):
+yield from gen
+value = (1, 2, 3)
+gen = MyGen()
+coro = yield_from_gen(gen)
+next(coro)
+coro.send(value)
+return gen.send_args != (value,)
+_YIELD_FROM_BUG = has_yield_from_bug()
+del has_yield_from_bug

(...)

+if _YIELD_FROM_BUG:
+# For for CPython issue #21209: using "yield from" and a custom
+# generator, generator.send(tuple) unpacks the tuple instead of passing
+# the tuple unchanged. Check if the caller is a generator using "yield
+# from" to decide if the parameter should be unpacked or not.
+def send(self, *value):
+frame = sys._getframe()
+caller = frame.f_back
+assert caller.f_lasti >= 0
+if caller.f_code.co_code[caller.f_lasti] != _YIELD_FROM:
+value = value[0]
+return self.gen.send(value)
+else:
+def send(self, value):
+return self.gen.send(value)

Hopefully, this code could be be removed from asyncio, since the bug was fixed 
(and asyncio is now only maintained in the Python stdlib, it's not longer a 
third party module).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 166286849048eccadecf02b242dbc4042b780944 by Victor Stinner in 
branch '3.9':
Add symbols of the stable ABI to python3dll.c (GH-23598) (GH-23801)
https://github.com/python/cpython/commit/166286849048eccadecf02b242dbc4042b780944


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +22659
pull_request: https://github.com/python/cpython/pull/23801

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:


New changeset fcc6935384b933fbe1a1ef659ed455a3b74c849a by Victor Stinner in 
branch 'master':
Add symbols of the stable ABI to python3dll.c (GH-23598)
https://github.com/python/cpython/commit/fcc6935384b933fbe1a1ef659ed455a3b74c849a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +22464
pull_request: https://github.com/python/cpython/pull/23598

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[Stefan Behnel]

Stefan Behnel  added the comment:

Adding to the list above:

"f_lasti" is used in "libpython.py", which is an almost exact copy of the one 
in CPython's "Tools/gdb/" for debugging Python code in gdb. If that 
implementation changes, we can probably adapt, especially since it uses runtime 
generated code. I don't find a C function for this necessary at this point, 
given that CPython will also need to access its own internals here in some way.

"f_localsplus" is used in the FASTCALL implementation and seems specific to 
Py3.6/7(?) which lacks the vectorcall protocol. So I guess it won't impact 3.8+ 
anymore. (The reason why its access is so complex is that StacklessPython has a 
slightly different frame struct, but we need to be binary(!) compatible with 
both.)

"f_back" is used in the coroutine implementation for correct exception 
traceback building, and the usage is pretty much copied from CPython. I doubt 
that there is much use for setting it outside of "code that tries to 
reimplement CPython behaviour", but Cython has to do exactly that here.

"f_lineno" is also used for traceback generation, because Cython creates frames 
when an exception occurs and needs to tell it what the current code line is. 
It's only used on newly created frames, although I can't guarantee that that 
will never change. Being able to read and set it seems reasonable.

"f_trace" is obviously used for tracing/profiling, and there isn't currently a 
good C-API for that, besides talking to frame struct fields (which is quite 
efficient when it comes to performance).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

> Please don't add any setter functions, that would a major change to the VM 
> and would need a PEP.

There is no such "major change". PyFrameObject structure was fully exposed in 
the public C API since the earliest Python version.

I don't see how adding setter is a major change, since it's already possible to 
directly modify *any* field of PyFrameObject without any kind of protection.

My plan is to have two milestones:

A) Make the structure opaque, so it's not longer possible to directly access it.

B) Deprecate or remove a few getter or setter functions, or move them to the 
internal C API.


I don't think that moving directly to step (B) is a realistic migration plan. 
So far, nobody analyzed all C extensions on PyPI to see how PyFrameObject is 
accessed.

I prefer to move slowly, step by step.

Breaking C extensions which currently *modify* directly PyFrameObject on 
purpose doesn't seem like a reasonable option to me.

--

I'm trying to distinguish functions which should be "safe"/"portable" from the 
ones which may be too "CPython specific":

* I added Include/pyframe.h which is included by Include/Python.h for 
"safe"/"portable" functions:

  * PyFrame_GetLineNumber()
  * PyFrame_GetCode()

* All other functions are added to Include/cpython/frameobject.h which is 
excluded from the limited C API:

  * PyFrame_GetBack()

Note: Compared to accessing directly PyFrameObject.f_code, PyFrame_GetCode() 
also avoids the issue of borrowed references since it returns a strong 
reference.

PyFrame_GetBack() looks specific to the current implementation of CPython. 
Another implementation might decide to not chain frames. Or maybe don't provide 
an easy way to traverse the chain of frames. Or at least, it might be a 
different API than PyFrame_GetBack().

--

> Also, could you remove PyFrame_GetLastInstr(PyFrameObject *frame)?

I didn't add it :-) It's the pending PR 19764.

I didn't merge it since it's unclear to me if it's a good idea to directly 
expose it or not. Cython uses it, but Cython also abuses CPython internals in 
general, usually for best performances :-)

*Maybe* one compromise would be to add a private _PyFrame_GetLastInstr() to 
ease migration to step (A) (replace direct access to the structure with 
function calls).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[Mark Shannon]

Mark Shannon  added the comment:

"maybe a few setter functions as well"
Please don't add any setter functions, that would a major change to the VM and 
would need a PEP.

Also, could you remove PyFrame_GetLastInstr(PyFrameObject *frame)?
The only purpose of `f_lasti` is to get the line number and that can be done 
directly via `PyFrame_GetLineNumber(PyFrameObject *frame)`

Perhaps Stefan can tell us why Cython needs to access the internals of the 
frame object.

--
nosy: +Mark.Shannon, scoder

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 703647732359200c54f1d2e695cc3a06b9a96c9a by Victor Stinner in 
branch 'master':
bpo-40421: Add PyFrame_GetBack() function (GH-19765)
https://github.com/python/cpython/commit/703647732359200c54f1d2e695cc3a06b9a96c9a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

I looked how Cython uses PyFrameObject:

* read f_lasti
* read/write f_back
* write f_lineno
* read f_localsplus
* read/write f_trace

Details:

* Cython/Debugger/libpython.py: code using the Python API of gdb to read 
PyFrameObject.f_lasti. It it used to compute the line number of a frame. The 
python_step() function puts a watch point on "f->f_lasti".
* Cython/Utility/Coroutine.c: set PyFrameObject.f_back using "f->f_back = 
tstate->frame;" and "Py_CLEAR(f->f_back);".
* Cython/Utility/ModuleSetupCode.c, __Pyx_PyFrame_SetLineNumber(): set 
PyFrameObject.f_lineno member. The limited C API flavor of this function does 
nothing, since this member cannot be set in the limited C API.
* Cython/Utility/ObjectHandling.c, __Pyx_PyFrame_GetLocalsplus(): complex 
implementation to access PyFrameObject.f_localsplus:

  // Initialised by module init code.
  static size_t __pyx_pyframe_localsplus_offset = 0;

  #include "frameobject.h"
  // This is the long runtime version of
  // #define __Pyx_PyFrame_GetLocalsplus(frame)  ((frame)->f_localsplus)
  // offsetof(PyFrameObject, f_localsplus) differs between regular C-Python and 
Stackless Python.
  // Therefore the offset is computed at run time from 
PyFrame_type.tp_basicsize. That is feasible,
  // because f_localsplus is the last field of PyFrameObject (checked by 
Py_BUILD_ASSERT_EXPR below).
  #define __Pxy_PyFrame_Initialize_Offsets()  \
((void)__Pyx_BUILD_ASSERT_EXPR(sizeof(PyFrameObject) == 
offsetof(PyFrameObject, f_localsplus) + Py_MEMBER_SIZE(PyFrameObject, 
f_localsplus)), \
 (void)(__pyx_pyframe_localsplus_offset = 
((size_t)PyFrame_Type.tp_basicsize) - Py_MEMBER_SIZE(PyFrameObject, 
f_localsplus)))
  #define __Pyx_PyFrame_GetLocalsplus(frame)  \
(assert(__pyx_pyframe_localsplus_offset), (PyObject **)(((char *)(frame)) + 
__pyx_pyframe_localsplus_offset))

* Cython/Utility/Profile.c, __Pyx_TraceLine(): read PyFrameObject.f_trace.

* Cython/Utility/Profile.c, __Pyx_TraceSetupAndCall(): set 
PyFrameObject.f_trace.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +19086
pull_request: https://github.com/python/cpython/pull/19765

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +19085
pull_request: https://github.com/python/cpython/pull/19764

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:


New changeset a42ca74fa30227e2f89a619332557cf093a937d5 by Victor Stinner in 
branch 'master':
bpo-40421: Add PyFrame_GetCode() function (GH-19757)
https://github.com/python/cpython/commit/a42ca74fa30227e2f89a619332557cf093a937d5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +19079
pull_request: https://github.com/python/cpython/pull/19757

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:


New changeset b8f704d2190125a7750b50cd9b67267b9c20fd43 by Victor Stinner in 
branch 'master':
bpo-40421: Add Include/cpython/code.h header file (GH-19756)
https://github.com/python/cpython/commit/b8f704d2190125a7750b50cd9b67267b9c20fd43


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +19077
pull_request: https://github.com/python/cpython/pull/19756

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 7c59d7c9860cdbaf4a9c26c9142aebd3259d046e by Victor Stinner in 
branch 'master':
bpo-40421: Add pyframe.h header file (GH-19755)
https://github.com/python/cpython/commit/7c59d7c9860cdbaf4a9c26c9142aebd3259d046e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +19076
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19755

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

New submission from STINNER Victor :

Similarly to bpo-39573 (make PyObject opaque) and bpo-39947 (make PyThreadState 
opaque), I propose to add getter functions to access PyFrameObject members 
without exposing the PyFrameObject structure in the C API.

The first step is to identify common usage of the PyFrameObject structure 
inside CPython code base to add getter functions, and maybe a few setter 
functions as well.

frameobject.h is not part of Python.h, but it's part of the public C API. The 
long term plan is to move PyFrameObject structure to the internal C API to hide 
implementation details from the public C API.

--
components: C API
messages: 367527
nosy: vstinner
priority: normal
severity: normal
status: open
title: [C API] Add getter functions for PyFrameObject and maybe move 
PyFrameObject to the internal C API
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com