Re: [pypy-dev] PyCodeObject incompatibility

[Carl Friedrich Bolz-Tereick]

Hi Dmitry,

I can only reiterate, we would need some information about what that
library is actually for to give more useful information at this point.
As Matti said, most of these APIs cannot be usefully implemented in
PyPy. most jitted functions do not *have* a filled frame object at all
in PyPy. so if you want to access the frames, you need to leave the
jit-produced machine code, which is very slow.

Cheers,

CF

On 15.02.22 04:15, Dmitry Kovner wrote:

Hi, again! First of all, thanks for the fast reply! The answer helped me
a lot. However, during the porting of the extension to the awesome PyPy,
I've got more questions:
1. The extension uses opcodes from cPython's opcode.h header file.
(https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/opcode.h
)
There is no such file in the implementation of C API in PyPy. What could
you recommend as the best alternative?
2. The extension uses some fields of PyThreadState structure. For
example, its frame field
(https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/cpython/pystate.h#L59
).
Is it right to use PyObject_GetAttrString() to access that field in PyPy
C API?
3. There is no function PyFrame_FastToLocals
(https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Objects/frameobject.c#L931
)
in the C API of PyPy. Are there any alternatives?
4. Functions PyEval_SetTrace(), PyEval_SetProfile(),
PyFrame_GetLineNumber() are defined only in stubs.py in PyPy.
(https://foss.heptapod.net/pypy/pypy/-/blob/branch/py3.7/pypy/module/cpyext/stubs.py#L918
)
Is there a way to get that information in PyPy except patching of its
source code?
5. Are there any analogues of all PyTrace_* constants
(https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/cpython/pystate.h#L26
)
in the PyPy C API?
6. There is no function PyFrame_Check()
(https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/frameobject.h#L53
)
in the PyPy C API. Are there any alternatives?

I'm really sorry for the long question! I hope the list of the problems
described above is full enough to port the extension and I will have no
more questions. :)

Best regards, Dmitrii

чт, 10 февр. 2022 г. в 23:07, Armin Rigo mailto:armin.r...@gmail.com>>:

Hi,

On Thu, 10 Feb 2022 at 14:03, Dmitry Kovner mailto:dmit...@lightrun.com>> wrote:
 > Hello! I'm trying to build a low-level C API extension of cPython
to be used in PyPy. The extension extensively uses some fields of
PyCodeObject

(https://github.com/python/cpython/blob/f87e616af038ee8963185e11b96841c81e8ef15a/Include/code.h#L23

):
co_firstlineno, co_stacksize, co_consts and so on. However, it seems
these fields are not defined in the similar structure of the PyPy
implementation:

https://foss.heptapod.net/pypy/pypy/-/blob/branch/py3.8/pypy/module/cpyext/include/code.h#L7

.
Is it possible to get values of these fields using PyPy C API somehow?

Yes, it's a limitation of PyPy not to expose these fields.  If you
want to write portable code that always works, the simplest is to call
`PyObject_GetAttrString(code, "__consts__")` etc.  Remember that you
need to call `Py_DECREF()` on the result at some point.


A bientôt,

Armin.


___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev


___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] PyCodeObject incompatibility

[Matti Picus]

On 15/2/22 05:15, Dmitry Kovner wrote:

Hi, again! First of all, thanks for the fast reply! The answer helped 
me a lot. However, during the porting of the extension to the awesome 
PyPy, I've got more questions:
1. The extension uses opcodes from cPython's opcode.h header file. 
(https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/opcode.h) 
There is no such file in the implementation of C API in PyPy. What 
could you recommend as the best alternative?
2. The extension uses some fields of PyThreadState structure. For 
example, its frame field 
(https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/cpython/pystate.h#L59). 
Is it right to use PyObject_GetAttrString() to access that field in 
PyPy C API?
3. There is no function PyFrame_FastToLocals 
(https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Objects/frameobject.c#L931) 
in the C API of PyPy. Are there any alternatives?
4. Functions PyEval_SetTrace(), PyEval_SetProfile(), 
PyFrame_GetLineNumber() are defined only in stubs.py in PyPy. 
(https://foss.heptapod.net/pypy/pypy/-/blob/branch/py3.7/pypy/module/cpyext/stubs.py#L918) 
Is there a way to get that information in PyPy except patching of its 
source code?
5. Are there any analogues of all PyTrace_* constants 
(https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/cpython/pystate.h#L26) 
in the PyPy C API?
6. There is no function PyFrame_Check() 
(https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/frameobject.h#L53) 
in the PyPy C API. Are there any alternatives?


I'm really sorry for the long question! I hope the list of the 
problems described above is full enough to port the extension and I 
will have no more questions. :)


Best regards, Dmitrii



Even if PyPy could expose the interfaces you desire (and it would be a 
lot of work to do that), you would lose out on speed since the C-API on 
PyPy is much much slower than pure python. If you really want to port 
your extension, you might want to rethink what it is doing and refactor 
it to do so in a PyPy-friendly way: pure python + CFFI where needed to 
interface to libraries that expose C interfaces. The C-API is a layer we 
bolted on top of PyPy to support popular libraries like Cython and 
NumPy, it is not meant for projects that probe CPython internals.



Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] PyCodeObject incompatibility

[Dmitry Kovner]

Hi, again! First of all, thanks for the fast reply! The answer helped me a
lot. However, during the porting of the extension to the awesome PyPy, I've
got more questions:
1. The extension uses opcodes from cPython's opcode.h header file. (
https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/opcode.h)
There is no such file in the implementation of C API in PyPy. What could
you recommend as the best alternative?
2. The extension uses some fields of PyThreadState structure. For example,
its frame field (
https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/cpython/pystate.h#L59).
Is it right to use PyObject_GetAttrString() to access that field in PyPy C
API?
3. There is no function PyFrame_FastToLocals (
https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Objects/frameobject.c#L931)
in the C API of PyPy. Are there any alternatives?
4. Functions PyEval_SetTrace(), PyEval_SetProfile(),
PyFrame_GetLineNumber() are defined only in stubs.py in PyPy. (
https://foss.heptapod.net/pypy/pypy/-/blob/branch/py3.7/pypy/module/cpyext/stubs.py#L918)
Is there a way to get that information in PyPy except patching of its
source code?
5. Are there any analogues of all PyTrace_* constants (
https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/cpython/pystate.h#L26)
in the PyPy C API?
6. There is no function PyFrame_Check() (
https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/frameobject.h#L53)
in the PyPy C API. Are there any alternatives?

I'm really sorry for the long question! I hope the list of the problems
described above is full enough to port the extension and I will have no
more questions. :)

Best regards, Dmitrii

чт, 10 февр. 2022 г. в 23:07, Armin Rigo :

> Hi,
>
> On Thu, 10 Feb 2022 at 14:03, Dmitry Kovner  wrote:
> > Hello! I'm trying to build a low-level C API extension of cPython to be
> used in PyPy. The extension extensively uses some fields of PyCodeObject (
> https://github.com/python/cpython/blob/f87e616af038ee8963185e11b96841c81e8ef15a/Include/code.h#L23):
> co_firstlineno, co_stacksize, co_consts and so on. However, it seems these
> fields are not defined in the similar structure of the PyPy
> implementation:
> https://foss.heptapod.net/pypy/pypy/-/blob/branch/py3.8/pypy/module/cpyext/include/code.h#L7.
> Is it possible to get values of these fields using PyPy C API somehow?
>
> Yes, it's a limitation of PyPy not to expose these fields.  If you
> want to write portable code that always works, the simplest is to call
> `PyObject_GetAttrString(code, "__consts__")` etc.  Remember that you
> need to call `Py_DECREF()` on the result at some point.
>
>
> A bientôt,
>
> Armin.
>
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] PyCodeObject incompatibility

[Armin Rigo]

Hi,

On Thu, 10 Feb 2022 at 14:20, Carl Friedrich Bolz-Tereick  wrote:
> are doing with them? But yes, as Armin write, accessing the the .co_*
> attributes with PyObject_GetAttrString is an approach!

Oops, sorry.  Python 3 renamed various attributes to use the
double-underscore convention, like on function objects, but skipped
code objects for some reason.  So I meant
`PyObject_GetAttrString(code, "co_consts")`.



Armin
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] PyCodeObject incompatibility

[Carl Friedrich Bolz-Tereick]

Hi Dmitry,

those fields are quite implementation-dependant, it's not clear to me
that their use can be just carried over from CPython in all cases. Could
you maybe tell us which exception module uses them or at least what you
are doing with them? But yes, as Armin write, accessing the the .co_*
attributes with PyObject_GetAttrString is an approach!

Cheers,

CF

On 10.02.22 07:01, Dmitry Kovner wrote:

Hello! I'm trying to build a low-level C API extension of cPython to be
used in PyPy. The extension extensively uses some fields of PyCodeObject
(https://github.com/python/cpython/blob/f87e616af038ee8963185e11b96841c81e8ef15a/Include/code.h#L23
):
 co_firstlineno, co_stacksize, co_consts
and so on. However, it seems these fields are not defined in the similar
structure of the PyPy  implementation:
https://foss.heptapod.net/pypy/pypy/-/blob/branch/py3.8/pypy/module/cpyext/include/code.h#L7
.
Is it possible to get values of these fields using PyPy C API somehow?

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev


___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] PyCodeObject incompatibility

[Armin Rigo]

Hi,

On Thu, 10 Feb 2022 at 14:03, Dmitry Kovner  wrote:
> Hello! I'm trying to build a low-level C API extension of cPython to be used 
> in PyPy. The extension extensively uses some fields of PyCodeObject 
> (https://github.com/python/cpython/blob/f87e616af038ee8963185e11b96841c81e8ef15a/Include/code.h#L23):
>  co_firstlineno, co_stacksize, co_consts and so on. However, it seems these 
> fields are not defined in the similar structure of the PyPy  implementation: 
> https://foss.heptapod.net/pypy/pypy/-/blob/branch/py3.8/pypy/module/cpyext/include/code.h#L7.
>  Is it possible to get values of these fields using PyPy C API somehow?

Yes, it's a limitation of PyPy not to expose these fields.  If you
want to write portable code that always works, the simplest is to call
`PyObject_GetAttrString(code, "__consts__")` etc.  Remember that you
need to call `Py_DECREF()` on the result at some point.


A bientôt,

Armin.
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] PyCodeObject incompatibility

[Dmitry Kovner]

Hello! I'm trying to build a low-level C API extension of cPython to be
used in PyPy. The extension extensively uses some fields of PyCodeObject (
https://github.com/python/cpython/blob/f87e616af038ee8963185e11b96841c81e8ef15a/Include/code.h#L23):
co_firstlineno, co_stacksize, co_consts
and so on. However, it seems these fields are not defined in the similar
structure of the PyPy  implementation:
https://foss.heptapod.net/pypy/pypy/-/blob/branch/py3.8/pypy/module/cpyext/include/code.h#L7.
Is it possible to get values of these fields using PyPy C API somehow?
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev