[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: Oh and obviously, it's not possible possible to define structures which *include* PyObject or PyVarObject if PyObject and PyVarObject become opaque. Example: typedef struct { PyObject ob_base; Py_ssize_t ob_size; /* Number of items in variable part

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: At commit cb15afcccffc6c42cbfb7456ce8db89cd2f77512, I am able to rename PyObject members (to make sure that the structure is not accessed directly), I only had to modify header files: * Py_REFCNT(), Py_SET_REFCNT() * Py_INCREF(), Py_DECREF() * Py_TYPE(),

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: I checked again the list of broken projects listed previously. Fixed: * Cython: https://github.com/cython/cython/commit/d8e93b332fe7d15459433ea74cd29178c03186bd * immutables: https://github.com/MagicStack/immutables/pull/52 * numpy: *

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: > boost https://bugzilla.redhat.com/show_bug.cgi?id=1896382 Fixed by: https://github.com/boostorg/python/commit/500194edb7833d0627ce7a2595fec49d0aae2484 -- ___ Python tracker

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset cb15afcccffc6c42cbfb7456ce8db89cd2f77512 by Victor Stinner in branch 'main': bpo-39573: Py_TYPE becomes a static inline function (GH-28128) https://github.com/python/cpython/commit/cb15afcccffc6c42cbfb7456ce8db89cd2f77512 --

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Change by STINNER Victor : -- pull_requests: +26567 pull_request: https://github.com/python/cpython/pull/28128 ___ Python tracker ___

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: See also bpo-44378: "Py_IS_TYPE(): cast discards ‘const’ qualifier from pointer target type". If Py_TYPE() is converted again to a static inline function which takes a "const PyObject*" type, Py_IS_TYPE() can be modified again at the same time to use

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Pablo Galindo Salgado added the comment: New changeset 6d518bb3a11f9b16098f45b21a13ebe8f537f045 by Pablo Galindo in branch 'main': bpo-44348: Revert "bpo-39573: Py_TYPE becomes a static inline function (GH-26493)" (GH-26596)

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Change by Pablo Galindo Salgado : -- nosy: +pablogsal nosy_count: 13.0 -> 14.0 pull_requests: +25180 pull_request: https://github.com/python/cpython/pull/26596 ___ Python tracker

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Change by Erlend E. Aasland : -- nosy: +erlendaasland ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: Ken Jin: Please open a separated issue for test_exceptions.test_recursion_in_except_handler(). It's not directly related to marking PyObject opaque, as William Pickard explained. See my notes on the stack size and stack overflow on a recursion error on

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

William Pickard added the comment: MSVC by default disables method inlining (/Ob0) when '/Od' is specified on the command line while the optimization options specify '/Ob2'. -- ___ Python tracker

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Ken Jin added the comment: @victor, git bisect tells me the change f3fa63ec75fdbb4a08a10957a5c631bf0c4a5970 caused test_exceptions.ExceptionTests.test_recursion_in_except_handler to stack overflow only on windows debug builds. 3 windows buildbots using python debug mode is affected. Python

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Change by Batuhan Taskaya : -- nosy: +BTaskaya nosy_count: 10.0 -> 11.0 pull_requests: +25148 pull_request: https://github.com/python/cpython/pull/26550 ___ Python tracker ___

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset f3fa63ec75fdbb4a08a10957a5c631bf0c4a5970 by Victor Stinner in branch 'main': bpo-39573: Py_TYPE becomes a static inline function (GH-26493) https://github.com/python/cpython/commit/f3fa63ec75fdbb4a08a10957a5c631bf0c4a5970 --

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Dong-hee Na added the comment: > So I propose again to convert Py_TYPE and Py_SIZE macros to static inline > functions +1 -- ___ Python tracker ___

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: Most projects broken by Py_TYPE and Py_SIZE changes have been fixed since last year. I succeeded to use my new pythoncapi_compat project on multiple C extensions. So I propose again to convert Py_TYPE and Py_SIZE macros to static inline functions:

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Change by STINNER Victor : -- pull_requests: +25089 pull_request: https://github.com/python/cpython/pull/26493 ___ Python tracker ___

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: > boost https://bugzilla.redhat.com/show_bug.cgi?id=1896382 I proposed https://github.com/boostorg/python/pull/330 fix. See also https://github.com/boostorg/python/pull/329 -- ___ Python tracker

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: > bitarray https://bugzilla.redhat.com/show_bug.cgi?id=1897536 I created https://github.com/ilanschnell/bitarray/pull/109 -- ___ Python tracker

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: > pyside2 https://bugzilla.redhat.com/show_bug.cgi?id=1898974 I proposed a fix upstream: https://bugreports.qt.io/browse/PYSIDE-1436 I also wrote a fix in Fedora: https://src.fedoraproject.org/rpms/python-pyside2/pull-request/7 The issue is now tracked in

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: > immutables: https://github.com/MagicStack/immutables/issues/46 I proposed a fix: https://github.com/MagicStack/immutables/pull/52 > mercurial https://bugzilla.redhat.com/show_bug.cgi?id=1897178 I proposed a fix:

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: Status: * Py_SET_REFCNT(), Py_SET_TYPE() and Py_SET_SIZE() functions added to Python 3.9. * Python and Cython have been modified to use Py_TYPE(), Py_SET_REFCNT(), Py_IS_TYPE(), etc. * pythoncapi_compat.h header file has been created to provide new

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Change by Andy Lester : -- nosy: -petdance ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: I created the https://github.com/pythoncapi/upgrade_pythoncapi project which updates automatically a C extension to newer C API. For example, it uses Py_TYPE(), Py_SIZE(), Py_REFCNT(), Py_SET_SIZE(), Py_SET_SIZE() and Py_SET_REFCNT() functions. --

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: > And breezy: > https://bugzilla.redhat.com/show_bug.cgi?id=1890880 (not yet reported > upstream) I reported the issue to breezy upstream: https://bugs.launchpad.net/brz/+bug/1904868 -- ___ Python tracker

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: > And breezy: > https://bugzilla.redhat.com/show_bug.cgi?id=1890880 (not yet reported > upstream) Oh, I didn't notice that this project is broken by the Py_REFCNT() change. I expected it to be broken by the Py_TYPE() change as others. Should we revert the

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset e0251787d85950538cf2490c2c73cc680b153940 by Miro Hrončok in branch 'master': bpo-39573: Remove What's new entry for Py_SIZE() (GH-23375) https://github.com/python/cpython/commit/e0251787d85950538cf2490c2c73cc680b153940 --

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Change by Miro Hrončok : -- pull_requests: +22268 pull_request: https://github.com/python/cpython/pull/23375 ___ Python tracker ___

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset 0e2ac21dd4960574e89561243763eabba685296a by Victor Stinner in branch 'master': bpo-39573: Convert Py_TYPE() and Py_SIZE() back to macros (GH-23366) https://github.com/python/cpython/commit/0e2ac21dd4960574e89561243763eabba685296a --

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: I wrote PR 23366 to revert the Py_TYPE() and Py_SIZE() changes: convert them back to macros to allow again "Py_TYPE(obj) = type;" and "Py_SIZE(obj) = size;" syntaxes. Miro Hrončok: > Another batch of broken projects: > PyPAM

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Change by STINNER Victor : -- pull_requests: +22259 pull_request: https://github.com/python/cpython/pull/23366 ___ Python tracker ___

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Miro Hrončok added the comment: Another batch of broken projects: PyPAM https://bugzilla.redhat.com/show_bug.cgi?id=1897264 bitarray https://bugzilla.redhat.com/show_bug.cgi?id=1897536 boost https://bugzilla.redhat.com/show_bug.cgi?id=1896382 duplicity

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: Miro: > I don't understand the rationale for this change in depth, but does the > benefit outweigh (yet another) backwards incompatibility? I wrote PEP 620 "Hide implementation details from the C API" to explain the rationale:

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: Miro: > This also breaks pycurl: > https://github.com/pycurl/pycurl/pull/660 Right, see my previous comment, another PR was already proposed in May! me: > This change broke pycurl: > https://github.com/pycurl/pycurl/pull/636 Merged pycurl fix:

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Neil Schemenauer added the comment: Correction: I think you *cannot* have it both ways. -- ___ Python tracker ___ ___

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Neil Schemenauer added the comment: > I don't understand the rationale for this change in depth, but > does the benefit outweigh (yet another) backwards incompatibility? I think you can have it both ways. Do you want a C API that is stable over a long period of CPython releases or do you

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Miro Hrončok added the comment: This also breaks pycurl: https://github.com/pycurl/pycurl/pull/660 And breezy: https://bugzilla.redhat.com/show_bug.cgi?id=1890880 (not yet reported upstream) I don't understand the rationale for this change in depth, but does the benefit outweigh (yet

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset 8182cc2e68a3c6ea5d5342fed3f1c76b0521fbc1 by Victor Stinner in branch 'master': bpo-39573: Use the Py_TYPE() macro (GH-21433) https://github.com/python/cpython/commit/8182cc2e68a3c6ea5d5342fed3f1c76b0521fbc1 --

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Change by STINNER Victor : -- pull_requests: +20580 pull_request: https://github.com/python/cpython/pull/21433 ___ Python tracker ___

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Change by William Pickard : -- nosy: +WildCard65 nosy_count: 9.0 -> 10.0 pull_requests: +20410 pull_request: https://github.com/python/cpython/pull/21262 ___ Python tracker

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: > Another idea would be to convert some C extensions of the standard library to > the limited C API. It would ensure that the limited C API contains enough > functions to be useful, but would also notify us directly if the API is > broken. I created

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: See also bpo-40881 "--with-valgrind broken": unicode_release_interned() still used "Py_REFCNT(s) += 1;". It's now fixed by commit c96a61e8163c2d25ed4ac77cf96201fd0bdb945c. -- ___ Python tracker

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: Note: numpy was updated to also the use the macros using ", (void)0": https://github.com/numpy/numpy/commit/f1671076c80bd972421751f2d48186ee9ac808aa -- ___ Python tracker

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset dc24b8a2ac32114313bae519db3ccc21fe45c982 by Victor Stinner in branch 'master': bpo-39573: Porting to Python 3.10: Py_SET_SIZE() macro (GH-20610) https://github.com/python/cpython/commit/dc24b8a2ac32114313bae519db3ccc21fe45c982 --

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: I proposed PR 20610 to enhance the documentation explaining how to port existing to code to Py_SET_SIZE() & cie. -- ___ Python tracker ___

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Change by STINNER Victor : -- pull_requests: +19838 pull_request: https://github.com/python/cpython/pull/20610 ___ Python tracker ___

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: numpy fix defines Py_SET_TYPE() and Py_SET_SIZE() on old Python versions: * https://github.com/numpy/numpy/commit/a96b18e3d4d11be31a321999cda4b795ea9eccaa * https://github.com/numpy/numpy/pull/16417 In the whole numpy code base, only 5 lines have to be

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: Similar macro for Py_SET_TYPE: #if PY_VERSION_HEX < 0x030900A4 # define Py_SET_TYPE(obj, size) do { Py_TYPE(obj) = (size); } while (0) #endif -- ___ Python tracker

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: To port code to Python 3.10, the following macro can be copied/pasted in your code. It defines Py_SET_SIZE() if it's not defined. #if PY_VERSION_HEX < 0x030900A4 # define Py_SET_SIZE(obj, size) do { Py_SIZE(obj) = (size); } while (0) #endif --

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: > bpo-39573: Convert Py_TYPE() to a static inline function (GH-20290) This change broke pycurl: https://github.com/pycurl/pycurl/pull/636 Extract of its current code: """ /* Initialize the type of the new type objects here; doing it here * is

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: > bpo-39573: Convert Py_TYPE() to a static inline function (GH-20290) This change broke two projects: * Cython: https://github.com/cython/cython/commit/d8e93b332fe7d15459433ea74cd29178c03186bd (FIXED) * immutables:

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset fe2978b3b940fe2478335e3a2ca5ad22338cdf9c by Victor Stinner in branch 'master': bpo-39573: Convert Py_REFCNT and Py_SIZE to functions (GH-20429) https://github.com/python/cpython/commit/fe2978b3b940fe2478335e3a2ca5ad22338cdf9c --

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Change by STINNER Victor : -- pull_requests: +19686 pull_request: https://github.com/python/cpython/pull/20429 ___ Python tracker ___

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Dong-hee Na added the comment: New changeset 7d847e29d76b178c2db66b180065771b4d90c78f by Dong-hee Na in branch 'master': bpo-39573: Fix buildbot failure for tupleobject.c (GH-20391) https://github.com/python/cpython/commit/7d847e29d76b178c2db66b180065771b4d90c78f --

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Change by Dong-hee Na : -- pull_requests: +19654 pull_request: https://github.com/python/cpython/pull/20391 ___ Python tracker ___

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Dong-hee Na added the comment: New changeset ad3252bad905d41635bcbb4b76db30d570cf0087 by Dong-hee Na in branch 'master': bpo-39573: Convert Py_TYPE() to a static inline function (GH-20290) https://github.com/python/cpython/commit/ad3252bad905d41635bcbb4b76db30d570cf0087 --

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Change by Dong-hee Na : -- pull_requests: +19565 pull_request: https://github.com/python/cpython/pull/20290 ___ Python tracker ___

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Change by Zackery Spytz : -- nosy: +ZackerySpytz nosy_count: 8.0 -> 9.0 pull_requests: +19294 pull_request: https://github.com/python/cpython/pull/19975 ___ Python tracker ___

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Dong-hee Na added the comment: New changeset 5e8ffe147710e449c2e935a4e2ff5cbd19828a8a by Hai Shi in branch 'master': bpo-39573: Use Py_IS_TYPE to check for types (GH-19882) https://github.com/python/cpython/commit/5e8ffe147710e449c2e935a4e2ff5cbd19828a8a --

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Change by hai shi : -- pull_requests: +19194 pull_request: https://github.com/python/cpython/pull/19882 ___ Python tracker ___ ___

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

Ronald Oussoren added the comment: The incompatibility mentioned in msg366473 is probably fixable by treating the PyObject header the same as the GC head structure. With some care this could mostly maintain binary compatibility by inserting some unused fields in PyObject_HEAD instead of the

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: PyType_FromSpec() and PyType_Spec API are not currently compatible with opaque PyObject. Example: --- #define PyObject_HEADPyObject ob_base; typedef struct { PyObject_HEAD ... } MyObject; static PyType_Spec type_spec = { .name = "MyObject",