[issue46600] Python built with clang -O0 allocates 10x more stack memory than clang -O3 on a Python function call

2022-02-01 Thread STINNER Victor
STINNER Victor added the comment: stack_overflow-4.py output depending on the compiler and compiler flags. gcc -O3 (./configure): --- test_python_call: 11904 calls before crash, stack: 704.1 bytes/call test_python_iterator: 17460 calls before crash, stack: 480.0 bytes/call test_python_getitem

[issue46600] Python built with clang -O0 allocates 10x more stack memory than clang -O3 on a Python function call

2022-02-01 Thread STINNER Victor
STINNER Victor added the comment: stack_overflow-4.py: Update script from bpo-30866 to measure stack memory usage before Python crash or raises a RecursionError. I had to modify the script since calling a Python function from a Python function no longer allocates (additional) memory

[issue46600] Python built with clang -O0 allocates 10x more stack memory than clang -O3 on a Python function call

2022-02-01 Thread STINNER Victor
STINNER Victor added the comment: See also bpo-30866: "Add _testcapi.stack_pointer() to measure the C stack consumption". -- ___ Python tracker <https://bugs.python.o

[issue46600] Python built with clang -O0 allocates 10x more stack memory than clang -O3 on a Python function call

2022-02-01 Thread STINNER Victor
STINNER Victor added the comment: Previous issues about stack memory usage, work done in 2017: * bpo-28870: Reduce stack consumption of PyObject_CallFunctionObjArgs() and like * bpo-29227: Reduce C stack consumption in function calls * bpo-29465: Modify _PyObject_FastCall() to reduce stack

[issue46600] Python built with clang -O0 allocates 10x more stack memory than clang -O3 on a Python function call

2022-02-01 Thread STINNER Victor
STINNER Victor added the comment: This issue is a follow-up of bpo-46542 "test_json and test_lib2to3 crash on s390x Fedora Clang 3.x buildbot". -- ___ Python tracker <https://bugs.python.o

[issue46600] Python built with clang -O0 allocates 10x more stack memory than clang -O3 on a Python function call

2022-02-01 Thread STINNER Victor
STINNER Victor added the comment: GH-31052 enables -Og when using clang and ./configure --with-pydebug and so the example uses 736 bytes instead of 9,104 bytes. -- ___ Python tracker <https://bugs.python.org/issue46

[issue46600] Python built with clang -O0 allocates 10x more stack memory than clang -O3 on a Python function call

2022-02-01 Thread STINNER Victor
Change by STINNER Victor : -- keywords: +patch pull_requests: +29234 stage: -> patch review pull_request: https://github.com/python/cpython/pull/31052 ___ Python tracker <https://bugs.python.org/issu

[issue46600] Python built with clang -O0 allocates 10x more stack memory than clang -O3 on a Python function call

2022-02-01 Thread STINNER Victor
New submission from STINNER Victor : Measure using this script on the main branch (commit 108e66b6d23efd0fc2966163ead9434b328c5f17): --- import _testcapi def f(): yield _testcapi.stack_pointer() print(_testcapi.stack_pointer() - next(f())) --- Stack usage depending on the compiler

[issue43540] importlib: Document how to replace load_module() in What's New in Python 3.10

2022-01-31 Thread STINNER Victor
STINNER Victor added the comment: Python 3.10 is released. It's too late, I close the issue. -- resolution: -> out of date stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.or

[issue46542] test_json and test_lib2to3 crash on s390x Fedora Clang 3.x buildbot

2022-01-31 Thread STINNER Victor
STINNER Victor added the comment: s390x Fedora Clang 3.x buildbot is back to green, I close the issue. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed versions: +Python 3.10, Python 3.9 ___ Python tracke

[issue42784] issues with object.h includes

2022-01-31 Thread STINNER Victor
STINNER Victor added the comment: There is an on-going work to move more and more header files into Include/cpython/ subdirectory, or even to the Include/internal/ directory. Examples: * bpo-35134: Move cellobject.h, classobject.h, context.h, funcobject.h, genobject.h and longintrepr.h

[issue46166] Get "self" args or non-null co_varnames from frame object with C-API

2022-01-31 Thread STINNER Victor
STINNER Victor added the comment: It would be nice to have a PyFrame_GetVariable(frame, "self") function: get the value of the "frame" variable of the specified frame object. -- ___ Python tracker <https://bug

[issue46542] test_json and test_lib2to3 crash on s390x Fedora Clang 3.x buildbot

2022-01-31 Thread STINNER Victor
STINNER Victor added the comment: The regression was introduced by GH-30855: "bpo-46329: Split calls into precall and call instructions" (commit 89fd7c34520aac493a8784a221366ed04452612b). -- ___ Python tracker <https://bugs.python.o

[issue46542] test_json and test_lib2to3 crash on s390x Fedora Clang 3.x buildbot

2022-01-31 Thread STINNER Victor
Change by STINNER Victor : -- pull_requests: +29218 pull_request: https://github.com/python/cpython/pull/31035 ___ Python tracker <https://bugs.python.org/issue46

[issue46542] test_json and test_lib2to3 crash on s390x Fedora Clang 3.x buildbot

2022-01-31 Thread STINNER Victor
STINNER Victor added the comment: test_lib2to3 crash if Python is built with clang when running test_all_project_files() of lib2to3.tests.test_all_fixers.Test_all. This test crash on processing the file: "Lib/lib2to3/tests/data/infinite_recursion.py". The GH-30855 change increased

[issue35081] Move internal headers to Include/internal/

2022-01-30 Thread STINNER Victor
Change by STINNER Victor : -- resolution: -> fixed status: open -> closed ___ Python tracker <https://bugs.python.org/issue35081> ___ ___ Python-bugs-list

[issue35134] Add a new Include/cpython/ subdirectory for the "CPython API" with implementation details

2022-01-30 Thread STINNER Victor
Change by STINNER Victor : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.or

[issue46582] Windows builds fail: fatal error RC1116: RC terminating after preprocessor errors

2022-01-30 Thread STINNER Victor
STINNER Victor added the comment: https://devguide.python.org/#status-of-python-branches -- ___ Python tracker <https://bugs.python.org/issue46582> ___ ___ Pytho

[issue46582] Windows builds fail: fatal error RC1116: RC terminating after preprocessor errors

2022-01-30 Thread STINNER Victor
STINNER Victor added the comment: Python 3.8 no longer accept bugfixes, I close this issue. -- resolution: -> wont fix stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.or

[issue40170] [C API] Prepare PyTypeObject structure for a stable ABI: avoid accessing members in the public API

2022-01-28 Thread STINNER Victor
STINNER Victor added the comment: I close the issue. While this issue is not fully fixed, it's a milestone of my "stable ABI" goal. I prefer to address remaining issues in new separted issues. This issues is not fully fixed, there are are still a 4 macros which access directly Py

[issue40170] [C API] Prepare PyTypeObject structure for a stable ABI: avoid accessing members in the public API

2022-01-28 Thread STINNER Victor
Change by STINNER Victor : -- title: [C API] Make PyTypeObject structure an opaque structure in the public C API -> [C API] Prepare PyTypeObject structure for a stable ABI: avoid accessing members in the public API ___ Python tracker <

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

2022-01-28 Thread STINNER Victor
STINNER Victor added the comment: Changes already done: * Macros converted to regular functions: * PyObject_GET_WEAKREFS_LISTPTR(); add internal inline _PyObject_GET_WEAKREFS_LISTPTR() * PyType_SUPPORTS_WEAKREFS(); add internal inline _PyType_SUPPORTS_WEAKREFS

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

2022-01-28 Thread STINNER Victor
STINNER Victor added the comment: The negative refcount issue has been fixed ;-) At commit 9a241271139a317597aca71d5971346b2cfe7dbd, Python now leaks exactly 0 reference count and 0 memory block at exit: $ ./python -I -X showrefcount -c pass [0 refs, 0 blocks

[issue39511] [subinterpreters] Per-interpreter singletons (None, True, False, etc.)

2022-01-28 Thread STINNER Victor
STINNER Victor added the comment: > On such platforms, the `PyGet_Foo` API can be on equal footing with the > legacy `Py_Foo` statics, i.e. both would do the same thing. That's how I've > done it in my experiment. The obvious problem is that on platforms without > compiler sup

[issue46417] Clear static types in Py_Finalize() for embedded Python

2022-01-28 Thread STINNER Victor
STINNER Victor added the comment: New changeset 9a241271139a317597aca71d5971346b2cfe7dbd by Victor Stinner in branch 'main': bpo-46417: _PyStructSequence_FiniType() updates _Py_RefTotal (GH-30988) https://github.com/python/cpython/commit/9a241271139a317597aca71d5971346b2cfe7dbd

[issue46449] Deep-freezed modules create inconsistency in sys.gettotalrefcount() (_Py_Reftotal)

2022-01-28 Thread STINNER Victor
STINNER Victor added the comment: With my additional GH-30988 fix, msg411075 example no longer leaks :-) --- Loop #1: 2 refs Loop #2: 2 refs Loop #3: 2 refs ... Loop #98: 2 refs Loop #99: 2 refs Loop #100: 2 refs --- I close the issue. Thanks Christian and Kumar for the fix

[issue46417] Clear static types in Py_Finalize() for embedded Python

2022-01-28 Thread STINNER Victor
Change by STINNER Victor : -- pull_requests: +29167 pull_request: https://github.com/python/cpython/pull/30988 ___ Python tracker <https://bugs.python.org/issue46

[issue46449] Deep-freezed modules create inconsistency in sys.gettotalrefcount() (_Py_Reftotal)

2022-01-28 Thread STINNER Victor
STINNER Victor added the comment: New changeset 5a9e423473bf2c4eb32a0982e8d73420875db1da by Kumar Aditya in branch 'main': bpo-46449: deepfreeze get_code() now returns strong ref (GH-30987) https://github.com/python/cpython/commit/5a9e423473bf2c4eb32a0982e8d73420875db1da

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

2022-01-28 Thread STINNER Victor
STINNER Victor added the comment: > `_Py_Identifier` has been useful but at this point there is a faster and > simpler approach we could take as a replacement: statically initialize the > objects as fields on `_PyRuntimeState` and reference them directly through a > macro.

[issue43536] 3.9.2 --without-pymalloc --with-pydebug --with-valgrind: test failed: test_posix

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: Without the test output, we cannot investigate the issue. I close it. -- nosy: +vstinner resolution: -> out of date stage: -> resolved status: open -> closed ___ Python tracker <https://bug

[issue40077] Convert static types to heap types: use PyType_FromSpec()

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: I marked bpo-23769 "valgrind reports leaks for test_zipimport" as duplicate of this issue. At exit, Python doesn't clear the static types of the _collections, itertools and _struct extensions: * itertools.accumulate * itertools.co

[issue23769] valgrind reports leaks for test_zipimport

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: > valgrind reports leaks for test_zipimport It's not strictly a memory leak: running the same code multiple time doesn't leak memory. It's just that Python allocates memory once and never releases it (doesn't releast it at exit). --- leak4.py: Upda

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: I marked bpo-35774 as a duplicate of this issue. -- ___ Python tracker <https://bugs.python.org/issue1635741> ___ ___ Pytho

[issue35774] ASAN, memory leak

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: > 1. Run python > 2. Ctrl + D It seems like this scenario has recently been fixed in bpo-1635741: $ ./python -I -X showrefcount Python 3.11.0a4+ (heads/main:18ea973c21, Jan 28 2022, 01:38:10) [GCC 11.2.1 20211203 (Red Hat 11.2.1-7)] on linux Type

[issue45113] [subinterpreters][C API] Add a new function to create PyStructSequence from Heap.

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: Raymond: > Is the subinterpreters work still on hold pending a PEP? While sub-interpreters is one motivation to convert static types to heap types, it's not the only one. IMO it's better to use the "embedded Python" use case to justify these ch

[issue45113] [subinterpreters][C API] Add a new function to create PyStructSequence from Heap.

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: Quick update on this closed issue. I landed on this issue from Erlend's SC request: https://github.com/python/steering-council/issues/99 > Oh! I assumed this bug wasn't resolved, but it is -- in bpo-34784. Sorry, I > should have checked

[issue46542] test_json and test_lib2to3 crash on s390x Fedora Clang 3.x buildbot

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: New changeset e7a6285f1be18992191599792524d3aa6aedfa55 by Victor Stinner in branch 'main': bpo-46542: test_json uses support.infinite_recursion() (GH-30972) https://github.com/python/cpython/commit/e7a6285f1be18992191599792524d3aa6aedfa55

[issue46542] test_json and test_lib2to3 crash on s390x Fedora Clang 3.x buildbot

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: I managed to reproduce the test_json crash on the "Fedora Stable ppc64le" buildbot using clang. Commands: --- ./configure --with-pydebug CC=clang make -j10 ./python -m test -v test_json --- I wrote GH-30

[issue46542] test_json and test_lib2to3 crash on s390x Fedora Clang 3.x buildbot

2022-01-27 Thread STINNER Victor
Change by STINNER Victor : -- keywords: +patch pull_requests: +29151 stage: -> patch review pull_request: https://github.com/python/cpython/pull/30972 ___ Python tracker <https://bugs.python.org/issu

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: New changeset 18ea973c21ee4a6adc26be41027881043fa498eb by Victor Stinner in branch 'main': bpo-40170: Remove PyHeapType_GET_MEMBERS() macro (GH-30942) https://github.com/python/cpython/commit/18ea973c21ee4a6adc26be41027881043fa498eb

[issue45476] [C API] PEP 674: Disallow using macros (Py_TYPE and Py_SIZE) as l-value

2022-01-27 Thread STINNER Victor
Change by STINNER Victor : -- title: [C API] PEP 674: Disallow using macros as l-value -> [C API] PEP 674: Disallow using macros (Py_TYPE and Py_SIZE) as l-value ___ Python tracker <https://bugs.python.org/issu

[issue45476] [C API] PEP 674: Disallow using macros as l-value

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: > * zodbpickle-2.2.0: src/zodbpickle/_pickle_33.c Technically, zodbpickle works on Python 3.11 and is not impacted by the Py_SIZE() change. _pickle_33.c redefines the Py_SIZE() macro to continue using as an l-value: https://github.com/zopefoundat

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: New changeset 0575551f69ba9c999835bfb176a543d468083c03 by Victor Stinner in branch 'main': bpo-40170: Move _Py_GetAllocatedBlocks() to pycore_pymem.h (GH-30943) https://github.com/python/cpython/commit/0575551f69ba9c999835bfb176a543d468083c03

[issue46417] Clear static types in Py_Finalize() for embedded Python

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: I close the issue. I cleared most static types at exit. Following work can be done in bpo-40077 or other issues. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Pytho

[issue46417] Clear static types in Py_Finalize() for embedded Python

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: New changeset 6c6a153dee132116611f2d5df0689a5a605f62b6 by Victor Stinner in branch 'main': bpo-46417: signal: move siginfo_type to the module state (GH-30964) https://github.com/python/cpython/commit/6c6a153dee132116611f2d5df0689a5a605f62b6

[issue46417] Clear static types in Py_Finalize() for embedded Python

2022-01-27 Thread STINNER Victor
Change by STINNER Victor : -- pull_requests: +29143 pull_request: https://github.com/python/cpython/pull/30964 ___ Python tracker <https://bugs.python.org/issue46

[issue40077] Convert static types to heap types: use PyType_FromSpec()

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: > And I found 135 more static types with this command Of these 135 static types, most are cleared since bpo-46417 was implemented: * 103 types are cleared by _PyTypes_FiniTypes() * 15 types are cleared by _PyIO_Fini() * the remaining 17 types are not clea

[issue40077] Convert static types to heap types: use PyType_FromSpec()

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: And I found 135 more static types with this command: --- grep -E '^PyTypeObject [a-zA-Z_0-9]+ *(;|= *{|) *$' $(find -name "*.c") --- Types: Objects/cellobject.c: PyCell_Type Objects/sliceobject.c: PyEllipsis_Type Objects/sliceobject.c: PySlice_Ty

[issue40077] Convert static types to heap types: use PyType_FromSpec()

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: I used the following shell command to search remaining static types: --- grep -E 'static PyTypeObject [a-zA-Z_0-9]+ *(;|= *{|) *$' $(find -name "*.c") --- I found 86 static types in 17 files: * PC/_msi.c: * msidb_Type * msiview_Type * r

[issue46551] Provide number of workers option for fast PGO build time

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: > When I tested with run -m test --pgo -j8, it doesn't affect to optimized > result with fast build time. You only test libregrtest.main and libregrtest.runtest_mp modules which don't execute code. Does it mean that running tests is useless to train t

[issue43916] Check that new heap types cannot be created uninitialised: add Py_TPFLAGS_DISALLOW_INSTANTIATION type flag

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: > 3.9 is still affected; we should fix those types first. I'm against backporting the new type flag, but we can try to set explicitly tp_set to NULL *after* calling PyType_Ready(). -- ___ Python tracker <

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: Thanks to recent enhancements, epecially in bpo-46417, the last memory blocks are now released at Python exit! The initial issue has been fixed!!! This bug was 15 years old! Current main branch: $ ./python -I -X showrefcount -c pass [-5 refs, 0 blocks] &q

[issue46449] Deep-freezed modules create inconsistency in sys.gettotalrefcount() (_Py_Reftotal)

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: The bpo-46476 added _Py_Deepfreeze_Fini() and _PyStaticCode_Dealloc() functions: commit c7f810b34d91a5c2fbe0a8385562015d2dd961f2. If we need to ajust _Py_RefTotal manually, *maybe* it can be done there? I don't understand well how static/immortal code

[issue46476] Not all memory allocated by _Py_Quicken() is released at Python exit

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: Thanks Kumar! Your change fixed my issue: $ ./python -I -X showrefcount -c pass [-5 refs, 0 blocks] Python no longer leaks memory at exit: it "leaks" exactly *zero* memory block. The negative reference count is likely caused by

[issue46476] Not all memory allocated by _Py_Quicken() is released at Python exit

2022-01-27 Thread STINNER Victor
STINNER Victor added the comment: New changeset c7f810b34d91a5c2fbe0a8385562015d2dd961f2 by Kumar Aditya in branch 'main': bpo-46476: Fix memory leak in code objects generated by deepfreeze (GH-30853) https://github.com/python/cpython/commit/c7f810b34d91a5c2fbe0a8385562015d2dd961f2

[issue45476] [C API] PEP 674: Disallow using macros as l-value

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: > recordclass-0.16.3: lib/recordclass/_dataobject.c + code generated by Cython I created: https://bitbucket.org/intellimath/recordclass/pull-requests/1/python-311-support-use-py_set_size -- ___ Python trac

[issue45476] [C API] PEP 674: Disallow using macros as l-value

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: > * scipy-1.7.3: scipy/_lib/boost/boost/python/object/make_instance.hpp This is a vendored the Boost.org python module which has already been fixed in boost 1.78.0 (commit: January 2021) by: https://github.com/boostorg/python/com

[issue45476] [C API] PEP 674: Disallow using macros as l-value

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: > * guppy3-3.1.2: src/sets/bitset.c and src/sets/nodeset.c I created: https://github.com/zhuyifei1999/guppy3/pull/40 -- ___ Python tracker <https://bugs.python.org/issu

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

2022-01-26 Thread STINNER Victor
Change by STINNER Victor : -- pull_requests: +29122 pull_request: https://github.com/python/cpython/pull/30943 ___ Python tracker <https://bugs.python.org/issue40

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: New changeset 6b491b9dc0b0fdfd1f07ea4e2151236186d8e7e6 by Victor Stinner in branch 'main': bpo-40170: Remove _Py_GetAllocatedBlocks() function (GH-30940) https://github.com/python/cpython/commit/6b491b9dc0b0fdfd1f07ea4e2151236186d8e7e6

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: New changeset af32b3ef1fbad3c2242627a14398320960a0cb45 by Victor Stinner in branch 'main': bpo-40170: PyType_SUPPORTS_WEAKREFS() becomes a regular function (GH-30938) https://github.com/python/cpython/commit/af32b3ef1fbad3c2242627a14398320960a0cb45

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

2022-01-26 Thread STINNER Victor
Change by STINNER Victor : -- pull_requests: +29121 pull_request: https://github.com/python/cpython/pull/30942 ___ Python tracker <https://bugs.python.org/issue40

[issue46542] test_json and test_lib2to3 crash on s390x Fedora Clang 3.x buildbot

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: These 2 tests seem to check for RecursionError. See my notes: https://pythondev.readthedocs.io/unstable_tests.html#unlimited-recursion -- ___ Python tracker <https://bugs.python.org/issue46

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

2022-01-26 Thread STINNER Victor
Change by STINNER Victor : -- pull_requests: +29119 pull_request: https://github.com/python/cpython/pull/30940 ___ Python tracker <https://bugs.python.org/issue40

[issue46542] test_json and test_lib2to3 crash on s390x Fedora Clang 3.x buildbot

2022-01-26 Thread STINNER Victor
New submission from STINNER Victor : s390x Fedora Clang 3.x buildbot: https://buildbot.python.org/all/#/builders/3/builds/1385 This change may be caused by this buildbot configuration change: https://github.com/python/buildmaster-config/commit/8fbb7492d4509df074750bc1a8ea69812ff53aae --- 0:05

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

2022-01-26 Thread STINNER Victor
Change by STINNER Victor : -- pull_requests: +29117 pull_request: https://github.com/python/cpython/pull/30938 ___ Python tracker <https://bugs.python.org/issue40

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: In the top 5000 PyPI projects, the _PyObject_SIZE() and _PyObject_VAR_SIZE() functions are used by 7 projects: * Cython-0.29.26 * frozendict-2.2.0: implement "sizeof" function, found in copies of Objects/dictobject.c file * JPype1-1.3.0 * nu

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: I searched for "_PyObject_DebugMallocStats" in top 5000 PyPI projects. There is a single project using it: guppy3. Extract of guppy3 src/heapy/xmemstats.c: ... dlptr__PyObject_DebugMallocStats = addr_of_symbol("_PyObject_DebugMallocStat

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: I searched for "_PyGC_FINALIZED" in top 5000 PyPI projects. It seems like only Cython is impacted. ddtrace and guppy3 use directly the internal C API. == Cython 0.29.26 == * Cython/Compiler/ModuleNode.py: finalised_check = '!_PyGC_FINALIZED(o)

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: TODO: * Macros still accessing directly PyTypeObject members: * PyHeapType_GET_MEMBERS() * PySequence_ITEM() * _PyObject_SIZE() * _PyObject_VAR_SIZE() * PyType_SUPPORTS_WEAKREFS() * Try again to apply "bpo-40170: PyType_HasFeature() now a

[issue38472] setup.py: GCC detection is broken when cross-compiling with a German locale

2022-01-26 Thread STINNER Victor
Change by STINNER Victor : -- title: GCC detection in setup.py is broken -> setup.py: GCC detection is broken when cross-compiling with a German locale ___ Python tracker <https://bugs.python.org/issu

[issue38472] GCC detection in setup.py is broken

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: The workaround for this bug is to build Python using the command: LC_ALL=C make rather than running: make -- ___ Python tracker <https://bugs.python.org/issue38

[issue38472] GCC detection in setup.py is broken

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: New changeset a9503ac39474a9cb1b1935ddf159c0d9672b04b6 by Victor Stinner in branch 'main': bpo-38472: setup.py uses LC_ALL=C to check the C compiler (GH-30929) https://github.com/python/cpython/commit/a9503ac39474a9cb1b1935ddf159c0d9672b04b6

[issue38472] GCC detection in setup.py is broken

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: FYI distutils.unixccompiler has a private _is_gcc() function: def _is_gcc(self, compiler_name): # clang uses same syntax for rpath as gcc return any(name in compiler_name for name in ("gcc", "g++", &q

[issue45476] [C API] PEP 674: Disallow using macros as l-value

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: > pickle5-0.0.12: pickle5/_pickle.c This project is a backport targeting Python 3.7 and older. I'm not sure if it makes sense to update to it to Python 3.11. It's the same for pysha3 which targets Python <

[issue45476] [C API] PEP 674: Disallow using macros as l-value

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: > datatable-1.0.0.tar.gz I created https://github.com/h2oai/datatable/pull/3231 -- ___ Python tracker <https://bugs.python.org/issu

[issue38472] GCC detection in setup.py is broken

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: Ok, I reproduced the issue. I wrote PR 30929 to fix it. The issue only occurs when cross-compiling Python with GCC and a German locale. I can reproduce the issue on Fedora 35: $ LC_ALL=de_DE gcc -E -v Es werden eingebaute Spezifikationen verwendet

[issue38472] GCC detection in setup.py is broken

2022-01-26 Thread STINNER Victor
Change by STINNER Victor : -- keywords: +patch pull_requests: +29108 stage: -> patch review pull_request: https://github.com/python/cpython/pull/30929 ___ Python tracker <https://bugs.python.org/issu

[issue43916] Mark static types newly converted to heap types as immutable: add Py_TPFLAGS_DISALLOW_INSTANTIATION type flag

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: Can we close this issue? Or is there a remaining task? -- ___ Python tracker <https://bugs.python.org/issue43916> ___ ___ Pytho

[issue43916] Mark static types newly converted to heap types as immutable: add Py_TPFLAGS_DISALLOW_INSTANTIATION type flag

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: In Python 3.11, 41 types are declared explicitly with the Py_TPFLAGS_DISALLOW_INSTANTIATION flag: * _curses_panel.panel * _dbm.dbm * _gdbm.gdbm * _hashlib.HASH * _hashlib.HASHXOF * _hashlib.HMAC * _md5.md5 * _multibytecodec.MultibyteCodec * _sha1.sha1

[issue43908] array.array should remain immutable: add Py_TPFLAGS_IMMUTABLETYPE flag

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: In Python 3.11, 68 heap types have the Py_TPFLAGS_IMMUTABLETYPE flag: * _blake2.blake2b * _blake2.blake2s * _bz2.BZ2Compressor * _bz2.BZ2Decompressor * _csv.Dialect * _csv.reader * _csv.writer * _dbm.dbm * _gdbm.gdbm * _hashlib.HASH * _hashlib.HASHXOF

[issue43908] array.array should remain immutable: add Py_TPFLAGS_IMMUTABLETYPE flag

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: > Should the immutable flag also be applied to the heap types converted in and > before Python 3.9 before closing this issue? I don't think that Python 3.9 should be changed. It's too late. IMO this issue is not important enough to introduce a new typ

[issue45476] [C API] PEP 674: Disallow using macros as l-value

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: > In the PyPI top 5000, I found two projects using PyDescr_TYPE() and > PyDescr_NAME() as l-value: M2Crypto and mecab-python3. In both cases, it was > code generated by SWIG I created bpo-46538 "[C API] Make the PyDescrObject structure opa

[issue46538] [C API] Make the PyDescrObject structure opaque: PyDescr_NAME() and PyDescr_TYPE()

2022-01-26 Thread STINNER Victor
Change by STINNER Victor : -- title: [C API] Make the PyDescrObject structure opaque -> [C API] Make the PyDescrObject structure opaque: PyDescr_NAME() and PyDescr_TYPE() ___ Python tracker <https://bugs.python.org/issu

[issue46538] [C API] Make the PyDescrObject structure opaque

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: > The problem of the PyDescr_SET_NAME() and PyDescr_SET_Type() approach is that > SWIG code is outdated: it doesn't initialized the PyDescrObject.d_qualname > member added to Python 3.3 (bpo-13577, commit > 9d57481f043cb9b94bfc45c1ee041415d

[issue35134] Add a new Include/cpython/ subdirectory for the "CPython API" with implementation details

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: New changeset d4a85f104bf9d2e368f25c9a567eaaa2cc39a96a by Victor Stinner in branch 'main': bpo-35134: Add Include/cpython/descrobject.h (GH-30923) https://github.com/python/cpython/commit/d4a85f104bf9d2e368f25c9a567eaaa2cc39a96a

[issue46538] [C API] Make the PyDescrObject structure opaque

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: Since making PyDescrObject opaque is not really worth it, I close this issue. I mean, it's worth it, but there are more important structures like PyObject, PyTypeObject or PyListObject. -- resolution: -> rejected stage: -> resolved status

[issue46538] [C API] Make the PyDescrObject structure opaque

2022-01-26 Thread STINNER Victor
Change by STINNER Victor : Added file: https://bugs.python.org/file50589/swig_pydescr_new.patch ___ Python tracker <https://bugs.python.org/issue46538> ___ ___ Python-bug

[issue46538] [C API] Make the PyDescrObject structure opaque

2022-01-26 Thread STINNER Victor
Change by STINNER Victor : Removed file: https://bugs.python.org/file50588/swig_pydescr_new.patch ___ Python tracker <https://bugs.python.org/issue46538> ___ ___ Pytho

[issue46538] [C API] Make the PyDescrObject structure opaque

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: swig_pydescr_new.patch: SWIG patch adding PyDescr_New() and using it. -- Added file: https://bugs.python.org/file50588/swig_pydescr_new.patch ___ Python tracker <https://bugs.python.org/issue46

[issue46538] [C API] Make the PyDescrObject structure opaque

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: pythoncapi_compat_pydescr_new.patch: pythoncapi_compat patch adding PyDescr_New() function with tests. -- Added file: https://bugs.python.org/file50587/pythoncapi_compat_pydescr_new.patch ___ Python tracker

[issue46538] [C API] Make the PyDescrObject structure opaque

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: pydescr_new.patch: Python patch adding the PyDescr_New() function. -- Added file: https://bugs.python.org/file50586/pydescr_new.patch ___ Python tracker <https://bugs.python.org/issue46

[issue46538] [C API] Make the PyDescrObject structure opaque

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: pydescr_set_type.patch: Python patch to add PyDescr_SET_TYPE() and PyDescr_SET_NAME() functions. -- keywords: +patch Added file: https://bugs.python.org/file50585/pydescr_set_type.patch ___ Python tracker <ht

[issue46538] [C API] Make the PyDescrObject structure opaque

2022-01-26 Thread STINNER Victor
New submission from STINNER Victor : While working on the PEP 674 implementation, I noticed that PyDescr_NAME() and PyDescr_TYPE() macros are used as l-value by two projects: M2Crypto and mecab-python3. Both are code generated by SWIG. M2Crypto-0.38.0/src/SWIG/_m2crypto_wrap.c and mecab

[issue35134] Add a new Include/cpython/ subdirectory for the "CPython API" with implementation details

2022-01-26 Thread STINNER Victor
Change by STINNER Victor : -- pull_requests: +29102 pull_request: https://github.com/python/cpython/pull/30923 ___ Python tracker <https://bugs.python.org/issue35

[issue46524] test_peg_generator takes 8 minutes on Windows

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: > Would it be possible to skip these tests when no changes to the parser > related code are made to speed up tests? Maybe, some CIs could export an environment variable which contains the list of modified files, and then each file would be able to

[issue45382] platform() is not able to detect windows 11

2022-01-26 Thread STINNER Victor
Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker <https://bugs.python.org/issue45382> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue46513] AC_C_CHAR_UNSIGNED from configure.ac confuses GCC 12+ by defining __CHAR_UNSIGNED__

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: In short, I did my checks on my side, and the merged change now LGTM. Thanks Christian for fixing it ;-) -- ___ Python tracker <https://bugs.python.org/issue46

[issue46513] AC_C_CHAR_UNSIGNED from configure.ac confuses GCC 12+ by defining __CHAR_UNSIGNED__

2022-01-26 Thread STINNER Victor
STINNER Victor added the comment: IMO making the assumption that "char" is signed or not in C code is bad. If Python has code like that, it must be signed to explicitly use one of these types: unsigned char or uint8_t, signed char or int8_t. Hopefully, Python can now use C99 si

[issue45382] platform() is not able to detect windows 11

2022-01-25 Thread STINNER Victor
STINNER Victor added the comment: I don't understand why we have to handle XML or JSON and encoding... just to get a version number. Why did Microsoft make it so complicated? sys.getwindowsversion() which exposes GetVersionEx() looks fine to me

<    2   3   4   5   6   7   8   9   10   11   >