[issue45847] Port module setup to PY_STDLIB_MOD() macro and addext()
Change by Anthony Sottile : -- nosy: -Anthony Sottile ___ Python tracker <https://bugs.python.org/issue45847> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45847] Port module setup to PY_STDLIB_MOD() macro and addext()
Anthony Sottile added the comment: I could, but it's very unlikely to get fixed given I believe 18.04 is in security-only fixes and backporting a pkg-config file seems unlikely this worked two days ago before this patch -- ___ Python tracker <https://bugs.python.org/issue45847> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45847] Port module setup to PY_STDLIB_MOD() macro and addext()
Anthony Sottile added the comment: the `tk-dev` package on ubuntu bionic does not ship with a `pkg-config` file for tk so it does not build properly there: ``` root@f0dd06a3e87c:/cpython# dpkg -L tk8.6-dev | grep pc root@f0dd06a3e87c:/cpython# ``` (a note: bionic reaches end of life in april 2023 so it is likely to still see significant use until then) -- ___ Python tracker <https://bugs.python.org/issue45847> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45847] Port module setup to PY_STDLIB_MOD() macro and addext()
Anthony Sottile added the comment: this appears to break the tkinter extension for ubuntu bionic (18.04) -- I'm not entirely sure on the correct fix here but I get the following from trying to build there: 2022-04-02T15:52:08.0910452Z Python build finished successfully! 2022-04-02T15:52:08.0911924Z The necessary bits to build these optional modules were not found: 2022-04-02T15:52:08.0913469Z _tkinter this is the tk I have available: root@f0dd06a3e87c:/# dpkg -l | grep libtk ii libtk8.6:amd64 8.6.8-4 amd64 Tk toolkit for Tcl and X11 v8.6 - run-time files -- nosy: +Anthony Sottile ___ Python tracker <https://bugs.python.org/issue45847> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47152] Reorganize the re module sources
Anthony Sottile added the comment: would it be possible to expose `parse_template` -- or at least some way to validate that a regex replacement string is correct prior to executing the replacement? I'm currently using that for my text editor: https://github.com/asottile/babi/blob/d37d7d698d560aef7c6a0d1ec0668672e039bd9a/babi/screen.py#L501 -- nosy: +Anthony Sottile ___ Python tracker <https://bugs.python.org/issue47152> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46707] Parser hanging on stacked { tokens
Change by anthony shaw : -- versions: +Python 3.10, Python 3.11 ___ Python tracker <https://bugs.python.org/issue46707> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46707] Parser hanging on stacked { tokens
New submission from anthony shaw :
Providing an (invalid) input to the parser causes an exponentially-slow DoS to
the Python executable in 3.10.
e.g.
python3.10 -c "{:"
takes ~2 seconds
python3.10 -c ":"
takes ~22 seconds
Tested this all the way up to
d```{{{ef f():y
which took over an hour
--
components: Parser
keywords: 3.10regression
messages: 412972
nosy: anthonypjshaw, lys.nikolaou, pablogsal
priority: normal
severity: normal
status: open
title: Parser hanging on stacked { tokens
type: crash
___
Python tracker
<https://bugs.python.org/issue46707>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46704] Parser API not checking for null-terminator
New submission from anthony shaw :
In tokenizer.c, the translate_newlines() function does a `strlen()` on the
input string, if the string is not null-terminated, e.g.
'\xbe' this leads to a heap-buffer-overflow. The overflow is not exploitable,
but if there are further changes to the parser, it might be worth using a
strlen() alternative, like strnlen().
static char *
translate_newlines(const char *s, int exec_input, struct tok_state *tok) {
int skip_next_lf = 0;
size_t needed_length = strlen(s) + 2, final_length;
This leads to a heap-buffer-overflow detected by ASAN in a simple reproducible
example, calling PyRun_StringFlags() from the LLVM fuzzer:
fuzz_target(47084,0x11356f600) malloc: nano zone abandoned due to inability to
preallocate reserved vm space.
Dictionary: 35 entries
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 3034498392
INFO: Loaded 1 modules (43 inline 8-bit counters): 43 [0x10a2b71e8,
0x10a2b7213),
INFO: Loaded 1 PC tables (43 PCs): 43 [0x10a2b7218,0x10a2b74c8),
INFO:1 files found in ../Tests/fuzzing/corpus
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than
4096 bytes
=
==47084==ERROR: AddressSanitizer: heap-buffer-overflow on address
0x60203131 at pc 0x00010bd1d555 bp 0x7ff7b5da0590 sp 0x7ff7b5d9fd50
READ of size 2 at 0x60203131 thread T0
#0 0x10bd1d554 in wrap_strlen+0x184
(libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x15554)
#1 0x10b12132b in translate_newlines+0x1b (Python:x86_64+0x5d32b)
#2 0x10b12071c in _PyParser_ASTFromString+0x1ac (Python:x86_64+0x5c71c)
#3 0x10b2f86de in PyRun_StringFlags+0x5e (Python:x86_64+0x2346de)
#4 0x10a25ec6b in CompileCode(char const*) fuzz_target.cpp:54
#5 0x10a25f247 in LLVMFuzzerTestOneInput fuzz_target.cpp:68
#6 0x10a27aff3 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*,
unsigned long) FuzzerLoop.cpp:611
#7 0x10a27c3c4 in
fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__1::vector >&) FuzzerLoop.cpp:804
#8 0x10a27c859 in fuzzer::Fuzzer::Loop(std::__1::vector >&) FuzzerLoop.cpp:857
#9 0x10a26aa5f in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char
const*, unsigned long)) FuzzerDriver.cpp:906
#10 0x10a298e42 in main FuzzerMain.cpp:20
#11 0x1134f44fd in start+0x1cd (dyld:x86_64+0x54fd)
0x60203131 is located 0 bytes to the right of 1-byte region
[0x60203130,0x60203131)
allocated by thread T0 here:
#0 0x10bd58a0d in wrap__Znam+0x7d
(libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x50a0d)
#1 0x10a27af02 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*,
unsigned long) FuzzerLoop.cpp:596
#2 0x10a27c3c4 in
fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__1::vector >&) FuzzerLoop.cpp:804
#3 0x10a27c859 in fuzzer::Fuzzer::Loop(std::__1::vector >&) FuzzerLoop.cpp:857
#4 0x10a26aa5f in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char
const*, unsigned long)) FuzzerDriver.cpp:906
#5 0x10a298e42 in main FuzzerMain.cpp:20
#6 0x1134f44fd in start+0x1cd (dyld:x86_64+0x54fd)
SUMMARY: AddressSanitizer: heap-buffer-overflow
(libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x15554) in wrap_strlen+0x184
Shadow bytes around the buggy address:
0x1c0405d0: fa fa 02 fa fa fa 02 fa fa fa 02 fa fa fa 02 fa
0x1c0405e0: fa fa 02 fa fa fa 02 fa fa fa 02 fa fa fa 02 fa
0x1c0405f0: fa fa 03 fa fa fa 03 fa fa fa 03 fa fa fa 03 fa
0x1c040600: fa fa 01 fa fa fa 01 fa fa fa 01 fa fa fa 01 fa
0x1c040610: fa fa 00 00 fa fa 00 fa fa fa 00 fa fa fa 00 00
=>0x1c040620: fa fa 00 fa fa fa[01]fa fa fa fd fa fa fa fd fd
0x1c040630: fa fa fd fa fa fa fd fa fa fa 00 fa fa fa 04 fa
0x1c040640: fa fa 00 00 fa fa 01 fa fa fa 01 fa fa fa 01 fa
0x1c040650: fa fa fd fa fa fa fd fa fa fa fd fd fa fa 01 fa
0x1c040660: fa fa 00 00 fa fa 01 fa fa fa fd fa fa fa fd fa
0x1c040670: fa fa 01 fa fa fa 06 fa fa fa 00 00 fa fa 06 fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user:f7
Container overflow: fc
Array cookie:ac
Intra object redzone:bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone:cb
Shadow gap: cc
==47084==ABORTING
MS: 0 ; base unit:
artifact_prefix='./'; Test unit written to
./crash-da39a3ee5e6b4b0d3255bfef95601890afd80709
Base64:
zsh: abort ./fuzz_target -dict=../Tests/fuzzing/python.dict -only_ascii=1
---
[issue46491] typing: allow Annotated in outermost scope
Anthony Sottile added the comment: 3.7.2 has another example where OrderedDict was added to typing I don't have any personal investment in this particular change but I've had quite a few unhappy consumers of libraries due to instability in typing apis between patch versions in the past (heh and they're especially frustrated because they don't care about type checking and just want functional software) it's also difficult and/or not cost effective to add every patch version to a CI matrix to catch these sorts of problems -- ___ Python tracker <https://bugs.python.org/issue46491> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46491] typing: allow Annotated in outermost scope
Anthony Sottile added the comment: to me this is the same as the Union[Pattern] / Union[Match] "fixes" that landed in 3.5.3 -- and the pain caused by using that and having CI pass (because of modern 3.5.x) but having spurious bug reports from users stuck on 3.5.2 or in 3.6.1 when NamedTuple was "fixed" to allow methods, again having CI pass with modern pythons but having frustrated users running 3.6.0 I forsee the same class of problems here with Annotated where it works great in 3.10.3 and 3.9.11 but anyone stuck on 3.10.2 or 3.9.10 or older will be broken -- ___ Python tracker <https://bugs.python.org/issue46491> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46491] typing: allow Annotated in outermost scope
Anthony Sottile added the comment: should this behaviour change be backported? it potentially creates an annoying edgecase where code seemingly works unless you use an older patch version since this isn't a bugfix I wouldn't expect this to land in 3.9 and 3.10 -- nosy: +Anthony Sottile ___ Python tracker <https://bugs.python.org/issue46491> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46147] Support AddressSanitizer in Windows build
anthony shaw added the comment: After some experimentation, this can be done if you enable ASAN in all projects except python3dll.vcxproj I think it would make sense (if this were supported) to have a flag in build.bat like there is for pgo `build.bat --asan` -- ___ Python tracker <https://bugs.python.org/issue46147> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46147] Support AddressSanitizer in Windows build
New submission from anthony shaw : I'd like to compile my C-extensions with ASAN for testing in Windows, but they cannot be loaded as the host python.exe process needs to be compiled with ASAN. https://docs.microsoft.com/en-us/cpp/sanitizers/asan?view=msvc-170#ide-msbuild The EnableASAN flag would be a setting in vcxproj files within the PCBuild solution, and a flag in the build.bat to enable it. -- components: Build, Windows messages: 409006 nosy: anthonypjshaw, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Support AddressSanitizer in Windows build type: enhancement ___ Python tracker <https://bugs.python.org/issue46147> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44295] self.assertDictContainsSubset warning is unhelpful
Anthony Sottile added the comment: looks like this was removed in 3.11 -- wondering if it's still worth correcting the deprecation warning location -- versions: +Python 3.10 -Python 3.11 ___ Python tracker <https://bugs.python.org/issue44295> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39214] Add curses.window.in_wch
Anthony Sottile added the comment: rebased this onto 3.11 -- versions: +Python 3.11 -Python 3.9 ___ Python tracker <https://bugs.python.org/issue39214> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45548] Update Modules/Setup
Anthony Sottile added the comment: I'm seeing some weird breakage in the deadsnakes builds, presumably due to this change: ``` ... 2021-10-27T08:55:21.9485959Z x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -g -fdebug-prefix-map=/tmp/code=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector -Wformat -Werror=format-security-std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden -I../Include/internal -IObjects -IInclude -IPython -I. -I../Include -Wdate-time -D_FORTIFY_SOURCE=2 -DPy_BUILD_CORE_BUILTIN -c ../Modules/_blake2/blake2module.c -o Modules/_blake2/blake2module.o 2021-10-27T08:55:22.0563233Z Assembler messages: 2021-10-27T08:55:22.0565823Z Fatal error: can't create Modules/_blake2/blake2module.o: No such file or directory 2021-10-27T08:55:22.0622105Z Makefile:2177: recipe for target 'Modules/_blake2/blake2module.o' failed ... ``` https://github.com/deadsnakes/python3.11-nightly/actions/runs/1389342824 I believe this is due to https://github.com/python/cpython/pull/29225/files#diff-aac314cb71e95912e95795a4f4a278f2e75a5bc28e93f52e338185c8e7f7d7eaL225 -- nosy: +Anthony Sottile ___ Python tracker <https://bugs.python.org/issue45548> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45165] alighment format for nullable values
Anthony Sottile added the comment: alignment was the only one I found which was applicable and didn't work `!s` "works" but it's kind of a hack (of course I want string formatting, I'm making a string!) -- then I would want to write `!s` everywhere which is cumbersome and ugly -- ___ Python tracker <https://bugs.python.org/issue45165> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45165] alighment format for nullable values
New submission from Anthony Sottile :
currently this works correctly:
```
>>> '%8s %8s' % (None, 1)
'None1'
```
but conversion to f-string fails:
```
>>> f'{None:>8} {1:>8}'
Traceback (most recent call last):
File "", line 1, in
TypeError: unsupported format string passed to NoneType.__format__
```
my proposal is to implement alignment `__format__` for `None` following the
same as for `str` for alignment specifiers
--
components: Interpreter Core
messages: 401582
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: alighment format for nullable values
type: behavior
versions: Python 3.10, Python 3.11
___
Python tracker
<https://bugs.python.org/issue45165>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39562] Asynchronous comprehensions don't work in asyncio REPL
Anthony Sottile added the comment: this maybe shouldn't have been backported to 3.8.x as the change in compiler flags appears to break pyc files in subtle ways: https://stackoverflow.com/q/68910329/812183 -- nosy: +Anthony Sottile ___ Python tracker <https://bugs.python.org/issue39562> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33671] Efficient zero-copy for shutil.copy* functions (Linux, OSX and Win)
Change by Anthony Sottile : -- nosy: +Anthony Sottile nosy_count: 19.0 -> 20.0 pull_requests: +26032 pull_request: https://github.com/python/cpython/pull/27516 ___ Python tracker <https://bugs.python.org/issue33671> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44692] Const folding in parser with negative numbers doesn't match float/int behaviour
anthony shaw added the comment: Thanks! I didn't realise it applied the negative operator instead of loading the constant value. -- ___ Python tracker <https://bugs.python.org/issue44692> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44692] Const folding in parser with negative numbers doesn't match float/int behaviour
Change by anthony shaw : -- title: Const unfolding in parser with negative numbers doesn't match float/int behaviour -> Const folding in parser with negative numbers doesn't match float/int behaviour ___ Python tracker <https://bugs.python.org/issue44692> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44692] Const unfolding in parser with negative numbers doesn't match float/int behaviour
New submission from anthony shaw : Powers with negative bases do not observe the same rules that float_pow/long_pow do when it comes to returning negative results/ Example: > -2 ** 2 -4 >>> x = -2 >>> y = 2 >>> x ** y 4 >>> -2. ** 2. -4.0 >>> x = -2. >>> y = 2. >>> x ** y 4.0 Tested on 3.8, 3.9 and 3.10, they all have the same bug. -- components: Parser messages: 397922 nosy: anthonypjshaw, lys.nikolaou, pablogsal priority: normal severity: normal status: open title: Const unfolding in parser with negative numbers doesn't match float/int behaviour type: behavior versions: Python 3.10, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue44692> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44467] profiling-compatible functools.wraps
New submission from Anthony Sottile :
this is a small proposal to add a new function to the functools module which
provides better profiling-compatible `@functools.wraps(...)`
the rationale comes from https://github.com/Yelp/named_decorator (which is dead
/ abandoned)
the tl;dr from there is any time a decorator is involved in a profile it
becomes very difficult to trace because everything becomes tangled around
common decorators (because function names are used from the code object)
here is the proposal and an initial implementation:
def wraps_with_name(func, decorator, **kwargs):
def wraps_with_name_decorator(wrapped):
new_name = f'{func.__name__}@{decorator.__name__}'
new_code = wrapped.__code__.replace(co_name=new_name)
# would be nice if `types.FunctionType` had a `.replace(...)` too!
new_wrapped = types.FunctionType(
new_code,
wrapped.__globals__,
new_name,
wrapped.__defaults__,
wrapped.__closure__,
)
return functools.wraps(func, **kwargs)(new_wrapped)
return better_wraps_decorator
the usage would be similar to `functools.wraps`, here is an example:
```python
def my_decorator(func):
@functools.wraps_with_name(func, my_decorator)
def my_decorator_inner(*args, **kwargs):
return func(*args, **kwargs)
return my_decorator
```
--
components: Library (Lib)
messages: 396199
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: profiling-compatible functools.wraps
type: enhancement
versions: Python 3.11
___
Python tracker
<https://bugs.python.org/issue44467>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44297] Frame with -1 line number
Anthony Sottile added the comment: here is a minimal reproduction: ```python def iterboom(): raise AssertionError yield 1 next(1 for x in iterboom()) ``` python 3.9: ``` $ python3.9 t.py Traceback (most recent call last): File "/tmp/rinohtype/t.py", line 5, in next(1 for x in iterboom()) File "/tmp/rinohtype/t.py", line 5, in next(1 for x in iterboom()) File "/tmp/rinohtype/t.py", line 2, in iterboom raise AssertionError AssertionError ``` ``` $ python3.10 t.py Traceback (most recent call last): File "/tmp/rinohtype/t.py", line 5, in next(1 for x in iterboom()) File "/tmp/rinohtype/t.py", line -1, in File "/tmp/rinohtype/t.py", line 2, in iterboom raise AssertionError AssertionError ``` -- ___ Python tracker <https://bugs.python.org/issue44297> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44459] 3.10b2 -> 3.10b3 regression in importlib.metadata for rinoh
New submission from Anthony Sottile : installed from git: ``` $ git remote -v origin https://github.com/brechtm/rinohtype.git (fetch) origin https://github.com/brechtm/rinohtype.git (push) $ git rev-parse HEAD 4054539bae53eaddd10291c8429a1a32aeeb4786 ``` working in 3.10 b2: ```console $ ./venv310/bin/python --version --version Python 3.10.0b2 (default, Jun 2 2021, 00:22:18) [GCC 9.3.0] $ ./venv310/bin/python -m rinoh usage: rinoh [-h] [-f FORMAT] [-o OPTION=VALUE] [-t NAME or FILENAME] [-s NAME or FILENAME] [-O FILENAME or DIRECTORY] [-p PAPER] [-i] [--list-templates] [--list-stylesheets] [--list-fonts [FILENAME]] [--list-formats] [--list-options FRONTEND] [--version] [--docs] [input] Render a structured document to PDF. positional arguments: input the document to render options: -h, --helpshow this help message and exit -f FORMAT, --format FORMAT the format of the input file (default: autodetect) -o OPTION=VALUE, --option OPTION=VALUE options to be passed to the input file reader -t NAME or FILENAME, --template NAME or FILENAME the document template or template configuration file to use (default: article) -s NAME or FILENAME, --stylesheet NAME or FILENAME the style sheet used to style the document elements (default: the template's default) -O FILENAME or DIRECTORY, --output FILENAME or DIRECTORY write the PDF output to FILENAME or to an existing DIRECTORY with a filename derived from the input filename (default: the current working directory) -p PAPER, --paper PAPER the paper size to render to (default: the template's default) -i, --install-resources automatically install missing resources (fonts, templates, style sheets) from PyPI --list-templates list the installed document templates and exit --list-stylesheetslist the installed style sheets and exit --list-fonts [FILENAME] list the installed fonts or, if FILENAME is given, write a PDF file displaying all the fonts --list-formatslist the supported input formats and exit --list-options FRONTEND list the options supported by the given frontend and exit --version show program's version number and exit --docsopen the online documentation in the default browser ``` broken in 3.10 b3: ```console $ ./venv/bin/python --version --version Python 3.10.0b3+ (heads/3.10:1b4addf3cb, Jun 18 2021, 17:21:48) [GCC 9.3.0] $ ./venv/bin/python -m rinoh Traceback (most recent call last): File "/home/asottile/workspace/cpython/prefix/lib/python3.10/runpy.py", line 187, in _run_module_as_main mod_name, mod_spec, code = _get_module_details(mod_name, _Error) File "/home/asottile/workspace/cpython/prefix/lib/python3.10/runpy.py", line 146, in _get_module_details return _get_module_details(pkg_main_name, error) File "/home/asottile/workspace/cpython/prefix/lib/python3.10/runpy.py", line 110, in _get_module_details __import__(pkg_name) File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/__init__.py", line 41, in from . import resource File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/resource.py", line 205, in from .template import DocumentTemplate File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/template.py", line 42, in from .stylesheets import sphinx File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/stylesheets/__init__.py", line 42, in .format(stylesheet.description, stylesheet)) File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/style.py", line 670, in __str__ for name, entry_point in self.installed_resources: File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/resource.py", line 54, in installed_resources for entry_point in ilm.entry_points()[cls.entry_point_group]: File "/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py", line 979, in entry_points return SelectableGroups.load(eps).select(**params) File "/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py", line 437, in load ordered = sorted(eps, key=by_group) File "/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py", line -1, in File "/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/_itertools.py", line 16, in unique_everseen k = key(element) File "/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py", line 600, in _normalized_name ret
[issue44297] Frame with -1 line number
Anthony Sottile added the comment:
here's the traceback pytest is trying to display and crashing:
```
Traceback (most recent call last):
File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/runner.py",
line 311, in from_call
result: Optional[TResult] = func()
File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/runner.py",
line 341, in
call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/python.py",
line 503, in collect
self._inject_setup_module_fixture()
File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/python.py",
line 516, in _inject_setup_module_fixture
self.obj, ("setUpModule", "setup_module")
File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/python.py",
line 291, in obj
self._obj = obj = self._getobj()
File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/python.py",
line 500, in _getobj
return self._importtestmodule()
File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/python.py",
line 578, in _importtestmodule
mod = import_path(self.fspath, mode=importmode)
File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/pathlib.py",
line 524, in import_path
importlib.import_module(module_name)
File
"/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/__init__.py",
line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File
"/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/assertion/rewrite.py",
line 170, in exec_module
exec(co, module.__dict__)
File "/tmp/rinohtype/tests/test_attribute.py", line 11, in
from rinoh.attribute import Attribute, Bool
File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/__init__.py",
line 41, in
from . import resource
File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/resource.py",
line 205, in
from .template import DocumentTemplate
File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/template.py",
line 42, in
from .stylesheets import sphinx
File
"/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/stylesheets/__init__.py",
line 42, in
.format(stylesheet.description, stylesheet))
File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/style.py", line
670, in __str__
for name, entry_point in self.installed_resources:
File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/resource.py",
line 54, in installed_resources
for entry_point in ilm.entry_points()[cls.entry_point_group]:
File
"/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py",
line 979, in entry_points
return SelectableGroups.load(eps).select(**params)
File
"/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py",
line 437, in load
ordered = sorted(eps, key=by_group)
File
"/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py",
line -1, in
File
"/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/_itertools.py",
line 16, in unique_everseen
k = key(element)
File
"/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py",
line 600, in _normalized_name
return Prepared.normalize(self.name)
File
"/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py",
line 841, in normalize
return re.sub(r"[-_.]+", "-", name).lower().replace('-', '_')
File "/home/asottile/workspace/cpython/prefix/lib/python3.10/re.py", line
187, in sub
return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object
```
--
___
Python tracker
<https://bugs.python.org/issue44297>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44297] Frame with -1 line number
Anthony Sottile added the comment: this appears to have regressed in 3.10 as well according to some reports on pytest: https://github.com/pytest-dev/pytest/pull/8227#issuecomment-864327090 -- nosy: +Anthony Sottile, pablogsal versions: +Python 3.10 ___ Python tracker <https://bugs.python.org/issue44297> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points
Anthony Sottile added the comment: they are directly coupled which is why I commented here the api redesign forces O(N) lookups and O(N) constructions which directly impact performance causing the regression -- resolution: fixed -> status: closed -> open ___ Python tracker <https://bugs.python.org/issue44246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points
Anthony Sottile added the comment: here's the performance regressions, they affect any callers of `distributions()` and are even worse on callers of the new apis. a call to distributions() is about 3x slower than in 3.9 here is the setup I am using: virtualenv venv39 -ppython3.9 venv39/bin/pip install flake8 pytest twine pre-commit virtualenv venv310 -ppython3.10 venv310/bin/pip install flake8 pytest twine pre-commit to test just the `distributions()` call I'm using the following: ```console $ venv39/bin/python -m timeit -n 20 -r 20 -s 'from importlib.metadata import entry_points' 'entry_points()' 20 loops, best of 20: 12.5 msec per loop $ venv310/bin/python -m timeit -n 20 -r 20 -s 'from importlib.metadata import entry_points' 'entry_points()' 20 loops, best of 20: 36.7 msec per loop ``` this is a less-extreme example, many applications have more dependencies installed -- but even in this case this is adding ~24ms startup to any application using `entry_points()` -- and it gets worse the return value of `entry_points()` alone isn't all that useful, next an application needs to retrieve its entry points. let's start for the somewhat normal case of retrieving a single category of entry points: ```console $ venv39/bin/python -m timeit -n 20 -r 20 -s 'from importlib.metadata import entry_points' 'entry_points()["flake8.extension"]' 20 loops, best of 20: 12.7 msec per loop $ venv310/bin/python -m timeit -n 20 -r 20 -s 'from importlib.metadata import entry_points' 'entry_points(name="flake8.extension")' 20 loops, best of 20: 37.1 msec per loop $ venv310/bin/python -m timeit -n 20 -r 20 -s 'from importlib.metadata import entry_points' 'entry_points().select(group="flake8.extension")' 20 loops, best of 20: 37.1 msec per loop ``` again, 3x slower and very real time to the end user (~24-25ms) now let's show an example usage that something like flake8 uses where multiple groups are requested (this is common for apps and plugin systems which provide multiple distinct functionalities) ```console $ venv39/bin/python -m timeit -n 20 -r 20 -s 'from importlib.metadata import entry_points' 'eps = entry_points(); eps["flake8.extension"]; eps["flake8.report"]' 20 loops, best of 20: 12.6 msec per loop $ venv310/bin/python -m timeit -n 20 -r 20 -s 'from importlib.metadata import entry_points' 'eps = entry_points(); eps.select(group="flake8.extension"); eps.select(group="flake8.report")' 20 loops, best of 20: 38.2 msec per loop ``` also slower, but an additional ms per call to `.select(...)` and it only gets worse with more and more packages installed here's the versions I'm using to ensure they are up to date: ```console $ venv39/bin/python --version --version Python 3.9.5 (default, May 19 2021, 11:32:47) [GCC 9.3.0] $ venv310/bin/python --version --version Python 3.10.0b2 (default, Jun 2 2021, 00:22:18) [GCC 9.3.0] ``` -- resolution: fixed -> status: closed -> open ___ Python tracker <https://bugs.python.org/issue44246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44295] self.assertDictContainsSubset warning is unhelpful
Change by Anthony Sottile : -- components: +Library (Lib) type: -> behavior ___ Python tracker <https://bugs.python.org/issue44295> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44307] date.today() is half as fast as datetime.now().date()
Anthony Sottile added the comment: *terry.reedy oops typo! -- ___ Python tracker <https://bugs.python.org/issue44307> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44307] date.today() is half as fast as datetime.now().date()
Anthony Sottile added the comment: @terry.reddy -- I believe your title change makes this more difficult to understand -- ___ Python tracker <https://bugs.python.org/issue44307> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44307] date.today() is 2x slower than datetime.now().date()
New submission from Anthony Sottile :
```console
$ python3.10 -m timeit -s 'from datetime import datetime'
'datetime.now().date()'
50 loops, best of 5: 708 nsec per loop
$ python3.10 -m timeit -s 'from datetime import date' 'date.today()'
20 loops, best of 5: 1.4 usec per loop
```
this surprised me so I dug into it -- it appears a fast path can be added to
`date.today()` to make it faster than `datetime.date.now()` -- though I'm
rather unfamiliar with the functions involved here
here is my ~sloppy patch attempting to add a fast path, I would need some
guidance to improve it and get it accepted:
```diff
$ git diff -w
diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c
index 8ef2dad37a..7eaa5d1740 100644
--- a/Modules/_datetimemodule.c
+++ b/Modules/_datetimemodule.c
@@ -2875,6 +2875,17 @@ date_fromtimestamp(PyObject *cls, PyObject *obj)
static PyObject *
date_today(PyObject *cls, PyObject *dummy)
{
+/* fast path, don't call fromtimestamp */
+if ((PyTypeObject *)cls == _DateType) {
+struct tm tm;
+time_t t;
+time();
+localtime_r(, );
+return new_date_ex(tm.tm_year + 1900,
+ tm.tm_mon + 1,
+ tm.tm_mday,
+ (PyTypeObject*)cls);
+} else {
PyObject *time;
PyObject *result;
_Py_IDENTIFIER(fromtimestamp);
@@ -2893,6 +2904,7 @@ date_today(PyObject *cls, PyObject *dummy)
Py_DECREF(time);
return result;
}
+}
/*[clinic input]
@classmethod
```
after this, `date.today()` is faster!
```console
$ ./python -m timeit -s 'from datetime import datetime' 'datetime.now().date()'
50 loops, best of 5: 764 nsec per loop
$ ./python -m timeit -s 'from datetime import date' 'date.today()'
50 loops, best of 5: 407 nsec per loop
```
\o/
--
components: Extension Modules
messages: 395061
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: date.today() is 2x slower than datetime.now().date()
type: performance
versions: Python 3.11
___
Python tracker
<https://bugs.python.org/issue44307>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44296] Should warnings.warn default to stacklevel=2?
New submission from Anthony Sottile : I have yet to come across a usecase where `stacklevel=1` makes sense -- usually it is more helpful to point at the calling code than the function which is itself warning my proposal is to update the default for `stacklevel=` from `1` to `2` an example bpo where this is relevant is bpo-44295 -- components: Library (Lib) messages: 394960 nosy: Anthony Sottile priority: normal severity: normal status: open title: Should warnings.warn default to stacklevel=2? type: behavior versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue44296> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44295] self.assertDictContainsSubset warning is unhelpful
Change by Anthony Sottile : -- keywords: +patch pull_requests: +25093 stage: -> patch review pull_request: https://github.com/python/cpython/pull/26497 ___ Python tracker <https://bugs.python.org/issue44295> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44295] self.assertDictContainsSubset warning is unhelpful
New submission from Anthony Sottile : it's missing stacklevel= -- mostly creating a bpo issue to link to -- messages: 394953 nosy: Anthony Sottile priority: normal severity: normal status: open title: self.assertDictContainsSubset warning is unhelpful versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue44295> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44277] cpython forks are spammed with dependabot PRs
New submission from Anthony Sottile : for example: https://github.com/asottile/cpython/pull/1 -- messages: 394842 nosy: Anthony Sottile priority: normal severity: normal status: open title: cpython forks are spammed with dependabot PRs ___ Python tracker <https://bugs.python.org/issue44277> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points
Anthony Sottile added the comment: it does not, it restores apis but in a way which requires a huge performance hit to avoid deprecation warnings it also still has the 2-500x performance regression I've stated above -- ___ Python tracker <https://bugs.python.org/issue44246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points
Anthony Sottile added the comment: I have also shown that the performance is indeed not better in the nominal case, as demonstrated in the first case -- ___ Python tracker <https://bugs.python.org/issue44246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points
Anthony Sottile added the comment: the toil is still present, the existing, good apis are deprecated and the new, bad apis are slow -- and the odd subclasses are still present -- ___ Python tracker <https://bugs.python.org/issue44246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points
Anthony Sottile added the comment: oops, tiny typo in those code examples, they should say `group=` instead of `name=` -- though the performance is unchanged: (first example) ```console $ ./venv39/bin/python t.py 0.6641988754272461 $ ./venv310/bin/python t.py 1.3172023296356201 ``` (second example) ```console $ ./venv39/bin/python t.py 0.014233589172363281 $ ./venv310/bin/python t.py 8.910593271255493 ``` -- ___ Python tracker <https://bugs.python.org/issue44246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points
Anthony Sottile added the comment:
the `.select(...)` api is at least twice as slow as indexing as well:
setup:
```
virtualenv venv39 -p python3.9
venv39/bin/pip install flake8 pytest pytest-randomly
virtualenv venv39 -p python3.10
venv310/bin/pip install flake8 pytest pytest-randomly
```
```python
import importlib.metadata
import sys
import time
def f():
eps = importlib.metadata.entry_points()
if sys.version_info >= (3, 10):
eps.select(name='console_scripts')
else:
eps['console_scripts']
t0 = time.time()
for _ in range(100):
f()
t1 = time.time()
print(f'{t1-t0}')
```
```
$ ./venv39/bin/python t.py
0.687570333480835
$ ./venv310/bin/python t.py
1.3486714363098145
```
it is *way* worse when involving multiple entry points:
```python
import importlib.metadata
import sys
import time
# moved outside of the loop, already showed this component is slower
eps = importlib.metadata.entry_points()
def f():
# common for plugin systems to look up multiple entry points
for ep in ('console_scripts', 'flake8.extension', 'pytest11'):
if sys.version_info >= (3, 10):
eps.select(name=ep)
else:
eps[ep]
t0 = time.time()
for _ in range(1):
f()
t1 = time.time()
print(f'{t1-t0}')
```
```console
$ ./venv39/bin/python t.py
0.01629471778869629
$ ./venv310/bin/python t.py
8.569908380508423
```
--
___
Python tracker
<https://bugs.python.org/issue44246>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points
Anthony Sottile added the comment: also https://github.com/miurahr/aqtinstall/issues/221 (this links to importlib-metadata tracker, not sure how you missed it) -- ___ Python tracker <https://bugs.python.org/issue44246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points
Anthony Sottile added the comment: I also need `.sort(key=...)` for what it's worth, the error in this issue was just the first encountered I think my only satisfactory outcome would be: - the original api returns actual dicts - the sub-api returns actual lists - the new select is implemented as a separate *new* api without changing the existing api -- ___ Python tracker <https://bugs.python.org/issue44246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points
Anthony Sottile added the comment: > Which types are sneaky and look like built-in types but do not act like them? well for starters, there's the tuple subclass which pretends to be a dict. but it violates substitutability for both `tuple` and `Mapping` so it's not useful in either contexts. mypy complains about incorrect types in overrides for both. the worst part of this is that the `__getitem__` moves from O(1) to O(N) (in some private code this makes importlib.metadata on 3.10 _10x slower than on 3.9_). next there's the `EntryPoints` tuple subclass which looks like a tuple but doesn't at all act like one (__getitem__ fails substitutability for example) -- this is an api break with 3.9 which returned a `list` (can't `.sort()` `.extend(...)`, etc. any more) > Given that backports are available I don't think this is appropriate. re-introducing a backport brings in a tree of dependencies that have been shaky at best with backward compatibility. in other words, using standard library importlib.metadata provides significantly improved compatibility and stability over the moving target backport (and its tree of dependencies, zipp being the one that breaks the most often from experience). you'll notice I closed the flake8 PRs specifically because I didn't want to reintroduce the backport. This backport also globally monkeypatches the import machinery breaking any other consumer as well -- in a tool as popular as flake8 I can't really make that global mutation decision for all of the other consumers. > The typing issues these were *trivially solved* by a dictionary comprehension using `entrypoint.name` -- it really did not need a full rework and break of the api to solve (could have just deprecated the `__iter__` which I actually suggested on the *original implementation* way back in 3.8) > The new APIs are not only easier to describe with types the types describing the new apis require *significant* `# type: ignore`s to pass mypy because they violate basic substitutability. they also cannot be used in any of the contexts they were appropriate for in <3.10 (Dict[str, List[EntryPoint]] or List[Entrypoint] depending on the api). > I'm not aware of a single breakage. I'm sorry but you have to have realized from the many issues on importlib-metadata or the many issues linking to your deprecation issue that there is significant toil caused directly by your change. cpython is meant to be a stable substrate to build upon, please do not force the community to shoulder the burden of your poor api decisions. > even without requiring an updated importlib_metadata backport. your proposed change introduced a different, unrelated package. not without its own maintenance problems (an additional dependency that has to ~work indefinitely, a hack at best to support this breaking change) > Given the amount of adoption already, reverting these changes is likely to > cause more disruption than moving forward with them. I disagree, bigger things have been reverted (see: __future__.annotations) > So far, the only example presented above appears contrived and not an actual > concern I promise you this is not a contrived case, if you look at your issue tracker it has been reported before and by others. For every issue reported there's likely tens or hundreds of others which are not reported. > It's common for Python minor releases to introduce undocumented changes that > are technically incompatible I've seen this as a rationalization for intentional surprise breaking changes but I don't buy it. Additionally your comments about (paraphrased) "the testsuite didn't demonstrate this usecase so I'm free to change it" are frankly a cop out -- your api returned a dict and a list, that's part of the public api -- changing that is a breaking change. -- ___ Python tracker <https://bugs.python.org/issue44246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points
Anthony Sottile added the comment: personally I think they should be reverted -- they were not committed in spirit with the backwards compatibility policy: https://www.python.org/dev/peps/pep-0387/#making-incompatible-changes - I don't think they were discussed thoroughly, and when opposition was presented it was not listened to thoroughly: https://github.com/python/importlib_metadata/pull/278 - the change significantly complicates importlib.metadata with lots of sneaky types (they all look like builtin types but do not act like them) - it simultaneously introduces new apis and old apis which will both be around for extended periods of time but immediately moves to DeprecationWarning - the new apis aren't remarkably better than the old apis -- the motivation was written as "typing issues" but without citing actual issues. in fact, the new items are significantly more complicated to type properly - the change breaks many significantly important projects, from perusing related issues it's at the very least flake8, pandas, virtualenv, astropy, pytest, hypothesis -- and hundreds more from a quick github code search -- ___ Python tracker <https://bugs.python.org/issue44246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points
Anthony Sottile added the comment: the "what's new" mentions nothing of this break, nor the aggressive deprecation warnings, nor the various shifted interfaces (such as this one): https://docs.python.org/3.10/whatsnew/3.10.html#importlib-metadata -- ___ Python tracker <https://bugs.python.org/issue44246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44247] bpo stacktrace linkifier does not understand 3.10+
New submission from Anthony Sottile : for instance in this issue: https://bugs.python.org/issue44246 it links to 3.1/... instead of 3.10/... -- messages: 394549 nosy: Anthony Sottile priority: normal severity: normal status: open title: bpo stacktrace linkifier does not understand 3.10+ ___ Python tracker <https://bugs.python.org/issue44247> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points
New submission from Anthony Sottile :
this is breaking code that's unfortunately out of my control (vendor) -- also
it looks really wrong
```python
import importlib.metadata
print('looks like a list:')
print(importlib.metadata.distribution('pip').entry_points)
print('first item:')
print(importlib.metadata.distribution('pip').entry_points[0])
```
output in 3.9:
```console
$ ./venv39/bin/python t.py
looks like a list:
[EntryPoint(name='pip', value='pip._internal.cli.main:main',
group='console_scripts'), EntryPoint(name='pip3',
value='pip._internal.cli.main:main', group='console_scripts'),
EntryPoint(name='pip3.8', value='pip._internal.cli.main:main',
group='console_scripts')]
first item:
EntryPoint(name='pip', value='pip._internal.cli.main:main',
group='console_scripts')
```
```console
$ venv310/bin/python t.py
looks like a list:
(EntryPoint(name='pip', value='pip._internal.cli.main:main',
group='console_scripts'), EntryPoint(name='pip3',
value='pip._internal.cli.main:main', group='console_scripts'),
EntryPoint(name='pip3.8', value='pip._internal.cli.main:main',
group='console_scripts'))
first item:
Traceback (most recent call last):
File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 217, in
__getitem__
return next(iter(self.select(name=name)))
StopIteration
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/y/t.py", line 5, in
print(importlib.metadata.distribution('pip').entry_points[0])
File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 219, in
__getitem__
raise KeyError(name)
KeyError: 0
```
--
messages: 394548
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: 3.10 beta 1: breaking change in importlib.metadata entry points
versions: Python 3.10
___
Python tracker
<https://bugs.python.org/issue44246>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44184] crash on windows invoking flake8
Anthony Sottile added the comment: that version of flake8 uses multiprocessing (even for 1 file) -- would the ast objects be involved in that way? (pyflakes also makes reference cyles to handle "parent" relationships) -- ___ Python tracker <https://bugs.python.org/issue44184> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44184] crash on windows invoking flake8
Anthony Sottile added the comment: would it maybe be helpful to bisect a history where the dataclasses / inspect import change is introduced earlier? this would perhaps help pinpoint the other commit which is causing this? -- ___ Python tracker <https://bugs.python.org/issue44184> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44184] crash on windows invoking flake8
Anthony Sottile added the comment: vstinner I showed the directions above, but here they are again: venv\Scripts\pip install flake8==3.6.0 venv\Scripts\pip install -e . --no-deps # ignore the conflict, but fix pyflakes -- ___ Python tracker <https://bugs.python.org/issue44184> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44184] crash on windows invoking flake8
Anthony Sottile added the comment: ah sorry, the branch got squash-merged this is the equivalent revision after the merge: f3b1b44bf3d2d5927004fa1c2fcf1ab2def816b9 -- ___ Python tracker <https://bugs.python.org/issue44184> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44184] crash on windows invoking flake8
Anthony Sottile added the comment: everything in this virtualenv is pure python so I don't think it's a faulty third party extension module -- ___ Python tracker <https://bugs.python.org/issue44184> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44184] crash on windows invoking flake8 under tox
Anthony Sottile added the comment: I can reproduce it outside of tox using: venv\Scripts\pip install flake8==3.6.0 venv\Scripts\pip install -e . --no-deps # ignore the conflict, but fix pyflakes C:\Users\asott\AppData\Local\Temp\y\pyflakes>venv\Scripts\flake8.exe setup.py C:\Users\asott\AppData\Local\Temp\y\pyflakes>echo %ERRORLEVEL% -1073741819 -- ___ Python tracker <https://bugs.python.org/issue44184> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44184] crash on windows invoking flake8 under tox
New submission from Anthony Sottile : I installed python using the installers from python.org -- I originally reproduced this using github actions using pyflakes's testsuite >ver Microsoft Windows [Version 10.0.19041.985] >venv\Scripts\python --version Python 3.10.0b1 C:\Users\asott\AppData\Local\Temp\y\pyflakes>venv\Scripts\python --version --version Python 3.10.0b1 (tags/v3.10.0b1:ba42175, May 3 2021, 20:22:30) [MSC v.1928 64 bit (AMD64)] to reproduce: ``` git clone https://github.com/pycqa/pyflakes cd pyflakes git checkout b02ba019e16f7c156ec63c2ea05c627a0fe86c48 ``` # install tox somehow ``` C:\python310\python -m venv venv venv\Scripts\pip install tox ``` here are the versions I have at that point: ``` >venv\Scripts\pip freeze --all appdirs==1.4.4 colorama==0.4.4 distlib==0.3.1 filelock==3.0.12 packaging==20.9 pip==21.1.1 pluggy==0.13.1 py==1.10.0 pyparsing==2.4.7 setuptools==56.0.0 six==1.16.0 toml==0.10.2 tox==3.23.1 virtualenv==20.4.6 ``` then run this a few times: `venv\Scripts\tox -e py310` even with `setenv = PYTHONFAULTHANDLER=1` I couldn't get a trace, though maybe that's a linux thing? it occasionally crashes like this: ``` py310 run-test: commands[2] | flake8 pyflakes setup.py ERROR: InvocationError for command 'C:\Users\asott\AppData\Local\Temp\y\pyflakes\.tox\py310\Scripts\flake8.EXE' pyflakes setup.py (exited with code 3221225477) ``` from some googling: > in hex is 0xc005 which is the Windows code for an access violation I don't do much development on windows so I'm passing the torch to someone who knows more :) -- messages: 393999 nosy: Anthony Sottile priority: normal severity: normal status: open title: crash on windows invoking flake8 under tox versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue44184> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44043] 3.10 b1 armhf Bus Error in hashlib test: test_gil
Anthony Sottile added the comment: oddly enough, when I add `-X faulthandler` it passes ___ I did some research on the error message and it looks like the ubuntu maintainers have found the same thing and reported it here: https://bugs.python.org/issue36445 this points at (intentional?) misaligned accesses being a problem on arm looks like they've provided a patch as well here: https://bugs.python.org/issue36515 -- ___ Python tracker <https://bugs.python.org/issue44043> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44043] 3.10 b1 armhf Bus Error in hashlib test: test_gil
New submission from Anthony Sottile : terribly sorry, I don't have much information to go off on this other than the build logs from deadsnakes. I retried this build twice and it seems it consistently fails in this position: https://launchpadlibrarian.net/537139233/buildlog_ubuntu-bionic-armhf.python3.10_3.10.0~b1-1+bionic2_BUILDING.txt.gz The relevant logs from the build: ``` ./python -m test --pgo --timeout=1200 || true 0:00:00 load avg: 2.37 Run tests sequentially (timeout: 20 min) 0:00:00 load avg: 2.37 [ 1/44] test_array 0:00:11 load avg: 2.23 [ 2/44] test_base64 0:00:15 load avg: 2.13 [ 3/44] test_binascii 0:00:16 load avg: 2.13 [ 4/44] test_binop 0:00:17 load avg: 2.13 [ 5/44] test_bisect 0:00:18 load avg: 2.13 [ 6/44] test_bytes 0:01:18 load avg: 1.41 [ 7/44] test_bz2 -- test_bytes passed in 1 min 0:01:25 load avg: 1.35 [ 8/44] test_cmath 0:01:28 load avg: 1.35 [ 9/44] test_codecs 0:01:52 load avg: 1.23 [10/44] test_collections 0:02:10 load avg: 1.16 [11/44] test_complex 0:02:16 load avg: 1.15 [12/44] test_dataclasses 0:02:23 load avg: 1.14 [13/44] test_datetime 0:03:18 load avg: 1.09 [14/44] test_decimal -- test_datetime passed in 54.9 sec 0:05:06 load avg: 1.01 [15/44] test_difflib -- test_decimal passed in 1 min 47 sec 0:05:24 load avg: 1.01 [16/44] test_embed 0:05:27 load avg: 1.01 [17/44] test_float 0:05:32 load avg: 1.01 [18/44] test_fstring 0:05:43 load avg: 1.00 [19/44] test_functools 0:05:47 load avg: 1.00 [20/44] test_generators 0:05:51 load avg: 1.00 [21/44] test_hashlib Fatal Python error: Bus error Current thread 0xf7901220 (most recent call first): File "/<>/Lib/test/test_hashlib.py", line 842 in test_gil File "/<>/Lib/unittest/case.py", line 549 in _callTestMethod File "/<>/Lib/unittest/case.py", line 592 in run File "/<>/Lib/unittest/case.py", line 652 in __call__ File "/<>/Lib/unittest/suite.py", line 122 in run File "/<>/Lib/unittest/suite.py", line 84 in __call__ File "/<>/Lib/unittest/suite.py", line 122 in run File "/<>/Lib/unittest/suite.py", line 84 in __call__ File "/<>/Lib/unittest/suite.py", line 122 in run File "/<>/Lib/unittest/suite.py", line 84 in __call__ File "/<>/Lib/test/support/testresult.py", line 169 in run File "/<>/Lib/test/support/__init__.py", line 959 in _run_suite File "/<>/Lib/test/support/__init__.py", line 1082 in run_unittest File "/<>/Lib/test/libregrtest/runtest.py", line 210 in _test_module File "/<>/Lib/test/libregrtest/runtest.py", line 246 in _runtest_inner2 File "/<>/Lib/test/libregrtest/runtest.py", line 282 in _runtest_inner File "/<>/Lib/test/libregrtest/runtest.py", line 154 in _runtest File "/<>/Lib/test/libregrtest/runtest.py", line 194 in runtest File "/<>/Lib/test/libregrtest/main.py", line 423 in run_tests_sequential File "/<>/Lib/test/libregrtest/main.py", line 521 in run_tests File "/<>/Lib/test/libregrtest/main.py", line 694 in _main File "/<>/Lib/test/libregrtest/main.py", line 641 in main File "/<>/Lib/test/libregrtest/main.py", line 719 in main File "/<>/Lib/test/__main__.py", line 2 in File "/<>/Lib/runpy.py", line 86 in _run_code File "/<>/Lib/runpy.py", line 196 in _run_module_as_main ``` -- components: Build messages: 392973 nosy: Anthony Sottile priority: normal severity: normal status: open title: 3.10 b1 armhf Bus Error in hashlib test: test_gil versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue44043> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43933] Regression in python3.10 with traceback frame having lineno of -1
Anthony Sottile added the comment: I've released pytest 6.2.4 for the other breakage (which is unrelated to this bpo -- it was the asname source position issue) -- ___ Python tracker <https://bugs.python.org/issue43933> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40465] Deprecate the optional *random* argument to random.shuffle()
Change by Anthony Sottile : -- nosy: -Anthony Sottile ___ Python tracker <https://bugs.python.org/issue40465> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43933] Regression in python3.10 with traceback frame having lineno of -1
Anthony Sottile added the comment: ah yeah that's the `asname` ast change -- if you use the unreleased main branch it has a fix for that -- ___ Python tracker <https://bugs.python.org/issue43933> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43994] change representation of match as / capture as `Name(..., ctx=Store())`
Anthony Sottile added the comment: at least for static analysis of other python constructs it's very convenient to know from a `Name` node alone whether it's being used in a read or write context -- without this information an ast traversal needs to maintain more information about whether it's a read or write context for pyflakes this is especially important as it needs to know what names are defined in scope (and referenced in scope) to produce diagnostic messages for other tools like `dead` / `vulture` it's useful to identify introduced and referenced names similarly the `as` in `with` does and the target for assignment expressions so I would expect the similar constructs in `match` to do so as well `Name` nodes are also useful for better diagnostic messages as they contain positioning information, which isn't easily extracted from `MatchAs`, etc. -- if I recall correctly, the `asname` for imports was recently extended to add this information for the same purpose -- ___ Python tracker <https://bugs.python.org/issue43994> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43994] change representation of match as / capture as `Name(..., ctx=Store())`
Anthony Sottile added the comment: and actually, now that I look close it would be useful for `MatchStar` and `MatchMapping` to also use a `Name(..., ctx=Store())` for their respective parameters as well -- ___ Python tracker <https://bugs.python.org/issue43994> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43994] change representation of match as / capture as `Name(..., ctx=Store())`
Anthony Sottile added the comment:
I'm suggesting instead of:
MatchAs(pattern=None, name='foo')
to have
MatchAs(pattern=None, name=Name('foo', ctx=Store()))
--
___
Python tracker
<https://bugs.python.org/issue43994>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43994] change representation of match as / capture as `Name(..., ctx=Store())`
New submission from Anthony Sottile : I'm looking at adding support to `match` for pyflakes, and the first impression I have is that `MatchAs` is unnecessarily different from `Name` with `ctx=Store()` if it were represented as the latter pyflakes would not require special handling of `match`, it would work the same as the current code I suspect other static analysis tools would benefit from a change as well -- messages: 392531 nosy: Anthony Sottile priority: normal severity: normal status: open title: change representation of match as / capture as `Name(..., ctx=Store())` versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue43994> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43933] Regression in python3.10 with traceback frame having lineno of -1
Anthony Sottile added the comment: can confirm that pytest no longer crashes -- ___ Python tracker <https://bugs.python.org/issue43933> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38605] [typing] PEP 563: Postponed evaluation of annotations: enable it by default in Python 3.11
Change by Anthony Sottile : -- nosy: +Anthony Sottile nosy_count: 14.0 -> 15.0 pull_requests: +24316 stage: resolved -> patch review pull_request: https://github.com/python/cpython/pull/25596 ___ Python tracker <https://bugs.python.org/issue38605> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43933] Regression in python3.10 with traceback frame having lineno of -1
New submission from Anthony Sottile :
This is breaking pytest for failed assertions:
https://github.com/pytest-dev/pytest/pull/8227
It also breaks the traceback in the output below
Here's a minimal example:
```python
class Boom:
def __enter__(self):
return self
def __exit__(self, *_):
raise AssertionError('boom!')
def main() -> int:
with Boom():
raise AssertionError('hi')
if __name__ == '__main__':
exit(main())
```
On python3.9 you get this:
```
Traceback (most recent call last):
File "/home/asottile/workspace/cpython/t.py", line 10, in main
raise AssertionError('hi')
AssertionError: hi
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/asottile/workspace/cpython/t.py", line 14, in
exit(main())
File "/home/asottile/workspace/cpython/t.py", line 10, in main
raise AssertionError('hi')
File "/home/asottile/workspace/cpython/t.py", line 5, in __exit__
raise AssertionError('boom!')
AssertionError: boom!
```
output in python3.10:
```
Traceback (most recent call last):
File "/home/asottile/workspace/cpython/t.py", line 10, in main
raise AssertionError('hi')
AssertionError: hi
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/asottile/workspace/cpython/t.py", line 14, in
exit(main())
File "/home/asottile/workspace/cpython/t.py", line -1, in main
File "/home/asottile/workspace/cpython/t.py", line 5, in __exit__
raise AssertionError('boom!')
AssertionError: boom!
```
Notice the second to last frame is now missing and invalid (line -1)
I bisected this and found this is the culprit:
https://github.com/python/cpython/commit/3bd6035b6baf1a7d51b7cc2c6bb2c81886236b67
https://github.com/python/cpython/pull/24202
```
3bd6035b6baf1a7d51b7cc2c6bb2c81886236b67 is the first bad commit
commit 3bd6035b6baf1a7d51b7cc2c6bb2c81886236b67
Author: Mark Shannon
Date: Wed Jan 13 12:05:43 2021 +
bpo-42908: Mark cleanup code at end of try-except and with artificial
(#24202)
* Mark bytecodes at end of try-except as artificial.
* Make sure that the CFG is consistent throughout optimiization.
* Extend line-number propagation logic so that implicit returns after
'try-except' or 'with' have the correct line numbers.
* Update importlib
Lib/test/test_dis.py |2 +-
Lib/test/test_sys_settrace.py | 40 +
Python/compile.c | 135 +-
Python/importlib.h| 3153 ++---
Python/importlib_external.h | 4489 -
Python/importlib_zipimport.h | 1013 +-
6 files changed, 4473 insertions(+), 4359 deletions(-)
bisect run success
```
which appears to be due to bpo-42908
--
messages: 391795
nosy: Anthony Sottile, Mark.Shannon, pablogsal
priority: normal
severity: normal
status: open
title: Regression in python3.10 with traceback frame having lineno of -1
versions: Python 3.10, Python 3.11
___
Python tracker
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43782] Failure to build from source on ppc64le on ubuntu xenial
Anthony Sottile added the comment: a rebuild succeeded so I'm ok closing this :shrug: -- stage: -> resolved status: pending -> closed ___ Python tracker <https://bugs.python.org/issue43782> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43833] Unexpected Parsing of Numeric Literals Concatenated with Boolean Operators
Anthony Sottile added the comment: here's quite a few other cases as well -- I'd love for this to be clarified in PEP8 such that I can rationalize crafting a rule for it in `pycodestyle` -- https://github.com/PyCQA/pycodestyle/issues/371 -- nosy: +Anthony Sottile ___ Python tracker <https://bugs.python.org/issue43833> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43737] Documentation of modulo operator should document behaviour clearly when second operator is negative
Anthony Flury added the comment: I fundamentally disagree with closing this - I know that this and many other 'quirks' catch beginners out, and the tutorial is what they use to learn.They don't look in the reference document - it is too dense in BNF definitions which turns a lot of people off from any other useful information - When I was researching this issue I didn't even think to look in the reference section. Looking in the library section for information about the built-in types is also non-obvious to beginners. If the tutorial doesn't contain the detail, and doesn't link to the detail then beginners are left entirely puzzled by the behavior. Given how difficult it is to search the documentation if you don't know exactly what you are looking for, then how beginners would know to look at the FAQ is beyond me. Having communicated with a number of beginners on a number of issues they had no idea that the FAQ even existed. The change didn't 'bury the tutorial in detail' - it added once sentence which linked to the FAQ. In fact all the change did was expand the FAQ entry by a few lines and link to the FAQ from the three places that are relevant. I think this is a small change and it should be the start of making the tutorial beginner friendly. -- resolution: not a bug -> status: closed -> open ___ Python tracker <https://bugs.python.org/issue43737> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43782] Failure to build from source on ppc64le on ubuntu xenial
Anthony Sottile added the comment: hmmm strange, the only changes in 3.9.4 are a revert -- perhaps this is a flakey bug in gcc and not actionable I've clicked rebuild on my build, maybe it'll magically fix it plus xenial is almost end of lifed so I doubt anyone cares about this strange architecture on an old platform anyway -- ___ Python tracker <https://bugs.python.org/issue43782> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43782] Failure to build from source on ppc64le on ubuntu xenial
New submission from Anthony Sottile : I realize this is unlikely to be a helpful report, but something that changed between 3.9.3 and 3.9.4 has caused the build to break on (admittedly a strange platform) ppc64le I attached the build log (zipped because otherwise it's too big ?) The live URL is here: https://launchpadlibrarian.net/532585040/buildlog_ubuntu-xenial-ppc64el.python3.9_3.9.4-1+xenial1_BUILDING.txt.gz Probably the most relevant part of the bug report is this bit, though I'm guessing so I don't really know what is useful and what is not. ``` Preprocessed source stored into /tmp/ccIkITd0.out file, please attach this to your bugreport. === BEGIN GCC DUMP === // Target: powerpc64le-linux-gnu // Configured with: ../src/configure -v --with-pkgversion='Ubuntu/IBM 5.4.0-6ubuntu1~16.04.12' --with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-5 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libquadmath --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-ppc64el/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-ppc64el --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-ppc64el --with-arch-directory=ppc64le --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-secureplt --with-cpu=power8 --ena ble-targets=powerpcle-linux --disable-multilib --enable-multiarch --disable-werror --with-long-double-128 --enable-checking=release --build=powerpc64le-linux-gnu --host=powerpc64le-linux-gnu --target=powerpc64le-linux-gnu // Thread model: posix // gcc version 5.4.0 20160609 (Ubuntu/IBM 5.4.0-6ubuntu1~16.04.12) // // ../Python/ceval.c: In function 'is_tstate_valid': // ../Python/ceval.c:5694:1: internal compiler error: Segmentation fault // } // ^ // Please submit a full bug report, // with preprocessed source if appropriate. // See for instructions. ``` -- components: Build files: buildlog.tgz messages: 390580 nosy: Anthony Sottile priority: normal severity: normal status: open title: Failure to build from source on ppc64le on ubuntu xenial versions: Python 3.9 Added file: https://bugs.python.org/file49946/buildlog.tgz ___ Python tracker <https://bugs.python.org/issue43782> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43737] Documentation of modulo operator should document behaviour clearly when second operator is negative
Change by Anthony Flury : -- assignee: -> docs@python components: +Documentation nosy: +docs@python ___ Python tracker <https://bugs.python.org/issue43737> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43737] Documentation of modulo operator should document behaviour clearly when second operator is negative
Change by Anthony Flury : -- type: -> enhancement ___ Python tracker <https://bugs.python.org/issue43737> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43737] Documentation of modulo operator should document behaviour clearly when second operator is negative
Change by Anthony Flury : -- pull_requests: +24015 pull_request: https://github.com/python/cpython/pull/25279 ___ Python tracker <https://bugs.python.org/issue43737> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43737] Documentation of modulo operator should document behaviour clearly when second operator is negative
Change by Anthony Flury : -- keywords: +patch pull_requests: +23957 stage: -> patch review pull_request: https://github.com/python/cpython/pull/25220 ___ Python tracker <https://bugs.python.org/issue43737> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43737] Documentation of modulo operator should document behaviour clearly when second operator is negative
Anthony Flury added the comment: I am working on a pull request for this. -- ___ Python tracker <https://bugs.python.org/issue43737> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43325] Documentation should warn that 'is' is not a safe comparison operator for most values.
Anthony Flury added the comment: Should the data structures page also link to the FAQ. The problem with the FAQ is that most beginners don't even know that == vs 'is' is actually a question they need to ask, and therefore they aren't likely to look at the FAQ in the first place. -- ___ Python tracker <https://bugs.python.org/issue43325> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43737] Documentation of modulo operator should document behaviour clearly when second operator is negative
Anthony Flury added the comment: I take your point about warnings etc - but when you come from other languages the Python behavior can initially be very surprising. The reference section has always seemed to be a very technical document, certainly not targeted at the usual audience of people using Python. Most Python users don't consider that int/float etc would be in the standard library either - for most users the built-ins are not the same. Can I suggest: 1) The identity that is mentioned on https://docs.python.org/3/reference/expressions.html#binary-arithmetic-operations should be made more clear - at the moment it is buried in a paragraph and easily missed. 2) The document https://docs.python.org/3/library/stdtypes.html#numeric-types-int-float-complex should also mention the identity and the need to preserve it. 3) A link fromhttps://docs.python.org/3/tutorial/introduction.html#numbers to the document https://docs.python.org/3/library/stdtypes.html#numeric-types-int-float-complex - the introductory tutorial should link to further detail where neccessary. -- ___ Python tracker <https://bugs.python.org/issue43737> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43737] Documentation of modulo operator should document behaviour clearly when second operator is negative
New submission from Anthony Flury : The behavior of a%b when a is positive and b is negative can be suprising. I understand that the behavior is so that the identity a = (a//b)*b + a%b can be preserved regardless of the signs of a or b. but the result is different from other languages which is why it can be surprising. I would be happy to do the grunt work to make the changes if some can suggest where. Do we - add a warning to the https://docs.python.org/3/tutorial/introduction.html#numbers page, or should we link to a new page that details and explains the behavior. Which is more 'pythonic' in terms of documentation ? -- messages: 390264 nosy: anthony-flury priority: normal severity: normal status: open title: Documentation of modulo operator should document behaviour clearly when second operator is negative ___ Python tracker <https://bugs.python.org/issue43737> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43567] regen.vcxproj cannot regenerate some necessary files
anthony shaw added the comment: Guido, regen.vcxproj targets 142 as the SDK version, which is most likely a mistake. The other projects are part of the main PCBuild.sln solution, which has a variable for the base SDK version. If you need to change it quickly, you can either open it in VS and right click on the project and change the properties to an older SDK, or edit the vcxproj file itself. I can make this change in a patch, but because its a standalone project, it doesn't share the base SDK version with the others. I've reviewed Steve's patch and it would fix this because it changes regen into a build target instead of a build project, so it doesn't specify the SDK version at all. Steve's implementation is also much, much cleaner! -- nosy: +anthony shaw ___ Python tracker <https://bugs.python.org/issue43567> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43614] Search is not beginner friendly
Anthony Flury added the comment: Is PR 25045 the correct Pull request - this Issue is a documentation change - the linked PR is related to Issue 43433 (a change to xmlrpc.client ?) -- ___ Python tracker <https://bugs.python.org/issue43614> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43614] Search is not beginner friendly
New submission from Anthony Flury :
A commonly asked question on Quora is 'What do *args and **kwargs' mean ?
While it is relatively easy for community to answer these questions the search
tool on the standard documentation doesn't make it easy.
I understand that 'args' and 'kwargs' are both naming conventions, they are
very common across the documentation, but searching on '*args' or '**kwargs'
doesn't actually find anything useful - it certainly doesn't place
'https://docs.python.org/3/tutorial/controlflow.html#arbitrary-argument-lists'
at or close to the top of the list.
It is my view that the documentation should be beginner friendly, but in this
case (and many other I guess) you have to know what to search for to find
something useful.
I note that even common phrases in Computing (such as 'variable arguments' or
'variable parameters') don't find anything useful. The term 'variadic' does
find the relevant page, but the link displayed in the search results lands on
the page (but not the relevant section) - and many beginners wont search for
'variadic'.
The index and search need to be improved to help beginners - specifically in
this case
* Search Index should include common conventional names (such as args, kwargs)
* Search Index should include common computing terms ('variable arguments' for
example - even if the documentation doesn't actually use that terminology).
* Search should link to the relevant section (and not just the page).
--
assignee: docs@python
components: Documentation
messages: 389442
nosy: anthony-flury, docs@python
priority: normal
severity: normal
status: open
title: Search is not beginner friendly
versions: Python 3.9
___
Python tracker
<https://bugs.python.org/issue43614>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8978] "tarfile.ReadError: file could not be opened successfully" if compiled without zlib
Anthony Sottile added the comment:
I took a stab at improving the error message (see the linked PR)
$ ./python -c 'import tarfile; tarfile.open("Lib/test/testtar.tar.xz")'
Traceback (most recent call last):
File "", line 1, in
File "/home/asottile/workspace/cpython/Lib/tarfile.py", line 1620, in open
raise ReadError(f"file could not be opened successfully:\n{error_msgs}")
tarfile.ReadError: file could not be opened successfully:
- method gz: ReadError('not a gzip file')
- method bz2: CompressionError('bz2 module is not available')
- method xz: CompressionError('lzma module is not available')
- method tar: ReadError('truncated header')
--
___
Python tracker
<https://bugs.python.org/issue8978>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8978] "tarfile.ReadError: file could not be opened successfully" if compiled without zlib
Change by Anthony Sottile : -- nosy: +Anthony Sottile nosy_count: 7.0 -> 8.0 pull_requests: +23611 pull_request: https://github.com/python/cpython/pull/24850 ___ Python tracker <https://bugs.python.org/issue8978> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43325] Documentation should warn that 'is' is not a safe comparison operator for most values.
New submission from Anthony Flury : A frequent bug for beginners is to assume that 'is' is somehow 'better' then '==' when comparing values, and it is certainly a cause for confusion amongst beginners as to why: [1,2] is [1,2] evaluates to False but 'a' is 'a' evaluates to True and many similar examples. As far as I can see the first mention of the 'is' operator is under Section 5 - More on conditionals : https://docs.python.org/3/tutorial/datastructures.html?highlight=comparison#comparing-sequences-and-other-types; and it is mentioned thus : The operators is and is not compare whether two objects are really the same object; this only matters for mutable objects like lists. As we know though this statement is misleading - it suggests that 'is' can be used to compare immutable values (ints, strings, tuples) etc, and while for some values of some immutables (small ints, shortish strings) 'is' can be used as an equivalent of '==' it wont be clear from this 'statement' that 'is' is simply not a safe way to compare values. There needs to be a warning here about 'is' and how it is not an equivalent to '==' in any general sense. -- assignee: docs@python components: Documentation messages: 387692 nosy: anthony-flury, docs@python priority: normal severity: normal status: open title: Documentation should warn that 'is' is not a safe comparison operator for most values. type: enhancement ___ Python tracker <https://bugs.python.org/issue43325> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43292] xml.ElementTree iterparse filehandle left open
anthony shaw added the comment:
Example script attached works perfectly on macOS
> python3.9 test.py
[('default',
'http://schemas.openxmlformats.org/officeDocument/2006/extended-properties'),
('vt', 'http://schemas.openxmlformats.org/officeDocument/2006/docPropsVTypes')]
[('default',
'http://schemas.openxmlformats.org/officeDocument/2006/extended-properties'),
('vt', 'http://schemas.openxmlformats.org/officeDocument/2006/docPropsVTypes')]
--
nosy: +anthonypjshaw
___
Python tracker
<https://bugs.python.org/issue43292>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43292] xml.ElementTree iterparse filehandle left open
anthony shaw added the comment: which version of Python were you doing this on? The function in question is https://github.com/python/cpython/blob/master/Lib/xml/etree/ElementTree.py#L1233-L1278 -- nosy: +anthonypjshaw ___ Python tracker <https://bugs.python.org/issue43292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3353] make built-in tokenizer available via Python C API
Anthony Sottile added the comment: I haven't looked into or thought about that yet, it might not be possible It might also make sense to build new tokenize.py apis avoiding the `readline()` api -- I always found it painful to work with -- ___ Python tracker <https://bugs.python.org/issue3353> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3353] make built-in tokenizer available via Python C API
Anthony Sottile added the comment: Either works for me, would you be able to point me to the starting bits as to how `_ast` becomes builtin? -- ___ Python tracker <https://bugs.python.org/issue3353> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3353] make built-in tokenizer available via Python C API
Anthony Sottile added the comment: private api sounds fine too -- I thought it was necessary to implement the module (as it needs external linkage) but if it isn't then even better -- ___ Python tracker <https://bugs.python.org/issue3353> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3353] make built-in tokenizer available via Python C API
Anthony Sottile added the comment: you already have that right now because the `tokenize` module is exposed. (except that every change to the tokenization requires it to be implemented once in C and once in python) it's much more frustrating when the two differ as well I don't think all the internals of the C tokenization need to be exposed, my main goals would be: - expose enough information to reimplement Lib/tokenize.py - replace Lib/tokenize.py with the C tokenizer and the reasons would be: - eliminate the (potential) drift and complexity between the two - get a fast tokenizer Unlike the AST, the tokenization changes much less frequently (last major addition I can remember is the `@` operator We can hide almost all of the details of the tokenization behind an opaque struct and getter functions -- ___ Python tracker <https://bugs.python.org/issue3353> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3353] make built-in tokenizer available via Python C API
Anthony Sottile added the comment: I'm interested in it because the `tokenize` module is painfully slow -- ___ Python tracker <https://bugs.python.org/issue3353> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3353] make built-in tokenizer available via Python C API
Anthony Sottile added the comment: Serhiy Storchaka is this still blocked? it's been a few years on either this or the linked issue and I'm reaching for this one :) -- nosy: +Anthony Sottile ___ Python tracker <https://bugs.python.org/issue3353> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43013] IDLE: update code, mostly by cleanups of 2.x or 2to3 artifacts
Anthony Sottile added the comment: > Anthony: removing 'set' from 'list(set(interable))' is wrong if 'set' were > added to remove duplicates. It's not removed, it's changed to a set comprehension (which was added in 2.7 and 3.0) pyupgrade is very battle tested, having been run on pip, pytest, and many open source projects -- ___ Python tracker <https://bugs.python.org/issue43013> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43013] IDLE: update code, mostly by cleanups of 2.x or 2to3 artifacts
Anthony Sottile added the comment:
just noticed this PR -- you might be interested in pyupgrade which automates
some of these things -- here's (for example) `git ls-files -- Lib/idlelib |
grep '\.py$' | xargs pyupgrade --py36-plus
```diff
diff --git a/Lib/idlelib/calltip_w.py b/Lib/idlelib/calltip_w.py
index 1e0404aa49..278546064a 100644
--- a/Lib/idlelib/calltip_w.py
+++ b/Lib/idlelib/calltip_w.py
@@ -25,7 +25,7 @@ def __init__(self, text_widget):
text_widget: a Text widget with code for which call-tips are desired
"""
# Note: The Text widget will be accessible as self.anchor_widget
-super(CalltipWindow, self).__init__(text_widget)
+super().__init__(text_widget)
self.label = self.text = None
self.parenline = self.parencol = self.lastline = None
@@ -54,7 +54,7 @@ def position_window(self):
return
self.lastline = curline
self.anchor_widget.see("insert")
-super(CalltipWindow, self).position_window()
+super().position_window()
def showtip(self, text, parenleft, parenright):
"""Show the call-tip, bind events which will close it and reposition
it.
@@ -73,7 +73,7 @@ def showtip(self, text, parenleft, parenright):
self.parenline, self.parencol = map(
int, self.anchor_widget.index(parenleft).split("."))
-super(CalltipWindow, self).showtip()
+super().showtip()
self._bind_events()
@@ -143,7 +143,7 @@ def hidetip(self):
# ValueError may be raised by MultiCall
pass
-super(CalltipWindow, self).hidetip()
+super().hidetip()
def _bind_events(self):
"""Bind event handlers."""
diff --git a/Lib/idlelib/debugger.py b/Lib/idlelib/debugger.py
index ccd03e46e1..53dba376af 100644
--- a/Lib/idlelib/debugger.py
+++ b/Lib/idlelib/debugger.py
@@ -49,9 +49,9 @@ def __frame2message(self, frame):
filename = code.co_filename
lineno = frame.f_lineno
basename = os.path.basename(filename)
-message = "%s:%s" % (basename, lineno)
+message = f"{basename}:{lineno}"
if code.co_name != "?":
-message = "%s: %s()" % (message, code.co_name)
+message = f"{message}: {code.co_name}()"
return message
@@ -213,7 +213,7 @@ def interaction(self, message, frame, info=None):
m1 = "%s" % str(type)
if value is not None:
try:
-m1 = "%s: %s" % (m1, str(value))
+m1 = "{}: {}".format(m1, str(value))
except:
pass
bg = "yellow"
diff --git a/Lib/idlelib/debugobj.py b/Lib/idlelib/debugobj.py
index 5a4c997884..71d01c7070 100644
--- a/Lib/idlelib/debugobj.py
+++ b/Lib/idlelib/debugobj.py
@@ -87,7 +87,7 @@ def GetSubList(self):
continue
def setfunction(value, key=key, object=self.object):
object[key] = value
-item = make_objecttreeitem("%r:" % (key,), value, setfunction)
+item = make_objecttreeitem(f"{key!r}:", value, setfunction)
sublist.append(item)
return sublist
diff --git a/Lib/idlelib/editor.py b/Lib/idlelib/editor.py
index a4d0c95362..5faebb5742 100644
--- a/Lib/idlelib/editor.py
+++ b/Lib/idlelib/editor.py
@@ -37,12 +37,12 @@
def _sphinx_version():
"Format sys.version_info to produce the Sphinx version string used to
install the chm docs"
major, minor, micro, level, serial = sys.version_info
-release = '%s%s' % (major, minor)
-release += '%s' % (micro,)
+release = f'{major}{minor}'
+release += f'{micro}'
if level == 'candidate':
-release += 'rc%s' % (serial,)
+release += f'rc{serial}'
elif level != 'final':
-release += '%s%s' % (level[0], serial)
+release += '{}{}'.format(level[0], serial)
return release
@@ -939,7 +939,7 @@ def update_recent_files_list(self, new_file=None):
rf_list = []
file_path = self.recent_files_path
if file_path and os.path.exists(file_path):
-with open(file_path, 'r',
+with open(file_path,
encoding='utf_8', errors='replace') as rf_list_file:
rf_list = rf_list_file.readlines()
if new_file:
@@ -1446,7 +1446,7 @@ def newline_and_indent_event(self, event):
else:
self.reindent_to(y.compute_backslash_indent())
else:
-assert 0, "bogus continuation type %r" % (c,)
+assert 0, f"bogus continuation type {c!r}"
return "break"
# This line starts a brand new
[issue43014] tokenize spends a lot of time in `re.compile(...)`
Anthony Sottile added the comment: attached out3.pstats / out3.svg which represent the optimization using lru_cache instead -- Added file: https://bugs.python.org/file49764/out3.svg ___ Python tracker <https://bugs.python.org/issue43014> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43014] tokenize spends a lot of time in `re.compile(...)`
Change by Anthony Sottile : Added file: https://bugs.python.org/file49763/out3.pstats ___ Python tracker <https://bugs.python.org/issue43014> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
