[issue40847] New parser considers empty line following a backslash to be a syntax error, old parser didn't

[Guido van Rossum]

Guido van Rossum  added the comment:

Sure looks like a tokenizer issue to me. For example this is broken in both 
versions:

pass
\

pass

It complains about an unexpected indent, but it should really be considered a 
blank line broken in two -- a backslash is supposed to just erase itself and 
the following newline.

https://docs.python.org/3/reference/lexical_analysis.html#explicit-line-joining

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40862] argparse.BooleanOptionalAction accept and silently discard its the const argument

[Raymond Hettinger]

Change by Raymond Hettinger :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40862] argparse.BooleanOptionalAction accept and silently discard its the const argument

[Raymond Hettinger]

Raymond Hettinger  added the comment:


New changeset d5e7348e4105d1d4a1c5bd1087f61041532ecbf3 by Miss Islington (bot) 
in branch '3.9':
bpo-40862: Raise TypeError when const is given to 
argparse.BooleanOptionalAction (GH-20623) (GH-20664)
https://github.com/python/cpython/commit/d5e7348e4105d1d4a1c5bd1087f61041532ecbf3


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40879] Strange regex cycle

[Tim Peters]

Tim Peters  added the comment:

Note that the relatively tiny pattern here extracts just a small piece of the 
regexp in question. As the output shows, increase the length of a string it 
fails to match by one character, and the time taken to fail approximately 
doubles: exponential-time behavior.

>>> import re
>>> c = re.compile(r'(?:[^\s()<>]+)+x')
>>> from time import perf_counter as now
>>> size = 1
>>> while True:
... s = 'a' * size
... start = now()
... junk = c.search(s)
... finish = now()
... print(size, finish - start)
... size += 1

1 9.90009110954e-06
2 1.180009998257e-05
3 1.120017197453e-05
4 1.209992187804e-05
5 1.520007098424e-05
6 1.569977414336e-05
7 2.11999194988e-05
8 3.3900053602e-05
9 4.959982774534e-05
10 7.77998547282e-05
11 0.0001481001830304
12 0.000340170905
13 0.000634899943317
14 0.001219100143504
15 0.00248249985066
16 0.00469410023127
17 0.00934219991863
18 0.0195416999067
19 0.0388015000526
20 0.076214100156
21 0.147214899945
22 0.2777167001336
23 0.649172200095
24 1.35531179
25 2.22982969982
26 4.98656629993
27 9.56792559995
28 19.0918107999
29 42.363349
30 83.5749305999
31 158.8824948998
...

The machine was far from quiet while running this, but it doesn't matter: the 
_point_ is dead obvious regardless.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40886] Add PYTHONLOGGING environment variable and -L cmdline argument

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +vinay.sajip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40847] New parser considers empty line following a backslash to be a syntax error, old parser didn't

[Lysandros Nikolaou]

Lysandros Nikolaou  added the comment:

This is limited to cases where the line continuation character is on an 
otherwise empty line. For example this works correctly:

$ cat t.py
print("hello world")
print("hello world 2") \

print("hello world 3")
$ ./python.exe t.py
hello world
hello world 2
hello world 3

The actual problem is at the tokenizer level, where a line with only a 
continuation character is not considered an empty line and thus two NEWLINE 
tokens get emitted, one after the other. The old parser was somehow working 
around this, probably by having this in the grammar:

file_input: (NEWLINE | stmt)* ENDMARKER

The PEG parser OTOH does not allow this.

The question now is, is it reasonable to change the tokenizer to consider a 
lone backslash an empty line? Do you also consider this a bug? Or should we 
change the new parser?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40879] Strange regex cycle

[Tim Peters]

Tim Peters  added the comment:

The repr truncates the pattern string, for display, if it's "too long". The 
only visual clue about that, though, is that the display is missing the pattern 
string's closing quote, as in the output you showed here. If you look at 
url_pat.pattern, though, you'll see that nothing has been lost.

I'm not sure why it does that.  As I vaguely recall, some years ago there was a 
crusade to limit maximum repr sizes because long output was considered to be "a 
security issue" (e.g., DoS attacks vis tricking logging/auditing facilities 
into writing giant strings when recording reprs).

In any case, that's all there is to that part.

For the rest, it's exceedingly unlikely that there's actually an infinite loop. 
Instead there's a messy regexp with multiple nested quantifiers, which are 
notorious for exhibiting exponential-time behavior and especially in 
non-matching cases. They can be rewritten to have linear-time behavior instead, 
but it's an effort I personally have no interest in pursuing here. See Jeffrey 
Friedl's "Mastering Regular Expressions" book for detailed explanations.

The reason I have no interest: it's almost always a losing idea to try to parse 
any aspect of HTML with regexps. Use an HTML parser instead (or for URLs 
specifically, see urllib.parse).

--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40886] Add PYTHONLOGGING environment variable and -L cmdline argument

[Bar Harel]

New submission from Bar Harel :

Per discussion on mailing list, I suggest adding a PYTHONLOGGING environment 
variable, and a matching -L cmdline argument.

When set to a logging level of choice, they will initiate basicConfig with the 
appropriate level.

For example, "py.exe -L info" will be equivalent to 
"logging.basicConfig(level='info')" on interpreter startup.

Sames as setting env var "PYTHONLOGGING=info".

This matches the current behavior of other settings, such as PYTHONWARNINGS and 
-W, allows to easily test programs without modifying them, and further 
completes the expected arguments available from the commandline.

Discussion on mailing list for reference:
https://mail.python.org/archives/list/python-id...@python.org/thread/I74LVJWJLE2LUCCZGOF5A5JDSDHJ6WX2/

--
components: Library (Lib)
messages: 370807
nosy: bar.harel
priority: normal
severity: normal
status: open
title: Add PYTHONLOGGING environment variable and -L cmdline argument
type: enhancement
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40879] Strange regex cycle

[Dennis Sweeney]

Dennis Sweeney  added the comment:

It looks like only the first 200 characters of the input string's repr are used 
as the compiled pattern's repr for some reason:

https://github.com/python/cpython/blob/master/Modules/_sre.c#L1294

I don't know if there is a good reason, especially since this violates 
eval(repr(pattern)) == pattern in a bad way:

>>> eval(repr(re.compile(STR_RE_URL)))
Traceback (most recent call last):
File "", line 1, in 
File "", line 1

re.compile('(?i)\\b((?:[a-z][\\w-]+:(?:/{1,3}|[a-z0-9%])|www\\d{0,3}[.]|[a-z0-9.\\-]+[.][a-z]{2,4}/)(?:[^\\s()<>]+|\\(([^\\s()<>]+|(\\([^\\s()<>]+\\)))*\\))+(?:\\(([^\\s()<>]+|(\\([^\\s()<>]+\\)))*\\)|[^\\s`!()\,
 re.IGNORECASE)


^
SyntaxError: EOL while scanning string literal

--
nosy: +Dennis Sweeney

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40879] Strange regex cycle

[Steven D'Aprano]

Steven D'Aprano  added the comment:

Wait, I'm sorry, do you mean this?

py> repr(r)[13:-16]

'?i)b((?:[a-z][w-]+:(?:/{1,3}|[a-z0-9%])|wwwd{0,3}[.]|[a-z0-9.-]+[.][a-z]{2,4}/)(?:[^s()<>]+|(([^s()<>]+|(([^s()<>]+)))*))+(?:(([^s()<>]+|(([^s()<>]+)))*)|[^s`!()\\'

Referring to the pattern being truncated in the repr? I would assume that's 
intentional.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40885] Cannot pipe GzipFile into subprocess

[Nehal Patel]

New submission from Nehal Patel :

The following code produces incorrect behavior:

with gzip.open("foo.gz") as gz:
res = subprocess.run("cat", stdin=gz, capture_output=True)

the contents of res.stdout are identical to the contents of "foo.gz" 

It seems the subprocess somehow gets a hold of the underlying file descriptor 
pointing to the compressed file, and ends up being fed the compressed bytes.

--
components: IO
messages: 370804
nosy: Nehal Patel
priority: normal
severity: normal
status: open
title: Cannot pipe GzipFile into subprocess
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40879] Strange regex cycle

[Steven D'Aprano]

Steven D'Aprano  added the comment:

> notice the stripped characters in the `repr`

Er, no. Your regex looks like line noise, and it hurts my brain to look at it 
:-)

If you have spotted a difference, can you tell us what characters are stripped? 
When I try running it, I don't get any characters stripped at all:

py> import re
py> STR_RE_URL = 
r"""(?i)\b((?:[a-z][\w-]+:(?:/{1,3}|[a-z0-9%])|www\d{0,3}[.]|[a-z0-9.\-]+[.][a-z]{2,4}/)(?:[^\s()<>]+|\(([^\s()<>]+|(\([^\s()<>]+\)))*\))+(?:\(([^\s()<>]+|(\([^\s()<>]+\)))*\)|[^\s`!()\[\]{};:'".,<>?«»“”‘’]))"""
py> r = re.compile(STR_RE_URL)
py> r.pattern == STR_RE_URL
True

--
nosy: +steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40880] Invalid read in pegen.c

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40883] parse_string.c: free "str"

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40884] Added defaults parameter for logging.Formatter

[Rémi Lapeyre]

Rémi Lapeyre  added the comment:

New features will go in Python 3.10 indeed.

--
nosy: +remi.lapeyre, vinay.sajip
versions:  -Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40880] Invalid read in pegen.c

[miss-islington]

miss-islington  added the comment:


New changeset 15fec5627ac343afd0bfa1e847746071982d5172 by Miss Islington (bot) 
in branch '3.9':
bpo-40880: Fix invalid read in newline_in_string in pegen.c (GH-20666)
https://github.com/python/cpython/commit/15fec5627ac343afd0bfa1e847746071982d5172


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40854] [Patch] Allow overriding sys.platlibdir

[Sandro Mani]

Sandro Mani  added the comment:

I'm on Fedora. My use case is for the mingw-python package I maintain there, 
see [1] for the full details. I believe (though I haven't investigated) that 
the previous downstream lib64 patch behaved slightly differently, or that 
something else between python-3.8 and earlier and python-3.9 changed, in that I 
didn't have lib64 appear in the site packages path previously when invoking 
python as detailed in [1], whereas it now does.

[1] https://src.fedoraproject.org/rpms/python3.9/pull-request/10#comment-0

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40883] parse_string.c: free "str"

[miss-islington]

miss-islington  added the comment:


New changeset 79e6c15aed9b4b50efd39ddaf1dc40c374b51213 by Miss Islington (bot) 
in branch '3.9':
bpo-40883: Fix memory leak in fstring_compile_expr in parse_string.c (GH-20667)
https://github.com/python/cpython/commit/79e6c15aed9b4b50efd39ddaf1dc40c374b51213


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40883] parse_string.c: free "str"

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 3.0 -> 4.0
pull_requests: +19886
pull_request: https://github.com/python/cpython/pull/20669

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40880] Invalid read in pegen.c

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 4.0 -> 5.0
pull_requests: +19887
pull_request: https://github.com/python/cpython/pull/20670

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40880] Invalid read in pegen.c

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset 2e6593db0086004a1ca7f7049218ff9573d473c2 by Pablo Galindo in 
branch 'master':
bpo-40880: Fix invalid read in newline_in_string in pegen.c (#20666)
https://github.com/python/cpython/commit/2e6593db0086004a1ca7f7049218ff9573d473c2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40883] parse_string.c: free "str"

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset a54096e30523534e8eebb8dc1011b4536ed237a8 by Pablo Galindo in 
branch 'master':
bpo-40883: Fix memory leak in fstring_compile_expr in parse_string.c (GH-20667)
https://github.com/python/cpython/commit/a54096e30523534e8eebb8dc1011b4536ed237a8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40884] Added defaults parameter for logging.Formatter

[Bar Harel]

Change by Bar Harel :


--
keywords: +patch
pull_requests: +19885
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/20668

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40884] Added defaults parameter for logging.Formatter

[Bar Harel]

New submission from Bar Harel :

TLDR; `logging.Formatter('%(ip)s %(message)s', defaults={"ip": None})`

Python's logging.Formatter allows the placement of custom fields, e.g.
`logging.Formatter("%(ip)s %(message)")`.

If a handler has a formatter with a custom field, all log records that go 
through the handler must have the custom field set using `extra={}`.
Failure to do so will result in exceptions thrown inside the logging library.

Custom fields are common, and are even suggested by the Python logging 
cookbook, where they are attached to the root logger.

There is, however, no way to specify default values for the custom fields. 
Quite a few issues arise from it.

For example, if I've set a formatter on the root logger with the custom field 
"%(ip)s", all logging messages sent by the asyncio library, will cause 
exceptions to raise.

Adding default values is possible using LoggerAdapter but will causes other 
issues as well as not solve the aforementioned problem.

Adding default values is possible using Filters, but cause confusion, isn't 
simple, and permanently modify the record object itself, which can cause issues 
if more handlers or formatters are attached.

>From a quick search, this feature was asked for many times in stackoverflow, 
>and even spawned up a few libraries such as "logaugment" in order to solve it.

I believe the solution offered, by using `defaults={}` is simple enough to not 
need discussion over python-ideas, yet common enough to justify the addition to 
the standard library.

I've provided a reference PR. It does not cause backwards compatibility issues, 
complies with all formatter styles (%, {}, $), passes all tests and is simple 
enough to both use and understand.

Not sure if 3.9 is feature-closed for small additions like this.

--
components: Library (Lib)
messages: 370796
nosy: bar.harel
priority: normal
severity: normal
status: open
title: Added defaults parameter for logging.Formatter
type: enhancement
versions: Python 3.10, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40883] parse_string.c: free "str"

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +19884
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/20667

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40869] errno missing descriptions

[YoSTEALTH]

Change by YoSTEALTH :


--
versions: +Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40880] Invalid read in pegen.c

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +19883
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/20666

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40883] parse_string.c: free "str"

[Stefan Krah]

New submission from Stefan Krah :

Also in test_decimal, there's a small leak here:


==10040== 24 bytes in 1 blocks are definitely lost in loss record 549 of 5,095
==10040==at 0x4C2DE56: malloc (vg_replace_malloc.c:299)
==10040==by 0x643B33: fstring_compile_expr (parse_string.c:594)
==10040==by 0x643B33: fstring_find_expr (parse_string.c:924)
==10040==by 0x643B33: fstring_find_literal_and_expr (parse_string.c:1076)
==10040==by 0x643B33: _PyPegen_FstringParser_ConcatFstring 
(parse_string.c:1293)
==10040==by 0x644569: fstring_parse (parse_string.c:1409)
==10040==by 0x644569: fstring_find_expr (parse_string.c:980)
==10040==by 0x644569: fstring_find_literal_and_expr (parse_string.c:1076)
==10040==by 0x644569: _PyPegen_FstringParser_ConcatFstring 
(parse_string.c:1293)
==10040==by 0x62CE94: _PyPegen_concatenate_strings (pegen.c:2003)
==10040==by 0x62EF52: strings_rule (parse.c:10834)
==10040==by 0x62EF52: atom_rule (parse.c:10674)
==10040==by 0x6389A2: t_primary_raw (parse.c:14042)
==10040==by 0x6389A2: t_primary_rule (parse.c:13839)
==10040==by 0x638D67: star_target_rule (parse.c:12684)
==10040==by 0x6392FC: star_targets_rule (parse.c:12501)
==10040==by 0x63BD7B: _tmp_135_rule (parse.c:23255)
==10040==by 0x63BD7B: _loop1_22_rule (parse.c:16468)
==10040==by 0x63BD7B: assignment_rule (parse.c:2116)
==10040==by 0x63BD7B: small_stmt_rule (parse.c:1508)
==10040==by 0x63DB44: simple_stmt_rule (parse.c:1406)
==10040==by 0x63F995: statement_rule (parse.c:1240)
==10040==by 0x63F995: _loop1_11_rule (parse.c:15835)
==10040==by 0x63F995: statements_rule (parse.c:1175)
==10040==by 0x63FB49: block_rule (parse.c:6127)

--
messages: 370795
nosy: lys.nikolaou, pablogsal, skrah
priority: normal
severity: normal
status: open
title: parse_string.c: free "str"

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40869] errno missing descriptions

[YoSTEALTH]

Change by YoSTEALTH :


--
type:  -> enhancement
versions: +Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40882] memory leak in multiprocessing.shared_memory.SharedMemory in Windows

[Eryk Sun]

New submission from Eryk Sun :

mmap.mmap in Windows doesn't support an exist_ok parameter and doesn't 
correctly handle the combination fileno=-1, length=0, and tagname with an 
existing file mapping. SharedMemory has to work around these limitations. 

Part of the workaround for the create=False case requires mapping a view via 
MapViewOfFile in order to get the size from VirtualQuerySize, since mmap.mmap 
requires it (needlessly if implemented right) when fileno=-1. This mapped view 
never gets unmapped, which means the shared memory will never be freed until 
the termination of all processes that have opened it with create=False. Also, 
at least in a 32-bit process, this wastes precious address space.

_winapi.UnmapViewOfFile needs to be implemented. Then the temporary view can be 
unmapped as follows:

self._name = name
h_map = _winapi.OpenFileMapping(_winapi.FILE_MAP_READ, False, name)
try:
p_buf = _winapi.MapViewOfFile(h_map, _winapi.FILE_MAP_READ, 0, 0, 0)
finally:
_winapi.CloseHandle(h_map)
try:
size = _winapi.VirtualQuerySize(p_buf)
finally:
_winapi.UnmapViewOfFile(p_buf)
self._mmap = mmap.mmap(-1, size, tagname=name)

[1]: 
https://docs.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-unmapviewoffile

--
components: Library (Lib), Windows
messages: 370794
nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: memory leak in multiprocessing.shared_memory.SharedMemory in Windows
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40881] --with-valgrind broken

[Stefan Krah]

New submission from Stefan Krah :

./configure --with-valgrind:

Objects/unicodeobject.c: In function ‘unicode_release_interned’:
Objects/unicodeobject.c:15672:26: error: lvalue required as left operand of 
assignment
 Py_REFCNT(s) += 1;
  ^
Objects/unicodeobject.c:15678:26: error: lvalue required as left operand of 
assignment
 Py_REFCNT(s) += 2;



Well, Py_REFCNT(s) is no longer an lvalue. :-)

--
messages: 370793
nosy: skrah, vstinner
priority: normal
severity: normal
status: open
title: --with-valgrind broken

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40880] Invalid read in pegen.c

[Lysandros Nikolaou]

Change by Lysandros Nikolaou :


--
nosy: +gvanrossum, pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40869] errno missing descriptions

[YoSTEALTH]

Change by YoSTEALTH :


--
keywords: +patch
pull_requests: +19882
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/20665

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40862] argparse.BooleanOptionalAction accept and silently discard its the const argument

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 2.0 -> 3.0
pull_requests: +19881
pull_request: https://github.com/python/cpython/pull/20664

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40862] argparse.BooleanOptionalAction accept and silently discard its the const argument

[Raymond Hettinger]

Raymond Hettinger  added the comment:


New changeset b084d1b97e369293d2d2bc0791e2135822c923a8 by Rémi Lapeyre in 
branch 'master':
bpo-40862: Raise TypeError when const is given to 
argparse.BooleanOptionalAction (GH-20623)
https://github.com/python/cpython/commit/b084d1b97e369293d2d2bc0791e2135822c923a8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40880] Invalid read in pegen.c

[Stefan Krah]

New submission from Stefan Krah :

>From test_decimal:

test_xor (test.test_decimal.PyIBMTestCases) ... ==17597== Invalid read of size 1
==17597==at 0x64A7E2: newline_in_string (pegen.c:940)
==17597==by 0x64A84E: bad_single_statement (pegen.c:958)
==17597==by 0x64AD59: _PyPegen_run_parser (pegen.c:1101)
==17597==by 0x64B044: _PyPegen_run_parser_from_string (pegen.c:1194)
==17597==by 0x5C6D56: PyPegen_ASTFromStringObject (peg_api.c:27)
==17597==by 0x52A2A9: Py_CompileStringObject (pythonrun.c:1259)
==17597==by 0x63CBF6: builtin_compile_impl (bltinmodule.c:819)
==17597==by 0x63AF08: builtin_compile (bltinmodule.c.h:249)
==17597==by 0x5F9446: cfunction_vectorcall_FASTCALL_KEYWORDS 
(methodobject.c:440)
==17597==by 0x4D2642: _PyObject_VectorcallTstate (abstract.h:114)
==17597==by 0x4D26A1: PyObject_Vectorcall (abstract.h:123)
==17597==by 0x4E3F26: call_function (ceval.c:5111)
==17597==  Address 0xadc82bf is 1 bytes before a block of size 22 alloc'd
==17597==at 0x4C3016F: realloc (vg_replace_malloc.c:826)
==17597==by 0x46A983: _PyMem_RawRealloc (obmalloc.c:121)
==17597==by 0x46B49E: PyMem_Realloc (obmalloc.c:623)
==17597==by 0x5C9565: translate_newlines (tokenizer.c:654)
==17597==by 0x5C98FE: PyTokenizer_FromUTF8 (tokenizer.c:751)
==17597==by 0x64AF7F: _PyPegen_run_parser_from_string (pegen.c:1169)
==17597==by 0x5C6D56: PyPegen_ASTFromStringObject (peg_api.c:27)
==17597==by 0x52A2A9: Py_CompileStringObject (pythonrun.c:1259)
==17597==by 0x63CBF6: builtin_compile_impl (bltinmodule.c:819)
==17597==by 0x63AF08: builtin_compile (bltinmodule.c.h:249)
==17597==by 0x5F9446: cfunction_vectorcall_FASTCALL_KEYWORDS 
(methodobject.c:440)
==17597==by 0x4D2642: _PyObject_VectorcallTstate (abstract.h:114)
==17597== 



*--cur dereferences one below p->tok->buf in the last iteration.

--
components: Interpreter Core
messages: 370791
nosy: lys.nikolaou, skrah
priority: normal
severity: normal
stage: needs patch
status: open
title: Invalid read in pegen.c
type: behavior
versions: Python 3.10, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39791] New `files()` api from importlib_resources.

[Jason R. Coombs]

Jason R. Coombs  added the comment:


New changeset a4fa9a95153a3800dea60b3029b2dcaf8a4f6acb by Miss Islington (bot) 
in branch '3.9':
bpo-39791: Refresh importlib.metadata from importlib_metadata 1.6.1. (GH-20659) 
(GH-20661)
https://github.com/python/cpython/commit/a4fa9a95153a3800dea60b3029b2dcaf8a4f6acb


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40867] Remove unused include in Module/_randommodule.c

[Raymond Hettinger]

Raymond Hettinger  added the comment:


New changeset 45af786e111aed5f687e1f0d8b45b6a5e678a6bc by Erlend Egeberg 
Aasland in branch 'master':
bpo-40867: Remove unused include from Module/_randommodule.c (GH-20635)
https://github.com/python/cpython/commit/45af786e111aed5f687e1f0d8b45b6a5e678a6bc


--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40867] Remove unused include in Module/_randommodule.c

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Thank you.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19468] Relax the type restriction on reloaded modules

[Brett Cannon]

Brett Cannon  added the comment:

Thanks for the PR, Furkan !

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40876] Clarify error message in csv module

[Joannah Nanjekye]

Joannah Nanjekye  added the comment:

PR merged. We can close this. Thanks Ram.

--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40876] Clarify error message in csv module

[Joannah Nanjekye]

Joannah Nanjekye  added the comment:


New changeset 235f918f44bb89e27190db2f1823d191dbd4ad28 by Ram Rachum in branch 
'master':
bpo-40876: Clarify error message in the csv module (GH-20653)
https://github.com/python/cpython/commit/235f918f44bb89e27190db2f1823d191dbd4ad28


--
nosy: +nanjekyejoannah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40879] Strange regex cycle

[Matt Miller]

New submission from Matt Miller :

I was evaluating a few regular expressions for parsing URL.  One such 
expression 
(https://daringfireball.net/2010/07/improved_regex_for_matching_urls) causes 
the `re.Pattern` to exhibit some strange behavior (notice the stripped 
characters in the `repr`):

```
>>> STR_RE_URL = 
>>> r"""(?i)\b((?:[a-z][\w-]+:(?:/{1,3}|[a-z0-9%])|www\d{0,3}[.]|[a-z0-9.\-]+[.][a-z]{2,4}/)(?:[^\s()<>]+|\(([^\s()<>]+|(\([^\s()<>]+\)))*\))+(?:\(([^\s()<>]+|(\([^\s()<>]+\)))*\)|[^\s`!()\[\]{};:'".,<>?«»“”‘’]))"""
>>> print(re.compile(STR_RE_URL))
re.compile('(?i)\\b((?:[a-z][\\w-]+:(?:/{1,3}|[a-z0-9%])|www\\d{0,3}[.]|[a-z0-9.\\-]+[.][a-z]{2,4}/)(?:[^\\s()<>]+|\\(([^\\s()<>]+|(\\([^\\s()<>]+\\)))*\\))+(?:\\(([^\\s()<>]+|(\\([^\\s()<>]+\\)))*\\)|[^\\s`!()\,
 re.IGNORECASE)
```

The reason I started looking at this was because the following string causes 
the same `re.Pattern` object's `.search()` method to loop forever for some 
reason:

```
>>> weird_str = 
>>> """AY:OhQOhQNhQLdLAX78N'7M&6K%4K#4K#7N&9P(JcHOiQE^=8P'F_DJdLC\@9P&D\;IdKHbJ@Z8AY7@Y7AY7B[9E_Jc@Jc:F_1PjRRlSOiLKeAKeAGa=D^:F`=Ga=Fa>> url_pat.search(weird_str)
```

The `.search(weird_str)` will never exit.


I assume the `.search()` taking forever is is an error in the expression but 
the fact that it causes the `repr` to strip some characters was something I 
thought should be looked into.

I have not tested this on any other versions of Python.

--
components: Library (Lib)
messages: 370784
nosy: Matt Miller
priority: normal
severity: normal
status: open
title: Strange regex cycle
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25150] 3.5: Include/pyatomic.h is incompatible with OpenMP (compilation of the third-party module fails on Python 3.5)

[STINNER Victor]

STINNER Victor  added the comment:

Python.h does not include pyatomic.h in Python 3.8: the header file moved to 
the internal C API. The initial issue is fixed, so I close again the issue.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39791] New `files()` api from importlib_resources.

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 3.0 -> 4.0
pull_requests: +19879
pull_request: https://github.com/python/cpython/pull/20661

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39791] New `files()` api from importlib_resources.

[miss-islington]

Change by miss-islington :


--
pull_requests: +19880
pull_request: https://github.com/python/cpython/pull/20662

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39791] New `files()` api from importlib_resources.

[Jason R. Coombs]

Jason R. Coombs  added the comment:


New changeset 161541ab45278df6603dd870113b10f13e4d9e16 by Jason R. Coombs in 
branch 'master':
bpo-39791: Refresh importlib.metadata from importlib_metadata 1.6.1. (GH-20659)
https://github.com/python/cpython/commit/161541ab45278df6603dd870113b10f13e4d9e16


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19468] Relax the type restriction on reloaded modules

[miss-islington]

miss-islington  added the comment:


New changeset fef1fae9df3b03510f9defb25bd0388135b4c591 by Furkan Önder in 
branch 'master':
bpo-19468: delete unnecessary instance check in importlib.reload() (GH-19424)
https://github.com/python/cpython/commit/fef1fae9df3b03510f9defb25bd0388135b4c591


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40874] Update to libmpdec-2.5.0

[Stefan Krah]

Stefan Krah  added the comment:


New changeset 83bff88b4b16fb30491faa9263bbd6f3df4bab56 by Miss Islington (bot) 
in branch '3.9':
bpo-40874: Update to libmpdec-2.5.0 (GH-20652)
https://github.com/python/cpython/commit/83bff88b4b16fb30491faa9263bbd6f3df4bab56


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40878] Use c99 on the aixtools bot

[Stefan Krah]

New submission from Stefan Krah :

There appears to be an xlc buildbot with libmpdec failures.

libmpdec uses C99 extern inline semantics. From the brief period that I had 
access to xlc I remember that xlc was quite picky about C99.

Actually all of Python uses C99. So I think xlc_r needs to be invoked as c99_r.

--
messages: 370779
nosy: Michael.Felt, skrah
priority: normal
severity: normal
status: open
title: Use c99 on the aixtools bot

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39791] New `files()` api from importlib_resources.

[Jason R. Coombs]

Change by Jason R. Coombs :


--
pull_requests: +19878
pull_request: https://github.com/python/cpython/pull/20659

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24670] os.chdir breaks result of os.path.abspath(__file__) and os.path.realpath(__file__)

[Brett Cannon]

Change by Brett Cannon :


--
nosy:  -brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24725] test_socket testFDPassEmpty fails on OS X 10.11+ with "Cannot allocate memory"

[Brett Cannon]

Change by Brett Cannon :


--
nosy:  -brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24792] zipimporter masks import errors

[Brett Cannon]

Change by Brett Cannon :


--
nosy:  -brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24914] Python: Not just OO style but this is not mentioned on python.org or in FAQ

[Brett Cannon]

Change by Brett Cannon :


--
assignee: docs@python -> brett.cannon
versions: +Python 3.10 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24914] Python: Not just OO style but this is not mentioned on python.org or in FAQ

[Brett Cannon]

Change by Brett Cannon :


--
keywords: +patch
nosy: +brett.cannon
nosy_count: 3.0 -> 4.0
pull_requests: +19877
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/20658

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34003] csv.DictReader can return basic dict instead of OrderedDict

[Éric Araujo]

Change by Éric Araujo :


--
nosy: +eric.araujo
nosy_count: 3.0 -> 4.0
pull_requests: +19876
pull_request: https://github.com/python/cpython/pull/20657

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39791] New `files()` api from importlib_resources.

[Jason R. Coombs]

Change by Jason R. Coombs :


--
pull_requests: +19875
pull_request: https://github.com/python/cpython/pull/20656

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24914] Python: Not just OO style but this is not mentioned on python.org or in FAQ

[Brett Cannon]

Change by Brett Cannon :


--
nosy:  -brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24929] _strptime.TimeRE should not enforce range in regex

[Brett Cannon]

Change by Brett Cannon :


--
nosy:  -brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25160] Stop using deprecated imp module; imp should now emit a real DeprecationWarning

[Brett Cannon]

Brett Cannon  added the comment:

imp has been moved to DeprecationWarning and I think its usage in the stdlib 
has been dealt with at this point.

--
resolution:  -> fixed
stage: needs patch -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25150] 3.5: Include/pyatomic.h is incompatible with OpenMP (compilation of the third-party module fails on Python 3.5)

[Brett Cannon]

Change by Brett Cannon :


--
nosy:  -brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25268] Support pointing frozen modules to the corresponding source files, if available.

[Brett Cannon]

Change by Brett Cannon :


--
nosy:  -brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40862] argparse.BooleanOptionalAction accept and silently discard its the const argument

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Right.  Go ahead :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40862] argparse.BooleanOptionalAction accept and silently discard its the const argument

[Rémi Lapeyre]

Rémi Lapeyre  added the comment:

argparse.BooleanOptionalAction was introduced in Python3.9 so there is no code 
there should be no code that already rely on it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40874] Update to libmpdec-2.5.0

[Stefan Krah]

Change by Stefan Krah :


--
versions: +Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40877] Code coverage is blocking a merge again

[Stefan Krah]

Stefan Krah  added the comment:

It finally went through after Travis-CI hanging for about an hour.

--
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40874] Update to libmpdec-2.5.0

[Stefan Krah]

Stefan Krah  added the comment:


New changeset 087d612efebe7c64e5f079b07e0454111859830e by Stefan Krah in branch 
'master':
bpo-40874: Update to libmpdec-2.5.0 (GH-20652)
https://github.com/python/cpython/commit/087d612efebe7c64e5f079b07e0454111859830e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40874] Update to libmpdec-2.5.0

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 3.0 -> 4.0
pull_requests: +19874
pull_request: https://github.com/python/cpython/pull/20654

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40854] [Patch] Allow overriding sys.platlibdir

[STINNER Victor]

STINNER Victor  added the comment:

Fedora and OpenSUSE Linux distributions were using a downstream patch which 
replaced /lib/ with /lib64/ for like 15 years.

Is this issue a new issue? Or did Fedora always had the issue for example?

Which Linux distribution are you using?

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40862] argparse.BooleanOptionalAction accept and silently discard its the const argument

[Raymond Hettinger]

Raymond Hettinger  added the comment:

ISTM this should be addressed through documentation rather than by breaking 
currently deployed code that may have specified a default that already matches 
what the action does.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40521] [subinterpreters] Make free lists and unicode caches per-interpreter

[STINNER Victor]

STINNER Victor  added the comment:

pyperformance comparaison between:

* commit dc24b8a2ac32114313bae519db3ccc21fe45c982 (before "Make tuple free list 
per-interpreter" change)
* PR 20645 (dict free lists) which cumulates all free lists changes (already 
commited + the PR)

Extract of the tested patch, new PyInterpreterState members:

diff --git a/Include/internal/pycore_interp.h b/Include/internal/pycore_interp.h
index f04ea330d0..b1a25e0ed4 100644
--- a/Include/internal/pycore_interp.h
+++ b/Include/internal/pycore_interp.h
(...)
@@ -157,6 +233,18 @@ struct _is {
 */
 PyLongObject* small_ints[_PY_NSMALLNEGINTS + _PY_NSMALLPOSINTS];
 #endif
+struct _Py_unicode_state unicode;
+struct _Py_float_state float_state;
+/* Using a cache is very effective since typically only a single slice is
+   created and then deleted again. */
+PySliceObject *slice_cache;
+
+struct _Py_tuple_state tuple;
+struct _Py_list_state list;
+struct _Py_dict_state dict_state;
+struct _Py_frame_state frame;
+struct _Py_async_gen_state async_gen;
+struct _Py_context_state context;
 };


Results:

$ python3 -m pyperf compare_to 2020-06-04_20-10-master-dc24b8a2ac32.json.gz 
2020-06-04_20-10-master-dc24b8a2ac32-patch-free_lists.json.gz -G 
Slower (10):
- chameleon: 20.1 ms +- 0.4 ms -> 23.1 ms +- 4.0 ms: 1.15x slower (+15%)
- logging_silent: 334 ns +- 51 ns -> 371 ns +- 70 ns: 1.11x slower (+11%)
- spectral_norm: 274 ms +- 37 ms -> 302 ms +- 55 ms: 1.10x slower (+10%)
- logging_format: 22.5 us +- 0.4 us -> 24.5 us +- 2.7 us: 1.09x slower (+9%)
- json_dumps: 26.6 ms +- 4.0 ms -> 28.7 ms +- 5.5 ms: 1.08x slower (+8%)
- sympy_sum: 390 ms +- 3 ms -> 415 ms +- 45 ms: 1.06x slower (+6%)
- float: 217 ms +- 3 ms -> 231 ms +- 30 ms: 1.06x slower (+6%)
- pidigits: 306 ms +- 32 ms -> 323 ms +- 47 ms: 1.06x slower (+6%)
- python_startup_no_site: 8.71 ms +- 0.77 ms -> 8.94 ms +- 0.91 ms: 1.03x 
slower (+3%)
- xml_etree_process: 130 ms +- 1 ms -> 133 ms +- 2 ms: 1.02x slower (+2%)

Faster (9):
- pickle_pure_python: 1.05 ms +- 0.16 ms -> 964 us +- 19 us: 1.09x faster (-9%)
- scimark_sparse_mat_mult: 11.4 ms +- 2.1 ms -> 10.5 ms +- 1.7 ms: 1.09x faster 
(-8%)
- hexiom: 19.5 ms +- 4.1 ms -> 18.0 ms +- 3.0 ms: 1.08x faster (-7%)
- telco: 15.7 ms +- 3.1 ms -> 14.5 ms +- 0.4 ms: 1.08x faster (-7%)
- unpickle: 31.8 us +- 5.7 us -> 29.5 us +- 4.9 us: 1.08x faster (-7%)
- scimark_lu: 292 ms +- 60 ms -> 274 ms +- 34 ms: 1.07x faster (-6%)
- django_template: 123 ms +- 16 ms -> 119 ms +- 2 ms: 1.04x faster (-3%)
- xml_etree_generate: 160 ms +- 4 ms -> 156 ms +- 3 ms: 1.02x faster (-2%)
- xml_etree_iterparse: 178 ms +- 3 ms -> 177 ms +- 2 ms: 1.01x faster (-1%)

Benchmark hidden because not significant (41): (...)


If we ignore differences smaller than 5%:

$ python3 -m pyperf compare_to 2020-06-04_20-10-master-dc24b8a2ac32.json.gz 
2020-06-04_20-10-master-dc24b8a2ac32-patch-free_lists.json.gz -G --min-speed=5
Slower (8):
- chameleon: 20.1 ms +- 0.4 ms -> 23.1 ms +- 4.0 ms: 1.15x slower (+15%)
- logging_silent: 334 ns +- 51 ns -> 371 ns +- 70 ns: 1.11x slower (+11%)
- spectral_norm: 274 ms +- 37 ms -> 302 ms +- 55 ms: 1.10x slower (+10%)
- logging_format: 22.5 us +- 0.4 us -> 24.5 us +- 2.7 us: 1.09x slower (+9%)
- json_dumps: 26.6 ms +- 4.0 ms -> 28.7 ms +- 5.5 ms: 1.08x slower (+8%)
- sympy_sum: 390 ms +- 3 ms -> 415 ms +- 45 ms: 1.06x slower (+6%)
- float: 217 ms +- 3 ms -> 231 ms +- 30 ms: 1.06x slower (+6%)
- pidigits: 306 ms +- 32 ms -> 323 ms +- 47 ms: 1.06x slower (+6%)

Faster (6):
- pickle_pure_python: 1.05 ms +- 0.16 ms -> 964 us +- 19 us: 1.09x faster (-9%)
- scimark_sparse_mat_mult: 11.4 ms +- 2.1 ms -> 10.5 ms +- 1.7 ms: 1.09x faster 
(-8%)
- hexiom: 19.5 ms +- 4.1 ms -> 18.0 ms +- 3.0 ms: 1.08x faster (-7%)
- telco: 15.7 ms +- 3.1 ms -> 14.5 ms +- 0.4 ms: 1.08x faster (-7%)
- unpickle: 31.8 us +- 5.7 us -> 29.5 us +- 4.9 us: 1.08x faster (-7%)
- scimark_lu: 292 ms +- 60 ms -> 274 ms +- 34 ms: 1.07x faster (-6%)

Benchmark hidden because not significant (46): (...)


Honestly, I'm surprised by these results. I don't see how these free lists 
change can make between 6 and 9 benchamrks faster (ex: 1.08x faster for 
telco!?). For me, it sounds like speed.python.org runner has some troubles. You 
can notice it if you look at the 3 last runs at https://speed.python.org/ : 
they are some spikes (in both directions, faster or slower) which are very 
surprising.

Pablo recently upgrade Ubuntu on the benchmark runner server. I don't know if 
it's related.

I plan to recompute all benchmarks run on the benchmark runner server since 
over the last years, pyperf and pyperformance were upgraded multiple times (old 
data were computed with old versions) and the system (Ubuntu) was upgraded 
(again, old data were computed with older Ubiuntu packages).

--

__

[issue40877] Code coverage is blocking a merge again

[Stefan Krah]

New submission from Stefan Krah :

Again code coverage prevents the merge button from going green:

https://travis-ci.org/github/python/cpython/jobs/695095365


"The job exceeded the maximum time limit for jobs, and has been terminated."

Why can this not run on buildbot.python.org?

--
messages: 370770
nosy: skrah
priority: normal
severity: normal
status: open
title: Code coverage is blocking a merge again

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40876] Clarify error message in csv module

[Ram Rachum]

Change by Ram Rachum :


--
keywords: +patch
pull_requests: +19873
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/20653

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40876] Clarify error message in csv module

[Ram Rachum]

New submission from Ram Rachum :

I was working with the csv module, and I vaguely remembered that you should 
open files in binary mode. So I did.

Then I saw this error message:

_csv.Error: iterator should return strings, not bytes (did you open the 
file in text mode?)

I read the end and thought "I didn't open it in text mode, what does it want 
from me?!" It took a careful reading to figure out that I was *supposed to* 
open it in text mode.

I'm going to open a PR to slightly change the text to:

_csv.Error: iterator should return strings, not bytes (the file should be 
opened in text mode)

--
components: Library (Lib)
messages: 370769
nosy: cool-RR
priority: normal
severity: normal
status: open
title: Clarify error message in csv module
type: behavior
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40875] Implement __repr__ for classes in csv module

[Ram Rachum]

New submission from Ram Rachum :

Why see this:



When you can see this: 



It could show columns=? if they weren't read yet. Any other info would be good 
too.

This applies to all classes in the csv module.

--
components: Library (Lib)
messages: 370768
nosy: cool-RR
priority: normal
severity: normal
status: open
title: Implement __repr__ for classes in csv module
type: enhancement
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35845] Can't read a F-contiguous memoryview in physical order

[jakirkham]

jakirkham  added the comment:

Sorry if I'm just misunderstanding the discussion here. Would it make sense to 
have an `order` keyword argument to `cast` as well? This seems useful when 
interpreting a flatten F-order `bytes` object (say on the receiving end of a 
transmission).

--
nosy: +jakirkham

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40874] Update to libmpdec-2.5.0

[Stefan Krah]

Change by Stefan Krah :


--
keywords: +patch
pull_requests: +19872
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/20652

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40874] Update to libmpdec-2.5.0

[Stefan Krah]

New submission from Stefan Krah :

Synopsis: There are no relevant new features for _decimal, but it would be too 
much work/error prone to have divergent code in libmpdec-2.5.0 and Python 3.9, 
especially for the Linux distributions.

I'll release libmpdec-2.5.0/libmpdec++-2.5.0 in a month or so. The standalone 
lib needs the new versions of mpd_qsqrt() and mpd_qdiv(), because it allows 
identical result/input args.  This is not needed for _decimal, but the 
distributions should have the correct version. 


In detail
=

   - Use Google style guide for header guards and includes.

   - Update mpdecimal.h for C++11.

   - Use minimum set of includes.

   - Whitespace fixes.

   - Add annotations to suppress false positives from static analyzers.

   - Small rewrite in base conversion functions to suppress false positives
 from static analyzers.

   - MSVC: make libmpdec /W4 warning free and replace UNUSED with void casts.

   - MSVC: C++ fixes in vccompat.h.

   - Make a couple of quiet functions safe for being called with a dirty status
 (irrelevant for _decimal and not recommended anyway -- always set the 
status
  to 0 before calling a quiet function).

   - Add the sqrt/div versions that are already in the Python libmpdec but not
 in the upstream libmpdec.  Also make them safe for identical result/operand
 arguments (irrelevant for _decimal, since Decimals are immutable).


New functions for the upcoming libmpdec++ (unused in _decimal)
==

  - mpd_qset_string_exact()

  - mpd_qset_i64_exact()

  - mpd_qset_u64_exact()

  - mpd_qcopy_cxx()

--
assignee: skrah
messages: 370766
nosy: doko, lukasz.langa, skrah
priority: normal
severity: normal
status: open
title: Update to libmpdec-2.5.0
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40873] Something wrong with html.unescape()

[Валентин Dreyk]

New submission from Валентин Dreyk :

import html
import xml.sax.saxutils as saxutils

print(saxutils.unescape("®hard"))  # ®hard
print(html.unescape("®hard"))  # ®hard



html.unescape() replace "®" to "®" even without ";" at the end.

--
components: Library (Lib)
messages: 370765
nosy: Валентин Dreyk
priority: normal
severity: normal
status: open
title: Something wrong with html.unescape()
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38035] shared_semaphores cannot be shared across unrelated processes

[Vinay Sharma]

Vinay Sharma  added the comment:

As suggested I have sent a mail to Python Ideas regarding this issue.
Link to Python Ideas Archive: 
https://mail.python.org/archives/list/python-id...@python.org/thread/X4AKFFMYEKW6GFOUMXMOJ2OBINNY2Q6L/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[Dong-hee Na]

Dong-hee Na  added the comment:


New changeset 3ad52e366fea37b02a3f619e6b7cffa7dfbdfa2e by Dong-hee Na in branch 
'master':
bpo-1635741: Port mmap module to multiphase initialization (GH-19459)
https://github.com/python/cpython/commit/3ad52e366fea37b02a3f619e6b7cffa7dfbdfa2e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40872] multiprocess.Lock is missing the locked() method

[Rémi Lapeyre]

Change by Rémi Lapeyre :


--
keywords: +patch
pull_requests: +19871
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/20651

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40872] multiprocess.Lock is missing the locked() method

[Rémi Lapeyre]

New submission from Rémi Lapeyre :

multiprocessing is supposed to be a drop-in replacement for the threading 
module and multiprocessing.Lock() is advertised as a "a close analog of 
threading.Lock." but it is missing the locked() method that returns whether the 
current status of the lock.

--
components: Library (Lib)
messages: 370762
nosy: remi.lapeyre
priority: normal
severity: normal
status: open
title: multiprocess.Lock is missing the locked() method
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40871] threading.Event.wait_unset()

[Jacob Kunnappally]

New submission from Jacob Kunnappally :

Just requesting a threading.Event.wait_unset(timeout=None) function. I would 
request the same for multiprocessing.

My use case:

I've made my own class that adds a little bit of IPC plumbing to the base 
Process class (ChildProcess). Each ChildProcess has a status that it can update 
to let other threads/processes know what it's doing at the moment. There is a 
configurable period when that updated status can be considered "fresh". In some 
cases, I would like a listening process to be able to ignore the "freshness" 
and only trigger some action only if the status updates while the listening 
process is waiting for it to update.

To do this, I need to be able to know when the status goes unfresh so that 
waiting for the status to update can begin in earnest. Right now I am polling 
manually, and that can't be the right answer.

Happy to clarify the above paragraphs. That's as best as I could think to 
describe it in text.

--
components: Library (Lib)
messages: 370761
nosy: Jacob Kunnappally
priority: normal
severity: normal
status: open
title: threading.Event.wait_unset()
type: enhancement
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19670] SimpleCookie Generates Non-RFC6265-Compliant Cookies

[Ido Michael]

Ido Michael  added the comment:

I think it can be closed?

--
nosy: +Ido Michael

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19670] SimpleCookie Generates Non-RFC6265-Compliant Cookies

[Ido Michael]

Change by Ido Michael :


--
nosy: +taleinat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36783] No documentation for _FromXandFold C API functions

[Roundup Robot]

Change by Roundup Robot :


--
nosy: +python-dev
nosy_count: 7.0 -> 8.0
pull_requests: +19870
pull_request: https://github.com/python/cpython/pull/20650

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40870] Custom AST can crash Python (debug build)

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

cross-linking pytest issue: https://github.com/pytest-dev/pytest/issues/7322

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36457] functools.singledispatchmethod interacts poorly with subclasses

[Dries Schaumont]

Dries Schaumont  added the comment:

I am interested in the fix, added myself to nosy list.

--
nosy: +dschaumont

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40521] [subinterpreters] Make free lists and unicode caches per-interpreter

[STINNER Victor]

STINNER Victor  added the comment:

> Have you done any performance analysis or tests of the cumulative effect of 
> all these changes?

No. It would be interesting to measure that using pyperformance.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40521] [subinterpreters] Make free lists and unicode caches per-interpreter

[Mark Shannon]

Mark Shannon  added the comment:

I'm worried about the performance impact of these changes, especially as many 
of the changes haven't been reviewed.

Have you done any performance analysis or tests of the cumulative effect of all 
these changes?

--
nosy: +Mark.Shannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40870] Custom AST can crash Python (debug build)

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
keywords: +patch
pull_requests: +19869
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/20649

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40521] [subinterpreters] Make free lists and unicode caches per-interpreter

[STINNER Victor]

STINNER Victor  added the comment:

bench_dict.patch: Microbenchmark on the C function PyDict_New() to measure the 
overhead of PR 20645.

--
Added file: https://bugs.python.org/file49216/bench_dict.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40870] Custom AST can crash Python (debug build)

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
components: +Interpreter Core
nosy: +pablogsal, serhiy.storchaka
type:  -> behavior
versions: +Python 3.10, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40870] Custom AST can crash Python (debug build)

[Batuhan Taskaya]

New submission from Batuhan Taskaya :

import ast
t = ast.fix_missing_locations(ast.Expression(ast.Name("True", ast.Load(
compile(t, "", "eval")

compilation of this AST can crash the interpreter for 3.8+

test_constant_as_name (test.test_ast.AST_Tests) ... python: 
Python/compile.c:3559: compiler_nameop: Assertion 
`!_PyUnicode_EqualToASCIIString(name, "None") && 
!_PyUnicode_EqualToASCIIString(name, "True") && 
!_PyUnicode_EqualToASCIIString(name, "False")' failed.
Fatal Python error: Aborted

I've encountered this while running test suite of 'pytest' with the current 
master, so I guess there are some usages related this out there. IMHO we should 
validate this on the PyAST_Validate step to prevent this kind of crashes.

--
messages: 370753
nosy: BTaskaya
priority: normal
severity: normal
status: open
title: Custom AST can crash Python (debug build)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40521] [subinterpreters] Make free lists and unicode caches per-interpreter

[STINNER Victor]

STINNER Victor  added the comment:

> bpo-40521: Make list free list per-interpreter (GH-20642)
> https://github.com/python/cpython/commit/88ec9190105c9b03f49aaef601ce02b242a75273

This change contains an interesting fix:

* _PyGC_Fini() clears gcstate->garbage list which can be stored in
  the list free list. Call _PyGC_Fini() before _PyList_Fini() to
  prevent leaking this list.

Maybe "Fini" functions should disable free lists to prevent following code to 
add something to a free list, during Python finalization.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40077] Convert static types to PyType_FromSpec()

[STINNER Victor]

Change by STINNER Victor :


--
components: +Subinterpreters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40724] Support buffer protocol with type specs

[Stefan Behnel]

Change by Stefan Behnel :


--
keywords: +patch
pull_requests: +19868
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/20648

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com