[issue38438] argparse "usage" overly-complex with nargs="*"

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Bob, would like to create a PR?

--
assignee:  -> rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38014] Python 3.7 does not compile

[Matej Cepl]

Change by Matej Cepl :


--
nosy:  -mcepl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38440] Possible news issues with IDLE

[Raymond Hettinger]

New submission from Raymond Hettinger :

I've been exercising the 3.8.0rc1 version of IDLE and noticed two anomalies but 
have not had a chance to investigate further:

* Periodically in a long shell session, the cursor jumps to the beginning of 
the cumulative text output.  This necessitates a lot of scrolling to get back 
down to the active part of the session.

* The pop-up message box for interrupting currently running code will appear 
several times in succession.  Dismissing the box doesn't clear a queue of 
events, so the message may occur several times in a row.

--
assignee: terry.reedy
components: IDLE
messages: 354418
nosy: rhettinger, terry.reedy
priority: normal
severity: normal
status: open
title: Possible news issues with IDLE
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38167] O_DIRECT read fails with 4K mmap buffer

[Josh Rosenberg]

Josh Rosenberg  added the comment:

Yeah, not a bug. The I/O subsystem was substantially rewritten between Python 2 
and Python 3, so you sometimes need to be more explicit about things like 
buffering, but as you note, once the buffering is correct, the code works; 
there's nothing to fix.

--
resolution:  -> not a bug
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38438] argparse "usage" overly-complex with nargs="*"

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17894] Edits to descriptor howto

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I am actively working on updates.  However, the work on Python 3.8's whatsnew 
and doc fixes are a more immediate priority.  Cumulatively, I've put an a lot 
of effort in to this and have made a continuous stream of improvements over the 
years including this year.  In a way, maintaining this entry is not much 
different from being a module maintainer who has the principal responsibility 
and authority over the updates.  This is something that I frequently teach and 
have battle tested course material that I'm looking forward to including in the 
entry.

Please give me space and don't shove. IMO some of the past posts on this 
subject are border-line abusive, leaving me feeling hounded and degraded 
(especially with respect to my writing style).  Please be kind and let me 
continue to improve my contribution at my own pace.  

Please don't re-open this issue. I have marked the patch as rejected because it 
is at odds with the improvements that I already have in progress and because 
some of it is essentially just a stylistic difference.

--
resolution:  -> rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13322] The io module doesn't support non-blocking files

[Matt Joiner]

Change by Matt Joiner :


--
nosy:  -anacrolix

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38422] Clarify docstrings of pathlib suffix(es)

[Brandt Bucher]

Change by Brandt Bucher :


--
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38014] Python 3.7 does not compile

[Teddy Thomas]

Change by Teddy Thomas :


--
nosy: +tthoma24

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38438] argparse "usage" overly-complex with nargs="*"

[Brandt Bucher]

Change by Brandt Bucher :


--
nosy: +brandtbucher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38355] ntpath.realpath() fails on sys.executable

[Kyle Stanley]

Change by Kyle Stanley :


--
stage: backport needed -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38359] pyw.exe opens console window in Windows 10

[Kyle Stanley]

Change by Kyle Stanley :


--
stage: backport needed -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38344] activate.bat else needs to be on the same line as the if

[Kyle Stanley]

Change by Kyle Stanley :


--
stage: patch review -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38439] IDLE menu icon is blurry on GNOME

[Miro Hrončok]

New submission from Miro Hrončok :

See the attached screenshot. All the GNOME apps have scalable icons, only the 
IDLE icon is scaled up from 48x48 pixels.

I've found similar issue32129, but apparently the accepted resolution was to 
change a bit of code somewhere, not to provide higher resolution icons.

--
assignee: terry.reedy
components: IDLE
files: idle.png
messages: 354415
nosy: hroncok, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE menu icon is blurry on GNOME
versions: Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file48655/idle.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38348] Make python -m ast more configurable

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:

I don't have any strong opinion either way, so it looks like we need to wait 
until someone else will ask for this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38428] Can't gracefully ctrl+C multiprocessing pool on Python3 & Windows

[Eryk Sun]

Eryk Sun  added the comment:

> Python 2.7 had totally different code, IIRC.

In 2.7 the wait() method of threading._Condition implements the timeout with a 
loop the calls time.sleep() with an increasing delay from 500 microseconds up 
to 50 ms, and no more than the remaining time.

> But the good news is that we should be able to use code similar to 
> what's in Modules/timemodule.c#L1878 (the pysleep() function) to also 
> wait on the event we use to emulate signals on Windows. 

If Python continues to use emulated condition variables, then the kernel wait 
in _PyCOND_WAIT_MS can easily be adapted to include the SIGINT event. However, 
isn't it planned at some point to remove the emulated condition-variable 
implementation in favor of native condition variables? If so and PyCOND_WAIT 
and PyCOND_TIMEDWAIT end up calling SleepConditionVariableSRW, there's no 
obvious way to interrupt this to support Ctrl+C and Ctrl+Break (given a custom 
SIGBREAK handler) in the main thread. 

In particular, in Windows 8+, SleepConditionVariableSRW begins by spinning in 
user mode for a fixed number of cycles. Then it calls NtWaitForAlertByThreadId 
to block until alerted by NtAlertThreadByThreadId, which gets called internally 
by Wake[All]ConditionVariable. It wouldn't be kosher to call 
NtAlertThreadByThreadId directly in order to alert the main thread because it's 
an undocumented system call, plus it's poking into the internal implementation 
details of SleepConditionVariableSRW and WakeConditionVariable, which could 
leave the variable in a bad state.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32561] Add API to io objects for cache-only reads/writes

[Nathaniel Smith]

Nathaniel Smith  added the comment:

> If you wanted to keep async disk access separate from the io module, then 
> what we'd have to do is to create a fork of all the code in the io module, 
> and add this feature to it.

Thinking about this again today, I realized there *might* be another option.

The tricky thing about supporting async file I/O is that users want the whole 
io module interface, and we don't want to have to reimplement all the 
functionality in TextIOWrapper, BufferedReader, BufferedWriter, etc. And we 
still need the blocking functionality too, for when we fall back to threads.

But, here's a possible hack. We could implement our own version of 'FileIO' 
that wraps around a real FileIO. Every operation just delegates to the 
underlying FileIO – but with a twist. Something like:

def wrapped_op(self, *args):
if self._cached_op.key == (op, args):
return self._cached_op.result
if MAGIC_THREAD_LOCAL.io_is_forbidden:
def cache_filler():
MAGIC_THREAD_LOCAL.io_is_forbidden = False
self._cached_op = self._real_file.op(*args)
raise IOForbiddenError(cache_filler)
return self._real_file.op(*args)

And then in order to implement an async operation, we do something like:

async def op(self, *args):
while True:
try:
# First try fulfilling the operation from cache
MAGIC_THREAD_LOCAL.io_is_forbidden = True
return self._io_obj.op(*args)
except IOForbiddenError as exc:
# We have to actually hit the disk
# Run the real IO operation in a thread, then try again
await in_thread(cache_filler)
finally:
del MAGIC_THREAD_LOCAL.io_is_forbidden

This is pretty convoluted: we keep trying the operation on the outer "buffered" 
object, seeing which low-level I/O operation it gets stuck on, doing that I/O 
operation, and trying again. There's all kinds of tricky non-local state here; 
like for example, there isn't any formal guarantee that the next time we try 
the "outer" I/O operation it will end up making exactly the same request to the 
"inner" RawIO object. If you try performing I/O operations on the same file 
from multiple tasks concurrently then you'll get all kinds of havoc. But if it 
works, then it does have two advantages:

First, it doesn't require changes to the io module, which is at least nice for 
experimentation.

And second, it's potentially compatible with the io_uring style of async disk 
I/O API. I don't actually know if this matters; if you look at the io_uring 
docs, the only reason they say they're more efficient than a thread pool is 
that they can do the equivalent of preadv(RWF_NOWAIT), and it's a lot easier to 
add preadv(RWF_NOWAIT) to a thread pool than it is to implement io_uring. But 
still, this approach is potentially more flexible than my original idea.

We'd still have to reimplement open() in order to set up our weird custom IO 
stacks, but hopefully that's not *too* bad, since it's mostly just a bunch of 
if statements to decide which wrappers to stick around the raw IO object.

My big concern is, that I'm not actually sure if this works :-).

The thing is, for this to work, we need 
TextIOWrapper/BufferedReader/BufferedWriter to be very well-behaved when the 
underlying operation raises an exception. In particular, if they're doing a 
complex operation that requires multiple calls to the underlying object, and 
the second call raises an exception, they need to keep the first call's results 
in their buffer so that next time they can pick up where they left off. And I 
have no idea if that's true.

I guess if you squint this is kind of like the non-blocking support in the io 
module – IOForbiddenError is like NonBlockingError. The big difference is that 
here, we don't have any "partial success" state at the low-level; either we do 
the operation immediately, or we punt and do the operation in a thread. Either 
way it completes as a single indivisible unit. So that might simplify things? 
From a quick skim of issue13322 it sounds like a lot of the problems with the 
current "non-blocking" mode come from these partial success states, but I 
haven't read it in detail.

--
title: Add API to io objects for non-blocking reads/writes -> Add API to io 
objects for cache-only reads/writes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36553] inspect.is_decorator_call(frame)

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38436] Improved performance for list addition.

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

>From the PR:

Where the benchmarks executed with CPU isolation? I am a bit suspicious of the

- pickle_list: 3.63 us +- 0.08 us -> 3.78 us +- 0.10 us: 1.04x slower (+4%)
If not, can you repeat them with CPU isolation and affinity? Check this for 
more info

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38438] argparse "usage" overly-complex with nargs="*"

[Eric V. Smith]

Eric V. Smith  added the comment:

I think [arg ...] would make sense.

Removing 3.5 and 3.6, since they're in security only mode. Adding 3.9.

--
nosy: +eric.smith
versions: +Python 3.9 -Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38437] Set GC_DEBUG for debug builds of the interpreter

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Thanks Victor and Tim!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38437] Set GC_DEBUG for debug builds of the interpreter

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38437] Set GC_DEBUG for debug builds of the interpreter

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset 320dd504ddf65efe946e6d1e89053ed2d7ebe1e9 by Pablo Galindo in 
branch 'master':
bpo-38437: Activate GC_DEBUG when PY_DEBUG is set (GH-16707)
https://github.com/python/cpython/commit/320dd504ddf65efe946e6d1e89053ed2d7ebe1e9


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38435] Start the deprecation cycle for subprocess preexec_fn

[Gregory P. Smith]

Gregory P. Smith  added the comment:

With task specific arguments.  cwd, start_new_session, group, extra_groups,
user, etc..

We cannot provide a general do everything replacement and should not try.
It not possible.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38437] Set GC_DEBUG for debug builds of the interpreter

[Tim Peters]

Tim Peters  added the comment:

+1.  This code got quite brittle when they decided to fit two pointers, a fat 
integer, and 3 flags into a struct with room for only the two pointers ;-)  
It's a mine field now.  Enabling one of the few automated mine detectors is 
thoroughly sensible.

--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38437] Set GC_DEBUG for debug builds of the interpreter

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> Under which condition can such list be corrupted?

If someone is adding/modifiying the gc and calls any of the functions that set 
the gc flags like (PREV_MASK_COLLECTING).

One example is that after calling move_unreachable(), the unreachable set has 
NEXT_MASK_UNREACHABLE and therefore is an invalid list (until 
move_legacy_finalizers removes the flag and makes it valid again). If you mess 
the order or violate any of the contract, the lists will be invalid. It helps a 
lot to know that you are not leaving the lists in invalid state by mistake or 
on the other hand, to check that they are valid when they are supposed to be 
valid.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38435] Start the deprecation cycle for subprocess preexec_fn

[STINNER Victor]

STINNER Victor  added the comment:

What is the recommanded way to replace preexec_fn?

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38437] Set GC_DEBUG for debug builds of the interpreter

[STINNER Victor]

STINNER Victor  added the comment:

Under which condition can such list be corrupted?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38438] argparse "usage" overly-complex with nargs="*"

[Bob Alexander]

New submission from Bob Alexander :

This program:

import argparse
ap = argparse.ArgumentParser()
ap.add_argument("arg", nargs="*")
ap.parse_args()

Gives usage message:
usage: help_complexity.py [-h] [arg [arg ...]]

It's correct, but unnecessarily complex and challenging to the user.
If I were manually writing the usage, arg... would do, or maybe [arg ...] to be 
consistent with other messages??

--
components: Library (Lib)
messages: 354402
nosy: bobjalex
priority: normal
severity: normal
status: open
title: argparse "usage" overly-complex with nargs="*"
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38437] Set GC_DEBUG for debug builds of the interpreter

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
stage: patch review -> 
versions:  -Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38437] Set GC_DEBUG for debug builds of the interpreter

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +16292
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/16707

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38437] Set GC_DEBUG for debug builds of the interpreter

[Pablo Galindo Salgado]

New submission from Pablo Galindo Salgado :

While working on bpo-38379 I had to manually set the GC_DEBUG macro to 1 to 
activate the extra checks that 'validate_list' does. These checks are super 
useful to make sure all the gc lists used are consistent and in the expected 
state with the expected masks.

For this reason, I propose to always activate GC_DEBUG for debug builds of the 
interpreter. It will have a performance impact, but the debugging benefits are 
substantial.

--
messages: 354401
nosy: pablogsal, vstinner
priority: normal
severity: normal
status: open
title: Set GC_DEBUG for debug builds of the interpreter
versions: Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38282] socketmodule.c: _FORTIFY_SOURCE=2 warning in AF_ALG case of getsockaddrarg()

[miss-islington]

Change by miss-islington :


--
pull_requests: +16291
pull_request: https://github.com/python/cpython/pull/16706

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38282] socketmodule.c: _FORTIFY_SOURCE=2 warning in AF_ALG case of getsockaddrarg()

[STINNER Victor]

STINNER Victor  added the comment:


New changeset d565fb9828ee9c494bb7a80057a08e4738273e30 by Victor Stinner in 
branch 'master':
bpo-38282: Rewrite getsockaddrarg() helper function (GH-16698)
https://github.com/python/cpython/commit/d565fb9828ee9c494bb7a80057a08e4738273e30


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36229] Avoid unnecessary copies for list, set, and bytearray ops.

[Brandt Bucher]

Change by Brandt Bucher :


--
pull_requests:  -16290

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38436] Improved performance for list addition.

[Brandt Bucher]

Change by Brandt Bucher :


--
keywords: +patch
pull_requests: +16289
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/16705

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36229] Avoid unnecessary copies for list, set, and bytearray ops.

[Brandt Bucher]

Change by Brandt Bucher :


--
pull_requests: +16290
pull_request: https://github.com/python/cpython/pull/16705

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38436] Improved performance for list addition.

[Brandt Bucher]

New submission from Brandt Bucher :

The attached PR adds a fast path for BINARY_ADD instructions involving two 
lists, where the left list has a refcount of exactly 1.

In this case, we instead do a PySequence_InPlaceConcat operation. This has the 
affect of avoiding quadratic complexity for list summations, by keeping only 
one intermediate result and extending it in-place.

For (potentially large) lists:

a + b + c + d ...

Currently executed as:

tmp = a + b
tmp = tmp + c
tmp = tmp + d
...

With this change:

tmp = a + b
tmp += c
tmp += d
...

Here are the pyperformance results (optimizations enabled):

Slower (6):
- pickle_list: 3.63 us +- 0.08 us -> 3.78 us +- 0.10 us: 1.04x slower (+4%)
- xml_etree_generate: 102 ms +- 2 ms -> 104 ms +- 3 ms: 1.02x slower (+2%)
- xml_etree_parse: 166 ms +- 4 ms -> 169 ms +- 9 ms: 1.02x slower (+2%)
- scimark_sparse_mat_mult: 5.42 ms +- 0.11 ms -> 5.47 ms +- 0.16 ms: 1.01x 
slower (+1%)
- pickle_dict: 23.6 us +- 0.8 us -> 23.8 us +- 0.4 us: 1.01x slower (+1%)
- scimark_fft: 413 ms +- 6 ms -> 416 ms +- 6 ms: 1.01x slower (+1%)

Faster (25):
- 2to3: 394 ms +- 18 ms -> 381 ms +- 8 ms: 1.03x faster (-3%)
- python_startup_no_site: 12.0 ms +- 0.2 ms -> 11.7 ms +- 0.2 ms: 1.03x faster 
(-3%)
- nqueens: 121 ms +- 2 ms -> 118 ms +- 3 ms: 1.03x faster (-2%)
- pathlib: 36.0 ms +- 1.3 ms -> 35.1 ms +- 0.9 ms: 1.02x faster (-2%)
- pickle_pure_python: 523 us +- 15 us -> 511 us +- 13 us: 1.02x faster (-2%)
- deltablue: 8.54 ms +- 0.25 ms -> 8.35 ms +- 0.27 ms: 1.02x faster (-2%)
- logging_silent: 220 ns +- 6 ns -> 215 ns +- 8 ns: 1.02x faster (-2%)
- raytrace: 566 ms +- 8 ms -> 554 ms +- 11 ms: 1.02x faster (-2%)
- hexiom: 11.2 ms +- 0.3 ms -> 11.0 ms +- 0.2 ms: 1.02x faster (-2%)
- go: 294 ms +- 6 ms -> 288 ms +- 5 ms: 1.02x faster (-2%)
- python_startup: 15.7 ms +- 0.4 ms -> 15.4 ms +- 0.5 ms: 1.02x faster (-2%)
- float: 136 ms +- 4 ms -> 134 ms +- 6 ms: 1.02x faster (-2%)
- scimark_sor: 232 ms +- 8 ms -> 228 ms +- 5 ms: 1.02x faster (-2%)
- fannkuch: 554 ms +- 8 ms -> 545 ms +- 9 ms: 1.02x faster (-2%)
- meteor_contest: 115 ms +- 2 ms -> 113 ms +- 3 ms: 1.02x faster (-2%)
- telco: 6.58 ms +- 0.22 ms -> 6.48 ms +- 0.15 ms: 1.02x faster (-2%)
- regex_effbot: 3.76 ms +- 0.08 ms -> 3.70 ms +- 0.09 ms: 1.02x faster (-2%)
- chaos: 129 ms +- 3 ms -> 127 ms +- 2 ms: 1.01x faster (-1%)
- regex_v8: 26.3 ms +- 0.4 ms -> 25.9 ms +- 0.6 ms: 1.01x faster (-1%)
- regex_compile: 200 ms +- 5 ms -> 197 ms +- 5 ms: 1.01x faster (-1%)
- logging_simple: 9.78 us +- 0.19 us -> 9.65 us +- 0.26 us: 1.01x faster (-1%)
- sqlite_synth: 3.24 us +- 0.10 us -> 3.20 us +- 0.06 us: 1.01x faster (-1%)
- unpickle: 16.4 us +- 0.5 us -> 16.2 us +- 0.5 us: 1.01x faster (-1%)
- logging_format: 10.8 us +- 0.2 us -> 10.7 us +- 0.3 us: 1.01x faster (-1%)
- json_loads: 30.2 us +- 0.8 us -> 29.9 us +- 0.6 us: 1.01x faster (-1%)

Benchmark hidden because not significant (14): json_dumps, nbody, pickle, 
pidigits, regex_dna, richards, scimark_lu, scimark_monte_carlo, spectral_norm, 
unpack_sequence, unpickle_list, unpickle_pure_python, xml_etree_iterparse, 
xml_etree_process

This is related to my earlier work in bpo-36229, however this is a much less 
invasive change that's limited to ceval.c, where our knowledge of context is 
much better.

--
components: Interpreter Core
messages: 354399
nosy: brandtbucher
priority: normal
severity: normal
status: open
title: Improved performance for list addition.
type: performance
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38417] Add support for settting umask in subprocess children

[Gregory P. Smith]

Gregory P. Smith  added the comment:

preexec_fn has been mentally and advisability deprecated for years. :) 
I'll mark it officially with pending deprecation in 3.9 destined to be removed 
in 3.11.  tracking that in its own rollup issue 
https://bugs.python.org/issue38435

As far as posix_spawn goes, I expect these kinds of between fork+exec features 
to be something that prevents posix_spawn from being usable.  As are many other 
things.  People who want to use posix_spawn will need to know that and seek to 
avoid those.  That's a documentation issue first and foremost.  Our existing 
POpen API is very old and wasn't designed to make that nice.

A new API could be made that *only* supports posix_spawn available features if 
you want an entrypoint that encourages the generally lower latency posix_spawn 
path.  (A subprocess.spawn function similar to subprocess.run perhaps?)  That 
should be taken up within its own enhancement issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38435] Start the deprecation cycle for subprocess preexec_fn

[Gregory P. Smith]

New submission from Gregory P. Smith :

subprocess's preexec_fn feature needs to enter PendingDeprecationWarning state 
in 3.9, to become a DeprecationWarning in 3.10, with a goal of removing it in 
3.11.

Rationale: We now live in a world full of threads, it is entirely unsafe to 
call back into the python interpreter within the forked child before exec per 
POSIX specification.

We've also already made preexec_fn no longer supported from CPython 
subinterpreters in 3.8.

If there are not already issues open for desired features of subprocess that do 
not yet have replacements or workarounds for *specific* actions that preexec_fn 
is being used for in your application, please open feature requests for those.  
(ex: calling umask is https://bugs.python.org/issue38417, and group, uid, gid 
setting has already landed in 3.9)

--
assignee: gregory.p.smith
messages: 354397
nosy: gregory.p.smith
priority: normal
severity: normal
stage: needs patch
status: open
title: Start the deprecation cycle for subprocess preexec_fn
type: enhancement
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17894] Edits to descriptor howto

[Ned Batchelder]

Change by Ned Batchelder :


--
resolution: rejected -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38014] Python 3.7 does not compile

[Chih-Hsuan Yen]

Change by Chih-Hsuan Yen :


--
nosy: +yan12125

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38434] sys.addaudithook event is not documented

[Steve Dower]

New submission from Steve Dower :

The sys.addaudithook raises an audit event 'sys.addaudithook'.

The docs for the function refer to the event, as it has slightly different 
semantics than most, but it is not named explicitly and does not appear in the 
table of events.

--
assignee: docs@python
components: Documentation
messages: 354396
nosy: docs@python, steve.dower
priority: normal
severity: normal
stage: needs patch
status: open
title: sys.addaudithook event is not documented
type: enhancement
versions: Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38432] ZeroDivisionError when inf is expected

[STINNER Victor]

STINNER Victor  added the comment:

That's not a bug: 64-bit float of IEEE 754 rounds 1e-324 to 0.0.

vstinner@apu$ python3
Python 3.7.4 (default, Jul  9 2019, 16:32:37) 
[GCC 9.1.1 20190503 (Red Hat 9.1.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 1e-324
0.0
>>> import sys
>>> sys.float_info
sys.float_info(max=1.7976931348623157e+308, max_exp=1024, max_10_exp=308, 
min=2.2250738585072014e-308, min_exp=-1021, min_10_exp=-307, dig=15, 
mant_dig=53, epsilon=2.220446049250313e-16, radix=2, rounds=1)
>>> sys.float_info.min
2.2250738585072014e-308
>>> sys.float_info.min.hex()
'0x1.0p-1022'

>>> 1/0.0
Traceback (most recent call last):
  File "", line 1, in 
ZeroDivisionError: float division by zero
>>> 1/1e-324
Traceback (most recent call last):
  File "", line 1, in 
ZeroDivisionError: float division by zero

--
nosy: +vstinner
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38433] 2.7.17rc1 tcl/tk version regression on Windows

[Terry J. Reedy]

New submission from Terry J. Reedy :

In my current 2.7.16+ Windows 10-64 repository build, and on at least one 
8.1-64 buildbot, the tcl/tk version is 8.5.19
>>> tk.Tk().tk.call('info', 'patchlevel')
'8.5.19'

In my installed 2.7.17rc1, it is the older 8.5.15.  It would seem to me that we 
should install the same latest 8.5 bugfix release that we test with.  (Or test 
with what we release ;-)  I presume the discrepancy is unintended due to an 
incomplete final 2.7 tcl/tk update.

--
components: Build, Tkinter
messages: 354394
nosy: benjamin.peterson, serhiy.storchaka, terry.reedy, zach.ware
priority: release blocker
severity: normal
status: open
title: 2.7.17rc1 tcl/tk version regression on Windows
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38428] Can't gracefully ctrl+C multiprocessing pool on Python3 & Windows

[Steve Dower]

Steve Dower  added the comment:

I think you're right about that change touching the right code, but I'm pretty 
sure it wasn't working before either. (Python 2.7 had totally different code, 
IIRC.)

But the good news is that we should be able to use code similar to what's in 
Modules/timemodule.c#L1878 (the pysleep() function) to also wait on the event 
we use to emulate signals on Windows. But perhaps there's a reason Antoine 
(nosied) didn't do that originally?

--
nosy: +davin, pitrou
stage:  -> test needed
type:  -> behavior
versions: +Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38431] dataclasses.InitVar breaks with typing.Optional

[Samuel Colvin]

Change by Samuel Colvin :


--
pull_requests: +16288
pull_request: https://github.com/python/cpython/pull/16702

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38431] dataclasses.InitVar breaks with typing.Optional

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Oh, Samuel already made a PR. Then I withdraw PR 16701.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38431] dataclasses.InitVar breaks with typing.Optional

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +16287
pull_request: https://github.com/python/cpython/pull/16701

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38432] ZeroDivisionError when inf is expected

[Behzad Seyfi]

New submission from Behzad Seyfi :

>>> 1/1e-323
inf

>>> 1/1e-324
Traceback (most recent call last):
  File "", line 1, in 
ZeroDivisionError: float division by zero

in a 1/x fraction, up to x = 1e-324 it is inf but when x = 1e-325 or little it 
throws ZeroDivisionError. It is not acceptable and accountable behaviour for 
inf definition.

--
components: Interpreter Core
messages: 354391
nosy: Behzad Seyfi
priority: normal
severity: normal
status: open
title: ZeroDivisionError when inf is expected
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38431] dataclasses.InitVar breaks with typing.Optional

[Samuel Colvin]

Change by Samuel Colvin :


--
keywords: +patch
pull_requests: +16286
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/16700

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38431] dataclasses.InitVar breaks with typing.Optional

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

git blame points to 
https://github.com/python/cpython/commit/01ee12ba35a333e8a6a25c4153c4a21838e9585c
 . I am tagging this as a 3.8 regression since the commit was not backported 
and 3.7 works fine. Feel free to remove it if it's a mistake.

--
keywords: +3.8regression
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38431] dataclasses.InitVar breaks with typing.Optional

[Samuel Colvin]

Samuel Colvin  added the comment:

This is a bug with the `__repr__` method on `InitVar`.

I'm working on a PR now.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38431] dataclasses.InitVar breaks with typing.Optional

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38431] dataclasses.InitVar breaks with typing.Optional

[Samuel Colvin]

New submission from Samuel Colvin :

The following code works fine with python 3.7 but breaks with 3.8:

```
import dataclasses
from typing import Optional

@dataclasses.dataclass
class TestingDataclass:
base_path: dataclasses.InitVar[Optional[str]] = None
```

Exception traceback:

```
Traceback (most recent call last):
  File "test.py", line 6, in 
class TestingDataclass:
  File "/usr/local/lib/python3.8/dataclasses.py", line 995, in dataclass
return wrap(cls)
  File "/usr/local/lib/python3.8/dataclasses.py", line 987, in wrap
return _process_class(cls, init, repr, eq, order, unsafe_hash, frozen)
  File "/usr/local/lib/python3.8/dataclasses.py", line 967, in _process_class
str(inspect.signature(cls)).replace(' -> None', ''))
  File "/usr/local/lib/python3.8/inspect.py", line 3050, in __str__
formatted = str(param)
  File "/usr/local/lib/python3.8/inspect.py", line 2568, in __str__
formatannotation(self._annotation))
  File "/usr/local/lib/python3.8/inspect.py", line 1202, in formatannotation
return repr(annotation)
  File "/usr/local/lib/python3.8/dataclasses.py", line 213, in __repr__
return f'dataclasses.InitVar[{self.type.__name__}]'
  File "/usr/local/lib/python3.8/typing.py", line 757, in __getattr__
raise AttributeError(attr)
AttributeError: __name__
```

The code runs fine with `str` instead of `Optional[str]`.

Tested locally with `Python 3.8.0rc1 (tags/v3.8.0rc1:34214de6ab, Oct 10 2019, 
16:15:14)`. The same error can be seen (in a more involved context) on travis 
[here](https://travis-ci.org/samuelcolvin/pydantic/jobs/596131963)

--
components: Library (Lib)
messages: 354388
nosy: samuelcolvin
priority: normal
severity: normal
status: open
title: dataclasses.InitVar breaks with typing.Optional
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37931] crash reimporting posix after Py_Finalize on mac

[Benoit Hudson]

Benoit Hudson  added the comment:

Ping! The patch has been ready for review for over a month.

If the patch can be backported to 2.7.x before that gets end-of-lifed it would 
be ideal.

--
versions: +Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38230] A Path Traversal vulnerability in test/ssl_servers.py

[STINNER Victor]

STINNER Victor  added the comment:

Lib/test/ssl_servers.py is an internal test helper and not designed to be used 
by any end-user. Don't use it ;-)

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36553] inspect.is_decorator_call(frame)

[Sylvain Marie]

Sylvain Marie  added the comment:

Quick update on this feature: for the following example to work:

from inspect import is_decorator_call

def set_hello_tag(tag='world'):
if is_decorator_call():
# called without parenthesis!
# the decorated object is `tag`
return set_hello_tag()(tag)   # note that `is_decorator_call` should 
not return True for this call
else:
def decorate(f):
setattr(f, 'hello', tag)  # set a hello tag on the decorated f
return f
return decorate


Then `is_decorator_call` should be callable without arguments (default to 
current frame) and should return `True` only if the current frame is the one 
directly following decorator application. In nested frames (such as the one 
obtained after first recursive call to `set_hello_tag` above, 
`is_decorator_call` should return `False`.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38230] A Path Traversal vulnerability in test/ssl_servers.py

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

I would propose closing since it's meant for tests and not for end users. I am 
also not sure if one of the tests depend on this behavior. Adding Victor as 
friendly ping since it's tagged as security.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37972] unittest.mock.call does not chain __getitem__ to another _Call object

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Closing as fixed since PRs were merged. Thanks Ben.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38282] socketmodule.c: _FORTIFY_SOURCE=2 warning in AF_ALG case of getsockaddrarg()

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +16285
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/16698

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38339] [3.5] The doc job of Travis CI fails on Python 3.5: needs at least Sphinx v1.8

[STINNER Victor]

STINNER Victor  added the comment:

Larry merged my fix:

commit edd9bc9cc99c919bdc87e5955ec8ad93c295fbe2
Author: Victor Stinner 
Date:   Tue Oct 8 19:12:21 2019 +0200

[3.7] Doc: Bump sphinx. (GH-10676) (GH-10803) (#16522)

(cherry picked from commit 2db96ae7444880d66d4ef65abab8a5e6ff328711)

Co-authored-by: Julien Palard 
(cherry picked from commit 23a98abd4256f931ed89b65ec6babd4f06dbff97)


The doc of job Travis CI is back to green ;-)

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38367] test_regrtest hanged on AMD64 Windows10 3.x

[STINNER Victor]

STINNER Victor  added the comment:

I hope that this bug has been fixed by my latest fix for bpo-37531.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38109] Missing constants in Lib/stat.py

[Ronan Lamy]

Ronan Lamy  added the comment:

Thanks Victor!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38353] Cleanup the path configuration implementation code (getpath.c)

[STINNER Victor]

STINNER Victor  added the comment:

I created this issue to attempt to modify _PyPathConfig to use PyWideStringList 
for module_search_paths, to support ":" character in a path. It would fix 
bpo-31210.

See also bpo-12989.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15498] Eliminate the use of deprecated OS X APIs in getpath.c

[STINNER Victor]

STINNER Victor  added the comment:

The deprecation warnings are not fixed yet. They were quickly mentioned at:
https://bugs.python.org/issue38429#msg354367

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38429] Failed to compile with --enable-framework on macOS on master

[Ronald Oussoren]

Ronald Oussoren  added the comment:

W.r.t. the deprecated APIs in getpath.c: Issue15498

That issue contains an ancient patch to switch to non-deprecated APIs, but we 
haven't had time yet to fully test these. The issue also mentions that Apple's 
install of Python already switched away from the deprecated APIs, but had some 
issues at the time.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38109] Missing constants in Lib/stat.py

[STINNER Victor]

STINNER Victor  added the comment:

I understand that backporting the change up to 3.7 makes your life easier, so I 
merged it. The change is safe and should not affect the regular use case (when 
the C extension _stat is used).

Thanks Ronan for this nice enhancement. It's good to see the stat module 
respect the PEP 399 ;-)

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38109] Missing constants in Lib/stat.py

[STINNER Victor]

STINNER Victor  added the comment:


New changeset f5ed41c1ae9a575e965d55c6a5e86fb59181eee8 by Victor Stinner (Miss 
Islington (bot)) in branch '3.7':
bpo-38109: Add missing constants to Lib/stat.py (GH-16665) (GH-16691)
https://github.com/python/cpython/commit/f5ed41c1ae9a575e965d55c6a5e86fb59181eee8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38109] Missing constants in Lib/stat.py

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 8ab11c433a81b6c7ef84a66e36fbe199915e2ff6 by Victor Stinner (Miss 
Islington (bot)) in branch '3.8':
bpo-38109: Add missing constants to Lib/stat.py (GH-16665) (GH-16690)
https://github.com/python/cpython/commit/8ab11c433a81b6c7ef84a66e36fbe199915e2ff6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14307] Make subclassing SocketServer simpler for non-blocking frameworks

[STINNER Victor]

STINNER Victor  added the comment:

> Hard to understand the reluctancy to _allow_ for a different way to handle 
> accept timeouts.  But this is also why I stopped contributing to core, 
> because it turned out to be more about lobbying than anything else.

To get a change merged into Python, you need to find someone who wants to be 
responsible for it: not only merge the change, but also maintain the code 
later. I touched socketserver recently for my work on making the Python CI more 
reliable. It was not easy to get a review. I ended to merge the changes with no 
review (if I recall properly).

I dislike socketserver design: it tries to be too generic. Like supporting TCP 
and UDP, support threading and forking models, etc.

To be honest, I didn't look at your issue. I only close it for bug triage, 
since there was no activity since 2016 and 2 core devs were against the change.

And you want to revisit the change, please reopen the issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38109] Missing constants in Lib/stat.py

[Ronan Lamy]

Ronan Lamy  added the comment:

Well, my interest in this is to reduce the divergence between PyPy and CPython, 
and, potentially, to help other implementations. So I actually care more about 
3.7 than about 3.9, which I probably won't look at before 2021.

That said, I don't *need* anything: the fix is already in PyPy and we'll carry 
it forward regardless of what CPython does.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38429] Failed to compile with --enable-framework on macOS on master

[STINNER Victor]

STINNER Victor  added the comment:

Thanks for the bug report Haruka Ma, it should now be fixed.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38353] Cleanup the path configuration implementation code (getpath.c)

[STINNER Victor]

STINNER Victor  added the comment:


New changeset b6e0fc74265db96fdd0e0b8151c768c583ad80f0 by Victor Stinner in 
branch 'master':
bpo-38353: Fix typos in calculate_argv0_path_framework() (GH-16695)
https://github.com/python/cpython/commit/b6e0fc74265db96fdd0e0b8151c768c583ad80f0


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38429] Failed to compile with --enable-framework on macOS on master

[STINNER Victor]

STINNER Victor  added the comment:


New changeset b6e0fc74265db96fdd0e0b8151c768c583ad80f0 by Victor Stinner in 
branch 'master':
bpo-38353: Fix typos in calculate_argv0_path_framework() (GH-16695)
https://github.com/python/cpython/commit/b6e0fc74265db96fdd0e0b8151c768c583ad80f0


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14307] Make subclassing SocketServer simpler for non-blocking frameworks

[Kristján Valur Jónsson]

Kristján Valur Jónsson  added the comment:

Nice necro :)
Socketserver is already subclassable and overridable for so many things.  Hard 
to understand the reluctancy to _allow_ for a different way to handle accept 
timeouts.  But this is also why I stopped contributing to core, because it 
turned out to be more about lobbying than anything else.  Anyway this is 
already implemented in PythonClassic, so no need for me to push it upstream :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38429] Failed to compile with --enable-framework on macOS on master

[STINNER Victor]

STINNER Victor  added the comment:

Hum, getpath.c seems to use deprecated APIs:

./Modules/getpath.c:1098:20: warning: 'NSModuleForSymbol' is deprecated: first 
deprecated in macOS 10.5 - dladdr() [-Wdeprecated-declarations]
pythonModule = NSModuleForSymbol(NSLookupAndBindSymbol("_Py_Initialize"));
   ^
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/mach-o/dyld.h:193:21:
 note: 'NSModuleForSymbol' has been explicitly marked deprecated here
extern NSModule NSModuleForSymbol(NSSymbol symbol) __API_UNAVAILABLE(ios, 
tvos, watchos)  __OSX_DEPRECATED(10.1, 10.5, "dladdr()");
^
./Modules/getpath.c:1098:38: warning: 'NSLookupAndBindSymbol' is deprecated: 
first deprecated in macOS 10.4 - dlsym() [-Wdeprecated-declarations]
pythonModule = NSModuleForSymbol(NSLookupAndBindSymbol("_Py_Initialize"));
 ^
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/mach-o/dyld.h:183:17:
 note: 'NSLookupAndBindSymbol' has been explicitly marked deprecated here
extern NSSymbol NSLookupAndBindSymbol(const char* symbolName)   
 __API_UNAVAILABLE(ios, tvos, watchos)  
__OSX_DEPRECATED(10...
^
./Modules/getpath.c:1101:27: warning: 'NSLibraryNameForModule' is deprecated: 
first deprecated in macOS 10.5 [-Wdeprecated-declarations]
const char* modPath = NSLibraryNameForModule(pythonModule);
  ^
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/mach-o/dyld.h:163:21:
 note: 'NSLibraryNameForModule' has been explicitly marked deprecated here
extern const char*  NSLibraryNameForModule(NSModule m) __API_UNAVAILABLE(ios, 
tvos, watchos)  __OSX_DEPRECATED(10.1, 10.5, "");
^
3 warnings generated.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17894] Edits to descriptor howto

[Ned Batchelder]

Ned Batchelder  added the comment:

I think it's a bad precedent to have pages in the official docs that belong to 
a single author.  If you want to maintain that kind of control, it should go on 
your blog.  Part of the agreement for having material added to the official 
docs should be the understanding that they are fair game for community 
contributions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32996] Improve What's New in 3.7

[miss-islington]

miss-islington  added the comment:


New changeset baf8a9b03a56ce1ee2284763b267ff56ab4507fe by Miss Islington (bot) 
in branch '3.8':
bpo-32996: Documentation fix-up. (GH-16646)
https://github.com/python/cpython/commit/baf8a9b03a56ce1ee2284763b267ff56ab4507fe


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32996] Improve What's New in 3.7

[miss-islington]

miss-islington  added the comment:


New changeset ca012360b103f2ab1acfa506aed032b0ed9b105c by Miss Islington (bot) 
in branch '3.7':
bpo-32996: Documentation fix-up. (GH-16646)
https://github.com/python/cpython/commit/ca012360b103f2ab1acfa506aed032b0ed9b105c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32996] Improve What's New in 3.7

[miss-islington]

Change by miss-islington :


--
pull_requests: +16284
pull_request: https://github.com/python/cpython/pull/16697

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32996] Improve What's New in 3.7

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:


New changeset d47f0dd2e85ce032aebfedbde18cdb2e728fa79f by Ivan Levkivskyi (M. 
Eric Irrgang) in branch 'master':
bpo-32996: Documentation fix-up. (GH-16646)
https://github.com/python/cpython/commit/d47f0dd2e85ce032aebfedbde18cdb2e728fa79f


--
nosy: +levkivskyi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32996] Improve What's New in 3.7

[miss-islington]

Change by miss-islington :


--
pull_requests: +16283
pull_request: https://github.com/python/cpython/pull/16696

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32561] Add API to io objects for non-blocking reads/writes

[Nathaniel Smith]

Nathaniel Smith  added the comment:

The proposal is to be able to run io module operations in two modes: the 
regular one, and one where performing actual I/O is forbidden – so if they go 
down the stack and can fulfill the operation from some in-memory buffer, great, 
they do that, and if not, they raise an error.

It turns out that this is actually the right primitive to enable async disk 
access. That's the only target use case, and there's no IO loop involved. If 
you wanted to keep async disk access separate from the io module, then what 
we'd have to do is to create a fork of all the code in the io module, and add 
this feature to it. Which doesn't seem like a good design :-).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38429] Failed to compile with --enable-framework on macOS on master

[Haruka Ma]

Haruka Ma  added the comment:

I'm now getting double free on the same function:

https://pastebin.com/zHWCtTuV (with debug on)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38430] Memory leak in ThreadPoolExecutor + run_in_executor

[Evgeny Nizhibitsky]

Evgeny Nizhibitsky  added the comment:

Oh, I see that in the initial code with leakage (it was heavy 
ThreadPoolExecutor + xgboost thing) there was an await but I must have lost it 
somewhere while reducing it to the minimal example and finished in the wrong 
direction.

Glad too see that it raised a discussion to prevent others from getting into 
this silent trap.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38429] Failed to compile with --enable-framework on macOS on master

[STINNER Victor]

STINNER Victor  added the comment:

Oh, I expected that we had macOS buildbots building Python with 
--enable-framework. Thanks for the reporting the issue.

I wrote PR 16695 which should fix the issue, but I cannot test my fix right now.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38429] Failed to compile with --enable-framework on macOS on master

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +16282
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/16695

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38353] Cleanup the path configuration implementation code (getpath.c)

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +16281
pull_request: https://github.com/python/cpython/pull/16695

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38430] Memory leak in ThreadPoolExecutor + run_in_executor

[STINNER Victor]

STINNER Victor  added the comment:

> If we convert run_in_executor() into async function we'll get a warning about 
> unawaited coroutine even without asyncio debug mode.

That sounds like a good solution :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38430] Memory leak in ThreadPoolExecutor + run_in_executor

[Andrew Svetlov]

Andrew Svetlov  added the comment:

The change is slightly not backward compatible but
1. It's pretty visible. In the worst-case instead of the memory leak people see 
a RuntimeWarning
2. We did it already for a part of API, e.g. loop.sock_read() and family

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38430] Memory leak in ThreadPoolExecutor + run_in_executor

[Andrew Svetlov]

Andrew Svetlov  added the comment:

> Can a Task be used instead of a Future in run_in_executor()?

I don't think that the task is required here. The problem is that 
run_in_executor is a function that returns asyncio future; that is in turn a 
wrapper around concurrent future object.

If we convert run_in_executor() into async function we'll get a warning about 
unawaited coroutine even without asyncio debug mode.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38430] Memory leak in ThreadPoolExecutor + run_in_executor

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Victor answered the first :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38430] Memory leak in ThreadPoolExecutor + run_in_executor

[STINNER Victor]

STINNER Victor  added the comment:

> Yuri, what should we do with the issue?

A Task emits a warning when it's not awaited. Can a Task be used instead of a 
Future in run_in_executor()?

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38430] Memory leak in ThreadPoolExecutor + run_in_executor

[Andrew Svetlov]

Andrew Svetlov  added the comment:

You MUST await a future returned from `loop.run_in_executor()` to avoid the 
leak.

Yuri, what should we do with the issue? I see the second similar report in the 
last half of the year.
Sure, we can add weakrefs somewhere in futures._chain_future() but I pretty 
sure that the proper fix is just enforcing `await` here, e.g.

1. rename existing run_in_executor() into a private _run_in_executor() function.

2. write async trampoline:
async def run_in_executor(self, executor, func, *args):
return await self._run_in_executor(executor, func, *args)

--
nosy:  -vstinner
resolution: not a bug -> 
stage: resolved -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38430] Memory leak in ThreadPoolExecutor + run_in_executor

[STINNER Victor]

STINNER Victor  added the comment:

Well, that's a common issue when using asyncio: you forgot await.

async def main(_loop):
while True:
with futures.ThreadPoolExecutor() as pool:
_loop.run_in_executor(pool, nop)
sys.stdout.write(f'\r{get_mem():0.3f}MB')

It should be: "await _loop.run_in_executor(pool, nop)" ;-)

Sadly, PYTHONASYNCIODEBUG=1 env var doesn't complain on this bug.

See: https://docs.python.org/dev/library/asyncio-dev.html#debug-mode

--
nosy: +vstinner
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38430] Memory leak in ThreadPoolExecutor + run_in_executor

[Evgeny Nizhibitsky]

New submission from Evgeny Nizhibitsky :

I have run into a memory leak caused by using run_in_executor + 
ThreadPoolExecutor while running some stability tests with custom web services.

It was 1 MB leaked for 1k requests made for my case and I've extracted the root 
cause and converted it into minimal script with both mentioned parts + just NOP 
function to "run".

The script can easily eat up to 1 GB of memory in less then 1 minute now. It 
uses external psutil library to report the memory allocated but it can be 
easily commented out and the leak will stay anyway.

One can found that script attached + Dockerfile/Makefile for reproducibility. 
I've also reproduced it in my own conda-based 3.7 environment as well as the 
master branch of cpython.

--
components: asyncio
files: python-leak.zip
messages: 354351
nosy: Evgeny Nizhibitsky, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: Memory leak in ThreadPoolExecutor + run_in_executor
type: resource usage
versions: Python 3.6, Python 3.7, Python 3.9
Added file: https://bugs.python.org/file48654/python-leak.zip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38429] Failed to compile with --enable-framework on macOS on master

[Ned Deily]

Change by Ned Deily :


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38416] test_largefile.TestSocketSendfile.test_it() failed with 15 min timeout on AMD64 FreeBSD 10-STABLE Non-Debug 3.x

[STINNER Victor]

Change by STINNER Victor :


--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38416] test_largefile.TestSocketSendfile.test_it() failed with 15 min timeout on AMD64 FreeBSD 10-STABLE Non-Debug 3.x

[Giampaolo Rodola']

Giampaolo Rodola'  added the comment:

Thanks.

--
resolution: fixed -> 
stage: resolved -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com