[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

[Yury Selivanov]

Yury Selivanov  added the comment:

> Imagine that we have a secondary copy of the bytecode in the cache inside the 
> code object and we mutate that instead. The key difference with the current 
> cache infrastructure is that we don't accumulate all the optimizations on the 
> same opcode, which can be very verbose. Instead, we change the generic opcode 
> to a more specialised to optimize and we change it back to deoptimize.

Yeah, I follow. As long as we keep the original list of opcodes we're good ;)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> - Rewriting code objects in place is wrong, IMO: you always need to have a 
> way to deoptimize the entire thing, so you need to keep the original one. It 
> might be that you have well defined and static types for the first 1 
> invocations and something entirely different on 10001. So IMO we need a 
> SpecializedCode object with the necessary bailout guards.

Imagine that we have a secondary copy of the bytecode in the cache inside the 
code object and we mutate that instead. The key difference with the current 
cache infrastructure is that we don't accumulate all the optimizations on the 
same opcode, which can be very verbose. Instead, we change the generic opcode 
to a more specialised to optimize and we change it back to deoptimize. The 
advantage is that BINARY_SUBSCRIPT for example won't be this gigantic block of 
text that will do different things depending if is specialising for dicts or 
lists or tuples, but we will have a different opcode for every of them, which I 
think is much easier to manage.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

[Yury Selivanov]

Yury Selivanov  added the comment:

Few thoughts in no particular order:

- I'd suggest implementing the cache for 2-3 more opcodes on top of the 
existing infrastructure to get more experience and then refactoring it to make 
it more generic.

- Generalizing LOAD_METHOD to work for methods with **kwargs, caching concrete 
operator implementations for opcodes like BINARY_ADD etc. are all possible on 
top of the current infra.

- Rewriting code objects in place is wrong, IMO: you always need to have a way 
to deoptimize the entire thing, so you need to keep the original one. It might 
be that you have well defined and static types for the first 1 invocations 
and something entirely different on 10001. So IMO we need a SpecializedCode 
object with the necessary bailout guards. But that's not a simple thing to 
implement, so unless someone will be working on this fulltime for a long time 
I'd suggest working off what we have now. (That said I'd take a close look at 
what Dino is building).

- There are multiple different approaches we can choose for optimizing CPython, 
ranging from hidden classes to a full blown JIT. I hope someone will do them 
one day. But IMO the current simple "opcode cache" (I wish we had a better 
name) mechanism we have would allow us to squeeze up to 15-25% median 
improvement in our benchmarks with relatively limited dev time. Maybe that's 
good enough for 3.10.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42097] Python 3.7.9 logging/threading/fork hang

[Gregory P. Smith]

Gregory P. Smith  added the comment:

If you use os.fork() or any of the multiprocessing start methods that call 
os.fork() with a process involving threads, this is expected behavior.  
os.fork() cannot be used in processes that have threads without potential for 
deadlock.

Specifically, make sure you explicitly call:

 multiprocessing.set_start_method('spawn')

before using multiprocessing in any application where threads exist.

It is possible to use 'forkserver' as well, but only if you ensure the 
multiprocessing forkserver process is started before your application has 
launched any threads.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Also, many of these ideas are not new, and many of them are inspired or taken 
from Yury's email 
(https://mail.python.org/pipermail/python-dev/2016-January/142945.html) but I 
wanted to add that I think that with some coordination between us we can 
achieve some excellent speedups for Python 3.10!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

To clarify what I mean with:

> - We could also do the same for operations like "some_container[]" if the 
> container is some builtin. We can substitute/specialize the opcode for 
> someone that directly uses built-in operations instead of the generic 
> BINARY_SUBSCR.

If a given function has a BINARY_SUBSCR opcode and when executing it a given 
number of times we see that the object to get the subscript is always a list, 
we change the BINARY_SUBSCR to BINARY_SUBSCR_FOR_LISTS and it that opcode we do 
a quick check for PyList_CheckExact and if is correct we call 
"Objects/listobject.c:list_subscript" directly and if is false we put back a 
BINARY_SUBSCR.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

[Pablo Galindo Salgado]

New submission from Pablo Galindo Salgado :

After https://bugs.python.org/issue42093 and https://bugs.python.org/issue26219 
is being clear that we can leverage some cache for different information in the 
evaluation loop to speed up CPython. This observation is also based on the fact 
that although Python is dynamic, there is plenty of code that does not exercise 
said dynamism and therefore factoring out the "dynamic" parts of the execution 
by using a cache mechanism can yield excellent results. 

So far we have two big improvements in performance for caching LOAD_ATTR and 
LOAD_GLOBAL (in some cases up to 10-14%) but I think we can do much much more. 
Here are some observations of what I think we can do:

* Instead of adding more caches using the current mechanism, which adds some 
inlined code in every opcode in the evaluation loop, we can try to formalize 
some kind of caching mechanism that has some better API that will allow adding 
more opcodes in the future. Having the code inline in ceval.c is going to 
become difficult to maintain if we keep adding more stuff directly there.

* Instead of handling the specialization in the same opcode as the original one 
(LOAD_ATTR is doing the slow and the fast path) we could mutate the original 
code object and replacing the slow and generic opcodes for the more specialized 
ones and these will also be in charge of changing it back to the generic and 
slow ones if the assumptions that activated them appear.

Obviously, mutating code objects is scary, so we could have some "specialized" 
version of the bytecode in the cache and use that if is present. Ideas that we 
could do with this cached stuff:

- For binary operators, we can grab both operands, resolve the addition 
function and cache that together with the types and the version tags and if the 
types and the version tags are the same, use directly the addition function 
instead of resolving it.

- For loading methods, we could cache the bound method as proposed by Yury 
originally here: 
https://mail.python.org/pipermail/python-dev/2016-January/142945.html.

- We could also do the same for operations like "some_container[]" if the 
container is some builtin. We can substitute/specialize the opcode for someone 
that directly uses built-in operations instead of the generic BINARY_SUBSCR.

The plan will be:

- Making some infrastructure/framework for the caching that allows us to 
optimize/deoptimize individual opcodes.
- Refactor the existing specialization for LOAD_GLOBAL/LOAD_ATTR to use said 
infrastructure.
- Thinking of what operations could benefit from specialization and start 
adding them one by one.

--
components: C API
messages: 379272
nosy: Mark.Shannon, methane, nascheme, pablogsal, vstinner, yselivanov
priority: normal
severity: normal
status: open
title: Caching infrastructure for the evaluation loop: specialised opcodes
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42057] peephole optimizer bug relating to JUMP_IF_NOT_EXC_MATCH

[Inada Naoki]

Inada Naoki  added the comment:

Thank you for reporting with reproducer.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32431] Two bytes objects of zero length don't compare equal

[Inada Naoki]

Change by Inada Naoki :


--
resolution:  -> not a bug
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42057] peephole optimizer bug relating to JUMP_IF_NOT_EXC_MATCH

[Inada Naoki]

Inada Naoki  added the comment:


New changeset 8f6787d93db1b6022db44b1e1d22460c2b74f60b by Inada Naoki in branch 
'3.9':
bpo-42057: Add a test case (GH-22878)
https://github.com/python/cpython/commit/8f6787d93db1b6022db44b1e1d22460c2b74f60b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42057] peephole optimizer bug relating to JUMP_IF_NOT_EXC_MATCH

[Inada Naoki]

Change by Inada Naoki :


--
pull_requests: +21820
pull_request: https://github.com/python/cpython/pull/22878

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41910] Document that object.__eq__ implements `a is b`

[Brett Cannon]

Brett Cannon  added the comment:

Thanks, Terry!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42057] peephole optimizer bug relating to JUMP_IF_NOT_EXC_MATCH

[Inada Naoki]

Inada Naoki  added the comment:


New changeset 07a44d9572c7746568a7fe2fbcd42127fd6d4019 by Inada Naoki in branch 
'3.9':
bpo-42057: Fix peephole optimizer (GH-22802)
https://github.com/python/cpython/commit/07a44d9572c7746568a7fe2fbcd42127fd6d4019


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41910] Document that object.__eq__ implements `a is b`

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Brett, I presume you want this closed.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42113] Replace _asyncio.TaskWakeupMethWrapper with PyCFunction

[Yury Selivanov]

Change by Yury Selivanov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41910] Document that object.__eq__ implements `a is b`

[Terry J. Reedy]

Terry J. Reedy  added the comment:


New changeset b2b3803081f07600710273b4f902b5be6e5596e7 by Miss Skeleton (bot) 
in branch '3.8':
bpo-41910: specify the default implementations of object.__eq__ and 
object.__ne__ (GH-22874) (#22877)
https://github.com/python/cpython/commit/b2b3803081f07600710273b4f902b5be6e5596e7


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41910] Document that object.__eq__ implements `a is b`

[Terry J. Reedy]

Terry J. Reedy  added the comment:


New changeset c3538b83816663d7b767391a375179a0ce923990 by Miss Skeleton (bot) 
in branch '3.9':
bpo-41910: specify the default implementations of object.__eq__ and 
object.__ne__ (GH-22874) (#22876)
https://github.com/python/cpython/commit/c3538b83816663d7b767391a375179a0ce923990


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41910] Document that object.__eq__ implements `a is b`

[miss-islington]

miss-islington  added the comment:


New changeset 3c69f0c933d4790855929f1fcd74e4a0fefb5d52 by Brett Cannon in 
branch 'master':
bpo-41910: specify the default implementations of object.__eq__ and 
object.__ne__ (GH-22874)
https://github.com/python/cpython/commit/3c69f0c933d4790855929f1fcd74e4a0fefb5d52


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41910] Document that object.__eq__ implements `a is b`

[miss-islington]

Change by miss-islington :


--
pull_requests: +21818
pull_request: https://github.com/python/cpython/pull/22876

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41910] Document that object.__eq__ implements `a is b`

[miss-islington]

Change by miss-islington :


--
pull_requests: +21819
pull_request: https://github.com/python/cpython/pull/22877

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42114] Documentation of ctypes.CDLL does not correspond to code

[Joseph Fox-Rabinovitz]

Joseph Fox-Rabinovitz  added the comment:

Last attempt before I give up:

ctypes.CDLL initializer defined in version 3.8 and beyond as

```
def __init__(self, name, mode=DEFAULT_MODE, handle=None,
 use_errno=False,
 use_last_error=False,
 winmode=None):
```

Documentation says `winmode=0`:

```
class ctypes.CDLL(name, mode=DEFAULT_MODE, handle=None, use_errno=False, 
use_last_error=False, winmode=0)
```

Loading of normal DLL from custom directory works when `winmode=0`, but not 
when `winmode=None`. To reproduce, any combination of adding the folder 
containing the DLL via `os.eviron['PATH'] += os.pathsep + ...`, 
`os.add_dll_directory(...)`, `sys.path.append(...)` does not change the 
behavior.

Worked prior to 3.8 because there was no `winmode` parameter, `mode` was passed 
in diretly, and `ctypes.DEFAULT_MODE == 0`.

I don't know whether it's better to update the code, the docs, or something 
else, but based on current info, would prefer updating the code.

Discovery triggered by https://stackoverflow.com/q/59330863/2988730.
Some info summarized in https://stackoverflow.com/a/64472088/2988730.

Link to docs: https://docs.python.org/3.{8,9,10}/library/ctypes.html#ctypes.CDLL
Link to GitHub code: 
https://github.com/python/cpython/blob/3.{8,9}/Lib/ctypes/__init__.py#L340

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42114] Documentation of ctypes.CDLL does not correspond to code

[Joseph Fox-Rabinovitz]

Joseph Fox-Rabinovitz  added the comment:

Company firewall mutilated the text. Here is another attempt:

ctypes.CDLL initializer defined in version 3.8 and beyond as

```
def __init__(self, name, mode=DEFAULT_MODE, handle=None,
 use_errno=False,
 use_last_error=False,
 winmode=None):
```

Documentation says `winmode=0`:

```
class ctypes.CDLL(name, mode=DEFAULT_MODE, handle=None, use_errno=False, 
use_last_error=False, winmode=0)
```

Loading of normal DLL from custom directory works when `winmode=0`, but not 
when `winmode=None`. To reproduce, any combination of adding the folder 
containing the DLL via `os.eviron['PATH'] += os.pathsep + ...`, 
`os.add_dll_directory(...)`, `sys.path.append(...)` does not change the 
behavior.

Worked prior to 3.8 because there was no `parameter, and `mode` was passed in 
direcCompany firewall mutilated the text. Here is another attempt:

ctypes.CDLL initializer defined in version 3.8 and beyond as

```
def __init__(self, name, mode=DEFAULT_MODE, handle=None,
 use_errno=False,
 use_last_error=False,
 winmode=None):
```

Documentation says `winmode=0`:

```
class ctypes.CDLL(name, mode=DEFAULT_MODE, handle=None, use_errno=False, 
use_last_error=False, winmode=0)
```

Loading of normal DLL from custom directory works when `winmode=0`, but not 
when `winmode=None`. To reproduce, any combination of adding the folder 
containing the DLL via `os.eviron['PATH'] += os.pathsep + ...`, 
`os.add_dll_directory(...)`, `sys.path.append(...)` does not change the 
behavior.

Worked prior to 3.8 because there was no `winmode` parameter, `mode` was passed 
in diretly, and `ctypes.DEFAULT_MODE == 0`.

I don't know whether it's better to update the code, the docs, or something 
else, but based on current info, would prefer updating the code.

Discovery triggered by https://stackoverflow.com/q/59330863/2988730.
Some info summarized in https://stackoverflow.com/a/64472088/2988730.

yink to docs: https://docs.python.org/3.{8,9,10}/library/ctypes.html#ctypes.CDLL
Link to GitHub code: 
https://github.com/python/cpython/blob/3.{8,9}/Lib/ctypes/__init__.py#L340t.ly, 
`ctyp. .DEFAULT_MODE == 0`.
I don't know whether it's better to update the code, the docs, or something 
else, but based on current info, would prefer updating the code.

Discovery triggered by https://stackoverflow.com/q/59330863/2988730.
Some info summarized in https://stackoverflow.com/a/64472088/2988730.

yink to docs: https://docs.python.org/3.{8,9,10}/library/ctypes.html#ctypes.CDLL
Link to GitHub code: 
https://github.com/python/cpython/blob/3.{8,9}/Lib/ctypes/__init__.py#L340

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42114] Documentation of ctypes.CDLL does not correspond to code

[Joseph Fox-Rabinovitz]

Change by Joseph Fox-Rabinovitz :


--
title: Documentation of -> Documentation of ctypes.CDLL does not correspond to 
code

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42114] Documentation of

[Joseph Fox-Rabinovitz]

New submission from Joseph Fox-Rabinovitz :

ctypes.CDLL initializer defined in version 3.8 and beyond as

```
def __init__(self, name, mode=DEFAULT_MODE, handle=None,
 use_errno=False,
 use_last_error=False,
 winmode=None):
```

Documentation says `winmode=0`:

```
class ctypes.CDLL(name, mode=DEFAULT_MODE, handle=None, use_errno=False, 
use_last_error=False, winmode=0)
```

Loading of normal DLL from custom directory works when `winmode=0`, but when 
`winmode=None` To reproduce, any combination of adding the fol += der 
containing the DLL to `os.evia ron['PATH']`, `os.add_dll_directory(...)`

Discoveryos.pathsep + ... triggered by https://stackoverflow.co, `sys.path.appen

Worked prior to 3.8 because there was no `winmode` parameter, and d(...)` does 
not change the behavior.m/q/`mode` was passed in directly, 
59330863/298873`ctypes0. .DEFAULT_MODE == 0`.

I don't know whether it's better to update the code, the docs, or something 
else, but based on current info, would prefer updating the code.Some info 
summarized in https://stackoverflow.com/a/{8,9,}64472088/2988730.

Link to do pagecscorresponding : 
https://docs.python.org/3.10/library/ctypes.html#ctypes.CDLL
730.
Link to GitHub code: 
https://github.com/python/cpython/blob/3.{8,9}/Lib/ctypes/__init__.py#L340

--
components: ctypes
messages: 379261
nosy: madphysicist
priority: normal
severity: normal
status: open
title: Documentation of
versions: Python 3.10, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35181] Doc: Namespace Packages: Inconsistent documentation of __loader__ being None

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

@maggyero - I haven't merged PR 10016, but I left some additional comments.  
Are you still interested in shepherding this PR?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41911] Language reference for expressions incorrectly specifies what type of object(s) are expected

[Brett Cannon]

Brett Cannon  added the comment:

It turns out the "expressions" page of the language reference makes multiple 
claims about types which do not hold, e.g. for multiplication, "The arguments 
must either both be numbers, or one argument must be an integer and the other 
must be a sequence", or for `&` that its arguments "must be integers".

So this is much larger than just comparison operators.

--
assignee: brett.cannon -> docs@python
nosy: +docs@python
title: Language reference incorrectly says comparison expressions return 
boolean values -> Language reference for expressions incorrectly specifies what 
type of object(s) are expected

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42113] Replace _asyncio.TaskWakeupMethWrapper with PyCFunction

[Vladimir Matveev]

Change by Vladimir Matveev :


--
keywords: +patch
pull_requests: +21817
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/22875

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42113] Replace _asyncio.TaskWakeupMethWrapper with PyCFunction

[Vladimir Matveev]

New submission from Vladimir Matveev :

`TaskWakeupMethWrapper` looks like a more limited version of `PyCFunction` so 
it can be replaced with one.
Pros: remove a bunch of code, use better calling convention
Cons: now `wakeup` object will expose slightly more properties but I'm not sure 
whether this is bad

--
components: asyncio
messages: 379258
nosy: asvetlov, v2m, yselivanov
priority: normal
severity: normal
status: open
title: Replace _asyncio.TaskWakeupMethWrapper with PyCFunction
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38980] Compile libpython with -fno-semantic-interposition

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset b451b0e9a772f009f4161f7a46476190d0d17ac1 by Pablo Galindo in 
branch 'master':
bpo-38980: Add -fno-semantic-interposition when building with optimizations 
(GH-22862)
https://github.com/python/cpython/commit/b451b0e9a772f009f4161f7a46476190d0d17ac1


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38980] Compile libpython with -fno-semantic-interposition

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35181] Doc: Namespace Packages: Inconsistent documentation of __loader__ being None

[miss-islington]

miss-islington  added the comment:


New changeset 6e842bcdf8a47fe081c9f2edc2b8875e1f3e2f18 by Miss Skeleton (bot) 
in branch '3.9':
bpo-35181: Correct importlib documentation for some module attributes (GH-15190)
https://github.com/python/cpython/commit/6e842bcdf8a47fe081c9f2edc2b8875e1f3e2f18


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42103] [security] DoS (MemError via CPU and RAM exhaustion) when processing malformed Apple Property List files in binary format

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

There are two issues here.

The simple one is building a large format string for struct.unpack(). It has 
simple solution: use f'>{n}{_BINARY_FORMAT[size]}'.

The hard issue is that read(n) allocates n bytes in memory even if there are 
not so many bytes in the file. It affects not only plistlib and should be fixed 
in the file implementation itself. There is an open issue for this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35181] Doc: Namespace Packages: Inconsistent documentation of __loader__ being None

[miss-islington]

miss-islington  added the comment:


New changeset 916ac9520108831d2099b13992a45884b112b193 by Miss Skeleton (bot) 
in branch '3.8':
bpo-35181: Correct importlib documentation for some module attributes (GH-15190)
https://github.com/python/cpython/commit/916ac9520108831d2099b13992a45884b112b193


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41910] Document that object.__eq__ implements `a is b`

[Brett Cannon]

Change by Brett Cannon :


--
keywords: +patch
pull_requests: +21816
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/22874

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35181] Doc: Namespace Packages: Inconsistent documentation of __loader__ being None

[miss-islington]

Change by miss-islington :


--
pull_requests: +21815
pull_request: https://github.com/python/cpython/pull/22873

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35181] Doc: Namespace Packages: Inconsistent documentation of __loader__ being None

[miss-islington]

Change by miss-islington :


--
pull_requests: +21814
pull_request: https://github.com/python/cpython/pull/22872

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35181] Doc: Namespace Packages: Inconsistent documentation of __loader__ being None

[miss-islington]

miss-islington  added the comment:


New changeset 27f1bd8787d24ac53cc3dc6ea5eb00b8a3499839 by Géry Ogam in branch 
'master':
bpo-35181: Correct importlib documentation for some module attributes (GH-15190)
https://github.com/python/cpython/commit/27f1bd8787d24ac53cc3dc6ea5eb00b8a3499839


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42112] ZipFIle.write remove slash at the beginning of member's path

[Matan Perelman]

Change by Matan Perelman :


--
keywords: +patch
pull_requests: +21813
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/22871

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39416] Document default numeric string formats

[miss-islington]

miss-islington  added the comment:


New changeset 224ed378b9aadad9dbbd890064677433188aecd9 by Miss Skeleton (bot) 
in branch '3.9':
Fix bpo-39416: Change "Numeric" to lower case; an english word, not a class 
name (GH-22867)
https://github.com/python/cpython/commit/224ed378b9aadad9dbbd890064677433188aecd9


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41052] Opt out serialization/deserialization for heap type

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I came to the same conclusion after trying to fix it. But we cannot just do it 
now. We have to fix bugs in Python 3.9 and support protocols 0 and 1 for some 
deprecation period.

Also protocols 0 and 1 are used in some third-party implementations for other 
programming languages, so they can be used for interoperability with other 
languages.

The proposed patch adds additional check. It uses the fact that the __new__ 
attribute for classes implemented in C is a bultin function with __self__ 
pointing to this class (it is created in tp_new_wrapper()).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41052] Opt out serialization/deserialization for heap type

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +21812
pull_request: https://github.com/python/cpython/pull/22870

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41959] Doc/library/asyncio-policy.rst grammar error

[Andrew Svetlov]

Change by Andrew Svetlov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39416] Document default numeric string formats

[miss-islington]

Change by miss-islington :


--
pull_requests: +21811
pull_request: https://github.com/python/cpython/pull/22869

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39416] Document default numeric string formats

[miss-islington]

Change by miss-islington :


--
pull_requests: +21810
pull_request: https://github.com/python/cpython/pull/22868

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39416] Document default numeric string formats

[miss-islington]

miss-islington  added the comment:


New changeset f8b1ccd63c94bcde1c15d56d24add89861b6ceee by kpinc in branch 
'master':
Fix bpo-39416: Change "Numeric" to lower case; an english word, not a class 
name (GH-22867)
https://github.com/python/cpython/commit/f8b1ccd63c94bcde1c15d56d24add89861b6ceee


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41959] Doc/library/asyncio-policy.rst grammar error

[Andrew Svetlov]

Andrew Svetlov  added the comment:


New changeset c3442fd8a7f6e34ba888efea085c3701b75c0b71 by Miss Skeleton (bot) 
in branch '3.9':
bpo-41959: Fix grammar around class asyncio.MultiLoopChildWatcher text 
(GH-22580) (#22865)
https://github.com/python/cpython/commit/c3442fd8a7f6e34ba888efea085c3701b75c0b71


--
nosy: +asvetlov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41959] Doc/library/asyncio-policy.rst grammar error

[miss-islington]

miss-islington  added the comment:


New changeset ba666747af83aa465ffa8b55efbbb7f992647e12 by Miss Skeleton (bot) 
in branch '3.8':
[3.8] bpo-41959: Fix grammar around class asyncio.MultiLoopChildWatcher text 
(GH-22580) (GH-22866)
https://github.com/python/cpython/commit/ba666747af83aa465ffa8b55efbbb7f992647e12


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41626] port shebang of tools from python2 to python3

[Ned Deily]

Change by Ned Deily :


--
Removed message: https://bugs.python.org/msg379245

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41626] port shebang of tools from python2 to python3

[Ned Deily]

Ned Deily  added the comment:

To address some of the concerns:

- The shebang line in Mac/BuildScript/build-installer.py can be safely removed.

- "Lib/idlelib/pyshell.py for example, maybe the IDLE entry point is just a 
symlink to that file"
It's not, at least for unix-y builds. In [install-prefix], an "idle*" script is 
installed of the form:

#![install-prefix]/bin/python3.[n]

from idlelib.pyshell import main
if __name__ == '__main__':
main()

So there is an absolute link to the correct interpreter and the shebang in 
pyshell.py is not used.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41626] port shebang of tools from python2 to python3

[Ned Deily]

Ned Deily  added the comment:

To address some of the concerns:

- The shebang line in Mac/BuildScript/build-installer.py can be safely removed.

- "Lib/idlelib/pyshell.py for example, maybe the IDLE entry point is just a 
symlink to that file"
It's not, at least for unix-y builds. In [install-prefix], an "idle*" script is 
installed of the form:

#![install-prefix]/bin/python3.10

from idlelib.pyshell import main
if __name__ == '__main__':
main()

So there is an abolute link to the correct interpreter and shebang is not used.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42096] zipfile.is_zipfile incorrectly identifying a gzipped file as a zip archive

[Irit Katriel]

Irit Katriel  added the comment:

Are you able to attach a file with which you see this problem?
Have you tried with newer Python versions?

--
nosy: +iritkatriel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41626] port shebang of tools from python2 to python3

[Éric Araujo]

Éric Araujo  added the comment:

This is a minor thing, and not a particularly visible bug, but it could create 
confusion about how to run scripts so could be fixed.

1) For stdlib modules, I think the intended usage is `python3 -m 
encodings.rot_13`; I don’t expect anyone to run `python3 
/usr/lib/python3.10/encodings/rot_13.py` to do a decoding in the terminal!
So for these easter eggs / hidden gems in the stdlib, I would delete the 
shebang and remove the executable bit.  There are 18 such modules, ignoring 
test, lib2to3, turtledemo and a few others named below.

1.1) There may be some exceptions: for Lib/idlelib/pyshell.py for example, 
maybe the IDLE entry point is just a symlink to that file, and removing the 
executable bit would break it.  So this can’t be a mass update but each case 
should be reviewed.

2) Some of the files in the PR are used during development of CPython: 
Tools/clinic/clinic.py Python/makeopcodetargets.py Parser/asdl_c.py 
Objects/typeslots.py Modules/_sha3/cleanup.py 
Lib/ctypes/macholib/fetch_macholib Mac/BuildScript/build-installer.py
I think some of these work with any Python, but some need the locally built 
interpreter to give correct results.
It could avoid confusion and mistakes to remove the shebangs and executable 
bits from the scripts that need a local Python, and make dure their 
documentation mentions `./python path/to/tool`.

2.1) One recent example, Tools/peg_generator/pegen/__main__.py is documented to 
work with Python 3.8+, and confusingly is both a __main__ module (to support 
python3 -m pegen and an executable script with shebang!)

2.2) In Tools/ssl, we have `/usr/bin/env python3` in make_ssl_data.py and 
`./python` in multissltests.py.  Are these working?!

3) The files in Tools/scripts are documented as «useful for building Python» or 
«generally useful scripts».  They nearly all executable and nearly all use 
`#!/usr/bin/env python3`, so fixing the few `python` references there seems ok. 
 Likewise for Tools/pynche/pynche.

3.1) For Tools/gdb/libpython.py I don’t know how it’s meant to be used; if it’s 
not distributed but only for debugging a locally built CPython, then removing 
the shebang seems correct.

(Oh and by the way, Tools/scripts/fixps.py is a tool to replace 
`/usr/local/bin/python` shebangs with `/usr/bin/env python`…  You can judge if 
that is useful and/or correct.)


So if things are changed (let’s get a couple opinions here), maybe this needs 
separate PRs for 1/2/3, and make sure to get reviews to ensure we don’t break 
IDLE / Mac installers / pegen / etc.

--
components: +Demos and Tools, Library (Lib)
nosy: +corona10, eric.araujo, ned.deily, petr.viktorin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39416] Document default numeric string formats

[Karl O. Pinc]

Change by Karl O. Pinc :


--
pull_requests: +21809
pull_request: https://github.com/python/cpython/pull/22867

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42112] ZipFIle.write remove slash at the beginning of member's path

[Matan Perelman]

Change by Matan Perelman :


--
components: Library (Lib)
nosy: matan1008
priority: normal
severity: normal
status: open
title: ZipFIle.write remove slash at the beginning of member's path
type: behavior
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42103] [security] DoS (MemError via CPU and RAM exhaustion) when processing malformed Apple Property List files in binary format

[Ronald Oussoren]

Ronald Oussoren  added the comment:

One Apple implementation of binary plist parsing is here: 
https://opensource.apple.com/source/CF/CF-550/CFBinaryPList.c.

That seems to work from a buffer (or mmap) of the entire file, making 
consistency checks somewhat easier, and I don't think they have a hardcoded 
limit on the number of items in the plist (but: it's getting late and might 
have missed that).

An easy fix for this issue is to limit the amount of values read by _read_ints, 
but that immediately begs the question what that limit should be. It is clear 
that 1B items in the array is too much (in this case 1B object refs for 
dictionary keys). 

I don't know what a sane limit would be. The largest plist file on my system is 
10MB. Limiting *n* to 20M would be easily enough to parse that file, and 
doesn't consume too much memory (about 160MB for the read buffer at most). 

Another thing the current code doesn't do is check that the "ref" argument to 
_read_object is in range. That's less problematic because the code will raise 
an exception regardless (IndexErrox, instead of the nicer InvalidFileException).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33754] f-strings should be part of the Grammar

[Eric V. Smith]

Eric V. Smith  added the comment:

Just some notes to consider before work starts on this in earnest:

We need to decide what sort of changes we'll accept, if any. For at least the 
first round of this, I'm okay with "absolutely no change will be acceptable".

For example, here's a good change (IMO): allowing f'{"\n" if cond else ""}'. 
I'd like to be able to use backslashes inside strings that are in an expression.

A questionable change: f'{'foo'}'. Nesting the same type of quotes.

I think we should be explicit about what we will accept, because editors, etc. 
will need to adapt. In msg318550 I mention that some external tools use the 
same lexer they use for strings to lex f-strings. Are we okay with break that?

And the f-string '=' feature maybe be hard to support. Although if we are able 
to support it, then I think the same solution will be applicable to string 
annotations without unparsing them.

--
nosy: +emilyemorehouse, pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42111] Make the xxlimited module an example of best extension module practices

[Petr Viktorin]

New submission from Petr Viktorin :

The "xxlimited" module (Modules/xxlimited.c) was added as part of PEP 
384 (Defining a Stable ABI), and is undocumented. As far as I can tell, 
it was added partly to test the stable ABI, and partly as an example of 
how to write a module (like "xx" from xxmodule.c).
In the last few years the module has not seen much maintenance, and I 
believe it's no longer a good example to follow: it works, but there are 
now better ways to do things.

I would like to take over maintenance of the module and make it into an 
example of how to write a low-level C extension with isolated module 
state, as described in PEP 630 (Isolating Extension Modules) -- an 
informational PEP that I plan to convert to a HOWTO doc when everything 
is ready.

The old module will be kept around to test the 3.5 stable ABI.

Past discussion: 
https://mail.python.org/archives/list/python-...@python.org/thread/FO3YPG3YLG2XF5FKHICJHNINSPY4OHEL/#YITRQXGUOIEHK22QP5K4C5E45QA356U3

--
components: C API
messages: 379241
nosy: petr.viktorin
priority: normal
severity: normal
status: open
title: Make the xxlimited module an example of best extension module practices
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41746] Add optional type information to asdl_seq objects

[Lysandros Nikolaou]

Lysandros Nikolaou  added the comment:

Last bit of work is now done.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41056] minor NULL pointer and sign issues reported by Coverity

[Paul Ganssle]

Change by Paul Ganssle :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41746] Add optional type information to asdl_seq objects

[Lysandros Nikolaou]

Lysandros Nikolaou  added the comment:


New changeset 2e5ca9e3f68b33abb7d2c66d22ffc18dec40641a by Lysandros Nikolaou in 
branch 'master':
bpo-41746: Cast to typed seqs in CHECK macros to avoid type erasure (GH-22864)
https://github.com/python/cpython/commit/2e5ca9e3f68b33abb7d2c66d22ffc18dec40641a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30155] Add ability to get tzinfo from a datetime instance in C API

[Paul Ganssle]

Change by Paul Ganssle :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42103] [security] DoS (MemError via CPU and RAM exhaustion) when processing malformed Apple Property List files in binary format

[Ronald Oussoren]

Ronald Oussoren  added the comment:

Thanks for the report. I can reproduce the issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41959] Doc/library/asyncio-policy.rst grammar error

[miss-islington]

Change by miss-islington :


--
pull_requests: +21807
pull_request: https://github.com/python/cpython/pull/22865

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41959] Doc/library/asyncio-policy.rst grammar error

[miss-islington]

miss-islington  added the comment:


New changeset caff2934f46510920a6169e192707d59e9c55f6b by Raúl Cumplido in 
branch 'master':
bpo-41959: Fix grammar around class asyncio.MultiLoopChildWatcher text 
(GH-22580)
https://github.com/python/cpython/commit/caff2934f46510920a6169e192707d59e9c55f6b


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41959] Doc/library/asyncio-policy.rst grammar error

[miss-islington]

Change by miss-islington :


--
pull_requests: +21808
pull_request: https://github.com/python/cpython/pull/22866

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41746] Add optional type information to asdl_seq objects

[Lysandros Nikolaou]

Change by Lysandros Nikolaou :


--
pull_requests: +21806
pull_request: https://github.com/python/cpython/pull/22864

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37048] ssl module: QUIC support for HTTP/3

[Jeremy Lainé]

Jeremy Lainé  added the comment:

The OpenSSL authors make a fair point, QUIC seems to be taking a long time to 
stabilize with little consideration for backwards compatibility at this stage.

As stated previously though it's perfectly feasible to implement a QUIC stack 
by linking to an unpatched OpenSSL if you're willing to implement a 
stripped-down TLS 1.3 engine yourself.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42025] zoneinfo: wrong time difference across transition when tzinfo is equal

[Paul Ganssle]

Paul Ganssle  added the comment:

Yeah, people are very confused by this, which is why I wrote that article.

Maybe there is a place for big warnings somewhere? I have been mulling over the 
possibility of proposing a backwards-incompatible (though minimally so, 
hopefully) change to arithmetic semantics that would work well with how people 
think this should work, but I think any such backwards-incompatible change 
would be a Big Deal™, and unfortunately quite hard to warn about with a proper 
deprecation period.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39416] Document default numeric string formats

[Éric Araujo]

Éric Araujo  added the comment:

Thanks for the patch!

--
nosy: +eric.araujo
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.10, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39416] Document default numeric string formats

[miss-islington]

miss-islington  added the comment:


New changeset ec62b47ebc8f29007942c8e9f3f260af91ca58cb by Miss Skeleton (bot) 
in branch '3.9':
[3.9] bpo-39416: Document some restrictions on the default string 
representations of numeric classes (GH-18111) (GH-22860)
https://github.com/python/cpython/commit/ec62b47ebc8f29007942c8e9f3f260af91ca58cb


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42107] Monotonic time on macOS 10.12+ should use mach_continuous_time()

[Ronald Oussoren]

Ronald Oussoren  added the comment:

See also #41303, which discusses using mach_continuous_time in another context.

If using a different clock is necessary I'd prefer to use 
clock_gettime(CLOCK_MONOTONIC_RAW) instead of mach_continuous_time(), as the 
former is a more cross-platform API.

BTW. Please hold off from merging a patch for this until the macOS 11 support 
code in #41100 is in, this is bound to cause a merge conflict otherwise.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39416] Document default numeric string formats

[miss-islington]

miss-islington  added the comment:


New changeset 89fac4c3748aa7eb23d09922331e90a62ce782fd by Miss Skeleton (bot) 
in branch '3.8':
[3.8] bpo-39416: Document some restrictions on the default string 
representations of numeric classes (GH-18111) (GH-22861)
https://github.com/python/cpython/commit/89fac4c3748aa7eb23d09922331e90a62ce782fd


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41100] Build failure on macOS 11 (beta)

[Ronald Oussoren]

Ronald Oussoren  added the comment:

https://github.com/python/cpython/pull/22855 is currently the most complete PR, 
and should be almost ready for merging (although there hasn't been any code 
review at this point).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42110] race condition in ThreadChildWatcher (default) and MultiLoopChildWatcher

[Alan Jenkins]

New submission from Alan Jenkins :

## Test program ##

import asyncio
import time
import os
import signal
import sys

# This bug happens with the default, ThreadedChildWatcher
# It also happens with MultiLoopChildWatcher,
# but not the other three watcher types.
#asyncio.set_child_watcher(asyncio.MultiLoopChildWatcher())

# Patch os.kill to call sleep(1) first,
# opening up the window for a race condition
os_kill = os.kill
def kill(p, n):
time.sleep(1)
os_kill(p, n)

os.kill = kill

async def main():
p = await asyncio.create_subprocess_exec(sys.executable, '-c', 'import sys; 
sys.exit(0)')
p.send_signal(signal.SIGTERM)
# cleanup
await p.wait()

asyncio.run(main())


## Test output ##

Traceback (most recent call last):
  File "", line 1, in 
  File "/home/alan-sysop/src/cpython/Lib/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
  File "/home/alan-sysop/src/cpython/Lib/asyncio/base_events.py", line 642, in 
run_until_complete
return future.result()
  File "", line 3, in main
  File "/home/alan-sysop/src/cpython/Lib/asyncio/subprocess.py", line 138, in 
send_signal
self._transport.send_signal(signal)
  File "/home/alan-sysop/src/cpython/Lib/asyncio/base_subprocess.py", line 146, 
in send_signal
self._proc.send_signal(signal)
  File "/home/alan-sysop/src/cpython/Lib/subprocess.py", line 2081, in 
send_signal
os.kill(self.pid, sig)
  File "", line 3, in kill
ProcessLookupError: [Errno 3] No such process


## Tested versions ##

* v3.10.0a1-121-gc60394c7fc
* python39-3.9.0-1.fc32.x86_64
* python3-3.8.6-1.fc32.x86_64


## Race condition ##

main thread vs  ThreadedChildWatcher._do_waitpid() thread

p=create_subprocess_exec(...)
waitpid(...)  # wait for process exit


p.send_signal(9)


## Result ##

A signal is sent to p.pid, after p.pid has been reaped by waitpid().  It might 
raise an error because p.pid no longer exists.

In the worst case the signal will be sent successfully - because an unrelated 
process has started with the same PID.


## How easy is it to reproduce? ##

It turns out the window for this race condition has been kept short, due to 
mitigations in the subprocess module.  IIUC, the mitigation protects against 
incorrect parallel use of a subprocess object by multiple threads.

def send_signal(self, sig):
# bpo-38630: Polling reduces the risk of sending a signal to the
# wrong process if the process completed, the Popen.returncode
# attribute is still None, and the pid has been reassigned
# (recycled) to a new different process. This race condition can
# happens in two cases [...]
self.poll()
if self.returncode is not None:
# Skip signalling a process that we know has already died.
return
os.kill(self.pid, sig)




## Possible workarounds ##

* SafeChildWatcher and FastChildWatcher should not have this defect.  However 
we use ThreadedChildWatcher and MultiLoopChildWatcher to support running event 
loops in different threads.

* PidfdChildWatcher should not have this defect.  It is only available on 
Linux, kernel version 5.3 or above.

It would be possible to avoid the ThreadedChildWatcher race by using native 
code and pthread_cancel(), so that the corresponding waitpid() call is canceled 
before sending a signal.  Except the current implementation of pthread_cancel() 
is also unsound, because of race conditions.

* https://lwn.net/Articles/683118/ "This is why we can't have safe cancellation 
points"
* https://sourceware.org/bugzilla/show_bug.cgi?id=12683 "Race conditions in 
pthread cancellation"

--
components: asyncio
messages: 379229
nosy: asvetlov, sourcejedi, yselivanov
priority: normal
severity: normal
status: open
title: race condition in ThreadChildWatcher (default) and MultiLoopChildWatcher
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19438] Where is NoneType in Python 3?

[Andrés Delfino]

Andrés Delfino  added the comment:

As per https://github.com/python/cpython/pull/22336 I believe this issue can be 
closed now.

My PR is not relevant to the problem stated by OP, so I'm "unlinking" it.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35181] Doc: Namespace Packages: Inconsistent documentation of __loader__ being None

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

Apologies for the long delay in reviewing this bug.  I'm looking at it now, 
however since Python 3.7 is in security-only mode, this will only apply to 
3.10, 3.9, and 3.8.

--
versions: +Python 3.10, Python 3.9 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42107] Monotonic time on macOS 10.12+ should use mach_continuous_time()

[Ned Deily]

Change by Ned Deily :


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42109] Use hypothesis for testing the standard library, falling back to stubs

[Paul Ganssle]

Change by Paul Ganssle :


--
keywords: +patch
pull_requests: +21805
pull_request: https://github.com/python/cpython/pull/22863

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42109] Use hypothesis for testing the standard library, falling back to stubs

[Paul Ganssle]

New submission from Paul Ganssle :

Following up on this year's language summit, I would like to propose that we 
start integrating property tests into the standard library. Zac Hatfield-Dodds 
(along with myself and Carl Friedrich Bolz-Tereick) have put together this 
repository of tests that we run against the standard library as is: 
https://github.com/Zac-HD/stdlib-property-tests

Here is the blog post covering the proposal from the language summit: 
https://pyfound.blogspot.com/2020/05/property-based-testing-for-python.html

The biggest challenges here are logistical:

1. Pulling in third party dependencies is difficult to do reliably on CI, but 
checking in hypothesis's entire dependency tree is probably not feasible.
2. We don't necessarily want to require developers running their tests locally 
to have to set up a hypothesis environment just to run the tests.
3. Hypothesis tests are not (by default) deterministic, which some are 
concerned may lead to flakiness by themselves.


To allay these concerns, I propose that we implement a compatibility interface 
for hypothesis that uses the third party module when it's installed, but 
otherwise falls back to stubs. The way I see the stubs working is that `@given` 
*without* `@example`s would simply skip the test. If you specify `@given` and 
one or more `@example`s, the test falls back to a simple parameterized test 
when hypothesis is not available.

At least at first, we won't attempt to add a mandatory PR job that runs with 
hypothesis installed. Instead, I'd like to run either an optional job on PR or 
have one or more buildbots that runs the hypothesis tests.

I would also like to suggest a policy of including at least one example in each 
property test, so that on PR at least some of the inputs are tested.

--
assignee: p-ganssle
components: Tests
messages: 379226
nosy: Zac Hatfield-Dodds, p-ganssle, terry.reedy
priority: normal
severity: normal
stage: patch review
status: open
title: Use hypothesis for testing the standard library, falling back to stubs
type: enhancement
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38980] Compile libpython with -fno-semantic-interposition

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Victor is on vacation for some weeks, so I am creating a PR to push this 
forward.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38980] Compile libpython with -fno-semantic-interposition

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +21804
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/22862

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42097] Python 3.7.9 logging/threading/fork hang

[Antoine Pitrou]

Change by Antoine Pitrou :


--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39416] Document default numeric string formats

[miss-islington]

Change by miss-islington :


--
pull_requests: +21803
pull_request: https://github.com/python/cpython/pull/22861

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39416] Document default numeric string formats

[miss-islington]

miss-islington  added the comment:


New changeset c60394c7fc9cc09b16e9675a3eeb5844b6d8523f by kpinc in branch 
'master':
bpo-39416: Document some restrictions on the default string representations of 
numeric classes (GH-18111)
https://github.com/python/cpython/commit/c60394c7fc9cc09b16e9675a3eeb5844b6d8523f


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39416] Document default numeric string formats

[miss-islington]

Change by miss-islington :


--
pull_requests: +21802
pull_request: https://github.com/python/cpython/pull/22860

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41052] Opt out serialization/deserialization for heap type

[Antoine Pitrou]

Antoine Pitrou  added the comment:

At some point, pickle protocols 0 and 1 should be deprecated and slated for 
removal.  Protocol 2 is 17 years ago already, and protocol 3 has become the 
default in Python 3.

That would probably be a better use of contributor time than trying to make 
heap types compatible with those obsolete protocols.

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41061] Incorrect expressions / assert with side effect in hashtable

[Christian Heimes]

Change by Christian Heimes :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37355] SSLSocket.read does a GIL round-trip for every 16KB TLS record

[Christian Heimes]

Change by Christian Heimes :


--
assignee: christian.heimes -> 
versions: +Python 3.10 -Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37048] ssl module: QUIC support for HTTP/3

[Christian Heimes]

Christian Heimes  added the comment:

OpenSSL 3.0.0 is not going support QUIC, 
https://www.openssl.org/blog/blog/2020/02/17/QUIC-and-OpenSSL/

--
versions: +Python 3.10 -Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36709] Asyncio SSL keep-alive connections raise errors after loop close.

[Christian Heimes]

Christian Heimes  added the comment:

This seems to be an asyncio problem.

--
assignee: christian.heimes -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37553] SendfileUsingSendTest tests timeout too short for Windows ARM32

[Irit Katriel]

Change by Irit Katriel :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35749] Rewrite asyncio signal handler

[Alan Jenkins]

Alan Jenkins  added the comment:

Here's a simple test case that fails on the main branch.

https://github.com/sourcejedi/cpython/commit/50184ea3b354fd775866d036ccee058ec6734927

> the patch assumes that python signal handler will be called *before* reading 
> from self-pipe. 
> Otherwise, a signal callback will be postponed up to next writing to the 
> pipe, which looks like a hidden bug.

I think it will be?  I can imagine it being a concern for future changes 
though.  I thought it could break if anyone takes seriously the comment "XXX 
Signals should be recorded per thread, now we have thread state." :-).

But if we're not confident, surely all we have to do is make sure to call 
_write_to_self() at the end of the python signal handler 
(_UnixSelectorEventLoop._sig_handler()).

It seems a pity not to have example code we can point to, that handles signals 
correctly.  Even if it's too tedious to explain the correct way in the official 
docs.

--
nosy: +sourcejedi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35324] ssl: FileNotFoundError when do handshake

[Christian Heimes]

Christian Heimes  added the comment:

I'm not able to reproduce the issue. Please re-open this issue if you are still 
having issue with more recent version of Python and OpenSSL.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33570] OpenSSL 1.1.1 / TLS 1.3 cipher suite changes

[Christian Heimes]

Change by Christian Heimes :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34915] LWPCookieJar.save() creates *.lwp file in 644 mode

[Christian Heimes]

Change by Christian Heimes :


--
assignee: christian.heimes -> 
nosy:  -christian.heimes
versions: +Python 3.10, Python 3.8, Python 3.9 -Python 2.7, Python 3.4, Python 
3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28022] SSL releated deprecation for 3.6

[Christian Heimes]

Change by Christian Heimes :


--
versions: +Python 3.10 -Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32549] Travis: Test with OpenSSL 1.1.0

[Christian Heimes]

Change by Christian Heimes :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26835] Add file-sealing ops to fcntl

[Christian Heimes]

Change by Christian Heimes :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31432] Documention for CERT_OPTIONAL is misleading

[Christian Heimes]

Change by Christian Heimes :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40592] `Shutil.which` incosistent with windows's `where`

[Christopher Marchfelder]

Christopher Marchfelder  added the comment:

@steve.dower Added the changes in the PR - could you please re-check? Thank you!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com