[issue46848] Use optimized string search function in mmap.find()

[benrg]

benrg  added the comment:

memmem isn't a standard C function, and some libraries don't have it, notably 
Microsoft's.

newlib's memmem seems to be the same as glibc's, but is under a BSD 3-clause 
license instead of LGPL. An older version of newlib's memmem (prior to 
2019-01-01) has the license "Permission to use, copy, modify, and distribute 
this software is freely granted, provided that this notice is preserved", and 
is still highly optimized and much better than a naive implementation.

Of course, bundling it would no longer be quite so "free".

Old newlib memmem: 
https://sourceware.org/git/?p=newlib-cygwin.git;a=blob_plain;f=newlib/libc/string/memmem.c;h=25704e467decff5971b34f4189ddfff04ac5fa8e

New newlib memmem: 
https://sourceware.org/git/?p=newlib-cygwin.git;a=blob_plain;f=newlib/libc/string/memmem.c

Helper file for both: 
https://sourceware.org/git/?p=newlib-cygwin.git;a=blob_plain;f=newlib/libc/string/str-two-way.h

--
nosy: +benrg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46843] PersistentTaskGroup API

[Joongi Kim]

Joongi Kim  added the comment:

Anoter case:

https://github.com/lablup/backend.ai-manager/pull/533
https://github.com/lablup/backend.ai-agent/pull/341

When shutting down the application, I'd like to explicitly cancel the shielded 
tasks, while keep them shielded before shutdown.

So I inserted `ptaskgroup.create_task()` inside `asyncio.shield()`, so that the 
tasks are not cancelled upon the cancellation of their callers but they get 
cancelled when the server shuts down.

This pattern is conveniently implemented with PersistentTaskGroup.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21761] [doc] language reference describes the role of module.__file__ inaccurately

[Stanley]

Change by Stanley :


--
keywords: +patch
nosy: +slateny
nosy_count: 6.0 -> 7.0
pull_requests: +29687
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/31565

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12165] [doc] clarify documentation of nonlocal

[Eryk Sun]

Eryk Sun  added the comment:

> Another problem with the current text is that it fails to exclude 
> enclosing class scopes

The nonlocal statement is only disallowed in module code (i.e. "exec" 
compilation mode) because it can never be nested. It's allowed in a class 
definition that has an outer function scope. A class body itself is never a 
nonlocal scope; it just has access to them. Here's a slightly modified version 
that includes class definitions:

"When the definition of a function or class is nested (enclosed) within the 
definitions of other functions, its nonlocal scopes are the local scopes of the 
enclosing functions. The nonlocal statement causes the listed identifiers to 
refer to names previously bound in nonlocal scopes. If a name is bound in more 
than one nonlocal scope, the nearest binding is used. If a name is not bound in 
any nonlocal scope, or if there is no nonlocal scope, a SyntaxError is raised.

The nonlocal statement applies to the entire scope of a function or class body. 
A SyntaxError is raised if a variable is used or assigned to prior to its 
nonlocal declaration in the scope."

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46845] dict: Use smaller entry for Unicode-key only dict.

[Inada Naoki]

Change by Inada Naoki :


--
keywords: +patch
pull_requests: +29686
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31564

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46844] Implicit binding of PersistentTaskGroup (or virtual event loops)

[Joongi Kim]

Joongi Kim  added the comment:

Updated the title to reduce confusion.

--
title: Context-based TaskGroup for legacy libraries -> Implicit binding of 
PersistentTaskGroup (or virtual event loops)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46844] Context-based TaskGroup for legacy libraries

[Joongi Kim]

Joongi Kim  added the comment:

I have added more about my stories in bpo-46843.

I think the suggestion of implicit taskgroup binding with the current 
asyncio.TaskGroup has no point but it would have more meaning with 
PersistentTaskGroup.

So, if we treat PersistentTaskGroup as a "nested, hierarchical virtual event 
loop" to repeat and group shutdown procedures for different task sets 
separately, the point may look a little bit clearer.

It is more like assigning a virtual event loop to different modules and 
libraries, while keeping the behavior of asyncio.create_task() same.  The 
difference is that the caller controls when these virtual loops are terminated 
and in what order.

Does this make sense better?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46606] Large C stack usage of os.getgroups() and os.setgroups()

[Inada Naoki]

Inada Naoki  added the comment:


New changeset ad6c7003e38a9f8bdf8d865fb5fa0f3c03690315 by Inada Naoki in branch 
'main':
bpo-46606: Remove redundant +1. (GH-31561)
https://github.com/python/cpython/commit/ad6c7003e38a9f8bdf8d865fb5fa0f3c03690315


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46843] PersistentTaskGroup API

[Joongi Kim]

Joongi Kim  added the comment:

Here is one another story.

When handling message queues in distributed applications, I use the following 
pattern frequently for graceful shutdown:
* Use a sentinel object to signal the end of queue.
* Enqueue the sentinel object when:
  - The server is shutting down. (i.e., cancelled explicitly)
  - The connection peer has sent an explicit termination message. (e.g., EOF)
* Wait until all enqueued messages before the sentinal object to be processed.
  - I'd like to impose a shutdown timeout on here using a persistent task 
group, by spawning all handler tasks of this queue into it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46843] PersistentTaskGroup API

[Joongi Kim]

Joongi Kim  added the comment:

I ended up with the following conclusion:
- The new abstraction should not cancel sibling tasks and itself upon unhandled 
execption but loudly report such errors (and the fallback error handler should 
be customizable).
- Nesting task groups will give additional benefits such as orderly shutdown of 
different task groups.  Empty up message queues before shutting down netweork 
connections, etc.

You may take my suggestion as "let's have a hierarchical nested virtual event 
loops to group tasks".  PersistentTaskGroup actually shares many 
characteristics with the event loop while itself is not an event loop.

So I came up with WeakSet with task decorators to handle exceptions by my own, 
and this is the current rudimentary implementation of PersistentTaskGroup in 
aiotools.

And I discovered from the additional search results that the same pattern 
---managing sporadic tasks using WeakSet and writing a proper cancellation loop 
of them---appear quite commonly in many different asyncio applications and 
libraries.

So that's why I think this should be an intrinsic/essential abstraction.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27583] [doc ] configparser: modifying default_section at runtime

[Stanley]

Change by Stanley :


--
keywords: +patch
nosy: +slateny
nosy_count: 5.0 -> 6.0
pull_requests: +29685
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31562

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46843] PersistentTaskGroup API

[Joongi Kim]

Joongi Kim  added the comment:

This particular experience, 
https://github.com/lablup/backend.ai-agent/pull/331, has actually motivated me 
to suggest PersistentTaskGroup.

The program subscribes the event stream of Docker daemon using aiohttp as an 
asyncio task, and this should be kept running throughout the whole application 
lifetime.  I first applied aiotools.TaskGroup to ensure shutdown of spawned 
event handler tasks, but I missed that it cancels all sibling tasks if one of 
the spawned tasks bubbles up an unhandled exception.  This has caused silent 
termination of the subscriber task and led to a bug.  We could debug this issue 
by inspecting aiomonitor and checking the existence of this task.  After this 
issue, I began to think we need a proper abstraction of a long-running task 
group (NOTE: the task group is long-running.  The lifetime of internal tasks 
does not matter).

Another case is that https://github.com/lablup/backend.ai/issues/330.

One of our customer site has suffered from excessive CPU usage by our program.  
We could identify the issue by aiomonitor, and the root cause was the 
indefinite accumulation of peridoically created asyncio tasks to measure the 
disk usage of user directories, when there are too many files in them.  Since 
the number of tasks have exceeded 10K, it was very difficult to group and 
distinguish individual asyncio tasks in aiomonitor.  I thought that it would be 
nice if we could group such tasks into long-running groups and view task 
statistics separately.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43364] Windows: Make UTF-8 mode more accessible

[Inada Naoki]

Change by Inada Naoki :


--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46606] Large C stack usage of os.getgroups() and os.setgroups()

[Inada Naoki]

Change by Inada Naoki :


--
pull_requests: +29684
pull_request: https://github.com/python/cpython/pull/31561

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46843] PersistentTaskGroup API

[Joongi Kim]

Joongi Kim  added the comment:

@gvanrossum As you mentioned, the event loop currently plays the role of the 
top-level task group already, even without introducing yet another top-level 
task.  For instance, asyncio.run() includes necessary shutdown procedures to 
cancel all belonging unfinished tasks and async generators.

However, I think we should provide an abstraction to organize the shutdown 
procedures in a *hierarchical* manner.  For example, we could cancel all event 
handler tasks before cancelling all HTTP handler tasks upon a web server 
shutdown.  This prevents any potential races between theses two different task 
sets.  I think you could agree with the necessity of orderly release of 
underlying resources during shutdown in general.  Currently 
asyncio.Task.all_tasks() is just a list created from WeakSet and we cannot 
guarantee which tasks will be cancelled first.

Yes, this can be done by manually writing codes to declare multiple WeakSets 
and a for-loop to cancel the contained tasks by enumerating over them, just 
like asyncio.run() does.  With the new addition of TaskGroup and 
ExceptionGroup, this code does not require core changes of Python.

But I believe that this hierarchical persistent task group abstraction should 
be an essential part of the API and asyncio tutorials when writing server 
applications.  asyncio.run() could be written by users, but I think the core 
devs have agreed with that it is an essential abstraction to be included in the 
stdlib.  I'd like argue that hierarchical persistent task groups is the same 
case.

Though I named it "PersistentTaskGroup" because it looks similar to TaskGroup, 
but this name may be misleading.  In PersistentTaskGroup, even when all tasks 
finish successfully, it does NOT terminate but keeps waiting for new tasks to 
be spawned.  It terminates only when the outer task is cancelled or its 
shutdown() method is called.  Note that belonging tasks may be either 
short-running or long-running, and this does not matter.  The point is to 
shutdown any remaining tasks in an orderly manner.  If you don't like the 
naming, please suggest alternatives.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46854] Failed to compile static python3.7.12

[aprpp]

New submission from aprpp <916495...@qq.com>:

I compile static version of python3.7.12, I added the static standard library 
that I want to compile in Modules/Setup, reference Modules/Setup.dist in python 
source, like this:

static

Modules that should always be present (non UNIX dependent):
array arraymodule.c # array objects
cmath cmathmodule.c _math.c # -lm # complex math library functions
math mathmodule.c _math.c # -lm # math library functions, e.g. sin()
_contextvars _contextvarsmodule.c # Context Variables
_struct _struct.c # binary structure packing/unpacking

But there are still many modules that fail to compile, these modules with no 
commented out build definitions in the Modules/Setup.dist file. How do I add 
these modules build definitions to the Modules/Setup, yes they compile 
successfully ?

Failed to build these modules:
_bz2  _ctypes   _ctypes_test
_decimal  _hashlib  _json
_lsprof   _lzma _multiprocessing
_opcode   _ssl  _testbuffer
_testimportmultiple   _testmultiphase   _uuid
_xxtestfuzz   ossaudiodev   xxlimited

--
components: Build
files: 截图.PNG
messages: 413958
nosy: aprpp
priority: normal
severity: normal
status: open
title: Failed to compile static python3.7.12
type: compile error
versions: Python 3.7
Added file: https://bugs.python.org/file50643/截图.PNG

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45991] Improve ambiguous docstrings in pkgutil

[Stanley]

Stanley  added the comment:

While it is ambiguous, when there's a path parameter I would default it to 
string unless otherwise specified. Maybe instead a note could be put in the 
Pathlib doc noting functions that accept path arguments might not accept Path 
objects?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12165] [doc] clarify documentation of nonlocal

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> I think this is more confusing then helpful.

I concur.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46793] expose expat XML billion laughs attack mitigation APIs

[sping]

sping  added the comment:

First mention at https://bugs.python.org/issue44394#msg395642

--
nosy: +sping

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20185] Derby #17: Convert 49 sites to Argument Clinic across 13 files

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 7d03c8be5af2f1559dbc35b775b3116dfd63cfb6 by Victor Stinner in 
branch 'main':
bpo-46852: Rename float.__set_format__() to float.__setformat__() (GH-31558)
https://github.com/python/cpython/commit/7d03c8be5af2f1559dbc35b775b3116dfd63cfb6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46852] Remove the float.__setformat__() method

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 7d03c8be5af2f1559dbc35b775b3116dfd63cfb6 by Victor Stinner in 
branch 'main':
bpo-46852: Rename float.__set_format__() to float.__setformat__() (GH-31558)
https://github.com/python/cpython/commit/7d03c8be5af2f1559dbc35b775b3116dfd63cfb6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +29683
pull_request: https://github.com/python/cpython/pull/31560

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46853] Python interpreter can get code from memory, it is not secure.

[i5-7200u]

New submission from i5-7200u :

Hi, Python Interpreter have a big security bug/error.

My friend and l am. We can give virus code to Python Interpreter.

we were looking for run a binary application from memory (byte array)

Later we find python, but we got it is security bug/error

example from my friend:
https://www.virustotal.com/gui/file/6fc3ad98c40e6962f3c29e07735f7ae25e50092c3d7595201740a954ad5f3cf4?nocache=1

https://github.com/ArdaKC/run-python-in-java

if we encrypt python virus code and give to java codes as byte array and we 
decrypt python virus code and give to python interpreter from memory then 
antiviruses never detect it (not comodo, comodo have a strong hips and auto 
conmaintent) but we dont it.

we just want to fix this bug. for more peoples security. Please.


This bug is reported by KCS Team.

--
components: Interpreter Core
files: afterexample.png
messages: 413952
nosy: i5-7200u
priority: normal
severity: normal
status: open
title: Python interpreter can get code from memory, it is not secure.
type: security
versions: Python 3.11
Added file: https://bugs.python.org/file50642/afterexample.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46567] Add Tcl/Tk builds for ARM64

[Steve Dower]

Steve Dower  added the comment:

Getting this done with some help from colleagues. Tcl and Tk have been updated 
to support it, and I've pulled down their patches into our source repo.

Hopefully I find time to get the build and setup updates done before the next 
alpha... adding Pablo just to say that if you feel like deferring by a day or 
two, I will fully support you :)

--
assignee:  -> steve.dower
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46752] Introduce task groups to asyncio and change task cancellation semantics

[Tin Tvrtković]

Change by Tin Tvrtković :


--
pull_requests: +29682
pull_request: https://github.com/python/cpython/pull/31559

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46852] Remove the float.__setformat__() method

[STINNER Victor]

STINNER Victor  added the comment:

We can keep the float.__getformat__() method, it doesn't harm. I change the 
issue title to only propose to remove the float.__setformat__() method.

--
title: Remove float.__get_format__() and  float.__set_format__() -> Remove the 
float.__setformat__() method

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46852] Remove float.__get_format__() and float.__set_format__()

[STINNER Victor]

STINNER Victor  added the comment:

I wrote GH-31558 to fix the typo in the method name.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20185] Derby #17: Convert 49 sites to Argument Clinic across 13 files

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +vstinner
nosy_count: 10.0 -> 11.0
pull_requests: +29681
pull_request: https://github.com/python/cpython/pull/31558

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46852] Remove float.__get_format__() and float.__set_format__()

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +29680
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31558

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46852] Remove float.__get_format__() and float.__set_format__()

[STINNER Victor]

STINNER Victor  added the comment:

Oh wait, I'm now confused by the method names. In Python 3.10, the correct 
names at:

* float.__getformat__() <= 4 underscores
* float.__set_format__() <= 5 underscores

It's even more confusing because the "set format" is only used in one place: 
test_float, and test_float uses... __setformat__() (4 underscores).

A typo a was introduced in Python 3.7 by:

commit b5c51d3dd95bbfde533655fb86ac0f96f771ba7b
Author: Serhiy Storchaka 
Date:   Sat Mar 11 09:21:05 2017 +0200

bpo-20185: Convert float object implementation to Argument Clinic. (#543)

Based on patch by Vajrasky Kok.

Since Python 3.7, the 4 "set format" tests are simply skipped!

$ ./python -m test -v test_float
(...)
test_getformat (test.test_float.FormatFunctionsTestCase) ... skipped 'requires 
__setformat__'
test_setformat (test.test_float.FormatFunctionsTestCase) ... skipped 'requires 
__setformat__'
(...)
test_double_specials_dont_unpack (test.test_float.UnknownFormatTestCase) ... 
skipped 'requires __setformat__'
test_float_specials_dont_unpack (test.test_float.UnknownFormatTestCase) ... 
skipped 'requires __setformat__'
(...)


Moreover, unittest.mock supports mocking __setformat__() (4 underscores).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46852] Remove float.__get_format__() and float.__set_format__()

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-46656: "Remove the Py_NO_NAN macro: require NAN to build Python 
3.11".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46656] Remove the Py_NO_NAN macro: require NAN to build Python 3.11

[STINNER Victor]

Change by STINNER Victor :


--
components: +Build, Interpreter Core
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
title: Compile fails if Py_NO_NAN is defined -> Remove the Py_NO_NAN macro: 
require NAN to build Python 3.11
versions:  -Python 3.10, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46852] Remove float.__get_format__() and float.__set_format__()

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I would not miss these methods.  Unless Mark says they are needed, +1 for 
removal.

--
nosy: +mark.dickinson, rhettinger, tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46656] Compile fails if Py_NO_NAN is defined

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 1b2611eb0283055835e5df632a7a735db8c894b8 by Victor Stinner in 
branch 'main':
bpo-46656: Remove Py_NO_NAN macro (GH-31160)
https://github.com/python/cpython/commit/1b2611eb0283055835e5df632a7a735db8c894b8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46656] Compile fails if Py_NO_NAN is defined

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 5f8b5e2f2150d2223ff9e286bd146de92ff16865 by Victor Stinner in 
branch 'main':
bpo-46656: Building Python now requires a C11 compiler (GH-31557)
https://github.com/python/cpython/commit/5f8b5e2f2150d2223ff9e286bd146de92ff16865


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46852] Remove float.__get_format__() and float.__set_format__()

[STINNER Victor]

New submission from STINNER Victor :

It has been decided to require IEEE 754 to build Python 3.11:
https://mail.python.org/archives/list/python-...@python.org/thread/J5FSP6J4EITPY5C2UJI7HSL2GQCTCUWN/

At Python startup, _PyFloat_InitState() checks the IEEE 754 format at runtime. 
It can be changed using float.__get_format__() and  float.__set_format__() 
methods.

These methods docstrings say that they only exist to test Python itself:

"You probably don't want to use this function. It exists mainly to be used in 
Python's test suite."

These methods are private and not documented.

I propose to remove them.

Once they will be removed, it will become possible to move the detection of the 
IEEE 754 format in the build step (./configure script) rather than doing the 
detection at runtime (slower). It would remove an "if" in _PyFloat_Pack4() and 
_PyFloat_Pack8(), and allow to specialize these functions for the detected 
format at build time. These functions are used by serialization formats: 
marshal, pickle and struct.

--
components: Interpreter Core
messages: 413943
nosy: vstinner
priority: normal
severity: normal
status: open
title: Remove float.__get_format__() and  float.__set_format__()
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46656] Compile fails if Py_NO_NAN is defined

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +29679
pull_request: https://github.com/python/cpython/pull/31557

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46851] Document multiprocessing.set_forkserver_preload

[Alex Waygood]

Alex Waygood  added the comment:

(3.7 and 3.8 are not accepting patches unless they relate to security issues, 
so I am removing them from the "versions" field.)

--
nosy: +AlexWaygood
versions:  -Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46851] Document multiprocessing.set_forkserver_preload

[Alex Waygood]

Change by Alex Waygood :


--
nosy:  -AlexWaygood

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46623] test_zlib: test_pair() and test_speech128() fail with s390x hardware accelerator

[STINNER Victor]

STINNER Victor  added the comment:

The Python 3.7 package on Fedora also skips these two tests using --ignore 
option of regrtest:

%ifarch s390x
--ignore test_speech128 \
--ignore test_pair \
%endif

* https://src.fedoraproject.org/rpms/python3.7/pull-request/38#request_diff
* 
https://src.fedoraproject.org/rpms/python3.7/blob/rawhide/f/python3.7.spec#_1168

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46623] test_zlib: test_pair() and test_speech128() fail with s390x hardware accelerator

[STINNER Victor]

STINNER Victor  added the comment:

Even if Python 3.9 and 3.10 are also affected, I prefer to not backport the 
change since it's not ideal.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46623] test_zlib: test_pair() and test_speech128() fail with s390x hardware accelerator

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 9475dc0b8d2a0db40278bbcb88a89b1265a77ec9 by Victor Stinner in 
branch 'main':
bpo-46623: Skip two test_zlib tests on s390x (GH-31096)
https://github.com/python/cpython/commit/9475dc0b8d2a0db40278bbcb88a89b1265a77ec9


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[STINNER Victor]

STINNER Victor  added the comment:


New changeset c9c178fdb1af01e441a6c83f3a21a67e5dd9f17d by Victor Stinner in 
branch 'main':
bpo-1635741: test_embed cheks that Python does not leak (GH-31555)
https://github.com/python/cpython/commit/c9c178fdb1af01e441a6c83f3a21a67e5dd9f17d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46851] Document multiprocessing.set_forkserver_preload

[Géry]

Change by Géry :


--
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46835] ImportError: bad magic number in ... does not indicate where is that file located

[Brett Cannon]

Change by Brett Cannon :


--
nosy: +brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45898] ctypes cfield.c defines duplicate ffi_type_* symbols

[Christian Heimes]

Christian Heimes  added the comment:

I have applied the patch to main branch (3.11). The workaround for Windows 
turned out to be more elaborate than initially anticipated. I'm not going to 
backport the fix to 3.10 and 3.9.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions:  -Python 3.10, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45898] ctypes cfield.c defines duplicate ffi_type_* symbols

[Christian Heimes]

Christian Heimes  added the comment:


New changeset 38f331d4656394ae0f425568e26790ace778e076 by Christian Heimes in 
branch 'main':
bpo-45898: Remove duplicate symbols from _ctypes/cfield.c (GH-29791)
https://github.com/python/cpython/commit/38f331d4656394ae0f425568e26790ace778e076


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45107] Improve LOAD_METHOD specialization

[Mark Shannon]

Mark Shannon  added the comment:


New changeset 2a6ece572ca38f989fca66f4c053cb16550bccd4 by Mark Shannon in 
branch 'main':
bpo-45107: Specialize `LOAD_METHOD` for instances with dict. (GH-31531)
https://github.com/python/cpython/commit/2a6ece572ca38f989fca66f4c053cb16550bccd4


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46851] Document multiprocessing.set_forkserver_preload

[Géry]

New submission from Géry :

I have just notice that the multiprocessing.set_forkserver_preload (which 
originates from multiprocessing.forkserver.set_forkserver_preload) is not 
documented:
https://github.com/python/cpython/blob/v3.10.2/Lib/multiprocessing/context.py#L180-L185

--
messages: 413934
nosy: docs@python, maggyero
priority: normal
severity: normal
status: open
title: Document multiprocessing.set_forkserver_preload
versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45991] Improve ambiguous docstrings in pkgutil

[Kevin Hock]

Kevin Hock  added the comment:

At best it is ambiguous, with the class being confused with Str being called 
Path. Looking up "AttributeError: 'PosixPath' object has no attribute 
'startswith'" gives a lot of results for similar issues, so I think the wording 
could be improved.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46833] Installer Wizard is unclear and has redundant settings

[Steve Dower]

Steve Dower  added the comment:

So I'm the "team" that designed it the first time (with input from others, of 
course, but it wasn't designed by a committee or anything), and the primary 
goal was to enable a single-click install for the majority of users, biased 
towards simplicity for users who are less-experienced at administering a 
Windows PC. This was balanced with the fact that many users would already have 
2.7 or 3.4 installed, and those installers had some poor, and irreversible, 
design flaws.

These flaws forced us to install py.exe for all users by default, otherwise the 
majority of people would end up installing it lower on PATH and would never get 
the updates. As a result, it would break, because the launcher included with 
3.4 wouldn't reliably detect 3.5, and we didn't have any option to detect or 
upgrade it.

However, this meant that admin permissions would be required for a default 
install, which is not acceptable. So the front page option to switch that 
setting was added so that non-admin users could easily disable the admin 
requirement.

The PATH option was added under protest, because too many users/teachers rely 
on it for their coursework and wanted it to be easier to find. If we could 
safely enable it by default we would, but it's fundamentally an unsafe 
operation. But it's popular, so that's why it's on the front page.

So unsurprisingly, I like your variant A the best, which is really just 
tweaking the Tools/msi/bundle/Default.thm file a bit. It maintains the short 
path for those who want it (which we believe is the majority of users), and 
hopefully makes it clearer for advanced users.

I dislike variant C because of the second page for simple installs. And for 
variant D we wouldn't offer per-machine installs for the runtime, and we 
wouldn't offer PATH options for the launcher, so it basically boils down to 
making the advanced button harder to find with less explanation.

So if you'd like to propose a PR with changes to the files in Tools/msi/bundle 
to match your actual UIs, feel free. It _should_ automatically build the 
installer if you've modified those files, and you _should_ be able to download 
it for testing, though if not it isn't hard to build locally with 
Tools/msi/build.bat.

(And of course, this would only apply to 3.11, so I've updated the version 
field.)

--
versions: +Python 3.11 -Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46430] intern strings in deepfrozen modules

[Guido van Rossum]

Guido van Rossum  added the comment:

> How it should be handled? Currently PyUnicode_InternInPlace ignores any 
> errors and does not return it. It would be backwards-incompatible to change 
> that, moreover as I explained in 
> https://github.com/python/cpython/pull/30683#discussion_r800648477 
> intern_strings only check if all the items are strings which will be always 
> true in case of deep-freeze so I don't think anything needs to be changed 
> here. I would be interested to know if I am missing something though.

The other functions you are calling *do* return errors. You should not ignore 
those. If any errors are reported the caller can decide what to do (e.g. call 
Py_FatalError().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46430] intern strings in deepfrozen modules

[STINNER Victor]

STINNER Victor  added the comment:

I wrote https://github.com/python/cpython/pull/31555 to make sure that Python 
doesn't leak at Python exit.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46430] intern strings in deepfrozen modules

[Dong-hee Na]

Change by Dong-hee Na :


--
pull_requests: +29678
pull_request: https://github.com/python/cpython/pull/31556

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +29677
pull_request: https://github.com/python/cpython/pull/31555

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46844] Context-based TaskGroup for legacy libraries

[Guido van Rossum]

Guido van Rossum  added the comment:

-1. Libraries that manage their own tasks should continue to do so, until they 
are ready to adopt TaskGroup. Trying to "own" (and wait for) tasks created by a 
library sounds like a recipe for disaster if the library wasn't expecting that.

Do you have a specific use case or scenario that has bitten you in the past? If 
you want to continue to argue for this feature we would need specifics (not a 
link to reams of code but a clear story telling of a problem you've encountered 
in real life in the past that your proposal might solve).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46430] intern strings in deepfrozen modules

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 4dc746310bd37ad6b381f9176acd167d445f4385 by Kumar Aditya in 
branch 'main':
bpo-46430: Fix memory leak in interned strings of deep-frozen modules (GH-31549)
https://github.com/python/cpython/commit/4dc746310bd37ad6b381f9176acd167d445f4385


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45459] Limited API support for Py_buffer

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 042f31da552c19054acd3ef7bb6cfd857bce172b by Victor Stinner in 
branch 'main':
bpo-45459: C API uses type names rather than structure names (GH-31528)
https://github.com/python/cpython/commit/042f31da552c19054acd3ef7bb6cfd857bce172b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46843] PersistentTaskGroup API

[Guido van Rossum]

Guido van Rossum  added the comment:

Could you just have a global task group that owns these long-running tasks? It 
could be embedded in a "toplevel" task that is created using 
asyncio.create_task() (which won't be deprecated). To shut down all 
long-running tasks at the end, just cancel that toplevel task.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3539] Problem with testembed make dependencies in certain circumstances

[Irit Katriel]

Irit Katriel  added the comment:

Closing as suggested by Martin.

--
nosy: +iritkatriel
resolution:  -> wont fix
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45459] Limited API support for Py_buffer

[STINNER Victor]

STINNER Victor  added the comment:


New changeset ec091bd47e2f968b0d1631b9a8104283a7beeb1b by Victor Stinner in 
branch 'main':
bpo-45459: Add pytypedefs.h header file (GH-31527)
https://github.com/python/cpython/commit/ec091bd47e2f968b0d1631b9a8104283a7beeb1b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46622] Add an async variant of lru_cache for coroutines.

[Raymond Hettinger]

Raymond Hettinger  added the comment:

[Andrew Svetlov]
> A third-party library should either copy all these 
> implementation details or import a private function from stdlib 

OrderedDict provides just about everything needed to roll lru cache variants.  
It simply isn't true this can only be done efficiently in the standard library.


[Serhiy]
> it would be simpler to add a decorator which wraps the result 
> of an asynchronous function into an object which can be awaited
> more than once:

This is much more sensible.

> It can be combined with lru_cache and cached_property any third-party
> caching decorator. No access to internals of the cache is needed.

Right.  The premise that this can only be done in the standard library was 
false.

> async_lru_cache() and async_cached_property() can be written 
> using that decorator. 

The task becomes trivially easy :-)  


[Andrew Svetlov]
> Pull Request is welcome!

ISTM it was premature to ask for a PR before an idea has been thought through.  
We risk wasting a user's time or committing too early before simpler, better 
designed alternatives emerge.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46622] Add an async variant of lru_cache for coroutines.

[Joongi Kim]

Change by Joongi Kim :


--
nosy: +achimnol

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46843] PersistentTaskGroup API

[Joongi Kim]

Joongi Kim  added the comment:

@yselivanov @asvetlov
I think this API suggestion would require more refining and discussion in 
depths, and probably it may be better to undergo the PEP writing and review 
process.  Or I might need to have a separate discussion thread somewhere else 
(maybe discuss.python.org?).

Since I'm just a newbie in terms of Python core/stdlib development, could one 
of you guide me with what you think as the right way?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46836] [C API] Move PyFrameObject to the internal C API

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests:  -29656

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46836] [C API] Move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

> PyAPI_FUNC(PyObject *) _PyEval_EvalFrameDefault(PyThreadState *tstate, struct 
> _interpreter_frame *f, int exc);

I created bpo-46850 "[C API] Move _PyEval_EvalFrameDefault() to the internal C 
API" for this issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46850] [C API] Move _PyEval_EvalFrameDefault() to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-46836: "[C API] Move PyFrameObject to the internal C API".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46850] [C API] Move _PyEval_EvalFrameDefault() to the internal C API

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +29676
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31532

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46850] [C API] Move _PyEval_EvalFrameDefault() to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

typo: "I propose to move most _PyEval private functions to the internal C API 
to clarify that they must *NOT* be used." :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46850] [C API] Move _PyEval_EvalFrameDefault() to the internal C API

[STINNER Victor]

New submission from STINNER Victor :

In Python 3.10, _PyEval_EvalFrameDefault() has the API:

PyObject* _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int 
throwflag);

In Python 3.11, bpo-44590 (commit ae0a2b756255629140efcbe57fc2e714f0267aa3 
"Lazily allocate frame objects (GH-27077)") changed it to:

PyObject* _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame 
*frame, int throwflag);

Problem: InterpreterFrame is part of the internal C API.

By the way, PyInterpreterState.eval_frame type (_PyFrameEvalFunction) also 
changed. This field type already changed in Python 3.9:

* ``PyInterpreterState.eval_frame`` (:pep:`523`) now requires a new mandatory
  *tstate* parameter (``PyThreadState*``).
  (Contributed by Victor Stinner in :issue:`38500`.)

Maybe the Python 3.11 change should be documented in What's New in Python 3.11, 
as it was in What's New in Python 3.9.


I propose to move most _PyEval private functions to the internal C API to 
clarify that they must be used.

--
components: C API
messages: 413918
nosy: vstinner
priority: normal
severity: normal
status: open
title: [C API] Move _PyEval_EvalFrameDefault() to the internal C API
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46836] [C API] Move PyFrameObject to the internal C API

[Petr Viktorin]

Petr Viktorin  added the comment:

OK, looking at it more carefully, it makes sense to do the change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46823] Add LOAD_FAST__LOAD_ATTR_INSTACE_VALUE combined opcode

[Mark Shannon]

Mark Shannon  added the comment:


New changeset a52d2528a405c1e2bfeb6470cb3313a5338dc45f by Dennis Sweeney in 
branch 'main':
bpo-46823: Implement LOAD_FAST__LOAD_ATTR_INSTANCE_VALUE superinstruction 
(GH-31484)
https://github.com/python/cpython/commit/a52d2528a405c1e2bfeb6470cb3313a5338dc45f


--
nosy: +Mark.Shannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46659] Deprecate locale.getdefaultlocale() function

[Marc-Andre Lemburg]

Marc-Andre Lemburg  added the comment:

Thanks, Victor.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46843] PersistentTaskGroup API

[Joongi Kim]

Joongi Kim  added the comment:

Some search results from cs.github.com with the input "asyncio task weakset", 
which may be replaced/simplified with PersistentTaskGroup:

- 
https://github.com/Textualize/textual/blob/38efc821737e3158a8c4c7ef8ecfa953dc7c0ba8/src/textual/message_pump.py#L43
- 
https://github.com/aiokitchen/aiomisc/blob/59abd4434e6d134537490db699f89a51df1e6bbc/aiomisc/entrypoint.py#L132
- 
https://github.com/anki/cozmo-python-sdk/blob/dd29edef18748fcd816550469195323842a7872e/src/cozmo/event.py#L102
- 
https://github.com/aio-libs/aiohttp-sse/blob/db7d49bfc8a4907d9a8e7696a85b9772e1c550eb/examples/graceful_shutdown.py#L50
- 
https://github.com/mosquito/aiormq/blob/9c6c0dfc771ea8f6e79b7532177640c2692c640f/aiormq/base.py#L18
https://github.com/mars-project/mars/blob/d1a14cc4a1cb96e40e1d81eef38113b0c9221a84/mars/lib/aio/_runners.py#L57

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46843] PersistentTaskGroup API

[Joongi Kim]

Joongi Kim  added the comment:

Example use cases:

* Implement an event iteration loop to fetch events and dispatch the handlers 
depending on the event type (e.g., WebSocket connections, message queues, etc.)
  - https://github.com/aio-libs/aiohttp/pull/2885
  - https://github.com/lablup/backend.ai-manager/pull/533
  - https://github.com/lablup/backend.ai-agent/pull/341
  - https://github.com/lablup/backend.ai-agent/pull/331
* Separate monitoring of event handler tasks by the event sources.
  - aiomonitor extension to count currently ongoing tasks and extract the most 
frequent task stack frames
* Separate the fallback exception handlers by each persistent task group, 
instead of using the single "global" event loop exception handler.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46849] Memory problems detected using Valgrind

[tongxiaoge]

New submission from tongxiaoge :

Reproduction steps:
1. Execute command: iotop -b -n 10 &
2. Execute the command in another session: valgrind /usr/sbin/iotop -b -n 5 > 
iotop_test

The output information is as follows:
[root@openEuler ~]# valgrind /usr/sbin/iotop -b -n 5 > iotop_test
==13750== Memcheck, a memory error detector
==13750== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==13750== Using Valgrind-3.16.0 and LibVEX; rerun with -h for copyright info
==13750== Command: /usr/sbin/iotop -b -n 5
==13750==
==13750== Conditional jump or move depends on uninitialised value(s)
==13750== at 0x49B2C40: PyUnicode_Decode (unicodeobject.c:3488)
==13750== by 0x49B335B: unicode_new (unicodeobject.c:15465)
==13750== by 0x4982C07: type_call (typeobject.c:1014)
==13750== by 0x492CA17: _PyObject_MakeTpCall (call.c:191)
==13750== by 0x48F1863: _PyObject_VectorcallTstate (abstract.h:116)
==13750== by 0x48F1863: _PyObject_VectorcallTstate (abstract.h:103)
==13750== by 0x48F1863: PyObject_Vectorcall (abstract.h:127)
==13750== by 0x48F1863: call_function (ceval.c:5075)
==13750== by 0x48F1863: _PyEval_EvalFrameDefault (ceval.c:3518)
==13750== by 0x48EAEE7: _PyEval_EvalFrame (pycore_ceval.h:40)
==13750== by 0x48EAEE7: function_code_fastcall (call.c:330)
==13750== by 0x492CBE7: _PyObject_FastCallDictTstate (call.c:118)
==13750== by 0x492CEEB: _PyObject_Call_Prepend (call.c:489)
==13750== by 0x498A007: slot_tp_init (typeobject.c:6964)
==13750== by 0x4982C4F: type_call (typeobject.c:1026)
==13750== by 0x492CA17: _PyObject_MakeTpCall (call.c:191)
==13750== by 0x48F1863: _PyObject_VectorcallTstate (abstract.h:116)
==13750== by 0x48F1863: _PyObject_VectorcallTstate (abstract.h:103)
==13750== by 0x48F1863: PyObject_Vectorcall (abstract.h:127)
==13750== by 0x48F1863: call_function (ceval.c:5075)
==13750== by 0x48F1863: _PyEval_EvalFrameDefault (ceval.c:3518)
==13750==
==13751== Warning: invalid file descriptor 1024 in syscall close()
==13751== Warning: invalid file descriptor 1025 in syscall close()
==13751== Warning: invalid file descriptor 1026 in syscall close()
==13751== Warning: invalid file descriptor 1027 in syscall close()
==13751== Use --log-fd= to select an alternative log fd.
==13751== Warning: invalid file descriptor 1028 in syscall close()
==13751== Warning: invalid file descriptor 1029 in syscall close()
==13752== Warning: invalid file descriptor 1024 in syscall close()
==13752== Warning: invalid file descriptor 1025 in syscall close()
==13752== Warning: invalid file descriptor 1026 in syscall close()
==13752== Warning: invalid file descriptor 1027 in syscall close()
==13752== Use --log-fd= to select an alternative log fd.
==13752== Warning: invalid file descriptor 1028 in syscall close()
==13752== Warning: invalid file descriptor 1029 in syscall close()
==13750==
==13750== HEAP SUMMARY:
==13750== in use at exit: 1,069,715 bytes in 10,017 blocks
==13750== total heap usage: 589,638 allocs, 579,621 frees, 128,672,782 bytes 
allocated
==13750==
==13750== LEAK SUMMARY:
==13750== definitely lost: 0 bytes in 0 blocks
==13750== indirectly lost: 0 bytes in 0 blocks
==13750== possibly lost: 1,042,483 bytes in 9,894 blocks
==13750== still reachable: 27,232 bytes in 123 blocks
==13750== suppressed: 0 bytes in 0 blocks
==13750== Rerun with --leak-check=full to see details of leaked memory
==13750==
==13750== Use --track-origins=yes to see where uninitialised values come from
==13750== For lists of detected and suppressed errors, rerun with: -s
==13750== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

Software version: python3-3.9.9 & python3.9.10  iotop-6.0

The above stack information is in Python 3.9.10 and this problem cannot be 
reproduced in Python 3.7.


So is it a python3  problem or an iotop problem?  How to fix it.

--
messages: 413912
nosy: sxt1001
priority: normal
severity: normal
status: open
title: Memory problems detected using Valgrind
type: resource usage
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43557] Deprecate getdefaultlocale(), getlocale() and normalize() functions

[STINNER Victor]

STINNER Victor  added the comment:

> Deprecating these functions is complex. I prefer to start with the least 
> controversial part: bpo-46659.

locale.getdefaultlocale() is now deprecated in Python 3.11.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46659] Deprecate locale.getdefaultlocale() function

[STINNER Victor]

STINNER Victor  added the comment:

locale.getdefaultlocale() is now deprecated.

calendar now uses locale.setlocale() instead of locale.getdefaultlocale().

The ANSI code page alias to MBCS now has better tests and better comments.

Thanks Eryk Sun for your very useful feedback!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Remove asyncore, asynchat and smtpd modules

[STINNER Victor]

STINNER Victor  added the comment:

Follow-up: PEP 594 schedules the removal of these 4 modules in Python 3.12, but 
the PEP remains a draft.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46843] PersistentTaskGroup API

[Joongi Kim]

Joongi Kim  added the comment:

I think people may ask "why in stdlib?".

My reasons are:
 - We are adding new asyncio APIs in 3.11 such as TaskGroup, so I think it is a 
good time to add another one, as long as it does not break existing stuffs.
 - I believe that long-running task sets are equally representative use-case 
for real-world asyncio applications, particularly for servers.  Why not to have 
intrinsic support for them?
 - PersistentTaskGroup is going to be universally adopted throughout my 70+K 
LoC asyncio codebase, for instance, in every aiohttp.Application context, 
plugin contexts and modules, etc.

Of course, the name "PersistentTaskGroup" may look quite long, and I'm 
completely open with alternative suggestions.  I also welcome suggestions on 
changes to its functional semantics based on your experience and knowledge.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46659] Deprecate locale.getdefaultlocale() function

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 4fccf910738d1442852cb900747e6dccb8fe03ef by Victor Stinner in 
branch 'main':
bpo-46659: Enhance LocaleTextCalendar for C locale (GH-31214)
https://github.com/python/cpython/commit/4fccf910738d1442852cb900747e6dccb8fe03ef


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43333] utf8 in BytesGenerator

[Chris]

Chris  added the comment:

found this issue while googling the error. Also having the same problem with 
as_bytes() breaking on non-ascii characters. 

I've tried policy=policy.default.clone(utf8=True) but it gives the same error. 

My sample.py file attached contains a string sample email - which has a 
character \u200d (https://unicode-table.com/en/200D/) - Zero Width Joiner in 
the body. 

UnicodeEncodeError: 'ascii' codec can't encode character '\u200d' in position 
70: ordinal not in range(128)

Any assistance on what I can do to solve it would be great. It seems I can 
parse 99% of the emails I've tried but this one has me confused.

--
nosy: +chrisstaunton1990
Added file: https://bugs.python.org/file50641/sample.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46843] PersistentTaskGroup API

[Joongi Kim]

Joongi Kim  added the comment:

So I have more things in mind.

Basically PersistentTaskGroup resemble TaskGroup in that:
 - It has the same "create_task()" method.
 - It has an explicit "cancel()" or "shutdown()" method.
 - Exiting of the context manager means that all tasks of it have either 
completed or cancelled.

TaskGroup is intended to be used for a short-lived set of tasks, while 
PersistentTaskGroup is intended for a long-running set of tasks though 
individual tasks may be short-lived.  Thus, adding globally accessible 
monitoring facility for plain TaskGroup would not be that useful.  In contrast, 
it is super-useful to have a monitoring feature in PersistentTaskGroup!

In aiomonitor, we can enumerate the currently running asyncio tasks by reading 
asyncio.Task.all_tasks().  This has saved my life several times when debugging 
real-world server applications.  I think we can go further by having 
asyncio.PersistentTaskGroup.all_task_groups() which works in the same way.  If 
we make different modules and libraries to use different persistent task 
groups, then we could keep track of their task statistics separately.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46844] Context-based TaskGroup for legacy libraries

[Joongi Kim]

Joongi Kim  added the comment:

Ok, let me be clear: Patching asyncio.create_task() to support this opt-in 
contextual task group binding is not an ultimate goal of this issue.  If it 
becomes possible to override/extend the task factory at runtime with any event 
loop implementation, then it's ok to implement this feature request as a 
3rd-party library.  I also don't want to bloat the stdlib with version-specific 
branches, if there are alternative ways to achieve the same goal.  I just 
wanted to check out your opinons and potential alternative approaches to 
implement it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46848] Use optimized string search function in mmap.find()

[Stefan Tatschner]

Change by Stefan Tatschner :


--
keywords: +patch
pull_requests: +29675
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31554

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46848] Use optimized string search function in mmap.find()

[Stefan Tatschner]

Stefan Tatschner  added the comment:

Sorry, I mean memmem(3). :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46848] Use optimized string search function in mmap.find()

[Stefan Tatschner]

New submission from Stefan Tatschner :

The mmap.find() in  function uses a naive loop to search string matches. This 
can be optimized “for free” by using libc's memmap(3) function instead.

The relevant file is Modules/mmapmodule.c, the relevant function is 
mmap_gfind().

--
messages: 413902
nosy: rumpelsepp
priority: normal
severity: normal
status: open
title: Use optimized string search function in mmap.find()

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46748] Python.h includes stdbool.h

[Petr Viktorin]

Change by Petr Viktorin :


--
keywords: +patch
pull_requests: +29674
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31553

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46841] Inline bytecode caches

[Mark Shannon]

Change by Mark Shannon :


--
pull_requests: +29673
pull_request: https://github.com/python/cpython/pull/31552

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46841] Inline bytecode caches

[Mark Shannon]

Mark Shannon  added the comment:

We need to decide what to do about dis.

I don't think we should have a `show_cache` option, as the caches are 
meaningless junk without quickening (maybe we should drop the CACHE opcode, and 
just use zeroes).

Instead we should have a `show_quickened` option, to show the quickened form, 
which we need to make clear is very much implementation defined.
E.g. Cinder might show the machine code as well.

That way, we can present the cache information as extra data on the quickened 
form, rather than junk instructions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46846] functools.partial objects should set __signature__ and _annotations__

[Alex Waygood]

Change by Alex Waygood :


--
nosy: +AlexWaygood

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46847] functools.update_wrapper doesn't understand partial objects and annotations

[Alex Waygood]

Change by Alex Waygood :


--
nosy: +AlexWaygood, Jelle Zijlstra, sobolevn

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44886] asyncio: create_datagram_endpoint() does not return a DatagramTransport

[Andrew Svetlov]

Change by Andrew Svetlov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37355] SSLSocket.read does a GIL round-trip for every 16KB TLS record

[Safihre]

Safihre  added the comment:

Could the new PR be reviewed? Thank you!
https://github.com/python/cpython/pull/31492
Documentation still needs updating, but would like feedback.

PS: Why not enable the setting the GitHub Actions workflow only need to be 
approved for new GitHub accounts instead of for *all* new contributors?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46844] Context-based TaskGroup for legacy libraries

[Andrew Svetlov]

Andrew Svetlov  added the comment:

I personally don't think that the described opt-in trick should be a part of 
asyncio.
A third-party library that patches asyncio.create_task() can be a useful thing 
though during the transition period.
We even cannot deprecate asyncio.create_task() right now: the minimal supported 
Python 3.7 thas now alternative, writing cross-version code is overcomplicated.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46847] functools.update_wrapper doesn't understand partial objects and annotations

[Larry Hastings]

New submission from Larry Hastings :

functools.update_wrapper currently copies over every attribute listed in the 
"assigned" parameter, which defaults to WRAPPER_ASSIGNMENTS, which means it 
copies the wrapped function's __annotations__ to the wrapper.  This is slightly 
wrong if the wrapper occludes an annotated parameter:

def foo(a: int, b: str, c: float):
print(a, b, c)

import functools

foo_a = functools.partial(foo, 3)
functools.update_wrapper(foo_a, foo)

print(foo_a.__annotations__)

In this case, foo_a.__annotations__ contains an annotation for a parameter 
named "a", even though foo_a doesn't have a parameter named "a".

This problem occurred to me just after I filed #46846; the two issues are 
definitely related.

--
components: Library (Lib)
messages: 413898
nosy: larry, rhettinger
priority: normal
severity: normal
stage: test needed
status: open
title: functools.update_wrapper doesn't understand partial objects and 
annotations
type: behavior
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46846] functools.partial objects should set __signature__ and _annotations__

[Larry Hastings]

New submission from Larry Hastings :

I ran across an interesting bug in issue #46761.  If you call 
functools.update_wrapper on a functools.partial object, inspect.signature will 
return the wrong (original) signature for the partial object.

We're still figuring that one out.  And, of course, it's telling that the bug 
has been there for a long time.  I suspect this isn't something that has 
inconvenienced a lot of people.

But: I suggest that it's time functools.partial participated in signature 
stuff.  Specifically, I think functools.partial should generate a new and 
correct __signature__ for the partial object.  And I propose it should also 
generate a new and correct __annotations__ for the partial, by removing all 
entries for parameters that are filled in by the partial object.

Right now inspect.signature has special support for functools.partial objects.  
It finds the underlying function, and .  Which means there's code in both 
modules that has to understand the internals of partial objects.  Just from a 
code hygiene perspective, it'd be better if all that logic lived under 
functools.

I wonder if functools.partial objects should generally do a better job of 
impersonating the original function.  Should they adopt the same __name__?  
__file__?  __qualname__?  My intuition is, it'd be nice if it did.  But I might 
be forgetting something important.

(I suspect everything I said about functools.partial also applies to 
functools.partialmethod.)

--
components: Library (Lib)
messages: 413897
nosy: larry, rhettinger
priority: normal
severity: normal
stage: test needed
status: open
title: functools.partial objects should set __signature__ and _annotations__
type: enhancement
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46761] functools.update_wrapper breaks the signature of functools.partial objects

[Larry Hastings]

Larry Hastings  added the comment:

Okay, so, I considered the problem for a while, and I have a reasonable theory 
about what follow_wrapper_chains was for in the first place.

If you have a generic decorator, like functools.cache(), it usually looks like 
this:

  def my_decorator(fn):
def generic_version(*args, **kwargs):
  args, kwargs = do_something(args, kwargs)
  return fn(*args, **kwargs)
return generic_version

  @my_decorator
  def add_five(i):
return i+5

If you take the signature of add_five, you'd get (*args, **kwargs), because 
that's the signature of the wrapper function returned by the decorator.  The 
decorator doesn't change the parameters of the function, but because of how 
decorators work it can occlude the proper function signature.  In that 
instance, follow_wrapper_chains does the right thing, and as a result you get a 
precise function signature.

(Of course, it would do the wrong thing if your hand-written decorator *also* 
behaved like a partial application, adding in its own hard-coded arguments, so 
that the resulting function signature changed.)

Still, obviously it's doing the wrong thing when it comes to 
functools.partial() functions.

My suspicion is that I'm the rare individual who actually uses update_wrapper 
on a functools.partial object.  So maybe we have the situation here where, 
yeah, it's a bug, and we can fix it without causing further breakage.

Maybe we can loop in someone who works on a popular runtime function 
introspection library (FastAPI, Pydantic) to see if they have any take on it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12165] [doc] clarify documentation of nonlocal

[Stanley]

Change by Stanley :


--
keywords: +patch
nosy: +slateny
nosy_count: 6.0 -> 7.0
pull_requests: +29672
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/31551

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46430] intern strings in deepfrozen modules

[Kumar Aditya]

Kumar Aditya  added the comment:

> Okay, let's change the error handling. @Kumar, can you handle that?

How it should be handled? Currently PyUnicode_InternInPlace ignores any errors 
and does not return it. It would be backwards-incompatible to change that, 
moreover as I explained in 
https://github.com/python/cpython/pull/30683#discussion_r800648477 
intern_strings only check if all the items are strings which will be always 
true in case of deep-freeze so I don't think anything needs to be changed here. 
I would be interested to know if I am missing something though.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com