[issue40116] Regression in memory use of shared key dictionaries for "compact dicts"

[Inada Naoki]

Inada Naoki  added the comment:

PyDict_Keys(), PyDict_Values(), and PyDict_Items() don't respect insertion 
order too.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46430] intern strings in deepfrozen modules

[Kumar Aditya]

Kumar Aditya  added the comment:

I have created a PR to fix the memory leak, See 
https://github.com/python/cpython/pull/31549

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46845] dict: Use smaller entry for Unicode-key only dict.

[Inada Naoki]

New submission from Inada Naoki :

Currently, PyDictKeyEntry is 24bytes (hash, key, and value).

We can drop the hash from entry when all keys are unicode, because unicode 
objects caches hash already.

This will cause some performance regression on microbenchmark because dict need 
one more indirect access to compare hash value.

On the other hand, this will reduce some RAM usage. Additionally, unlike 
docstrings and annotations, this includes much **hot** RAM. It will make Python 
more cache efficient.

This is work in progress code: https://github.com/methane/cpython/pull/43
pypeformance result is in the PR too.

--
components: Interpreter Core
messages: 413892
nosy: Mark.Shannon, methane, rhettinger
priority: normal
severity: normal
status: open
title: dict: Use smaller entry for Unicode-key only dict.
type: performance
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40116] Regression in memory use of shared key dictionaries for "compact dicts"

[Inada Naoki]

Change by Inada Naoki :


--
pull_requests: +29671
pull_request: https://github.com/python/cpython/pull/31550

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46844] Context-based TaskGroup for legacy libraries

[Joongi Kim]

Joongi Kim  added the comment:

Ah, and this use case also requires that TaskGroup should have an option like 
`return_exceptions=True` which makes it not to cancel sibling tasks upon 
unhandled exceptions, as I suggested in PersistentTaskGroup (bpo-46843).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46430] intern strings in deepfrozen modules

[Kumar Aditya]

Change by Kumar Aditya :


--
pull_requests: +29670
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31549

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46844] Context-based TaskGroup for legacy libraries

[Joongi Kim]

Joongi Kim  added the comment:

An example would be like:

tg = asyncio.TaskGroup()
...
async with tg:
  with asyncio.TaskGroupBinder(tg):  # just a hypothetical API
asyncio.create_task(...) # equivalent to tg.create_task(...)
await some_library.some_work()   # all tasks are bound to tg
  asyncio.create_task(...)   # fire-and-forget (not bound to tg)

If TaskGroup supports enumeration/counting of its own tasks and asyncio allows 
enumeration of TaskGroups just like asyncio.Task.all_tasks(), we could extend 
aiomonitor to provide per-taskgroup statistics.

In my projects, we have multiple cases to find and fix bugs in customer sites 
using aiomonitor and I'm willing to improve aiomonitor to support task groups 
as well.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46844] Context-based TaskGroup for legacy libraries

[Joongi Kim]

Joongi Kim  added the comment:

My propsal is to opt-in the taskgroup binding for asyncio.create_task() under a 
specific context, not changing the defautl behavior.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46844] Context-based TaskGroup for legacy libraries

[Andrew Svetlov]

Andrew Svetlov  added the comment:

-1

Now bare `create_task()` does fire-and-forget action.
After the proposed change it will fail loudly.
Even if this behavior is better it is not backward compatible.
People start blaming and asking "how to return everything back?"

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30677] [doc] mention that os.mkdir() raises FileNotFound if path does not exist

[Stanley]

Change by Stanley :


--
keywords: +patch
nosy: +slateny
nosy_count: 3.0 -> 4.0
pull_requests: +29669
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31548

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46844] Context-based TaskGroup for legacy libraries

[Joongi Kim]

Joongi Kim  added the comment:

It is also useful to write debugging/monitoring codes for asyncio applications. 
 For instance, we could "group" tasks from different libraries and count them.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46844] Context-based TaskGroup for legacy libraries

[Joongi Kim]

Joongi Kim  added the comment:

Conceptually it is similar to replace malloc using LD_PRELOAD or 
LD_LIBRARY_PATH manipulation.  When I cannot modify the executable/library 
binaries, this allows replacing the functionality of specific functions.

If we could assign a specific (persistent) task group to all asyncio tasks 
spawned by a black-box code (when the black-box itself does not use task 
groups), we could achieve the full application-level transparency on the timing 
of task cancellation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46844] Context-based TaskGroup for legacy libraries

[Joongi Kim]

Joongi Kim  added the comment:

The main benefit is that any legacy code that I cannot modify can be upgraded 
to TaskGroup-based codes, which offers a better machinary for exception 
handling and propagation.

There may be different ways to visit this issue: allow replacing the task 
factory in asyncio at runtime.  Then I could just implement my own snippet to 
transfer the "ownership" of the task to a specific task group.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40116] Regression in memory use of shared key dictionaries for "compact dicts"

[Inada Naoki]

Inada Naoki  added the comment:

I found regression caused by GH-28520.

```
class C:
def __init__(self, n):
if n:
self.a = 1
self.b = 2
self.c = 3
else:
self.c = 1
self.b = 2
self.a = 3


o1 = C(True)
o2 = C(False)
print(o2.__dict__)  # {'c': 1, 'b': 2, 'a': 3}

d1 = {}
d1.update(o2.__dict__)  # {'a': 3, 'b': 2, 'c': 1}
print(d1)
```

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45991] Improve ambiguous docstrings in pkgutil

[Stanley]

Stanley  added the comment:

Could you expand a bit on why 'list of paths' in pkgutil is understood by 
default to be 'list of PosixPath paths'? I would interpret it by default to be 
string paths if I saw it somewhere without context

--
nosy: +slateny

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46844] Context-based TaskGroup for legacy libraries

[Yury Selivanov]

Yury Selivanov  added the comment:

> I believe that this approach would allow more control over tasks implicitly 
> spawned by 3rd-party libraries that cannot control.

Please elaborate. I'm not sure what are the benefits of this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46844] Context-based TaskGroup for legacy libraries

[Joongi Kim]

New submission from Joongi Kim :

Along with bpo-46843 and the new asyncio.TaskGroup API, I would like to suggest 
addition of context-based TaskGroup feature.

Currently asyncio.create_task() just creates a new task directly attached to 
the event loop, while asyncio.TaskGroup.create_task() creates a new task 
managed by the TaskGroup instance.

It would be ideal to all existing asyncio codes to migrate to use TaskGroup, 
but this is impractical.

An alternative approach is to implicitly bind asyncio.create_task() under a 
specific context to a specific task group, probably using contextvars.

I believe that this approach would allow more control over tasks implicitly 
spawned by 3rd-party libraries that cannot control.

How about your thoughts?

--
components: asyncio
messages: 413881
nosy: achimnol, asvetlov, gvanrossum, yselivanov
priority: normal
severity: normal
status: open
title: Context-based TaskGroup for legacy libraries
type: enhancement
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34429] [doc] On Windows tempfile.TemporaryFile behaves like NamedTemporaryFile

[Stanley]

Change by Stanley :


--
keywords: +patch
nosy: +slateny
nosy_count: 4.0 -> 5.0
pull_requests: +29668
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31547

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46843] PersistentTaskGroup API

[Joongi Kim]

New submission from Joongi Kim :

I'm now tracking the recent addition and discussion of TaskGroup and 
cancellation scopes. It's interesting! :)

I would like to suggest to have a different mode of operation in 
asyncio.TaskGroup, which I named "PersistentTaskGroup".

AFAIK, TaskGroup targets to replace asyncio.gather, ensuring completion or 
cancellation of all tasks within the context manager scope.

I believe that a "safe" asyncio application should consist of a nested tree of 
task groups, which allow us to explicitly state when tasks of different 
purposes and contexts terminate.  For example, a task group for database 
transactions should be shutdown before a task group for HTTP handlers is 
shutdown.

To this end, in server applications with many sporadically spawned tasks 
throughout the whole process lifetime, there are different requirements for a 
task group that manages such task sets.  The tasks should *not* be cancelled 
upon the unhandled exceptions of sibling tasks in the task group, while we need 
an explicit "fallback" exception handler for those (just like 
"return_exceptions=True" in asyncio.gather).  The tasks belong to the task 
group but their references should not be kept forever to prevent memory leak 
(I'd suggest using weakref.WeakSet).  When terminating the task group itself, 
the ongoing tasks should be cancelled.  The cancellation process upon 
termination may happend in two phases: cancel request with initial timeout + 
additional limited waiting of cancellations.  (This is what Guido has mentioned 
in the discussion in bpo-46771.)

An initial sketch of PersistentTaskGroup is on aiotools:
https://github.com/achimnol/aiotools/blob/main/src/aiotools/ptaskgroup.py
Currently has no two-phase cancellation because it would require Python 3.11 
with asyncio.Task.uncancel().

As Andrew has left a comment 
(https://github.com/achimnol/aiotools/issues/29#issuecomment-997437030), I 
think it is the time to revisit the concrete API design and whether to include 
PersistentTaskGroup in the stdlib or not.

--
components: asyncio
messages: 413880
nosy: achimnol, asvetlov, gvanrossum, yselivanov
priority: normal
severity: normal
status: open
title: PersistentTaskGroup API
type: enhancement
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46839] Process finished with exit code -1073741819 (0xC0000005)

[Eryk Sun]

Eryk Sun  added the comment:

I tried to get more information for you by installing 2.7.11 (64-bit because of 
the given fault) and unassembling python27.dll at the fault offset. But it 
doesn't help. Whatever the bug is, it ended up jumping to an address that's in 
the middle of an instruction. This final bit of nonsense is what killed the 
process with an access violation, but it's not directly related to the problem.

Here's the address that triggered the access violation:

0:000> ? python27 + a4f81
Evaluate expression: 1368149889 = `518c4f81

The nearest function is rawiobase_readall():

0:000> ln python27 + a4f81
(`518c4f10)   python27!rawiobase_readall+0x71   |
(`518c51a0)   python27!resize_buffer

But 0x518c4f81 isn't the address of an instruction in this function:

0:000> u python27!rawiobase_readall python27!rawiobase_readall+80
python27!rawiobase_readall:
[...]
`518c4f7f 48894318mov qword ptr [rbx+18h],rax
`518c4f83 48894310mov qword ptr [rbx+10h],rax
`518c4f87 48894320mov qword ptr [rbx+20h],rax
`518c4f8b 48837f10fe  cmp qword ptr 
[rdi+10h],0FFFEh
`518c4f90 4c89642440  mov qword ptr [rsp+40h],r12

It's inside of the first MOV instruction.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46842] py to pyc location mapping with sys.pycache_prefix isn't 1-to-1 on Windows

[benrg]

New submission from benrg :

`importlib._bootstrap_external` contains this comment:

# We need an absolute path to the py file to avoid the possibility of
# collisions within sys.pycache_prefix [...]
# [...] the idea here is that if we get `Foo\Bar`, we first
# make it absolute (`C:\Somewhere\Foo\Bar`), then make it root-relative
# (`Somewhere\Foo\Bar`), so we end up placing the bytecode file in an
# unambiguous `C:\Bytecode\Somewhere\Foo\Bar\`.

The code follows the comment, but doesn't achieve the goal: 
`C:\Somewhere\Foo\Bar` and `D:\Somewhere\Foo\Bar` collide. There is also no 
explicit handling of UNC paths, with the result that `\\Somewhere\Foo\Bar` maps 
to the same location.

I think that on Windows the code should use a mapping like

C:\Somewhere\Foo\Bar  ==>  C:\Bytecode\C\Somewhere\Foo\Bar
D:\Somewhere\Foo\Bar  ==>  C:\Bytecode\D\Somewhere\Foo\Bar
\\Somewhere\Foo\Bar   ==>  C:\Bytecode\UNC\Somewhere\Foo\Bar

The lack of double-slash prefix handling also matters on Unixy platforms that 
give it a special meaning. Cygwin is probably affected by this. I don't know 
whether there are any others.

--
components: Library (Lib), Windows
messages: 413878
nosy: benrg, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: py to pyc location mapping with sys.pycache_prefix isn't 1-to-1 on 
Windows
type: behavior
versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46826] prefixes argument to site.getsitepackages() missing documentation

[Addison Snelling]

Change by Addison Snelling :


--
keywords: +patch
pull_requests: +29667
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31546

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13475] Add '--mainpath'/'--nomainpath' command line options to override sys.path[0] initialisation

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-16202 "sys.path[0] security issues".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46761] functools.update_wrapper breaks the signature of functools.partial objects

[Ofey Chan]

Ofey Chan  added the comment:

Thank you Larry.  It can never be too careful to deal with language issues!

> why the code behaves like this--is this behavior a genuine bug, or is it 
> actually a bugfix for some worse behavior?

In my view, there's rather an concept needing clarify, than a genuine bug.  I 
look into the file blame, and it shows the processing order around 
`follow_wrapper_chains` might be carefully arranged...

- The `follow_wrapper_chains` functionality is added in issue 13266: 
https://bugs.python.org/issue13266
- When `signature()` first implemented, `__wrapped__` is handled then 
`partial`, this order never changed.
   - Issue #15008: Implement PEP 362 "Signature Objects": 
https://github.com/python/cpython/commit/7c7cbfc00fc8d655fc267ff57f8084357858b1db

> will fixing the bug cause problems for Python users? and if so, can we still 
> fix the bug while mitigating the damage to people who are unfortunately 
> depending on the bug?

It's really a heavy responsibility!  Slow down is right... A clear explaination 
and plan should be constructed before any action is taken.

So I may study PEP 362 to get some context, about why the code is this way.  
And I'm wondering what else can I do?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46841] Inline bytecode caches

[Brandt Bucher]

Change by Brandt Bucher :


--
keywords: +patch
pull_requests: +29666
pull_request: https://github.com/python/cpython/pull/31543

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46841] Inline bytecode caches

[Brandt Bucher]

New submission from Brandt Bucher :

...as discussed in https://github.com/faster-cpython/ideas/discussions/263.

My plan is for this initial PR to lay the groundwork, then to work on porting 
over the existing opcode caches one-by-one. Once that's done, we can clean up 
lots of the "old" machinery.

--
assignee: brandtbucher
components: Interpreter Core
messages: 413875
nosy: Mark.Shannon, brandtbucher
priority: normal
severity: normal
stage: patch review
status: open
title: Inline bytecode caches
type: performance
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes

[Guido van Rossum]

Guido van Rossum  added the comment:


New changeset 7fce1063b6e5a366f8504e039a8ccdd6944625cd by Tin Tvrtković in 
branch 'main':
bpo-46771: Implement task cancel requests counter (GH-31513)
https://github.com/python/cpython/commit/7fce1063b6e5a366f8504e039a8ccdd6944625cd


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46754] Improve Python Language Reference based on [Köhl 2020]

[Jelle Zijlstra]

Jelle Zijlstra  added the comment:

Some specific points from the document:

"While the PLR explicitly states that “x < y calls x.__lt__(y)” [20, 3.3.1.] 
this is actually false." They had to read the implementation to actually figure 
out how this works.

"If no expression is provided, the PLR states that “the last exception that was 
active in the current scope” should be re-raised. Unfortunately, the PLR stays 
unspecific on what it means for an exception to be “the last exception that was 
active in the current scope.”"

"The description of this name binding and resolution process in the PLR [20, 
4.2.] is unfortunately not particularly clear"

"While generator objects seem not to be specified as part of the PLR, the 
documentation of the inspect module5 describes them."

(I didn't read the whole thing, but searched for "PLR".)

Here's a few similar things I noticed while reading through the reference 
recently:

- 
https://docs.python.org/3.10/reference/datamodel.html#the-standard-type-hierarchy
 is supposed to be a list of primitive types but has a few problems:
  - Lists "numbers.Number" as a standard type, when it's an ABC that's really 
not part of the core language
  - Lists "I/O objects" as a kind of standard object, but there are many kinds 
of file-like objects, and I'd argue they're not part of the core language, but 
of the standard library.
  - On the other hand, some builtin types (range, map, filter, zip) are not 
listed.
- https://docs.python.org/3.10/reference/expressions.html#subscriptions doesn't 
actually give the rules about when __class_getitem__ is called vs. __getitem__. 
That's in 
https://docs.python.org/3.10/reference/datamodel.html#class-getitem-versus-getitem.
- The text at https://docs.python.org/3.10/reference/expressions.html#calls 
doesn't cover positional-only parameters.

--
nosy: +Jelle Zijlstra

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46430] intern strings in deepfrozen modules

[Dong-hee Na]

Change by Dong-hee Na :


--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13475] Add '--mainpath'/'--nomainpath' command line options to override sys.path[0] initialisation

[STINNER Victor]

STINNER Victor  added the comment:

The problem of long option names is that they cannot be used in a Unix shebang: 
"#!/usr/bin/python3 --long-option".

I propose to add the -P option to not add sys.path[0]: I wrote GH-31542.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13475] Add '--mainpath'/'--nomainpath' command line options to override sys.path[0] initialisation

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +29665
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31542

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46840] xmlrpc.client.ServerProxy shows password in __repr__ when using basic authentication

[Jerome Perrin]

New submission from Jerome Perrin :

>>> import xmlrpc.client
>>> xmlrpc.client.ServerProxy('https://login:passw...@example.com')


Because this repr is included in error messages, this can lead to leaking the 
password:

>>> xmlrpc.client.ServerProxy('https://login:passw...@example.com').method()
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python3.7/xmlrpc/client.py", line 1112, in __call__
return self.__send(self.__name, args)
  File "/usr/lib/python3.7/xmlrpc/client.py", line 1452, in __request
verbose=self.__verbose
  File "/usr/lib/python3.7/xmlrpc/client.py", line 1154, in request
return self.single_request(host, handler, request_body, verbose)
  File "/usr/lib/python3.7/xmlrpc/client.py", line 1187, in single_request
dict(resp.getheaders())
xmlrpc.client.ProtocolError: 

--
components: Library (Lib)
messages: 413870
nosy: perrinjerome
priority: normal
severity: normal
status: open
title: xmlrpc.client.ServerProxy shows password in __repr__ when using basic 
authentication

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24313] json fails to serialise numpy.int64

[Nikolay Markov]

Nikolay Markov  added the comment:

Just ran into this. Are there any updates? Is there any task to contribute to 
regarding this?

--
nosy: +mxposed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13475] Add '--mainpath'/'--nomainpath' command line options to override sys.path[0] initialisation

[STINNER Victor]

STINNER Victor  added the comment:

See also "Python flag/envvar not to put current directory to sys.path (but 
don’t ignore PYTHONPATH)" discussion:
https://discuss.python.org/t/python-flag-envvar-not-to-put-current-directory-to-sys-path-but-dont-ignore-pythonpath/4235/2

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23041] csv needs more quoting rules

[Samwyse]

Samwyse  added the comment:

I just signed the contributor agreement. (Thought I had done that last year but 
I don’t see any emails. Is there any place to check?)

I agree that round-tripping should Bebe possible for any value of quoting.

Hopefully this will finally get done before its eighth birthday.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46623] test_zlib: test_pair() and test_speech128() fail with s390x hardware accelerator

[STINNER Victor]

STINNER Victor  added the comment:

I checked os.uname() value on two buildbots, by looking at their 
test.pythoninfo:

os.uname: posix.uname_result(sysname='Linux', nodename='ztcpip3.pok.ibm.com', 
release='3.10.0-1160.53.1.el7.s390x', version='#1 SMP Thu Dec 16 04:33:52 EST 
2021', machine='s390x')

os.uname: posix.uname_result(sysname='Linux', nodename='dje', 
release='5.10.0-11-s390x', version='#1 SMP Debian 5.10.92-1 (2022-01-18)', 
machine='s390x')

So the machine is 's390x'.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46754] Improve Python Language Reference based on [Köhl 2020]

[Guido van Rossum]

Guido van Rossum  added the comment:

Another:

- The description of this name binding and resolution process
  in the PLR [20, §4.2.] is unfortunately not particularly clear.

(I found this to be the case too, and wrote up what I found:
https://gvanrossum.github.io/formal/scopesblog.html
Hopefully it matches what Kohl derived.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46606] Large C stack usage of os.getgroups() and os.setgroups()

[STINNER Victor]

STINNER Victor  added the comment:

>  n++; // Avoid malloc(0)
> grouplist = PyMem_New(gid_t, n+1);

FYI PyMem_New(0) is well specified and doesn't return NULL:
https://docs.python.org/dev/c-api/memory.html#c.PyMem_Malloc

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46754] Improve Python Language Reference based on [Köhl 2020]

[Guido van Rossum]

Guido van Rossum  added the comment:

A few examples of issues brought up by Kohl:

- While the PLR explicitly states that “x < y calls x.__lt__(y)”
  [20, §3.3.1.] this is actually false.
  There are cases where x < y does not call x.__lt__(y)
  and there are other cases where x.__lt__(y) is called
  but more than that happens.

- If no expression is provided, the PLR states that
  “the last exception that was active in the current scope”
  should be re-raised. Unfortunately, the PLR stays
  unspecific on what it means for an exception to be
  “the last exception that was active in the current scope.
  [...]
  Instead, raise re-raises the exception that *is active*
  in the respective execution context

(Perhaps unrelated, but indicative of how out of date the PLR is: in 
executionmodel.rst there's still a mention of and even an index entry for 
restricted execution, a feature that was removed in some early Python 2 
release.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46794] Please update bundled libexpat to 2.4.6 with security fixes (5 CVEs)

[Ned Deily]

Ned Deily  added the comment:


New changeset 15d7594d9974cfef10e65cbb01161168c42abe9d by Miss Islington (bot) 
in branch '3.7':
bpo-46794: Bump up the libexpat version into 2.4.6 (GH-31487) (GH-31521)
https://github.com/python/cpython/commit/15d7594d9974cfef10e65cbb01161168c42abe9d


--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46839] Process finished with exit code -1073741819 (0xC0000005)

[Zachary Ware]

Zachary Ware  added the comment:

0xc005 appears to be an access violation, but beyond that there's not much 
we can help with here.

For a start, Python 2.7 has been out of support for over 2 years now, and as 
far as I can easily tell Anaconda2 is no longer supported either.  It is highly 
recommended to update to Python 3.9 or later if at all possible.

Also, it seems likely that the problem is actually in a third-party extension 
module rather than Python itself, though it's impossible to tell for sure from 
what information is here.

For further assistance, you can try the Users category of discuss.python.org, 
the python-l...@python.org mailing list, or a forum specific to Anaconda 
(though I'm not familiar with one myself).

--
nosy: +zach.ware
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46838] Parameters and arguments parser syntax error improvments

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Thanks a lot for the suggestions! I will try to give these a go to see if we 
can get them implemented. Parameter parsing is a bit hairy so not sure how 
lucky we will be but all of them make sense!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46839] Process finished with exit code -1073741819 (0xC0000005)

[sami]

New submission from sami :

Hi, 
I am running my python code for large data set and currently, I received this 
exit code: "Process finished with exit code -1073741819 (0xC005)"

I monitored via Event Viewer and found the following error. I wonder how I can 
solve this issue? Thanks

Faulting application name: python.exe, version: 0.0.0.0, time stamp: 0x56abcaee
Faulting module name: python27.dll, version: 2.7.11150.1013, time stamp: 
0x56abcaed
Exception code: 0xc005
Fault offset: 0x000a4f81
Faulting process id: 0x29d0
Faulting application start time: 0x01d8287ff03c7476
Faulting application path: C:\Users\sami\Anaconda2\python.exe
Faulting module path: C:\Users\sami\Anaconda2\python27.dll

--
messages: 413859
nosy: fa.sami
priority: normal
severity: normal
status: open
title: Process finished with exit code -1073741819 (0xC005)
type: crash

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46752] Introduce task groups to asyncio and change task cancellation semantics

[Tin Tvrtković]

Change by Tin Tvrtković :


--
nosy: +tinchester
nosy_count: 6.0 -> 7.0
pull_requests: +29664
pull_request: https://github.com/python/cpython/pull/31513

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46838] Parameters and arguments parser syntax error improvments

[Andrej Klychin]

Andrej Klychin  added the comment:

I also sometimes write def foo(pos_only, /*, kwarg): pass. Perhaps this can be 
special cased with 
SyntaxError: expected comma between / and *

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44136] Remove pathlib flavours

[Barney Gale]

Barney Gale  added the comment:

^ just to bring my previous comment up-to-date:

I'm no longer pursuing adding `os.path.isreserved()` and `os.path.fileuri()` 
functions, or modifying `normpath()`. At least, not for now!

Instead, my plan is to move flavour functionality as follows (as classmethods 
with underscore prefixes):

  _Flavour--> PurePath
  _PosixFlavour   --> PurePath
  _WindowsFlavour --> PureWindowsPath

As a result, PurePath will use POSIX syntax by default. This is fully 
backwards-compatible, as users can't create PurePath objects! 
PurePath.__new__() instantiates PurePosixPath or PureWindowsPath. But it will 
matter for future user subclasses of PurePath/Path, where we usually want POSIX 
syntax.

I think there will be three PRs involved. PR 30320 and PR 30321 move 
_Flavour.make_uri() and _Flavour.is_reserved() respectively; these are 
reasonably standalone. If/when they land, I'll make a larger PR that moves the 
remaining methods into PurePath and PureWindowsPath.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46838] Parameters and arguments parser syntax error improvments

[Andrej Klychin]

New submission from Andrej Klychin :

I saw that pablogsal welcomed improvments to the parser's suggestions, so here 
are the messages for parameters and arguments lists I think should be written 
instead of the current generic "invalid syntax".

>>> def foo(*arg, *arg): pass
SyntaxError: * argument may appear only once

>>> def foo(**arg, **arg): pass
SyntaxError: ** argument may appear only once

>>> def foo(arg1, /, arg2, /, arg3): pass
SyntaxError: / may appear only once

>>> def foo(*args, /, arg): pass
SyntaxError: / must be ahead of *

>>> def foo(/, arg): pass
SyntaxError: at least one argument must precede /

>>> def foo(arg=): pass
SyntaxError: expected default value expression

>>> def foo(*args=None): pass
SyntaxError: * argument cannot have default value

>>> def foo(**kwargs=None): pass
SyntaxError: ** argument cannot have default value

>>> foo(*args=[0])
SyntaxError: cannot assign to iterable argument unpacking

>>> foo(**args={"a": None})
SyntaxError: cannot assign to keyword argument unpacking

>>> foo(arg=)
SyntaxError: expected argument value expression

--
components: Parser
messages: 413856
nosy: Andy_kl, lys.nikolaou, pablogsal
priority: normal
severity: normal
status: open
title: Parameters and arguments parser syntax error improvments
type: enhancement
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46430] intern strings in deepfrozen modules

[STINNER Victor]

STINNER Victor  added the comment:

I tried to hack _PyStaticCode_Dealloc() to free strings, but since immortal 
objects are half-baken today, calling Py_DECREF() does crash.

Py_SETREF() should increase _Py_RefTotal if the old object is immortal and he 
new object is not.

Maybe this change should be reverted until Eric's PEP 683 "Immortal Objects, 
Using a Fixed Refcount" is implemented.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46836] [C API] Move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

Stefan Behnel:
> I haven't looked fully into this yet, but I *think* that Cython can get rid 
> of most of the direct usages of PyFrameObject by switching to the new 
> InterpreterFrame struct instead. It looks like the important fields have now 
> been moved over to that.

InterpreterFrame is part of the internal C API. As I wrote, IMO it's fine for 
now that Cython uses the internal C API.

> That won't improve the situation regarding the usage of CPython internals, 
> but it's probably worth keeping in mind before we start adding new API 
> functions that work on frame objects.

Right.

My hope is also that making the structure internal should help to identify 
which members should be exposed with getter functions, or even setter functions 
(Mark would prefer to no add setter functions).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46836] [C API] Move PyFrameObject to the internal C API

[Stefan Behnel]

Stefan Behnel  added the comment:

I haven't looked fully into this yet, but I *think* that Cython can get rid of 
most of the direct usages of PyFrameObject by switching to the new 
InterpreterFrame struct instead. It looks like the important fields have now 
been moved over to that.

That won't improve the situation regarding the usage of CPython internals, but 
it's probably worth keeping in mind before we start adding new API functions 
that work on frame objects.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46430] intern strings in deepfrozen modules

[STINNER Victor]

STINNER Victor  added the comment:

PyUnicode_InternInPlace() in intern_strings() can convert an immortal string to 
a regular Python strong reference, whereas _PyStaticCode_Dealloc() doesn't 
bother with clearing co_names, co_consts and co_localsplusnames.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46430] intern strings in deepfrozen modules

[Guido van Rossum]

Guido van Rossum  added the comment:

That's pretty mysterious. The deep-freeze code isn't on the stack for either of 
those, but allocation of new unicode string objects is. I'm guessing these are 
somehow leaked by the interning, but I don't follow yet how.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45459] Limited API support for Py_buffer

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +29663
pull_request: https://github.com/python/cpython/pull/31539

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 8a716bc62c8205bb9deeda17422b7e69204b6897 by Victor Stinner in 
branch 'main':
bpo-40421: What's New in Python 3.11: PyFrameObject.f_lasti (GH-31536)
https://github.com/python/cpython/commit/8a716bc62c8205bb9deeda17422b7e69204b6897


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[miss-islington]

miss-islington  added the comment:


New changeset 763e23e11e785162f9d68f83539404af8e863748 by Miss Islington (bot) 
in branch '3.9':
bpo-40421: Fix PyFrame_GetCode() documentation (GH-31535)
https://github.com/python/cpython/commit/763e23e11e785162f9d68f83539404af8e863748


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[miss-islington]

miss-islington  added the comment:


New changeset b0de6299a840a397d4fe3e6c98159d9f258d3295 by Miss Islington (bot) 
in branch '3.10':
bpo-40421: Fix PyFrame_GetCode() documentation (GH-31535)
https://github.com/python/cpython/commit/b0de6299a840a397d4fe3e6c98159d9f258d3295


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46823] Add LOAD_FAST__LOAD_ATTR_INSTACE_VALUE combined opcode

[Brandt Bucher]

Change by Brandt Bucher :


--
nosy: +brandtbucher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45107] Improve LOAD_METHOD specialization

[Brandt Bucher]

Change by Brandt Bucher :


--
nosy: +brandtbucher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46754] Improve Python Language Reference based on [Köhl 2020]

[Brandt Bucher]

Change by Brandt Bucher :


--
nosy: +brandtbucher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46430] intern strings in deepfrozen modules

[STINNER Victor]

STINNER Victor  added the comment:

> Sure, but "leaks" caused by deep-freezing cannot be solved by freeing up the 
> deep-frozen memory -- the solution must be to update the accounting somewhere.

Python allocates memory (ex: with PyObject_Malloc()) which is not released at 
exit. Two examples from Valgrind:

==196803== 50 bytes in 1 blocks are still reachable in loss record 1 of 87
==196803==at 0x484486F: malloc (vg_replace_malloc.c:381)
==196803==by 0x544E29: _PyMem_RawMalloc (obmalloc.c:101)
==196803==by 0x545E0E: PyObject_Malloc (obmalloc.c:700)
==196803==by 0x587159: PyUnicode_New (unicodeobject.c:1448)
==196803==by 0x58C4CF: get_latin1_char (unicodeobject.c:2148)
==196803==by 0x58D7F7: _PyUnicode_FromUCS1 (unicodeobject.c:2450)
==196803==by 0x58E374: PyUnicode_FromKindAndData (unicodeobject.c:2525)
==196803==by 0x69402B: r_object (marshal.c:1150)
==196803==by 0x694295: r_object (marshal.c:1212)
==196803==by 0x694AE0: r_object (marshal.c:1393)
==196803==by 0x694295: r_object (marshal.c:1212)
==196803==by 0x694AA4: r_object (marshal.c:1387)
==196803==by 0x6950F3: read_object (marshal.c:1524)
==196803==by 0x6953E7: PyMarshal_ReadObjectFromString (marshal.c:1641)
==196803==by 0x763223: _Py_Get_Getpath_CodeObject (getpath.c:792)
==196803==by 0x76337F: _PyConfig_InitPathConfig (getpath.c:847)
==196803==by 0x68CD04: config_init_import (initconfig.c:1967)
==196803==by 0x68CE4E: _PyConfig_InitImportConfig (initconfig.c:2000)
==196803==by 0x69E594: init_interp_main (pylifecycle.c:1103)
==196803==by 0x69EBF8: pyinit_main (pylifecycle.c:1216)
==196803==by 0x69EDCE: Py_InitializeFromConfig (pylifecycle.c:1247)
==196803==by 0x6D5346: pymain_init (main.c:67)
==196803==by 0x6D64F8: pymain_main (main.c:692)
==196803==by 0x6D65A1: Py_BytesMain (main.c:725)
==196803==by 0x41D7B5: main (python.c:15)

and

==196803== 3,336 bytes in 60 blocks are still reachable in loss record 87 of 87
==196803==at 0x484486F: malloc (vg_replace_malloc.c:381)
==196803==by 0x544E29: _PyMem_RawMalloc (obmalloc.c:101)
==196803==by 0x545E0E: PyObject_Malloc (obmalloc.c:700)
==196803==by 0x587159: PyUnicode_New (unicodeobject.c:1448)
==196803==by 0x59AEB9: unicode_decode_utf8 (unicodeobject.c:5162)
==196803==by 0x59B5B0: PyUnicode_DecodeUTF8Stateful (unicodeobject.c:5292)
==196803==by 0x58D296: PyUnicode_FromString (unicodeobject.c:2322)
==196803==by 0x5CFFAE: PyUnicode_InternFromString (unicodeobject.c:15650)
==196803==by 0x4E8B61: descr_new (descrobject.c:885)
==196803==by 0x4E8C9B: PyDescr_NewMethod (descrobject.c:934)
==196803==by 0x56694C: type_add_method (typeobject.c:5643)
==196803==by 0x566A92: type_add_methods (typeobject.c:5689)
==196803==by 0x569166: type_ready_fill_dict (typeobject.c:6165)
==196803==by 0x569A02: type_ready (typeobject.c:6421)
==196803==by 0x569B59: PyType_Ready (typeobject.c:6457)
==196803==by 0x544363: _PyTypes_InitTypes (object.c:1952)
==196803==by 0x69D12D: pycore_init_types (pylifecycle.c:704)
==196803==by 0x69D8FC: pycore_interp_init (pylifecycle.c:831)
==196803==by 0x69DC31: pyinit_config (pylifecycle.c:887)
==196803==by 0x69E355: pyinit_core (pylifecycle.c:1050)
==196803==by 0x69ED32: Py_InitializeFromConfig (pylifecycle.c:1240)
==196803==by 0x6D5346: pymain_init (main.c:67)
==196803==by 0x6D64F8: pymain_main (main.c:692)
==196803==by 0x6D65A1: Py_BytesMain (main.c:725)
==196803==by 0x41D7B5: main (python.c:15)

Before the commit c0a5ebeb1239020f2ecc199053bb1a70d78841a1, Python didn't leak 
this memory.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44337] Port LOAD_ATTR to adaptive interpreter

[Brandt Bucher]

Brandt Bucher  added the comment:


New changeset 281ea9c39146a00cdf3fa2b3d0be60e2a39278ce by Brandt Bucher in 
branch 'main':
bpo-44337: Shrink the LOAD_ATTR/STORE_ATTR caches (GH-31517)
https://github.com/python/cpython/commit/281ea9c39146a00cdf3fa2b3d0be60e2a39278ce


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[miss-islington]

Change by miss-islington :


--
pull_requests: +29662
pull_request: https://github.com/python/cpython/pull/31538

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 78859e58e4e016286e648d1dc155e0f6cebfa6ff by Victor Stinner in 
branch 'main':
bpo-40421: Fix PyFrame_GetCode() documentation (GH-31535)
https://github.com/python/cpython/commit/78859e58e4e016286e648d1dc155e0f6cebfa6ff


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 5.0 -> 6.0
pull_requests: +29661
pull_request: https://github.com/python/cpython/pull/31537

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

> I found the uwsgi project on PyPI which uses f_lasti with PyCode_Addr2Line().

Oh, plugins/python/profiler.c uses that to define PyFrame_GetLineNumber() on 
Python older than 2.7, Python 3.0 and Python 3.1. In 2022, it's no longer 
relevant.

But well, there might be other code in the wild using PyCode_Addr2Line() with 
f_lasti, so IMO it's worth it to document the suggestion ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

Mark Shannon:
> The only purpose of `f_lasti` is to get the line number and that can be done 
> directly via `PyFrame_GetLineNumber(PyFrameObject *frame)`

I found the uwsgi project on PyPI which uses f_lasti with PyCode_Addr2Line(). I 
wrote GH-31536 to suggest using PyFrame_GetLineNumber() in What's New in Python 
3.11.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +29660
pull_request: https://github.com/python/cpython/pull/31536

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

Ned Batchelder:
> I went ahead and changed the coverage.py code to this: (...)

I proposed a coverage PR using PyObject_GetAttrString(frame, "f_lasti") which 
should works on all Python versions:
https://github.com/nedbat/coveragepy/pull/1331

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes

[Guido van Rossum]

Guido van Rossum  added the comment:

I will now merge GH-31513 (cancel counts). Once that's in you can merge main 
into your timeout PR (GH-31394) and then that can land soon (I'd like to review 
it once more).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Clear, thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46430] intern strings in deepfrozen modules

[Guido van Rossum]

Guido van Rossum  added the comment:

> Not leaking memory at exit matters when Python is embedded
> in an application.

Sure, but "leaks" caused by deep-freezing cannot be solved by freeing up the 
deep-frozen memory -- the solution must be to update the accounting somewhere.

Where is the existence of Py_None accounted for (since it's statically 
allocated, or at least used to be)? That's likely where we'd have to do 
something about the deep-frozen objects.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45390] asyncio.Task doesn't propagate CancelledError() exception correctly.

[Guido van Rossum]

Guido van Rossum  added the comment:

We should really stop appending to a closed issue.

Anyway, raising ExceptionGroup is backwards incompatible, since "except 
CancelledError" wouldn't cancel the group.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46430] intern strings in deepfrozen modules

[STINNER Victor]

STINNER Victor  added the comment:

> Presumably the leak is imaginary

Well, you can check with your favorite memory debugger like Valgrind if you 
don't trust Python internal memory debugger :-)

Not leaking memory at exit matters when Python is embedded in an application.

I don't think that it's related to the error handling which is stripped in 
release mode.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45390] asyncio.Task doesn't propagate CancelledError() exception correctly.

[Thomas Grainger]

Thomas Grainger  added the comment:

there could be multiple messages here

perhaps it could be:

```
finally:
# Must reacquire lock even if wait is cancelled
cancelled = []
while True:
try:
await self.acquire()
break
except exceptions.CancelledError as e:
cancelled.append(e)

if len(cancelled) > 1:
raise ExceptionGroup("Cancelled", cancelled)
if cancelled:
raise cancelled[0]
```

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

> The docs for PyFrame_GetCode say it's returning an "int". 

Oh. I  missed your comment. I created GH-31535 to fix the return type in the 
documentation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +29659
pull_request: https://github.com/python/cpython/pull/31535

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46736] Generate HTML 5 with SimpleHTTPRequestHandler.list_directory

[Benjamin Peterson]

Benjamin Peterson  added the comment:


New changeset 0bb40a42d71873ea267aace8c92a02d66fe36dc2 by Dong-hee Na in branch 
'main':
closes bpo-46736: SimpleHTTPRequestHandler now uses HTML5. (GH-31533)
https://github.com/python/cpython/commit/0bb40a42d71873ea267aace8c92a02d66fe36dc2


--
nosy: +benjamin.peterson
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes

[Yves Duprat]

Change by Yves Duprat :


--
nosy: +yduprat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46761] functools.update_wrapper breaks the signature of functools.partial objects

[Larry Hastings]

Larry Hastings  added the comment:

Ofey, I appreciate your enthusiasm, but you should probably slow down.  Fixing 
the bug is probably going to be the easy part here.  But we're not to that 
stage quite yet.  First, we need to determine

* why the code behaves like this--is this behavior a genuine bug, or is it 
actually a bugfix for some worse behavior?
* will fixing the bug cause problems for Python users? and if so, can we still 
fix the bug while mitigating the damage to people who are unfortunately 
depending on the bug?

The next step is not to write a bugfix for this exact behavior, it's to 
determine why the code is the way it is.  If it was genuinely just a mistake, 
and we can simply fix it and people will thank us, then we may have a use for 
your patch.  But, generally, people who work on Python are smart, and they 
don't tend to commit dumb mistakes, so we can't simply assume it's a simple bug 
and fix it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46835] ImportError: bad magic number in ... does not indicate where is that file located

[Miro Hrončok]

Change by Miro Hrončok :


--
keywords: +patch
pull_requests: +29658
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31534

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46430] intern strings in deepfrozen modules

[Guido van Rossum]

Guido van Rossum  added the comment:

Okay, let's change the error handling. @Kumar, can you handle that?

@Victor, the refleak is unrelated to the error handling right? Presumably the 
leak is imaginary -- the deep-frozen interned strings should be accounted for 
somehow. @Kumar do you need help investigating this?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46837] lstrip and strip not working as expected

[Jigar Gajjar]

Change by Jigar Gajjar :


--
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46837] lstrip and strip not working as expected

[Jigar Gajjar]

New submission from Jigar Gajjar :

Code:

my_string = 'Auth:AWS'
print(my_string.lstrip('Auth:'))


Actual Output:

WS

Excepted Output:

AWS

--
messages: 413831
nosy: jigar030
priority: normal
severity: normal
status: open
title: lstrip and strip not working as expected
type: behavior
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45390] asyncio.Task doesn't propagate CancelledError() exception correctly.

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Also Future.result() and Future.exception() can raise a CancelledError. So a 
CancelledError raised in a task may not contain a message passed to 
Task.cancel().

import asyncio
import random

async def main():
fut = asyncio.Future()
fut.cancel()
async def job():
if random.random() < 0.5:
await asyncio.sleep(2)
fut.result()
await asyncio.sleep(5)
task = asyncio.create_task(job())
await asyncio.sleep(1)
task.cancel("cancel task")
await task

asyncio.run(main())

You need to catch a CancelledError raised in a coroutine and re-raise a new 
CancelledError with the specified cancel message if the task was cancelled.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46836] [C API] Move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

Petr Viktorin:
> So, this will break Cython and gevent,

This change doesn't break Cython and gevent: they are already broken.


> but (unlike the optimization work that broke f_code/f_frame) it won't provide 
> any value to users?

The problem is that the C API changed silently: existing code which gets 
directly PyFrameObject.f_back still compiles successfully, but it will no 
longer work in some cases.

See bpo-46356 "[C API] Enforce usage of PyFrame_GetBack()" for more details.

The intent of moving the structure to the internal C API is to clarify its 
status: we provide no backward compatibility warranty, you are on our own if 
you use it.

It's also a way to promote the usage of the new clean public C API: it is now 
reliable, whereas accessing directly PyFrameObject members break at each Python 
version.

The internal C API cannot be used easily on purpose: you have to opt-in for 
this API by defining the Py_BUILD_CORE_MODULE macro and you need to use 
different #include. It's a way to enforce the usage of the clean public C API.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46836] [C API] Move PyFrameObject to the internal C API

[Petr Viktorin]

Petr Viktorin  added the comment:

So, this will break Cython and gevent, but (unlike the optimization work that 
broke f_code/f_frame) it won't provide any value to users?

I don't see how that's a good idea.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45412] [C API] Remove Py_OVERFLOWED(), Py_SET_ERRNO_ON_MATH_ERROR(), Py_ADJUST_ERANGE1()

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-46670: "Build Python with -Wundef: don't use undefined macros".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45412] [C API] Remove Py_OVERFLOWED(), Py_SET_ERRNO_ON_MATH_ERROR(), Py_ADJUST_ERANGE1()

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 9bbdde218005f552304d9954bb97e3f9185edded by Victor Stinner in 
branch 'main':
bpo-45412: Add _PY_SHORT_FLOAT_REPR macro (GH-31171)
https://github.com/python/cpython/commit/9bbdde218005f552304d9954bb97e3f9185edded


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46668] encodings: the "mbcs" alias doesn't work

[STINNER Victor]

STINNER Victor  added the comment:

commit ccbe8045faf6e63d36229ea4e1b9298572cda126
Author: Victor Stinner 
Date:   Tue Feb 22 22:04:07 2022 +0100

bpo-46659: Fix the MBCS codec alias on Windows (GH-31218)

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46430] intern strings in deepfrozen modules

[STINNER Victor]

STINNER Victor  added the comment:

_PyStaticCode_InternStrings() error handling is based on assert(): that's 
really bad. It can crash Python (exit with abort()) at the first memory 
allocation failure. Why not returning -1 on error?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46430] intern strings in deepfrozen modules

[STINNER Victor]

STINNER Victor  added the comment:

> This change introduced a memory leak at Python exit.

It's regression related to bpo-1635741 which I fixed recently:
https://mail.python.org/archives/list/python-...@python.org/thread/E4C6TDNVDPDNNP73HTGHN5W42LGAE22F/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46430] intern strings in deepfrozen modules

[STINNER Victor]

STINNER Victor  added the comment:

> New changeset c0a5ebeb1239020f2ecc199053bb1a70d78841a1 by Kumar Aditya in 
> branch 'main':
> bpo-46430: Intern strings in deep-frozen modules  (GH-30683)

This change introduced a memory leak at Python exit.

Before:

$ ./python -X showrefcount -c pass
[0 refs, 0 blocks]

After:

$ ./python -X showrefcount -c pass
[0 refs, 344 blocks]

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46829] Confusing CancelError message if multiple cancellations are scheduled

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

For reference, the msg parameter of Task.cancel() was added in issue31033.

It seems that the initial use case was for debugging. I do not see how it 
differs from the following example:

r = random.random()
if r < 0.5:
x = 0
else:
x = 0
1/x

In the traceback we see the line where an error occurred but we do not see a 
line which lead to this error.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes

[Guido van Rossum]

Guido van Rossum  added the comment:

To make this cleanly interact with timeout, TaskGroup etc., the CancelOnEvent 
class should have a "did-I-cancel" flag which is set in the _cancel_on_event() 
callback. Then if that flag is set it should call .uncancel(), and if that 
returns a value > 0, it should bubble the CancelledError out; otherwise it can 
raise EventRaised (if the condition is set).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46622] Add an async variant of lru_cache for coroutines.

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

async_lru_cache() and async_cached_property() can be written using that 
decorator. The implementation of async_lru_cache() is complicated because the 
interface of lru_cache() is complicated. But it is simpler than using 
_lru_cache_wrapper().

def async_lru_cache(maxsize=128, typed=False):
if callable(maxsize) and isinstance(typed, bool):
user_function, maxsize = maxsize, 128
return lru_cache(maxsize, typed)(reawaitable(user_function))

def decorating_function(user_function):
return lru_cache(maxsize, typed)(reawaitable(user_function))

return decorating_function

def async_cached_property(user_function):
return cached_property(reawaitable(user_function))

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46736] Generate HTML 5 with SimpleHTTPRequestHandler.list_directory

[Dong-hee Na]

Change by Dong-hee Na :


--
keywords: +patch
pull_requests: +29657
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31533

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46833] Installer Wizard is unclear and has redundant settings

[Christian Buhtz]

Christian Buhtz  added the comment:

In the attachment you will find a PDF with variants A to D on each side.

I tried to think into the design decisions made by the team who created the 
current installer. I am not sure of course but I tried to take this (assumed) 
decisions into account.

Variant A:
This is nearest to the current release version.
 - The settings on page 1 are moved onto that page directly to the bottom of 
"Install Now" and before(!) "Customize installation".
 - The "all users" option is now available for the interpreter and launcher.
Modifying the position of the two checkboxes make‘s it more clear to which 
decision way (simple or customized/advanced installation) they belong.

Variant B:
The same as A but Page 2 and 3 (from A) are joined together. If there is enough 
room on the wizard dialog this would be OK. But very important is to visually 
separate the two sections "Interpreter" and "Launcher" on that one dialog. You 
could do that with bigger bold text like headings or you could use a horizontal 
bar.

Variant C (would be my second favourite):
Page 1 is more minimal. The user only have to make a decision between simple 
installation and advanced/customize installation.

Variant D (my favourite):
Page one offers the simple options about "PATH" and "all users" for interpreter 
and launcher. This should be separated in a visual way of course.
btw: From a technical point of view I do not see an advantage of separating the 
decision about "PATH" and "all users" between python and py. I would assume if 
py should goes to PATH and installed for "all users" the interpreter should 
treated the same.
More important on D is that the way to the "advanced" (currently named 
"customize") installation way is "hidden" behind a simple GUI button. A lot of 
other installers doing it the same way. It is just a simple button. Not big, no 
special colours or something like that. The page 1 of the current release 
version of the installer is to much bling-bling. ;)

Some more Notes and Thoughts

„Customize“ is not a good term, because it is still possible to „customize“ the 
installation on that first page (the two check boxes on the bottom) without 
clicking „Customize installation“ and
When clicking on „Customize installation“ the next (2nd) page is named 
„Optional Features“ which is different from „Customization“. I would suggest 
"Advanced" or "Expert".
It is similar with “Advanced” on page 3. What is the difference between 
“Advanced” and “Customize”?

Add a „What is the py launcher for“ link to the wizard.

Add a „What is pip launcher for“ link to the wizard.

In the What-for-pages: Do not describe what py/pip can do but describe what the 
user can do with it. Modify the perspectives/view points! I would help you to 
review this texts.

Use horizontal bars in the GUI to better visualise the separate ways/topics. 
E.g. in Variant B on page 2.

I have some more detailed suggestions about modified wording. But I think at 
this point it is enough. :)

--
Added file: https://bugs.python.org/file50640/py_installer.pdf

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46829] Confusing CancelError message if multiple cancellations are scheduled

[Guido van Rossum]

Guido van Rossum  added the comment:

But that example is made-up. Is there a real-world situation where you need to 
know the call site, and it wouldn't be obvious from other log messages?

Directly cancelling a task without also somehow catching the cancellation (like 
in the timeout or task group cases) feels like an odd practice to me.

And passing the cancel message through is complex (as we've seen in recent PRs).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46622] Add an async variant of lru_cache for coroutines.

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I think that it would be simpler to add a decorator which wraps the result of 
an asynchronous function into an object which can be awaited more than once:

def reawaitable(func):
@wraps(func)
def wrapper(*args, **kwargs):
return CachedAwaitable(func(*args, **kwargs))
return wrapper

It can be combined with lru_cache and cached_property any third-party caching 
decorator. No access to internals of the cache is needed.

@lru_cache()
@reawaitable
async def coro(...):
...

@cached_property
@reawaitable
async def prop(self):
...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43311] bpo-43311: PyInterpreterState_New use thread-specific data tstate before key create .

[STINNER Victor]

Change by STINNER Victor :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com