date:20211117



Change by Dong-hee Na :


--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45795] urllib http client vulnerable to DOS attack

2021-11-17 Thread Senthil Kumaran



Senthil Kumaran  added the comment:

> Timeout value should not be reset after client receives a data(bytes), 
> because it can easily be abused to achieve DOS.

Interesting. I looked the server example.

Does clients like curl have something like this too?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45788] Doc sys.prefix lib & include directories on Windows and Mac

2021-11-17 Thread Terry J. Reedy



Change by Terry J. Reedy :


--
keywords: +patch
pull_requests: +27847
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/29606

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45816] Python does not support standalone MSVC v143 (VS 2022) Build Tools

2021-11-17 Thread Ma Lin



Ma Lin  added the comment:

They are LNK1268 error:

LINK : fatal error LNK1268: inconsistent option 'pdbthreads:5' specified with 
/USEPROFILE but not with /GENPROFILE [e:\dev\cpython\PCbuild\_queue.vcx
proj]

LINK : fatal error LNK1268: inconsistent option 'pdbthreads:1' specified with 
/USEPROFILE but not with /GENPROFILE [e:\dev\cpython\PCbuild\_asyncio.v
cxproj]

LINK : fatal error LNK1268: inconsistent option 'pdbthreads:5' specified with 
/USEPROFILE but not with /GENPROFILE [e:\dev\cpython\PCbuild\_elementtr
ee.vcxproj]

LINK : fatal error LNK1268: inconsistent option 'cgthreads:8' specified with 
/USEPROFILE but not with /GENPROFILE [e:\dev\cpython\PCbuild\_hashlib.vc
xproj]

...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45836] unittest.TestCase.assertWarns raises RuntimeEror if sys.modules changes size (Python 3.9)

2021-11-17 Thread Dennis Sweeney



Dennis Sweeney  added the comment:

https://github.com/python/cpython/pull/29605 was just opened as a backport

--
nosy: +Dennis Sweeney

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29620] unittest.TestCase.assertWarns raises RuntimeEror if sys.modules changes size



Change by miss-islington :


--
pull_requests: +27846
pull_request: https://github.com/python/cpython/pull/29605

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45836] unittest.TestCase.assertWarns raises RuntimeEror if sys.modules changes size (Python 3.9)

2021-11-17 Thread Patrick Bourke



New submission from Patrick Bourke :

Hi all,

Forgive me if this is not the correct way to report this, but we have run into 
the issue from #29620 ( https://bugs.python.org/issue29620 ) on Python 3.9.

The fix appears to be present in the tip of 3.8:

https://github.com/python/cpython/blob/c37a0d9c0ae4aa0d9135fac9a58afc7b34ff71d6/Lib/unittest/case.py#L254

and 3.10:

https://github.com/python/cpython/blob/0ef308a2890571c850c624fb99ac00f8951363c6/Lib/unittest/case.py#L255

but missing from 3.9:

https://github.com/python/cpython/blob/0ef308a2890571c850c624fb99ac00f8951363c6/Lib/unittest/case.py#L255

Here is our test case:

import unittest
import graspologic


class MyTestCase(unittest.TestCase):
def test_something(self):
with self.assertWarns(UserWarning):
pass


if __name__ == '__main__':
unittest.main()

fails with:

==
ERROR: test_something (__main__.MyTestCase)
--
Traceback (most recent call last):
  File 
"C:\Users\pbourke\AppData\Roaming\JetBrains\PyCharm2021.2\scratches\test.py", 
line 7, in test_something
with self.assertWarns(UserWarning):
  File 
"C:\Users\pbourke\.pyenv\pyenv-win\versions\3.9.7\lib\unittest\case.py", line 
255, in __enter__
for v in sys.modules.values():
RuntimeError: dictionary changed size during iteration


The module triggering the error appears to be IPython.utils.io, which is 
included somewhere in our dependencies.

My CPython is:
Python 3.9.7 (tags/v3.9.7:1016ef3, Aug 30 2021, 20:19:38) [MSC v.1929 64 bit 
(AMD64)]

--
components: Library (Lib)
messages: 406516
nosy: pbourke
priority: normal
severity: normal
status: open
title: unittest.TestCase.assertWarns raises RuntimeEror if sys.modules changes 
size (Python 3.9)
type: behavior
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40059] Provide a toml module in the standard library

2021-11-17 Thread Shantanu



Shantanu  added the comment:

Hopefully tomli being less than a year old should not be a blocker, after all, 
TOML v1.0.0 is itself less than a year old.

Despite tomli being new code, it's pretty widely used. Reverse dependencies 
include pip, pytest, mypy, black, flit, coverage, setuptools-scm, cibuildwheel. 
All in all, it should be well proven in practice by the time it comes to make 
3.11 stable.

It also helps that it's easy to anticipate the interface: dump, dumps, load, 
loads. Overall, hopefully risk here should be well within the budget of a 
provisional module.

That's all to say, if and when the Steering Council decides that there's work 
to be done here, let me know how I can help :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45826] unittest.assertRaisesRegex is broken in Python 3.11 and leading to crashing if tested regex does not match name.


Łukasz Langa  added the comment:

Thanks, Dennis! ✨  ✨

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45826] unittest.assertRaisesRegex is broken in Python 3.11 and leading to crashing if tested regex does not match name.


Łukasz Langa  added the comment:


New changeset 8eabe60108b536b942c791b5d3dc3c3020497aac by Łukasz Langa in 
branch '3.10':
[3.10] bpo-45826: Fix a crash in suggestions.c by checking for `traceback is 
None` (GH-29590) (GH-29602)
https://github.com/python/cpython/commit/8eabe60108b536b942c791b5d3dc3c3020497aac


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45429] [Windows] time.sleep() should use CREATE_WAITABLE_TIMER_HIGH_RESOLUTION


Łukasz Langa  added the comment:


New changeset fc4474e45eecbea8e88095f28c98c5d56438d841 by Dong-hee Na in branch 
'main':
bpo-45429: Merge whatsnew about time.sleep (GH-29589)
https://github.com/python/cpython/commit/fc4474e45eecbea8e88095f28c98c5d56438d841


--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45820] Parser can segfault if an error happens before reading any input


Łukasz Langa  added the comment:

Thanks, Pablo! ✨  ✨

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45820] Parser can segfault if an error happens before reading any input


Łukasz Langa  added the comment:


New changeset 00ee14e814d35587ac55f89c7de871a01360c876 by Miss Islington (bot) 
in branch '3.9':
[3.9] bpo-45820: Fix a segfault when the parser fails without reading any input 
(GH-29580) (GH-29584)
https://github.com/python/cpython/commit/00ee14e814d35587ac55f89c7de871a01360c876


--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45820] Parser can segfault if an error happens before reading any input



miss-islington  added the comment:


New changeset b455df59a8eca1b0c0793bc11a116ffc2829b175 by Miss Islington (bot) 
in branch '3.10':
bpo-45820: Fix a segfault when the parser fails without reading any input 
(GH-29580)
https://github.com/python/cpython/commit/b455df59a8eca1b0c0793bc11a116ffc2829b175


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45822] Py_CompileString does not respect the coding cookie with the new parser if flags are empty


Łukasz Langa  added the comment:


New changeset 0ef308a2890571c850c624fb99ac00f8951363c6 by Pablo Galindo Salgado 
in branch '3.9':
bpo-45822: Respect PEP 263's coding cookies in the parser even if flags are not 
provided (GH-29582) (GH-29585)
https://github.com/python/cpython/commit/0ef308a2890571c850c624fb99ac00f8951363c6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45822] Py_CompileString does not respect the coding cookie with the new parser if flags are empty


Łukasz Langa  added the comment:


New changeset e3aa9fd77bf474bb3e8a7a1d1bd1ebf45147945a by Pablo Galindo Salgado 
in branch '3.10':
[3.10] bpo-45822: Respect PEP 263's coding cookies in the parser even if flags 
are not provided (GH-29582) (GH-29586)
https://github.com/python/cpython/commit/e3aa9fd77bf474bb3e8a7a1d1bd1ebf45147945a


--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45826] unittest.assertRaisesRegex is broken in Python 3.11 and leading to crashing if tested regex does not match name.


Change by Łukasz Langa :


--
pull_requests: +27845
pull_request: https://github.com/python/cpython/pull/29602

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45826] unittest.assertRaisesRegex is broken in Python 3.11 and leading to crashing if tested regex does not match name.


Łukasz Langa  added the comment:


New changeset 5d90c467c02ffefdb13c1abc83a171db1a99ffad by Dennis Sweeney in 
branch 'main':
bpo-45826: Fix a crash in suggestions.c by checking for `traceback is None` 
(GH-29590)
https://github.com/python/cpython/commit/5d90c467c02ffefdb13c1abc83a171db1a99ffad


--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42540] Debug pymalloc crash when using os.fork() [regression]


Łukasz Langa  added the comment:

Thanks, Sam! ✨  ✨

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45833] NamedTemporaryFile deleted before enclosing context manager exit

2021-11-17 Thread Zachary Ware



Zachary Ware  added the comment:

It's a bit convoluted, but the file is actually deleted before the `os.stat` 
call.  Because there are no references to anything but the `name` (which is 
just a string), the `_GeneratorContextManager` (result of `my_tmp_file`) and 
the `_TemporaryFileWrapper` (result of `my_tmp_file().__enter__()`) are both 
destroyed.  Because `NamedTemporaryFile` is called with `delete=True` 
(default), the `_TemporaryFileWrapper` has a `_closer` attribute which is a 
`_TemporaryFileCloser`, which calls `self.close()` in `__del__`, which deletes 
the file.

If a reference to the result of `my_tmp_file()` is saved anywhere along the 
way, none of the objects are destroyed and the file still exists.  This also 
wouldn't happen in an implementation without reference counting.

--
nosy: +zach.ware
resolution:  -> not a bug
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42540] Debug pymalloc crash when using os.fork() [regression]


Łukasz Langa  added the comment:


New changeset 87787c8774221c81602b31b0e0dc0678ad494e91 by Sam Gross in branch 
'3.9':
[3.9] bpo-42540: reallocation of id_mutex should not force default allocator 
(GH-29564) (GH-29600)
https://github.com/python/cpython/commit/87787c8774221c81602b31b0e0dc0678ad494e91


--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45832] Misleading membersip expression documentation

2021-11-17 Thread Harald Husum



Harald Husum  added the comment:

Might i then suggest mentioning in the docs that this assumes the invariance, 
combined with a backlink to the definition, instead of a full repeat?

I'm sure the hash invariant is well known to you guys, working on python, but I 
was genuinely surprised by this behaviour.

Not sure I should be ashamed or not, given that my job has revolved around 
python for the last couple of years.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45835] Race condition in test_queue can lead to test failures



Change by Sam Gross :


--
pull_requests: +27844
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/29601

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45831] _Py_DumpASCII() writes characters one by one leading to corrupted tracebacks


Łukasz Langa  added the comment:

Thanks, Victor! ✨  ✨

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45831] _Py_DumpASCII() writes characters one by one leading to corrupted tracebacks


Change by Łukasz Langa :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45831] _Py_DumpASCII() writes characters one by one leading to corrupted tracebacks


Łukasz Langa  added the comment:


New changeset ac89f8cab79800195687dd141de472f90c626ec3 by Miss Islington (bot) 
in branch '3.9':
bpo-45831: _Py_DumpASCII() uses a single write() call if possible (GH-29596) 
(GH-29597)
https://github.com/python/cpython/commit/ac89f8cab79800195687dd141de472f90c626ec3


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45831] _Py_DumpASCII() writes characters one by one leading to corrupted tracebacks


Łukasz Langa  added the comment:


New changeset 4ffde90dccd741b04a448f2e44f0b82a41b6fe96 by Miss Islington (bot) 
in branch '3.10':
bpo-45831: _Py_DumpASCII() uses a single write() call if possible (GH-29596) 
(GH-29598)
https://github.com/python/cpython/commit/4ffde90dccd741b04a448f2e44f0b82a41b6fe96


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45835] Race condition in test_queue can lead to test failures



Change by Sam Gross :


--
keywords: +patch
Added file: https://bugs.python.org/file50447/issue45835_repro.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45835] Race condition in test_queue can lead to test failures



New submission from Sam Gross :

The test_queue suite has a race condition that can lead to test failures in 
test_many_threads, test_many_threads_nonblock, and test_many_threads_timeout. 
Consumers are signaled to exit by a sentinel value (None). The sentinel values 
are at the end of the input list, but that doesn't mean they are necessarily 
enqueued at the end of the inter-thread queue when there are multiple "feeder" 
threads.

In particular, a feeder thread may be delayed in enqueueing a non-sentinel 
value. The other feeder threads may finish popping and enqueueing the remaining 
values including all the sentinels, leading to the delayed non-sentinel value 
arriving AFTER all the sentinels. The "consumer" threads exit before processing 
all the values leading to the assertion error in run_threads() in test_queue.py:

  self.assertTrue(q.empty())

I will attach a patch that adds a delay in feed() to make the race condition 
occur more frequently so that the issue is easier to reproduce.

--
components: Tests
messages: 406498
nosy: colesbury
priority: normal
severity: normal
status: open
title: Race condition in test_queue can lead to test failures
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45832] Misleading membersip expression documentation

2021-11-17 Thread Eric V. Smith


Eric V. Smith  added the comment:

I don’t think repeating the hash invariant in multiple places adds anything, I 
think it would just add clutter. I also think the existing docs are easier to 
understand than the version with the hashing containers split out.

--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45796] Using tab to cycle through tkinter widgets breaks foreground styling

2021-11-17 Thread Ned Deily



Ned Deily  added the comment:

For the record, the Tk issue opened for this is at:
https://core.tcl-lang.org/tk/tktview?name=300bad1beb

--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45796] Using tab to cycle through tkinter widgets breaks foreground styling

2021-11-17 Thread Ned Deily



Change by Ned Deily :


--
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45795] urllib http client vulnerable to DOS attack

2021-11-17 Thread Ned Deily



Change by Ned Deily :


--
nosy: +orsenthil

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45834] Move runtime except: check to the parser

2021-11-17 Thread Irit Katriel



New submission from Irit Katriel :

The compiler has a runtime check that an "except:" without type can only be the 
last in the try block:

https://github.com/python/cpython/blob/15409c720be0503131713e3d3abc1acd0da07378/Python/compile.c#L3251

This could be detected by the parser instead.

--
components: Parser
messages: 406495
nosy: iritkatriel, lys.nikolaou, pablogsal
priority: normal
severity: normal
status: open
title: Move runtime except: check to the parser
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45833] NamedTemporaryFile deleted before enclosing context manager exit

2021-11-17 Thread Brian McCutchon



New submission from Brian McCutchon :

Consider the following code:

# Copyright 2021 Google LLC.
# SPDX-License-Identifier: Apache-2.0
import contextlib
import os

@contextlib.contextmanager
def my_tmp_file():
  with tempfile.NamedTemporaryFile('w') as f:
yield f

os.stat(my_tmp_file().__enter__().name)  # File not found
os.stat(contextlib.ExitStack().enter_context(my_tmp_file()).name)  # Same

I would expect the file to still exist, as __exit__ has not been called and I 
can't see why the file would have been closed. Also, it performs as expected 
when using NamedTemporaryFile directly, but not when it is nested in another 
context manager. It also performs as expected when my_tmp_file() or 
contextlib.ExitStack() is used in a "with" statement.

--
components: Library (Lib)
messages: 406494
nosy: Brian McCutchon
priority: normal
severity: normal
status: open
title: NamedTemporaryFile deleted before enclosing context manager exit
type: behavior
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42540] Debug pymalloc crash when using os.fork() [regression]



miss-islington  added the comment:


New changeset 1079b3e3cb3eba7062e174ecc6c0ab20c2d0722e by Miss Islington (bot) 
in branch '3.10':
bpo-42540: reallocation of id_mutex should not force default allocator 
(GH-29564)
https://github.com/python/cpython/commit/1079b3e3cb3eba7062e174ecc6c0ab20c2d0722e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42540] Debug pymalloc crash when using os.fork() [regression]



Change by Sam Gross :


--
pull_requests: +27843
pull_request: https://github.com/python/cpython/pull/29600

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Proliferation of Python packaging formats

2021-11-17 Thread Mats Wichmann

On 11/17/21 07:44, Chris Angelico wrote:

On Thu, Nov 18, 2021 at 12:20 AM Skip Montanaro
wrote:

Is the proliferation of packaging formats in Python as nutzo as this author
believes?

https://drewdevault.com/2021/11/16/Python-stop-screwing-distros-over.html

Asking because I've never been in the business of releasing "retail" Python
applications or packages.

It's a problem if you feel the need to release in every single
possible way. The XKCD in that example is a showcase of what happens
when you attempt to install in every possible way, which is usually a
consequence of everyone releasing in just one way.

He's blaming the PSF, but that's not really fair. The PSF has never
said "hey everyone, make sure you package for conda as well as posting
on PyPI". Part of the proliferation comes because there are multiple
completely independent distributions, and in the XKCD shown there, he
has installed Python at least five times from completely different
sources (the OS-provided one, two from Homebrew, one from Anaconda,
and one from python.org), and maybe installed pip into one of those as
well. Duh, of course that's going to cause problems.

There are people working on the problems, but the author doesn't seem to
like the progress being made. Here's one jumping point for those
interested who haven't otherwise followed this stuff:

https://discuss.python.org/t/graceful-cooperation-between-external-and-python-package-managers-pep-668/10302

Some of the problems in Python ecosystem, which categorically does *not*
just consist of machines running a Linux distro, are in fact caused by
the choices the Linux distros make in packaging. The model of "get
everything you need from your distro's packages" doesn't come close to
cutting it these days, so there has to be coexistence. Anecdotally, I
have a module I use in a couple of projects and I forgot that on my main
machine I actually had that supplied by a Fedora package (silly me), and
surprise - it's been dropped in F35.

Yes, it's true that the PyPA does not set itself up to be the final
picker of winners, so this comment isn't just off the wall:

"P.S. PEP-517 and 518 are a start, but are very disappointing in how
little they address distro problems. These PEPs are designed to tolerate
the proliferation of build systems, which is exactly what needs to stop.
Python ought to stop trying to avoid hurting anyone’s feelings and pick
one."

That doesn't seem likely to happen, and in fact there's probably
strength in having multiple implementations of a PEP (in other worlds,
like W3C, multiple implementations are mandatory before a standard can
be accepted).

--
https://mail.python.org/mailman/listinfo/python-list

[issue42540] Debug pymalloc crash when using os.fork() [regression]



STINNER Victor  added the comment:


New changeset 736684b1bb67369a2e95a9f621752deead44e7ef by Sam Gross in branch 
'main':
bpo-42540: reallocation of id_mutex should not force default allocator 
(GH-29564)
https://github.com/python/cpython/commit/736684b1bb67369a2e95a9f621752deead44e7ef


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42540] Debug pymalloc crash when using os.fork() [regression]



Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 3.0 -> 4.0
pull_requests: +27842
pull_request: https://github.com/python/cpython/pull/29599

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21876] os.rename(src, dst) does nothing when src and dst files are hard-linked

2021-11-17 Thread Eryk Sun



Eryk Sun  added the comment:

> It's nice if os.rename() can be atomic. 

How about fixing the behavior in shutil.move()? Currently it tries 
os.rename(src, dst) without checking for this case. For example:

>>> os.path.samefile('src', 'dst')
True
>>> shutil.move('src', 'dst')
'dst'
>>> os.path.exists('src')
True
>>> os.stat('src').st_nlink
2

---

Note that there's nothing to fix, and not much to document in Windows. It works 
as expected. A slight difference from the general case is that os.replace() is 
required only when replacing a different file, not a hard link for the same 
file.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45832] Misleading membersip expression documentation

2021-11-17 Thread Raymond Hettinger



Raymond Hettinger  added the comment:

This section presumes that the usual hash invariant holds:  a==b implies 
hash(a)==hash(b).  We could repeat that here but I don't think it makes the 
docs better or more useable to require that docs repeat the same facts in 
multiple places.

Alternatively, the sentence could be split to cover both cases:

"""
For sequence container types such as list, tuple, or collections.deque,
the expression `x in y` is equivalent to `any(x is e or x == e for e in y)`.
For container that use hashing, such as dict, set, or frozenset, 
the expression `x in y` is equivalent to `any(x is e or x == e for e in y if 
hash(x) == hash(e))`.
"""

While that is more precise, it borders on being pedantic and likely doesn't 
make the average reader better off.

Consider submitting a feature request to pandas suggesting that they harmonize 
their hash functions with their counterparts in numpy.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45831] _Py_DumpASCII() writes characters one by one leading to corrupted tracebacks



Change by miss-islington :


--
pull_requests: +27841
pull_request: https://github.com/python/cpython/pull/29598

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45831] _Py_DumpASCII() writes characters one by one leading to corrupted tracebacks



Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 2.0 -> 3.0
pull_requests: +27840
pull_request: https://github.com/python/cpython/pull/29597

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45831] _Py_DumpASCII() writes characters one by one leading to corrupted tracebacks


Łukasz Langa  added the comment:


New changeset b919d8105c4d77f00509b6d3ab2073f09db640de by Victor Stinner in 
branch 'main':
bpo-45831: _Py_DumpASCII() uses a single write() call if possible (GH-29596)
https://github.com/python/cpython/commit/b919d8105c4d77f00509b6d3ab2073f09db640de


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40059] Provide a toml module in the standard library

2021-11-17 Thread Michał Górny


Michał Górny  added the comment:

I've already asked that, and the author said it's fine but with the deprecated 
text file support removed:

https://github.com/hukkin/tomli/issues/141#issuecomment-968056905

That said, for consistency with json and so on we'd probably want to combine 
tomli and tomli_w to get dumping support as well.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45116] Performance regression 3.10b1: inlining issue in the big _PyEval_EvalFrameDefault() function with Visual Studio (MSC)

2021-11-17 Thread neonene



neonene  added the comment:

>This essentially disables PGO.

Thank you for the suggestion. I'll take another experimental aproach to reduce 
the size of 3.11 evalfunc for stronger validation.


>@neonene what's the importance of PR29565?

While we are talking about function size, I would like to use around PR29565 
for consistent reporting. I think any commit is okay to reproduce the issue.

And please ignore the patch to build.bat.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45518] Invalid example for typing

2021-11-17 Thread Bozhi You



Bozhi You  added the comment:

My bad. Didn't notice the version constraint. Thank you, Logan!

--
stage:  -> resolved
status: pending -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41735] Thread locks in zlib module may go wrong in rare case

2021-11-17 Thread Gregory P. Smith



Change by Gregory P. Smith :


--
assignee:  -> gregory.p.smith
resolution: later -> 
stage: resolved -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45832] Misleading membersip expression documentation

2021-11-17 Thread Harald Husum



New submission from Harald Husum :

https://docs.python.org/3/reference/expressions.html#membership-test-operations

> For container types such as list, tuple, set, frozenset, dict, or 
> collections.deque, the expression `x in y` is equivalent to `any(x is e or x 
> == e for e in y)`.

Yet:

```py
import pandas as pd
import numpy as np

pd_0_dt = pd.Timedelta(0)
np_0_dt = np.timedelta64(0)

cm = (pd_0_dt, np_0_dt)

d1 = {np_0_dt: pd_0_dt}
d2 = {pd_0_dt: np_0_dt}

def test_membership_doc_claim(candidate_members, dct):
for m in candidate_members:
if m in dct:
assert any(m is e or m == e for e in dct)

if any(m is e or m == e for e in dct):
assert m in dct

if __name__ == "__main__":
test_membership_doc_claim(cm, d1) # Fails
test_membership_doc_claim(cm, d2) # Fails

```

Not too surprised, given the td.__hash__() implementation differs between these 
classes, but they are considered equal none the less.

Unsure whether it is the dict implementation or the doc claim that needs to 
budge here.

--
assignee: docs@python
components: Documentation
messages: 406485
nosy: docs@python, eric.araujo, ezio.melotti, harahu, mdk, willingc
priority: normal
severity: normal
status: open
title: Misleading membersip expression documentation
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40059] Provide a toml module in the standard library

2021-11-17 Thread Brett Cannon



Brett Cannon  added the comment:

> I'd love it if we could make this happen for Python 3.11. What can I do to 
> help, e.g. would it be helpful if I drafted a PEP?

A PEP isn't necessarily required right now as 
https://discuss.python.org/t/how-do-we-want-to-manage-additions-removals-to-the-stdlib/10681
 needs to be resolved by the steering council first (it's on our agenda).

> Several projects have switched to using tomli 
> (https://github.com/hukkin/tomli), which is fully compatible with TOML v1.0.0 
> and is maybe 800 LoC with claimed 100% branch coverage.

Part of the issue with tomli is how new it is; 
https://pypi.org/project/tomli/#history shows it is only 6 months old and we 
typically require a project be at least a year old before we consider pulling 
it in.

If you want to help you could ask the author of tomli what they think about it 
being added to the stdlib and whether they support that idea.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45691] Partial moving of core objects to interpreter state is incorrect at best, unsafe at worse.



STINNER Victor  added the comment:

> many classes are allocated statically

Right. Changing that is an hard problem :-( See for example bpo-40601 "[C API] 
Hide static types from the limited C API".

I tried once to "free" / reset static types in Py_Finalize(), but it's hard to 
implement properly :-(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21876] os.rename(src, dst) does nothing when src and dst files are hard-linked



STINNER Victor  added the comment:

It's nice if os.rename() can be atomic. It can be atomic if it is implemented 
as a single syscall and the source and the destination are on the same 
filesystem.

If we modify the implement to check if src and/or dst is a hardlink, we will 
need more syscalls which cannot be atomic. What if another process replaces src 
or dst before or after the rename() syscall?

I suggest to document the issue. Or maybe we need a new flavor which keeps the 
current behavior.

"Atomic write" is an old topic and no one managed to implement it in a portable 
way. So maybe you can ignore the atomicity constraint.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40222] "Zero cost" exception handling



Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45831] _Py_DumpASCII() writes characters one by one leading to corrupted tracebacks



STINNER Victor  added the comment:

> This is also a performance issue

Yep. On Windows, it already happened to me to see characters written one by one 
when the machine and the terminal was super slow. I expect faster dump if an 
ASCII string can be written at once.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45831] _Py_DumpASCII() writes characters one by one leading to corrupted tracebacks



Change by STINNER Victor :


--
keywords: +patch
nosy: +vstinner
nosy_count: 1.0 -> 2.0
pull_requests: +27839
pull_request: https://github.com/python/cpython/pull/29596

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45831] _Py_DumpASCII() writes characters one by one leading to corrupted tracebacks


New submission from Łukasz Langa :

This is also a performance issue but I'm mostly concerned about the stdout 
corruption aspect. When more than one thread aborts at the same time, output 
ends up looking like the example below on the terminal. Unless you understand 
the issue is output written out byte-by-byte, you might get misled into seeing 
memory corruption.


Fatal Python error: Aborted

Fatal Python error: Fatal Python error: Fatal Python error: AbortedAborted



AbortedCurrent thread 0x

000110cf4dc0 (most recent call first):
Current thread 0xCurrent thread 0x  File 000110cf4dc0Current thread 0x 
(most recent call first):
000110cf4dc110cf4dc0 (most recent call first):
 (most recent call first):
"  File   File   File 
/tmp"""tctmtmppmpyp//tcchppoy/nytc-hptrohyonoton-t-rh-ormooaoontt-i--rnmomo/aatiLi-nni/m/baLLriiinbab/rrLraayirr/byyr//FFaFrrrayam/mrFeearwmaowermwoeorkrwkkossr//kPPsyy/ttPhsyo/tnhP.hyotnoh.nof.nrff.arfmarreaammwmeeoewwrowkro/orVrkkek///rVVseVierorsesnirsioson/ns3si.//o331.n1s.111///l13il.bi1/b1//pl/ypiltybht/ihopbno/n3yp3.t.1hy11o1/n/m3mu.tul1lth1ti/oipmnrpu3ol.rc1toe1cs/esmsiunsliignt/pigspr/yorncocscehysnrseocsihnsnirigoz/nesi.yzpnyen."cgph/yrs"oy,
 line nn95i, line  in c95z_ in e__h.e_pnertnyoet"ernr_i__z_
, line
e  File   File .95p in y_""_//"ttemmnppt//, line ccepp95yy in 
ttr_h_ho_on_-ner-nr
ooto  File ote-rm"_a/_ti
nm  File 
p//Lcitbp"yr-ta/hrtym/pFm/racaipmnye/owLtoihbrokrnnas-r/-Pyry/rtFoohooorttn-a.mfma-eirmnwaamoiernwk/osL//rikPL/yiVtbherroansrb.iyfo/nrFasr/ma3emw.e1owr1o/kr/lkiVsbe/r/Pspyyittohhnoosnn/.33f..r11a11m//elmwiuolbrt/kpi/pyVtrehorocsenis3os.n1is1n//g3/m.uq1ul1te/iulpeirsb.o/cppeyys"ts,
 line h102io in nng3g/.qe1utr
e1u  File a/"rme/uytsl/.mtppFi/yrpc"rpa, line yomt102ehcw in eosgnose-rirtnk
gs  File o//o"Pt/qty-mumpea/ticunpey/hLtosih.bopnrnya-".rr, line yfo/r102oF in 
argtaemm-temew
woaor  File 
rikkns/"L//Pi/ybtVtrmhapeor/nyc/.pFfyrrtaahmmoerwensow-rorkorsok/t/Pi-Vymoetarhisonnin.o/sfnLrs//ai33mb.er.1w11o1ar/rk/l/lyViieb/brF/srp/aipyomntyhseo/tn3w3.ho1.r11o1/k/lsnic3ob//n.cpPu1yr1yrt/ehmntuothl/otfinup.rtfourrcaeenss3s/.ip1nr1gom/c/qecusoesne.uwpceyuosr."rp,
 line ry236ke in /"_nVpt, line re/o102fcr in sueitgsuesr_toewn
...

--
components: Interpreter Core
messages: 406480
nosy: lukasz.langa
priority: normal
severity: normal
stage: patch review
status: open
title: _Py_DumpASCII() writes characters one by one leading to corrupted 
tracebacks
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45116] Performance regression 3.10b1: inlining issue in the big _PyEval_EvalFrameDefault() function with Visual Studio (MSC)



Guido van Rossum  added the comment:

@neonene what's the importance of PR29565?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45830] Custom pickler memory leak

2021-11-17 Thread Douglas Raillard



New submission from Douglas Raillard :

The following script exhibits the memory leak. It only happens if 
"dispatch_table" is set _before_ calling super().__init__, which is pretty 
unexpected.


import pickle
import io
import gc
import tracemalloc

tracemalloc.start(10)
snap = tracemalloc.take_snapshot()

class MyPickler(pickle.Pickler):
def __init__(self, *args, **kwargs):
# Swapping the next 2 lines "solves" the memory leak for some reason
self.dispatch_table = dict()
super().__init__(*args, **kwargs)

l=[]
for i in range(1):
if i % 1000 == 0:
print('='*80)
snap2 = tracemalloc.take_snapshot()
stats=snap2.compare_to(snap, 'lineno')
for s in stats[:10]:
print(s)
snap = snap2

f = io.BytesIO()
MyPickler(f)
gc.collect()


The output of the last iteration is as follow. The leak of 562 kiB is apparent:

testmem.py:12: size=562 KiB (+62.5 KiB), count=9000 (+1000), average=64 
B
/usr/lib/python3.10/tracemalloc.py:125: size=2376 B (-72 B), count=33 
(-1), average=72 B
/usr/lib/python3.10/tracemalloc.py:129: size=72 B (+72 B), count=1 
(+1), average=72 B
/usr/lib/python3.10/tracemalloc.py:502: size=252 B (+28 B), count=9 
(+1), average=28 B
/usr/lib/python3.10/tracemalloc.py:498: size=2104 B (+0 B), count=36 
(+0), average=58 B
/home/dourai01/Work/lisa/lisa/testmem.py:10: size=1844 B (+0 B), 
count=9 (+0), average=205 B
/usr/lib/python3.10/tracemalloc.py:193: size=1680 B (+0 B), count=35 
(+0), average=48 B
/usr/lib/python3.10/tracemalloc.py:547: size=1256 B (+0 B), count=3 
(+0), average=419 B
/usr/lib/python3.10/tracemalloc.py:226: size=832 B (+0 B), count=2 
(+0), average=416 B
/usr/lib/python3.10/tracemalloc.py:173: size=800 B (+0 B), count=2 
(+0), average=400 B

If "dispatch_table" is set after calling super().__init__, there is no leak 
anymore:

/usr/lib/python3.10/tracemalloc.py:135: size=740 B (+740 B), count=7 
(+7), average=106 B
/usr/lib/python3.10/tracemalloc.py:125: size=2088 B (-656 B), count=29 
(-4), average=72 B
/usr/lib/python3.10/tracemalloc.py:136: size=320 B (+320 B), count=1 
(+1), average=320 B
/usr/lib/python3.10/tracemalloc.py:132: size=0 B (-256 B), count=0 (-1)
/usr/lib/python3.10/tracemalloc.py:129: size=72 B (+72 B), count=1 
(+1), average=72 B
/usr/lib/python3.10/tracemalloc.py:498: size=2008 B (+48 B), count=34 
(+1), average=59 B
/usr/lib/python3.10/tracemalloc.py:193: size=1584 B (+48 B), count=33 
(+1), average=48 B
/usr/lib/python3.10/tracemalloc.py:502: size=196 B (-28 B), count=7 
(-1), average=28 B
/usr/lib/python3.10/tracemalloc.py:126: size=84 B (+28 B), count=3 
(+1), average=28 B

--
components: Library (Lib)
messages: 406478
nosy: douglas-raillard-arm
priority: normal
severity: normal
status: open
title: Custom pickler memory leak
type: resource usage
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44525] Implement CALL_FUNCTION adaptive interpreter optimizations



Change by Mark Shannon :


--
pull_requests: +27838
pull_request: https://github.com/python/cpython/pull/29595

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45829] Remove C stack use by specializing BINARY_SUBSCR, STORE_SUBSCR, LOAD_ATTR, and STORE_ATTR



Guido van Rossum  added the comment:

That's a good one too, and perhaps simpler.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45829] Remove C stack use by specializing BINARY_SUBSCR, STORE_SUBSCR, LOAD_ATTR, and STORE_ATTR



Mark Shannon  added the comment:

I don't think it matter much which we do first.
I happened to do BINARY_SUBSCR first.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45816] Python does not support standalone MSVC v143 (VS 2022) Build Tools

2021-11-17 Thread Steve Dower



Steve Dower  added the comment:

Could you post the errors?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45116] Performance regression 3.10b1: inlining issue in the big _PyEval_EvalFrameDefault() function with Visual Studio (MSC)

2021-11-17 Thread Steve Dower



Steve Dower  added the comment:

> -set pgo_job=-m test --pgo
> +set pgo_job=-c"pass"

This essentially disables PGO. You won't get anything valid or useful from 
analysing its results if you don't give it a somewhat reasonable profile 
(preferably one that exercises the interpreter loop, which "pass" does not).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Symbolic links on Windows

2021-11-17 Thread ast


Le 17/11/2021 à 13:10, Python a écrit :

ast wrote:

Hello,

It seems that symbolic links on Windows are not
well reconized by modules os or pathlib.

I have a file named json.txt on my destop. With a
drag and drop right click on it I create a link
automatically named: json.txt - Raccourci.lnk


This is not a symbolic link.

Symbolic links were introduced in Windows Vista/Windows
Server 2008. A .lnk file is just a regular file containing
the path of the target (you can open it and read it if you
wish) that is interpreted in a specific way by the graphical
Shell and some other applications.




Yes you are right

On Windows symbolic links are created in a cmd tool:
e.g

mklink json2.txt json.txt

And it is something different than shortcut files (suffix .lnk)
--
https://mail.python.org/mailman/listinfo/python-list

Symbolic links on Windows

2021-11-17 Thread ast


Hello,

It seems that symbolic links on Windows are not
well reconized by modules os or pathlib.

I have a file named json.txt on my destop. With a
drag and drop right click on it I create a link
automatically named: json.txt - Raccourci.lnk

Then:

>>> from pathlib import Path

>>> p2 = Path('C:/Users/jm/desktop/json.txt - Raccourci.lnk')

>>> p2
WindowsPath('C:/Users/jm/desktop/json.txt - Raccourci.lnk')

>>> p2.exists()
True

>>> p2.is_file()
True

>>> p2.is_symlink()
False

With this last command I was expecting True and for
p2_is_file() I was expecting False



With os, it's the same

import os

>>> os.path.exists(p2)
True

>>> os.path.isfile(p2)
True

os.path.islink(p2)
False


What's wrong plz ?
--
https://mail.python.org/mailman/listinfo/python-list

Re: Proliferation of Python packaging formats

2021-11-17 Thread Jon Ribbens via Python-list

On 2021-11-17, Skip Montanaro  wrote:
> Is the proliferation of packaging formats in Python as nutzo as this author
> believes?
>
> https://drewdevault.com/2021/11/16/Python-stop-screwing-distros-over.html
>
> Asking because I've never been in the business of releasing "retail" Python
> applications or packages.

Well the first paragraph is ridiculous. I've never heard of half of the
things he lists as being necessary to deal with, and half the remainder
are just words relating to packages, i.e. you could make a similar list
for any language.

The other major problem with that post is that it gives no examples
or even clues as to what the author's actual problem is...

On the other hand it is true that Python's packaging system is confusing
and very badly documented, and I doubt the vast majority of people have
any idea of what the difference between 'distutils' and 'setuptools' is
(I certainly don't), except inasmuch as 'setuptools' (and 'wheel') is
something you have to remember to manually update after creating a
virtual environment before installing your actual packages.

It's also true that you have to remember with Python that you basically
cannot use 'pip' to install anything globally as it will interfere with
the operating system's packaging. You must use virtual envs for
everything, or the operating system's provided packages.

Also PEP 518's choice of TOML is absolutely risible, a language about
which the only positive thing can be said is that it's not as bad as
YAML, and for which Python doesn't even have a built-in parser -
something that should have absolutely ruled it out as an option.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue45829] Remove C stack use by specializing BINARY_SUBSCR, STORE_SUBSCR, LOAD_ATTR, and STORE_ATTR



Guido van Rossum  added the comment:

Of these, presumably LOAD_GETATTR is by far the most used, so should we try 
that first?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45829] Remove C stack use by specializing BINARY_SUBSCR, STORE_SUBSCR, LOAD_ATTR, and STORE_ATTR



Change by Guido van Rossum :


--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45825] Heap Segmentation Fault

2021-11-17 Thread Bill Borskey

Bill Borskey  added the comment:

No worries. I find bugs in my day job, thought this might be a useful segfault 
but it segfaults because it’s incrementing that reference count on the pyobj 
that don’t exist. So pretty lame. I did spend an hour tracking it down so I 
thought I’d let y’all know in case you wanted to fix it. 

Cheers 

> On Nov 17, 2021, at 6:08 AM, Zachary Ware  wrote:
> 
> 
> Zachary Ware  added the comment:
> 
> In general, as soon as you touch ctypes you're on your own :).  ctypes has no 
> protections for this kind of case, so you need to protect yourself.
> 
> If you came across this some way that *should* have been safe, please provide 
> more information. But just passing a random memory address to ctypes is 
> likely to cause segfaults or worse.
> 
> --
> nosy: +zach.ware
> resolution:  -> not a bug
> status: open -> pending
> 
> ___
> Python tracker 
> 
> ___

--
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45573] Use pkg-config autoconf macros to detect flags for Modules/Setup

2021-11-17 Thread Christian Heimes



Change by Christian Heimes :


--
pull_requests: +27837
pull_request: https://github.com/python/cpython/pull/29594

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Proliferation of Python packaging formats

2021-11-17 Thread Chris Angelico

On Thu, Nov 18, 2021 at 12:20 AM Skip Montanaro
 wrote:
>
> Is the proliferation of packaging formats in Python as nutzo as this author
> believes?
>
> https://drewdevault.com/2021/11/16/Python-stop-screwing-distros-over.html
>
> Asking because I've never been in the business of releasing "retail" Python
> applications or packages.
>

It's a problem if you feel the need to release in every single
possible way. The XKCD in that example is a showcase of what happens
when you attempt to install in every possible way, which is usually a
consequence of everyone releasing in just one way.

He's blaming the PSF, but that's not really fair. The PSF has never
said "hey everyone, make sure you package for conda as well as posting
on PyPI". Part of the proliferation comes because there are multiple
completely independent distributions, and in the XKCD shown there, he
has installed Python at least five times from completely different
sources (the OS-provided one, two from Homebrew, one from Anaconda,
and one from python.org), and maybe installed pip into one of those as
well. Duh, of course that's going to cause problems.

Here's the reality: OS-provided Pythons are always going to exist, and
they will always have a place (because I should be able to say "apt
install " without having to worry about whether that's going to
download from PyPI), and they'll never have 100% of everything exactly
how you want it, so you will always have more than one place where
packages can be installed, more than one package format. And it's not
the PSF's fault, nor the PSF's responsibility.

The massive complexities come when people aren't satisfied with the
status quo, and create a new thing. The blog mentions tox, flit,
conda, and poetry, all of which are third-party distributions or
package managers. Is the PSF supposed to tell people "don't create new
package managers for any Python code"? Seriously?

Blog dismissed as whining.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue45116] Performance regression 3.10b1: inlining issue in the big _PyEval_EvalFrameDefault() function with Visual Studio (MSC)

2021-11-17 Thread neonene



neonene  added the comment:

Here are the 3 steps to reproduce with minimal pgo training. (vs2019)

1. Download the source archive of PR29565 and extract.
   
https://github.com/python/cpython/archive/6a84d61c55f2e543cf5fa84522d8781a795bba33.zip

2. Apply the following patch.

==
--- PCbuild/build.bat
+++ PCbuild/build.bat
@@ -66 +66 @@
-set pgo_job=-m test --pgo
+set pgo_job=-c"pass"
--- PCbuild/pyproject.props
+++ PCbuild/pyproject.props
@@ -47,2 +47,3 @@
   /utf-8 %(AdditionalOptions)
+  /d2inlinelogfull:_PyEval_EvalFrameDefault 
%(AdditionalOptions)
 
==

3. Build [Rebuild]

   PCbuild\build --no-tkinter --pgo > build.log [-r]

   According to the inlining section in the log, any function that has one or 
more conditional expressions got "reject" from inliner.

   > Inlinee for function _PyEval_EvalFrameDefault 
   >  -_Py_EnsureFuncTstateNotNULL (pgo hard reject)
   >  ...
   >  _Py_INCREF (pgu decision)
   >  _Py_INCREF (pgu decision)
   >  -_Py_XDECREF (pgo hard reject)
   >  -_Py_XDECREF (pgo hard reject)
   >  -_Py_DECREF (pgo hard reject)
   >  -_Py_DECREF (pgo hard reject)
   >  ...


Profiling scores can be shown on VS2019 Command Prompt.

   pgomgr PCbuild\amd64\python311.pgd /summary [/detail] > largefile.txt

   * pgomgr.exe (or profile itself) has an issue.
 https://developercommunity.visualstudio.com/t/1560909


Unused opcodes in this training

   ROT_THREE, DUP_TOP_TWO, UNARY_POSITIVE, UNARY_NEGATIVE,
   BINARY_OP_ADD_FLOAT, UNARY_INVERT, BINARY_OP_MULTIPLY_INT,
   BINARY_OP_MULTIPLY_FLOAT, GET_LEN, MATCH_MAPPING, MATCH_SEQUENCE,
   MATCH_KEYS, LOAD_ATTR_SLOT, LOAD_METHOD_CLASS, GET_AITER, GET_ANEXT,
   BEFORE_ASYNC_WITH, END_ASYNC_FOR, STORE_ATTR_SLOT,
   STORE_ATTR_WITH_HINT, GET_YIELD_FROM_ITER, PRINT_EXPR, YIELD_FROM,
   GET_AWAITABLE, LOAD_ASSERTION_ERROR, SETUP_ANNOTATIONS, UNPACK_EX,
   DELETE_ATTR, DELETE_GLOBAL, ROT_N, COPY, DELETE_DEREF,
   LOAD_CLASSDEREF, MATCH_CLASS, SET_UPDATE, DO_TRACING

   I managed to activate inliner experimentally by removing the 36 op-cases 
from switch and merging/removing many macros.


Static instruction counts of _PyEval_EvalFrameDefault()

   PR29565   : 6882 (down to 4400 with above change)

   PR29482   : 7035
   PR29482~1 : 7742
   3.10.0+   : 3980 (well inlined sharing DISPATCH macro)
   3.10.0: 5559
   3.10b1: 5680
   3.10a7: 4117 (well inlined)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45825] Heap Segmentation Fault

2021-11-17 Thread Zachary Ware



Zachary Ware  added the comment:

In general, as soon as you touch ctypes you're on your own :).  ctypes has no 
protections for this kind of case, so you need to protect yourself.

If you came across this some way that *should* have been safe, please provide 
more information. But just passing a random memory address to ctypes is likely 
to cause segfaults or worse.

--
nosy: +zach.ware
resolution:  -> not a bug
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Proliferation of Python packaging formats

2021-11-17 Thread Skip Montanaro

Is the proliferation of packaging formats in Python as nutzo as this author
believes?

https://drewdevault.com/2021/11/16/Python-stop-screwing-distros-over.html

Asking because I've never been in the business of releasing "retail" Python
applications or packages.

Skip
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue45512] [sqlite3] simplify "isolation level"



Change by Erlend E. Aasland :


--
pull_requests: +27836
pull_request: https://github.com/python/cpython/pull/29593

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45512] [sqlite3] simplify "isolation level"



Erlend E. Aasland  added the comment:

> Now we close this issue right?

Not yet ;) I have an AC tweak I'd like to add. Wait for a PR anytime soon.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45512] [sqlite3] simplify "isolation level"



Dong-hee Na  added the comment:

@erlendaasland

Now we close this issue right?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45512] [sqlite3] simplify "isolation level"



Dong-hee Na  added the comment:


New changeset e002bbc6cce637171fb2b1391ffeca8643a13843 by Dong-hee Na in branch 
'main':
bpo-45512: Simplify manage isolation level (GH-29562)
https://github.com/python/cpython/commit/e002bbc6cce637171fb2b1391ffeca8643a13843


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40059] Provide a toml module in the standard library



Change by Erlend E. Aasland :


--
nosy: +erlendaasland

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39026] Include/cpython/pystate.h contains non-relative of initconfig.h include causing macOS Framework include failure

2021-11-17 Thread Gaige Paulsen



Gaige Paulsen  added the comment:

Short version is not yet. I spent time on it but ran into an issue getting the 
full build on the machine I was building on. I expect to try it again before 
the weekend.

Is there a binary copy of the framework from the CI somewhere? If so, I can 
certainly do a much faster check against building with the framework, saving me 
the step I'm having problem with right now.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45824] CSV module document does not include how to append files

2021-11-17 Thread Eric V. Smith



Eric V. Smith  added the comment:

I agree with Raymond.

--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45829] Remove C stack use by specializing BINARY_SUBSCR, STORE_SUBSCR, LOAD_ATTR, and STORE_ATTR



Change by Mark Shannon :


--
keywords: +patch
pull_requests: +27835
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/29592

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45816] Python does not support standalone MSVC v143 (VS 2022) Build Tools

2021-11-17 Thread Ma Lin



Ma Lin  added the comment:

There are 5 link errors when building the PGO build.
Command: build --pgo

--
nosy: +malin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45512] [sqlite3] simplify "isolation level"



Dong-hee Na  added the comment:


New changeset 5f9247e36a0213b0dcfd43533db5cf6570895cfd by Erlend Egeberg 
Aasland in branch 'main':
bpo-45512: Extend `sqlite3` test suite regarding isolation levels (GH-29576)
https://github.com/python/cpython/commit/5f9247e36a0213b0dcfd43533db5cf6570895cfd


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45256] Remove the usage of the C stack in Python to Python calls



Mark Shannon  added the comment:

https://bugs.python.org/issue45829 is the related issue for special methods

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45829] Remove C stack use by specializing BINARY_SUBSCR, STORE_SUBSCR, LOAD_ATTR, and STORE_ATTR



New submission from Mark Shannon :

We can remove the C stack use and general overhead of calling special methods 
implemented in Python for attribute access and indexing.

Each operation has a special method that implements it. When that special 
method is implemented in Python, we should avoid the `tp_xxx` slot machinery 
and use the same mechanism we use for normal calls to Python functions. 

* BINARY_SUBSCR: `__getitem__`
* STORE_SUBSCR: `__setitem__`
* LOAD_ATTR: `__getattribute__` (and maybe `__getattr__`)
* STORE_ATTR: `__setattr__`

It probably isn't worth bothering with the deletion forms.

The getters (`__getitem__` and `__getattribute__`) are relatively simple, as 
the call returns the result.

The setters are a bit more complicated as the return value needs to be 
discarded, so an additional frame which discards the result of the call needs 
to be inserted.

--
assignee: Mark.Shannon
components: Interpreter Core
messages: 406461
nosy: Mark.Shannon, brandtbucher, pablogsal
priority: normal
severity: normal
status: open
title: Remove C stack use by specializing BINARY_SUBSCR, STORE_SUBSCR, 
LOAD_ATTR, and STORE_ATTR
type: performance
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45828] [sqlite3] use unraisable exceptions in callbacks



Change by Erlend E. Aasland :


--
keywords: +patch
pull_requests: +27834
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/29591

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45828] [sqlite3] use unraisable exceptions in callbacks



Change by Erlend E. Aasland :


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45643] SIGSTKFLT is missing from the signals module on Linux

2021-11-17 Thread Gareth Rees



Gareth Rees  added the comment:

Tagging vstinner as you have touched Modules/signalmodule.c a few times in the 
last year. What do you think?

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45828] [sqlite3] use unraisable exceptions in callbacks



New submission from Erlend E. Aasland :

In order to print tracebacks from exceptions in SQLite callbacks, the sqlite3 
extension module provides sqlite3.enable_callback_tracebacks(flag). Setting the 
flag to True instructs the sqlite3 extension module to PyErr_Print() if an 
exception occurs during a callback. Else, PyErr_Clear() is called.

>From the sqlite3.enable_callback_tracebacks() docs:

By default you will not get any tracebacks in user-defined functions,
aggregates, converters, authorizer callbacks etc. If you want to debug
them, you can call this function with flag set to True. Afterwards, you
will get tracebacks from callbacks on sys.stderr. Use False to disable the
feature again.


Few other exceptions use a similar approach:

$ grep -r PyErr_Print Modules 
Modules/_tkinter.c:PyErr_Print();
Modules/_testcapimodule.c:PyErr_Print();
Modules/main.c:PyErr_Print();
Modules/main.c:PyErr_Print();
Modules/main.c:PyErr_Print();
Modules/_io/bytesio.c:PyErr_Print();
Modules/_sqlite/connection.c:PyErr_Print();
Modules/_sqlite/cursor.c:PyErr_Print();
Modules/_xxtestfuzz/fuzzer.c:PyErr_Print();
Modules/_xxtestfuzz/fuzzer.c:PyErr_Print();
Modules/_xxtestfuzz/fuzzer.c:PyErr_Print();
Modules/_xxtestfuzz/fuzzer.c:PyErr_Print();
Modules/_xxtestfuzz/fuzzer.c:PyErr_Print();
Modules/_xxtestfuzz/fuzzer.c:PyErr_Print();
Modules/_xxtestfuzz/fuzzer.c:PyErr_Print();
Modules/_ctypes/callbacks.c:PyErr_Print();


We get a higher hit for unraisable exceptions:

$ grep -r PyErr_WriteUnraisable Modules | wc -l
  45


AFAICS, using unraisable exceptions is a better approach.


Current behaviour:

Python 3.10.0 (v3.10.0:b494f5935c, Oct  4 2021, 14:59:20) [Clang 12.0.5 
(clang-1205.0.22.11)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> cx = sqlite3.connect(":memory:")
>>> cx.set_trace_callback(lambda stmt: 5/0)
>>> cx.execute("select 1")

>>> sqlite3.enable_callback_tracebacks(True)
>>> cx.execute("select 1")
Traceback (most recent call last):
  File "", line 1, in 
ZeroDivisionError: division by zero



With unraisable exceptions:

Python 3.11.0a2+ (heads/sqlite-unraisable-exceptions-dirty:de29590d6a, Nov 
17 2021, 10:29:19) [Clang 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> cx = sqlite3.connect(":memory:")
>>> cx.set_trace_callback(lambda stmt: 5/0)
>>> cx.execute("select 1")

>>> sqlite3.enable_callback_tracebacks(True)
>>> cx.execute("select 1")
Exception ignored in:  at 0x10b4e3ee0>
Traceback (most recent call last):
  File "", line 1, in 
ZeroDivisionError: division by zero



The user experience is mostly unchanged; we get one extra line, telling us that 
the exception was ignored. Also, users can now use sys.unraisablehook:

>>> sys.unraisablehook = lambda unraisable: print(unraisable)
>>> cx.execute("select 1")
UnraisableHookArgs(exc_type=, 
exc_value=ZeroDivisionError('division by zero'), exc_traceback=, err_msg=None, object= at 0x10b4e3ee0>)



The only question I have, is if we should deprecate 
sqlite3.enable_callback_tracebacks() after switching to unraisable exceptions.

--
components: Extension Modules
messages: 406459
nosy: erlendaasland
priority: normal
severity: normal
status: open
title: [sqlite3] use unraisable exceptions in callbacks
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28806] Improve the netrc library

2021-11-17 Thread Andrew Svetlov



Change by Andrew Svetlov :


--
versions: +Python 3.11 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40059] Provide a toml module in the standard library

2021-11-17 Thread Shantanu



Shantanu  added the comment:

I'd love it if we could make this happen for Python 3.11. What can I do to 
help, e.g. would it be helpful if I drafted a PEP?

Several projects have switched to using tomli 
(https://github.com/hukkin/tomli), which is fully compatible with TOML v1.0.0 
and is maybe 800 LoC with claimed 100% branch coverage.

There's also tomlkit, which has been around for longer, but is perhaps larger 
and more feature-ful than we'd want in stdlib, e.g. it allows you to roundtrip 
dump and load TOML documents preserving comments, whitespace, etc.

--
nosy: +hauntsaninja

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28806] Improve the netrc library

2021-11-17 Thread Andrew Svetlov



Andrew Svetlov  added the comment:


New changeset 15409c720be0503131713e3d3abc1acd0da07378 by Emmanuel Arias in 
branch 'main':
bpo-28806: Continue work: improve the netrc library (GH-26330)
https://github.com/python/cpython/commit/15409c720be0503131713e3d3abc1acd0da07378


--
nosy: +asvetlov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45126] [sqlite3] cleanup and harden Connection and Cursor init