[issue43349] [doc] incorrect tuning(7) manpage link

[Erlend Egeberg Aasland]

Erlend Egeberg Aasland  added the comment:

> FreeBSD only (?) system call [...]

Correction, it's not a sys call. It belongs to the misc manpage section.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41837] Upgrade installers to OpenSSL 1.1.1j

[miss-islington]

Change by miss-islington :


--
pull_requests: +23464
pull_request: https://github.com/python/cpython/pull/24679

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41837] Upgrade installers to OpenSSL 1.1.1j

[miss-islington]

Change by miss-islington :


--
pull_requests: +23463
pull_request: https://github.com/python/cpython/pull/24678

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41837] Upgrade installers to OpenSSL 1.1.1j

[Ned Deily]

Ned Deily  added the comment:


New changeset 0242494a156970186cbc4121ccf03aefbddea716 by Ned Deily in branch 
'master':
bpo-41837: Update macOS installer build to use OpenSSL 1.1.1j. (GH-24677)
https://github.com/python/cpython/commit/0242494a156970186cbc4121ccf03aefbddea716


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43349] [doc] incorrect tuning(7) manpage link

[Erlend Egeberg Aasland]

New submission from Erlend Egeberg Aasland :

In https://docs.python.org/3.10/library/resource.html#resource.RLIMIT_SWAP, 
tuning(7) points to https://manpages.debian.org/tuning(7), however this is a 
FreeBSD only (?) system call, so the link is incorrect.

I suggest linking to either:
- https://docs.freebsd.org/en/books/handbook/config/
- https://www.freebsd.org/cgi/man.cgi?query=tuning=7=html

--
assignee: docs@python
components: Documentation
messages: 387845
nosy: docs@python, erlendaasland
priority: normal
severity: normal
status: open
title: [doc] incorrect tuning(7) manpage link
versions: Python 3.10, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41837] Upgrade installers to OpenSSL 1.1.1j

[Ned Deily]

Change by Ned Deily :


--
pull_requests: +23462
pull_request: https://github.com/python/cpython/pull/24677

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42627] urllib.request.getproxies() misparses Windows registry proxy settings

[NateScarlet]

Change by NateScarlet :


--
nosy: +NateScarlet

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43103] Add configure --without-static-libpython to not build libpython3.10.a

[Ned Deily]

Ned Deily  added the comment:


New changeset 0608425944932f46b544afea04ae6d301a76f5f2 by Ned Deily in branch 
'master':
bpo-43103: Fix build failure with macOS framework builds. (GH-24676)
https://github.com/python/cpython/commit/0608425944932f46b544afea04ae6d301a76f5f2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43229] freeze searches libpython3.9.so in /usr/lib instead /usr/lib/x86_64-linux-gnu

[Christian Bachmaier]

Christian Bachmaier  added the comment:

Anyone?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43103] Add configure --without-static-libpython to not build libpython3.10.a

[Ned Deily]

Change by Ned Deily :


--
nosy: +ned.deily
nosy_count: 2.0 -> 3.0
pull_requests: +23461
pull_request: https://github.com/python/cpython/pull/24676

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: weirdness with list()

[Greg Ewing]

On 28/02/21 1:17 pm, Cameron Simpson wrote:

[its length in bytes] is presented via the object's __len__ method,



BUT... It also has a __iter__ value, which like any Box iterates over
the subboxes.


You're misusing __len__ here. If an object is iterable and
also has a __len__, its __len__ should return the number of
items you would get if you iterated over it. Anything else
is confusing and can lead to trouble, as you found here.


But is there a cleaner way to do this?


Yes. Give up on using __len__ to get the length in bytes,
and provide another way to do that.

--
Greg

--
https://mail.python.org/mailman/listinfo/python-list

[issue42603] Tkinter: pkg-config is not used to get location of tcl and tk headers/libraries

[Ned Deily]

Ned Deily  added the comment:


New changeset a65b050516a4ec8f5fc591119b94ceaaa5f7afe3 by Ned Deily in branch 
'master':
bpo-42603: Add whatsnew and ACKS entries. (GH-24675)
https://github.com/python/cpython/commit/a65b050516a4ec8f5fc591119b94ceaaa5f7afe3


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Packaging/MANIFEST.in: Incude All, Exclude .gitignore

[Abdur-Rahmaan Janhangeer]

Greetings list,

SInce i have a .gitignore, how do i exclude
all files and folders listed by my gitignore?
How do i include everything by default?

Kind Regards,

Abdur-Rahmaan Janhangeer
about  | blog

github 
Mauritius
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue42603] Tkinter: pkg-config is not used to get location of tcl and tk headers/libraries

[Ned Deily]

Change by Ned Deily :


--
pull_requests: +23460
pull_request: https://github.com/python/cpython/pull/24675

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43347] IDLE crashes in macOS Apple chip, maybe completions

[Ned Deily]

Ned Deily  added the comment:

What version of Python (and Tcl/Tk) are you running on your Intel Macn that 
does not fail? Currently, we use Tcl/Tk 8.6.11 with the macOS 64-bit universal2 
installer variant as of Python 3.9.2 and 3.10.0a4; previously, they used 
8.6.10. All other current installer variants use 8.6.8.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43271] AMD64 Windows10 3.x crash with Windows fatal exception: stack overflow

[Guido van Rossum]

Guido van Rossum  added the comment:

Thanks! That's new info, at least, it wasn't clear to me from Victor's
original comment that this was for a debug build (even though as you say it
can be deduced from context).

Do you feel like submitting a PR along the lines I suggested?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43347] IDLE crashes in macOS Apple chip, maybe completions

[Terry J. Reedy]

Change by Terry J. Reedy :


--
priority: normal -> high

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43347] IDLE crashes in macOS Apple chip, maybe completions

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Completions are in a Listbox.  listboxes are used for completions, stackviewer, 
and a few places in the settings dialog, in particular the keys list.  And 
maybe the drop down list for themes and keysets.  The latter two are the only 
one normally gray.  Try the different settings tabs.

--
nosy: +ned.deily
title: IDLE crashes in -> IDLE crashes in macOS Apple chip, maybe completions

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43343] argparse.REMAINDER missing in online documentation for 3.9.x

[Chigozirim Chukwu]

Chigozirim Chukwu  added the comment:

Alright thanks for the info everyone

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43348] XMLRPC behaves strangely under pythonw, not under python

[Tim Magee]

New submission from Tim Magee :

Summary: I run a SimpleXMLRPCServer in pythonw. When I call an exposed function 
the call appears to be made twice and the connection ends abnormally.

This is Python 3.8.3 Windows 64-bit, with the pywin32 additions, and under 
Windows 7. Boo, hiss, I know -- I won't have access to a win10 machine for a 
week or so, but I get the impression that that won't affect this behaviour, and 
the post-3.8.3 changelog doesn't have anything that (to me) looks relevant in 
it.

I've made a small reproduction that will run on one Windows machine.
You need two small source files (see below):
  * demosrv.py - the server
  * democli.py - the client
The method, in a CMD window:
  * run: start "demosrv" python demosrv.py
[ runs in its own console window ]
  * run: python democli.py
[ one instance of notepad appears and the client exits normally ]
  * Ctrl+C in the demosrv window to stop that
  * run: pythonw demosrv.py
[ runs in background ]
  * run: python democli.py
[ TWO notepad instances pop up ]
[ the client crashes with an unexpected connection ]
I've pasted in the two files and an example session below.

This is very much cut-down from the original.  Before deciding to submit I used 
logging at the top of the exposed function, which appeared to show it being 
called twice.  Logging immediately before the call in the client showed only 
one call.

Thanks,
Tim

#--8< demosrv.py 8<--
from xmlrpc.server import SimpleXMLRPCServer
from subprocess import Popen, DEVNULL

def notepad_me():
try:
cmd = ['c:\\windows\\notepad.exe']
p = Popen(cmd, stdin=DEVNULL, stdout=DEVNULL, stderr=DEVNULL)
return True
except Exception as e:
return False

class NetServer(SimpleXMLRPCServer):
def __init__(self, address):
super(NetServer, self).__init__(address)
self.register_introspection_functions()
self.register_function(notepad_me)

with NetServer(("0.0.0.0", )) as server:
server.serve_forever()

#---8< democli.py 8<-
import sys
from xmlrpc.client import ServerProxy, Error

def go():
try:
with ServerProxy("http://localhost:/;) as proxy:
if proxy.notepad_me():
return 0
except Error as e:
pass
return 1
sys.exit(go())
#
d:\temp\demo>REM Run the server under python.exe

d:\temp\demo>start "demosrv" python demosrv.py
d:\temp\demo>python democli.py

d:\temp\demo>REM That ran up one instance of notepad
d:\temp\demo>REM Ctrl+C to break out the running demosrv.py

d:\temp\demo>pythonw demosrv.py
d:\temp\demo>REM that runs in the background

d:\temp\demo>python democli.py
Traceback (most recent call last):
  File "democli.py", line 13, in 
sys.exit(go())
  File "democli.py", line 7, in go
if proxy.notepad_me():
  File "C:\Python\lib\xmlrpc\client.py", line 1109, in __call__
return self.__send(self.__name, args)
  File "C:\Python\lib\xmlrpc\client.py", line 1450, in __request
response = self.__transport.request(
  File "C:\Python\lib\xmlrpc\client.py", line 1153, in request
return self.single_request(host, handler, request_body, verbose)
  File "C:\Python\lib\xmlrpc\client.py", line 1166, in single_request
resp = http_conn.getresponse()
  File "C:\Python\lib\http\client.py", line 1332, in getresponse
response.begin()
  File "C:\Python\lib\http\client.py", line 303, in begin
version, status, reason = self._read_status()
  File "C:\Python\lib\http\client.py", line 272, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

d:\temp\demo>REM as you see that ran amok
d:\temp\demo>REM and it created two instances of notepad

--
components: Windows
messages: 387837
nosy: paul.moore, steve.dower, tim.golden, tim_magee, zach.ware
priority: normal
severity: normal
status: open
title: XMLRPC behaves strangely under pythonw, not under python
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42128] Structural Pattern Matching (PEP 634)

[Guido van Rossum]

Guido van Rossum  added the comment:


New changeset a22bca6b1e2f3dc11a58f3e5ef5c22e26b8a2d80 by Daniel F Moisset in 
branch 'master':
bpo-42128: Add documentation for pattern matching (PEP 634) (#24664)
https://github.com/python/cpython/commit/a22bca6b1e2f3dc11a58f3e5ef5c22e26b8a2d80


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42603] Tkinter: pkg-config is not used to get location of tcl and tk headers/libraries

[Ned Deily]

Ned Deily  added the comment:

Thanks for the PR. Merged for inclusion in 3.10.0a6.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42603] Tkinter: pkg-config is not used to get location of tcl and tk headers/libraries

[Ned Deily]

Ned Deily  added the comment:


New changeset d20279494a3a311c7aefa313174c45d32aa7f5d1 by Manolis 
Stamatogiannakis in branch 'master':
bpo-42603: Use pkg-config to get TCL/TK paths for tkinter. (GH-23721)
https://github.com/python/cpython/commit/d20279494a3a311c7aefa313174c45d32aa7f5d1


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30837] Mac OS High Sierra Beta - Python Crash

[Jason R. Coombs]

Jason R. Coombs  added the comment:

All good. Thanks for the references. As with the prior issues, I'm going to 
just suppress those failures.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30837] Mac OS High Sierra Beta - Python Crash

[Ned Deily]

Ned Deily  added the comment:

If the crashes are occurring while the multiprocessing fork tests are running, 
it's likely the issue described in Issue33725. In Python 3.8, the default start 
method for multiprocessing on macOS was changed to be "spawn" rather than 
"fork" because of the inherent issues with fork/no-exec on macOS. You could try 
backporting that to your 3.6.

https://docs.python.org/3.8/library/multiprocessing.html#contexts-and-start-methods

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30837] Mac OS High Sierra Beta - Python Crash

[Jason R. Coombs]

Jason R. Coombs  added the comment:

I started tracking it down in https://github.com/jaraco/keyring/issues/494 and 
found I'd previously delved deep into a related issue 
(https://github.com/jaraco/keyring/issues/281).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30837] Mac OS High Sierra Beta - Python Crash

[Jason R. Coombs]

Jason R. Coombs  added the comment:

I've stumbled into this issue and it's started occurring reliably on keyring in 
Python 3.6 as found on Github CI 
(https://github.com/jaraco/keyring/runs/1994652010?check_suite_focus=true).

It's not a highly-reproducible environment. I can't replicate it on my local 
machine, so there's probably something about how Github CI has changed the 
environment that's started to trigger the error.

It happens in the multiprocessing tests.

--
nosy: +jaraco
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43266] "String conversion and formatting" formatting messes up array subscripting

[Carol Willing]

Carol Willing  added the comment:


New changeset 4ebd637dde7d7e56b18926d2a70657abcac73df0 by Miss Islington (bot) 
in branch '3.8':
closes bpo-43266: Improve array formatting. (GH-24573) (GH-24585)
https://github.com/python/cpython/commit/4ebd637dde7d7e56b18926d2a70657abcac73df0


--
nosy: +willingc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36346] Prepare for removing the legacy Unicode C API

[Carol Willing]

Carol Willing  added the comment:


New changeset 346afeb159a5a8b2e260d3de740019b64352660e by Miss Islington (bot) 
in branch '3.8':
bpo-36346: Document removal schedule of deprecate APIs (GH-20879) (#24626)
https://github.com/python/cpython/commit/346afeb159a5a8b2e260d3de740019b64352660e


--
nosy: +willingc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42128] Structural Pattern Matching (PEP 634)

[Carol Willing]

Carol Willing  added the comment:


New changeset a8e2615aa885a121b8cf8e9cd07696207356a25a by Pablo Galindo in 
branch 'master':
bpo-42128: Add documentation for the new match-based AST nodes (GH-24673)
https://github.com/python/cpython/commit/a8e2615aa885a121b8cf8e9cd07696207356a25a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43271] AMD64 Windows10 3.x crash with Windows fatal exception: stack overflow

[David Bolen]

David Bolen  added the comment:

I hadn't tested release mode earlier, since the commit only removed the pragma 
in debug builds, but I just built a release build with the commit in place on 
the worker and it seems fine.

So barring stack changes (enlarging or improving usage) it appears the optimize 
pragma is required on Windows, at least in order to pass the current test suite.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43271] AMD64 Windows10 3.x crash with Windows fatal exception: stack overflow

[Guido van Rossum]

Guido van Rossum  added the comment:

Thanks David!

So the crash only happens in debug mode?

PEP 651 promises to take reduce the C stack usage in the future. Therefore, I 
would be okay with rolling this back for the time being.
If we do roll it back, maybe add a comment to ceval.c explaining
that even in debug mode this file is optimized?

(The change to the pragma can stay, those two letters have been unused for a 
decade.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43347] IDLE crashes in

[Raymond Hettinger]

New submission from Raymond Hettinger :

I'm running a python.org Mac build on Apple silicon.

Python 3.10.0a5 (v3.10.0a5:b0478d738a, Feb  2 2021, 16:30:44) 
[Clang 12.0.0 (clang-1200.0.32.29)] on darwin

Periodically, Python freezes and requires a force quit.  I'm not certain but 
think the trigger event is the display of a pop-up box showing attribute 
completions.  The box is correctly shaped but filled with a solid light gray.

I suspect the problem is partly in the O/S because after the freeze, I see 
something similar when clicking the system  icon, so the effect extends beyond 
Python.  However, the problem does not occur on my Mac with Intel silicon and 
it does not happen with Python 3.9 running on Apple silicon.  

This has been occurring for a while and has persisted through O/S updates.  
Today, it happened on Big Sur 11.2.2.   

Will try to get a screen shot and more details when this happens again.

The event occurs often enough to make IDLE unusable for classroom 
demonstrations or for development.

--
assignee: terry.reedy
components: IDLE, Tkinter
messages: 387824
nosy: rhettinger, ronaldoussoren, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE crashes in
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: weirdness with list()

[MRAB]

On 2021-02-28 23:28, Peter Otten wrote:

On 28/02/2021 23:33, Marco Sulla wrote:

On Sun, 28 Feb 2021 at 01:19, Cameron Simpson  wrote:

My object represents an MDAT box in an MP4 file: it is the ludicrously
large data box containing the raw audiovideo data; for a TV episode it
is often about 2GB and a movie is often 4GB to 6GB.
[...]
That length is presented via the object's __len__ method
[...]

I noticed that it was stalling, and investigation revealed it was
stalling at this line:

 subboxes = list(self)

when doing the MDAT box. That box (a) has no subboxes at all and (b) has
a very large __len__ value.

BUT... It also has a __iter__ value, which like any Box iterates over
the subboxes. For MDAT that is implemented like this:

 def __iter__(self):
 yield from ()

What I was expecting was pretty much instant construction of an empty
list. What I was getting was a very time consuming (10 seconds or more)
construction of an empty list.


I can't reproduce, Am I missing something?

marco@buzz:~$ python3
Python 3.6.9 (default, Jan 26 2021, 15:33:00)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

class A:

... def __len__(self):
... return 1024**3
... def __iter__(self):
... yield from ()
...

a = A()
len(a)

1073741824

list(a)

[]




It takes milliseconds to run list(a)


Looks like you need at least Python 3.8 to see this. Quoting
https://docs.python.org/3/whatsnew/3.8.html:

"""
The list constructor does not overallocate the internal item buffer if
the input iterable has a known length (the input implements __len__).
This makes the created list 12% smaller on average. (Contributed by
Raymond Hettinger and Pablo Galindo in bpo-33234.)
"""


I'm not seeing a huge problem here:

Python 3.9.2 (tags/v3.9.2:1a79785, Feb 19 2021, 13:44:55) [MSC v.1928 64 
bit (AMD64)] on win32

Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> class A:
... def __len__(self):
... return 1024**3
... def __iter__(self):
... yield from ()
...
>>> a = A()
>>> len(a)
1073741824
>>> s = time.time()
>>> list(a)
[]
>>> print(time.time() - s)
0.16294455528259277
--
https://mail.python.org/mailman/listinfo/python-list

Re: weirdness with list()

[Cameron Simpson]

On 01Mar2021 00:28, Peter Otten <__pete...@web.de> wrote:
>On 28/02/2021 23:33, Marco Sulla wrote:
>>I can't reproduce, Am I missing something?
>>
>>marco@buzz:~$ python3
>>Python 3.6.9 (default, Jan 26 2021, 15:33:00)
>>[GCC 8.4.0] on linux
>>Type "help", "copyright", "credits" or "license" for more information.
>class A:
>>... def __len__(self):
>>... return 1024**3
>>... def __iter__(self):
>>... yield from ()
>>...
>a = A()
>len(a)
>>1073741824
>list(a)
>>[]
>
>>
>>It takes milliseconds to run list(a)
>
>Looks like you need at least Python 3.8 to see this. Quoting
>https://docs.python.org/3/whatsnew/3.8.html:
>"""
>The list constructor does not overallocate the internal item buffer if
>the input iterable has a known length (the input implements __len__).
>This makes the created list 12% smaller on average. (Contributed by
>Raymond Hettinger and Pablo Galindo in bpo-33234.)
>"""

That may also explain why I hadn't noticed this before, eg last year.

I do kind of wish __length_hint__ overrode __len__ rather than the other 
way around, if it's doing what I think it's doing.

Cheers,
Cameron Simpson 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: weirdness with list()

[Cameron Simpson]

On 28Feb2021 10:51, Peter Otten <__pete...@web.de> wrote:
>On 28/02/2021 01:17, Cameron Simpson wrote:
>>I noticed that it was stalling, and investigation revealed it was
>>stalling at this line:
>>
>> subboxes = list(self)
>>
>>when doing the MDAT box. That box (a) has no subboxes at all and (b) has
>>a very large __len__ value.
[...]
>
>list(iter(self))
>
>should work, too. It may be faster than the explicit loop, but also
>defeats the list allocation optimization.

Yes, very neat. I went with [subbox for subbox in self] last night, but 
the above is better.

[...]
>>Still, thoughts? I'm interested in any approaches that would have let 
>>me
>>make list() fast while keeping __len__==binary_length.
>>
>>I'm accepting that __len__ != len(__iter__) is a bad idea now, though.
>
>Indeed. I see how that train wreck happened -- but the weirdness is not
>the list behavior.

I agree. The only weirdness is that list(empty-iterable) took a very 
long time. Weirdness in the eye of the beholder I guess.

>Maybe you can capture the intended behavior of your class with two
>classes, a MyIterable without length that can be converted into MyList
>as needed.

Hmm. Maybe.

What I've done so far is:

The afore mentioned [subbox for subbox in self] which I'll replace with 
your nicer one today.

Given my BinaryMixin a transcribed_length method which measures the 
length of the binary transcription. For small things that's actually 
fairly cheap, and totally general. By default it is aliased to __len__, 
which still seems a natural thing - the length of the binary object is 
the number of bytes required to serialise it.

The alias lets me override transcribed_length() for bulky things like 
MDAT where (a) transcription _is_ expensive and (b) the source data may 
not be present anyway ("skip" mode), but the measurement of the data 
from the parse is recorded.

And I can disassociate __len__ from transcribed_length() if need be in 
subclasses. I've not done that, given the iter() shuffle above.

Cheers,
Cameron Simpson 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: weirdness with list()

[Peter Otten]

On 28/02/2021 23:33, Marco Sulla wrote:

On Sun, 28 Feb 2021 at 01:19, Cameron Simpson  wrote:

My object represents an MDAT box in an MP4 file: it is the ludicrously
large data box containing the raw audiovideo data; for a TV episode it
is often about 2GB and a movie is often 4GB to 6GB.
[...]
That length is presented via the object's __len__ method
[...]

I noticed that it was stalling, and investigation revealed it was
stalling at this line:

 subboxes = list(self)

when doing the MDAT box. That box (a) has no subboxes at all and (b) has
a very large __len__ value.

BUT... It also has a __iter__ value, which like any Box iterates over
the subboxes. For MDAT that is implemented like this:

 def __iter__(self):
 yield from ()

What I was expecting was pretty much instant construction of an empty
list. What I was getting was a very time consuming (10 seconds or more)
construction of an empty list.


I can't reproduce, Am I missing something?

marco@buzz:~$ python3
Python 3.6.9 (default, Jan 26 2021, 15:33:00)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

class A:

... def __len__(self):
... return 1024**3
... def __iter__(self):
... yield from ()
...

a = A()
len(a)

1073741824

list(a)

[]




It takes milliseconds to run list(a)


Looks like you need at least Python 3.8 to see this. Quoting
https://docs.python.org/3/whatsnew/3.8.html:

"""
The list constructor does not overallocate the internal item buffer if 
the input iterable has a known length (the input implements __len__). 
This makes the created list 12% smaller on average. (Contributed by 
Raymond Hettinger and Pablo Galindo in bpo-33234.)

"""



--
https://mail.python.org/mailman/listinfo/python-list

[issue43296] [sqlite3] Fix sqlite3_value_blob() usage

[Erlend Egeberg Aasland]

Change by Erlend Egeberg Aasland :


--
keywords: +patch
pull_requests: +23459
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24674

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43346] subprocess.run() sometimes ignores timeout in Windows

[Eryk Sun]

Eryk Sun  added the comment:

Demo Popen() methods, for discussion:

def _read_output(self, fileobj):
handle = msvcrt.get_osfhandle(fileobj.fileno())
output = self._fileobj2output[fileobj]

while True:
try:
size = _winapi.PeekNamedPipe(handle)[0] or 1
data = _winapi.ReadFile(handle, size)[0]
except BrokenPipeError:
break
except OSError as e:
if e.winerror == _winapi.ERROR_OPERATION_ABORTED:
# Should this be mapped to InterruptedError
# (EINTR) in PC/errmap.h?
break
raise
output.append(data)

fileobj.close()


def _communicate(self, input, endtime, orig_timeout):
if not self._communication_started:
self._fileobj2thread = {}
self._fileobj2output = {}
if self.stdout:
self._fileobj2output[self.stdout] = []
if self.stderr:
self._fileobj2output[self.stderr] = []

stdout = self._fileobj2output.get(self.stdout)
stderr = self._fileobj2output.get(self.stderr)

thread_list = []
for fileobj in (self.stdin, self.stdout, self.stderr):
if fileobj is None:
continue
if fileobj in self._fileobj2thread:
thread = self._fileobj2thread[fileobj]
else:
if fileobj == self.stdin:
target, args = self._stdin_write, (input,)
else:
target, args = self._read_output, (fileobj,)
thread = threading.Thread(target=target, args=args,
daemon=True)
thread.start()
self._fileobj2thread[fileobj] = thread
thread_list.append(thread)

for thread in thread_list:
thread.join(self._remaining_time(endtime))
if thread.is_alive():
self._check_timeout(endtime, orig_timeout,
stdout, stderr, skip_check_and_raise=True)

# Join partial reads.
if stdout is not None:
stdout = b''.join(stdout)
if stderr is not None:
stderr = b''.join(stderr)

if self.text_mode:
if stdout is not None:
stdout = self._translate_newlines(stdout,
self.stdout.encoding, self.stdout.errors)
if stderr is not None:
stderr = self._translate_newlines(stderr,
self.stderr.encoding, self.stderr.errors)

return (stdout, stderr)


def _cancel_io(self):
if not hasattr(self, '_fileobj2thread'):
return
for fileobj in (self.stdin, self.stdout, self.stderr):
thread = self._fileobj2thread.get(fileobj)
if thread is None or not thread.is_alive():
continue
try:
handle = _winapi.OpenThread(
_winapi.TERMINATE_THREAD, False, thread.ident)
except OSError:
pass
else:
try:
try:
_winapi.CancelSynchronousIo(handle)
except OSError:
pass
finally:
_winapi.CloseHandle(handle)


def __exit__(self, exc_type, value, traceback):
if _mswindows:
self._cancel_io()

# rest unchanged...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43346] subprocess.run() sometimes ignores timeout in Windows

[Eryk Sun]

New submission from Eryk Sun :

subprocess.run() handles TimeoutExpired by terminating the process and waiting 
on it. In POSIX, the exception object contains the partially read stdout and 
stderr bytes. For example:

cmd = 'echo spam; echo eggs >&2; sleep 2'
try: p = subprocess.run(cmd, shell=True, capture_output=True,
text=True, timeout=1)
except subprocess.TimeoutExpired as e: ex = e
 
>>> ex.stdout, ex.stderr
(b'spam\n', b'eggs\n')

In Windows, subprocess.run() has to finish reading output with a second 
communicate() call, after which it manually sets the exception's stdout and 
stderr attributes.

The poses the problem that the second communicate() call may block 
indefinitely, even though the child process has terminated.

The primary issue is that the pipe handles may be inherited by one or more 
descendant processes (e.g. via shell=True), which are all regarded as potential 
writers that keep the pipe from closing. Reading from an open pipe that's empty 
will block until data becomes available. This is generally desirable for 
efficiency, compared to polling in a loop. But in this case, the downside is 
that run() in Windows will effectively ignore the given timeout.

Another problem is that _communicate() writes the input to stdin on the calling 
thread with a single write() call. If the input exceeds the pipe capacity (4 
KiB by default -- but a pipesize 'suggested' size could be supported), the 
write will block until the child process reads the excess data. This could 
block indefinitely, which will effectively ignore a given timeout. The POSIX 
implementation, in contrast, correctly handles a timeout in this case.

Another problem is that Popen.__exit__() closes the stdout, stderr, and stdin 
files without regard to the _communicate() worker threads. This may seem 
innocuous, but if a worker thread is blocked on synchronous I/O with one of 
these files, WinAPI CloseHandle() will also block if it's closing the last 
handle for the file in the current process. (In this case, the kernel I/O 
manager has a close procedure that waits to acquire the file for the current 
thread before performing various housekeeping operations, primarily in the 
filesystem, such as clearing byte-range locks set by the current process.) A 
blocked close() is easy to demonstrate. For example:

args = 'python -c "import time; time.sleep(99)"'
p = subprocess.Popen(args, shell=True, stdout=subprocess.PIPE)
try: p.communicate(timeout=1)
except: pass

p.kill() # terminates the shell process -- not python.exe
with p: pass # stdout.close() blocks until python.exe exits

I think the Windows implementation of Popen._communicate() needs to be 
redesigned as follows: 

* read in chunks, with a size from 1 byte up to the maximum available,
  as determined by _winapi.PeekNamedPipe()
* write to the child's stdin on a separate thread
* after communicate() has started, ensure that synchronous I/O in worker
  threads has been canceled via CancelSynchronousIo() before closing 
  the pipes.

The _winapi module would need to wrap OpenThread() and CancelSynchronousIo(), 
plus define the TERMINATE_THREAD (0x0001) access right.

With the proposed changes, subprocess.run() would no longer special case 
TimeoutExpired for Windows.

--
components: Library (Lib), Windows
messages: 387822
nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: subprocess.run() sometimes ignores timeout in Windows
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29753] Ctypes Packing Bitfields Incorrectly - Linux

[miss-islington]

miss-islington  added the comment:


New changeset 0d7ad9fb38c041c46094087b0cf2c8ce44916b11 by Filipe Laíns in 
branch 'master':
bpo-29753: fix merging packed bitfields in ctypes struct/union (GH-19850)
https://github.com/python/cpython/commit/0d7ad9fb38c041c46094087b0cf2c8ce44916b11


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37146] opcode cache for LOAD_GLOBAL emits false alarm in memory leak hunting

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37146] opcode cache for LOAD_GLOBAL emits false alarm in memory leak hunting

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset af5fa13ef6f648fc7a7a33a7556db13887e7d643 by Pablo Galindo in 
branch 'master':
bpo-37146: Deactivate opcode cache only when using huntrleaks in the test suite 
(GH-24643)
https://github.com/python/cpython/commit/af5fa13ef6f648fc7a7a33a7556db13887e7d643


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42128] Structural Pattern Matching (PEP 634)

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +23458
pull_request: https://github.com/python/cpython/pull/24673

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: weirdness with list()

[Marco Sulla]

On Sun, 28 Feb 2021 at 01:19, Cameron Simpson  wrote:
> My object represents an MDAT box in an MP4 file: it is the ludicrously
> large data box containing the raw audiovideo data; for a TV episode it
> is often about 2GB and a movie is often 4GB to 6GB.
> [...]
> That length is presented via the object's __len__ method
> [...]
>
> I noticed that it was stalling, and investigation revealed it was
> stalling at this line:
>
> subboxes = list(self)
>
> when doing the MDAT box. That box (a) has no subboxes at all and (b) has
> a very large __len__ value.
>
> BUT... It also has a __iter__ value, which like any Box iterates over
> the subboxes. For MDAT that is implemented like this:
>
> def __iter__(self):
> yield from ()
>
> What I was expecting was pretty much instant construction of an empty
> list. What I was getting was a very time consuming (10 seconds or more)
> construction of an empty list.

I can't reproduce, Am I missing something?

marco@buzz:~$ python3
Python 3.6.9 (default, Jan 26 2021, 15:33:00)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> class A:
... def __len__(self):
... return 1024**3
... def __iter__(self):
... yield from ()
...
>>> a = A()
>>> len(a)
1073741824
>>> list(a)
[]
>>>

It takes milliseconds to run list(a)
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue42128] Structural Pattern Matching (PEP 634)

[Brandt Bucher]

Brandt Bucher  added the comment:

Thanks Pablo!

> If you'd like to take it on feel free, if not I'll create a PR next weekend 
> with tests (probably after release, though I believe it is not a blocker as 
> is).

No problem, I'm pretty busy too. Next weekend is fine.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41972] bytes.find consistently hangs in a particular scenario

[Dennis Sweeney]

Change by Dennis Sweeney :


--
pull_requests: +23457
pull_request: https://github.com/python/cpython/pull/24672

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40469] TimedRotatingFileHandler rotating on use not time

[Raymond Hettinger]

Change by Raymond Hettinger :


--
nosy: +vinay.sajip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42128] Structural Pattern Matching (PEP 634)

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I can pick up the AST docs PR

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42128] Structural Pattern Matching (PEP 634)

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

> @BTaskaya, do you think you'll have time to open a PR with your AST validator 
> this weekend? It looks good to me (assuming tests pass).

Unfortunately not. I believe it still lacks tests for invalid cases, but other 
than that should work. If you'd like to take it on feel free, if not I'll 
create a PR next weekend with tests (probably after release, though I believe 
it is not a blocker as is).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43271] AMD64 Windows10 3.x crash with Windows fatal exception: stack overflow

[David Bolen]

David Bolen  added the comment:

I don't think it's actually any change in ceval per se, or any new buffers, 
just how the compiler code generation has changed due to this commit.

Based on some local testing, the triggering issue is the exclusion of the 
optimization pragma entirely in debug mode.  It appears that the existing code 
required the pragma to be enabled to stay within the available stack space.

A quick reproduction is running test_pickletools.  It fails abruptly and 
quickly, at least on the worker, without the pragma, but runs to completion if 
enabled.

Perhaps that aspect of the commit should be reverted, maybe with a separate 
flag for debugging ceval (but not normal testing)?  Alternatively, is there a 
way to increase the stack in debug mode to compensate?  Though I guess that 
would risk missing a release-build only stack issue.

--
nosy: +db3l

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42128] Structural Pattern Matching (PEP 634)

[Brandt Bucher]

Brandt Bucher  added the comment:

@BTaskaya, do you think you'll have time to open a PR with your AST validator 
this weekend? It looks good to me (assuming tests pass).

Also, we should add the AST docs to our documentation to-do list (should be 
just adding entries for ast.Match, ast.MatchAs, ast.MatchOr, and 
ast.match_case).

I'll try to find time to review the open doc PRs today.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42129] Support resources in namespace packages

[Jason R. Coombs]

Change by Jason R. Coombs :


--
keywords: +patch
pull_requests: +23456
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24670

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43344] RotatingFileHandler breaks file type associations

[Raymond Hettinger]

Raymond Hettinger  added the comment:

+1 The ability to customize the filenames seems like a reasonable feature 
request.  Perhaps an optional *format* option could introduced.

--
nosy: +rhettinger, vinay.sajip
versions: +Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41972] bytes.find consistently hangs in a particular scenario

[Tim Peters]

Tim Peters  added the comment:


New changeset 73a85c4e1da42db28e3de57c868d24a089b8d277 by Dennis Sweeney in 
branch 'master':
bpo-41972: Use the two-way algorithm for string searching (GH-22904)
https://github.com/python/cpython/commit/73a85c4e1da42db28e3de57c868d24a089b8d277


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43343] argparse.REMAINDER missing in online documentation for 3.9.x

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Chigozirim, thanks for the report.

REMAINDER is now an undocumented legacy feature.  It isn't deprecated because 
we don't plan to remove it.  It isn't documented because it is buggy in a way 
that isn't easy to fix.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[Announce] - preview of the new python extension "pymsgque"

[aotto1968]

Hello everybody,

  PYTHON has a new application server
  → https://nhi1.selfhost.co/nhi1/

  SOMETHING that writes code for you
  → https://nhi1.selfhost.co/wiki/NHI1_-_the_POWER_of_programming

have fun
--
https://mail.python.org/mailman/listinfo/python-list

[issue43343] argparse.REMAINDER missing in online documentation for 3.9.x

[hai shi]

hai shi  added the comment:

Hi, Chigozirim Chukwu. Thanks for your report.

There have releated comment in: https://bugs.python.org/issue17050#msg315716

--
nosy: +paul.j3, rhettinger, shihai1991
versions: +Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43251] [sqlite3] sqlite3_column_name() failures should raise MemoryError

[Berker Peksag]

Change by Berker Peksag :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43251] [sqlite3] sqlite3_column_name() failures should raise MemoryError

[Berker Peksag]

Berker Peksag  added the comment:


New changeset 2183d06bc8a481098d62a4ebed8d6982b3d1602a by Erlend Egeberg 
Aasland in branch 'master':
bpo-43251: sqlite3_column_name() failures now raise MemoryError (GH-24609)
https://github.com/python/cpython/commit/2183d06bc8a481098d62a4ebed8d6982b3d1602a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42484] get_obs_local_part fails to handle empty local part

[Zackery Spytz]

Change by Zackery Spytz :


--
keywords: +patch
nosy: +ZackerySpytz
nosy_count: 3.0 -> 4.0
pull_requests: +23455
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24669

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Twisted 21.2.0 Release Announcement

[Craig Rodrigues]

On behalf of Twisted Matrix Laboratories, I am honored to announce the
release of Twisted 21.2.0!

There are two major announcements for this release:

- Python 2.7 support has been dropped.  Twisted 21.2.0 supports Python 3.5.3 
and higher only
- This will be the last Twisted release to support Python 3.5.

Twisted 21.2.0 brings the following:
- twist dns --pyzone now works on Python 3
- twisted.web.twcgi now works on Python 3
- all tests pass on all platforms on Python 3.8, including asyncio tests on 
Windows
- tests pass on Python 3.9 (thanks to Michał Górny)
- type hints, as specified in PEP 484, are now used in functions throughout the 
codebase
- mypy (static type checker) is now run over the codebase during CI to help 
detect problems, and help improve the use of types
- code in is now formatted with Black (thanks to Tom Most)
- many CI and automation improvements (thanks to Adi Roiban and Thomas Grainger)
- many more fixes (225 tickets closed!)


You can install by running this command:

  python -m pip install Twisted==21.2.0

The full tarball is available at:
https://pypi.org/project/Twisted/21.2.0/#files

The full NEWS file with all changes is at:
https://github.com/twisted/twisted/blob/twisted-21.2.0/NEWS.rst
___
Python-announce-list mailing list -- python-announce-list@python.org
To unsubscribe send an email to python-announce-list-le...@python.org
https://mail.python.org/mailman3/lists/python-announce-list.python.org/
Member address: arch...@mail-archive.com

[issue27777] cgi.FieldStorage can't parse simple body with Content-Length and no Content-Disposition

[Christoph Zwerschke]

Christoph Zwerschke  added the comment:

Just created a test case for this problem after a pentest provoked this error 
on one of my web apps. Then I found this bug report which already has a similar 
test case attached.

The problem is that read_binary() as the name says reads binary data, but then 
writes it to a file which may or may not be binary, depending on whether 
self._binary_file is set, which depends on whether a filename was set via the 
content-disposition header.

Jakub's patch looks good and works for me. Please merge this!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Tkinter new window contentent when button is clicked.

[Bischoop]

On 2021-02-25, MRAB  wrote:
>> 
> The trick is to put the "pages" on top of each other and then show the 
> appropriate one, something like this:
> import tkinter as tk
>
> def on_next_page():
>  # Brings page 2 to the top.
>  frame_2.tkraise()
>
> def on_previous_page():
>  # Brings page 1 to the top.
>  frame_1.tkraise()
>
> def on_finish():
>  # Closes the dialog.
>  root.destroy()
>
> root = tk.Tk()
>
> # Page 1.
> frame_1 = tk.Frame(root)
> tk.Label(frame_1, text='Page 1').pack()
> tk.Button(frame_1, text='Next', command=on_next_page).pack()
>
> # Page 2.
> frame_2 = tk.Frame()
> tk.Label(frame_2, text='Page 2').pack()
> tk.Button(frame_2, text='Previous', command=on_previous_page).pack()
> tk.Button(frame_2, text='Finish', command=on_finish).pack()
>
> # Put the pages on top of each other.
> frame_1.grid(row=0, column=0, sticky='news')
> frame_2.grid(row=0, column=0, sticky='news')
>
> # Bring page 1 to the top.
> frame_1.tkraise()
>
> tk.mainloop()


Great, thanks for reply, I'll look into that.


--
Thanks

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue42129] Support resources in namespace packages

[Jason R. Coombs]

Jason R. Coombs  added the comment:

I've pushed [this 
branch](https://github.com/python/cpython/tree/feature/42129-resources-namespace-packages),
 which includes fixes for the above two identified issues. Still one issue 
remains:

ERROR: test_package_has_no_reader_fallback 
(test.test_importlib.test_resource.ResourceCornerCaseTests)
--
Traceback (most recent call last):
  File 
"/Users/jaraco/code/public/cpython/Lib/test/test_importlib/test_resource.py", 
line 98, in test_package_has_no_reader_fallback
self.assertFalse(resources.is_resource(module, 'A'))
  File "/Users/jaraco/code/public/cpython/Lib/importlib/resources.py", line 
157, in is_resource
package_contents = set(contents(package))
  File "/Users/jaraco/code/public/cpython/Lib/importlib/resources.py", line 
174, in contents
transversable = _common.from_package(package)
  File "/Users/jaraco/code/public/cpython/Lib/importlib/_common.py", line 75, 
in from_package
reader = spec.loader.get_resource_reader(spec.name)
AttributeError: 'object' object has no attribute 'get_resource_reader'

This same test passes on importlib_resources, and the difference seems to be 
rooted in how [from_package resolves the package spec using a compatibility 
wrapper](https://github.com/python/importlib_resources/blob/1401cc48e5077088036aa7e729c8995ffbbb9e88/importlib_resources/_common.py#L76).
 On the backport, this causes the package without a resource reader to have a 
resource reader and return a degenerate value:

```
> /Users/jaraco/code/public/importlib_resources/importlib_resources/_py3.py(139)is_resource()
-> package_contents = set(contents(package))
(Pdb) _common.from_package(package)
PosixPath('/path/which/shall/not/be')
(Pdb) from . import _compat
(Pdb) 
_compat.package_spec(package).loader.get_resource_reader('any').files().is_dir()
False
```

This means that the compatibility shim in from_package is masking test failure 
in the backport, and probably the best course of action will be to unmask that 
failure in the backport and figure out the best behavior there.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: weirdness with list()

[Peter Otten]

On 28/02/2021 01:17, Cameron Simpson wrote:

I just ran into a surprising (to me) issue with list() on an iterable
object.

My object represents an MDAT box in an MP4 file: it is the ludicrously
large data box containing the raw audiovideo data; for a TV episode it
is often about 2GB and a movie is often 4GB to 6GB. For obvious reasons,
I do not always want to load that into memory, or even read the data
part at all when scanning an MP4 file, for example to recite its
metadata.

So my parser has a "skip" mode where it seeks straight past the data,
but makes a note of its length in bytes. All good.

That length is presented via the object's __len__ method, because I want
to know that length later and this is a subclass of a suite of things
which return their length in bytes this way.

So, to my problem:

I've got a walk method which traverses the hierarchy of boxes in the MP4
file. Until some minutes ago, it looked like this:

   def walk(self):
 subboxes = list(self)
 yield self, subboxes
 for subbox in subboxes:
   if isinstance(subbox, Box):
 yield from subbox.walk()

somewhat like os.walk does for a file tree.

I noticed that it was stalling, and investigation revealed it was
stalling at this line:

 subboxes = list(self)

when doing the MDAT box. That box (a) has no subboxes at all and (b) has
a very large __len__ value.

BUT... It also has a __iter__ value, which like any Box iterates over
the subboxes. For MDAT that is implemented like this:

 def __iter__(self):
 yield from ()

What I was expecting was pretty much instant construction of an empty
list. What I was getting was a very time consuming (10 seconds or more)
construction of an empty list.

I believe that this is because list() tries to preallocate storage. I
_infer_ from the docs that this is done maybe using
operator.length_hint, which in turn consults "the actual length of the
object" (meaning __len__ for me?), then __length_hint__, then defaults
to 0.

I've changed my walk function like so:

   def walk(self):
 subboxes = []
 for subbox in self:
   subboxes.append(subbox)
 ##subboxes = list(self)


list(iter(self))

should work, too. It may be faster than the explicit loop, but also
defeats the list allocation optimization.


and commented out the former list(self) incantation. This is very fast,
because it makes an empty list and then appends nothing to it. And for
your typical movie file this is fine, because there are never _very_
many immediate subboxes anyway.

But is there a cleaner way to do this?

I'd like to go back to my former list(self) incantation, and modify the
MDAT box class to arrange something efficient. Setting __length_hint__
didn't help: returning NotImplemeneted or 0 had no effect, because
presumably __len__ was consulted first.

Any suggestions? My current approach feels rather hacky.

I'm already leaning towards making __len__ return the number of subboxes
to match the iterator, especially as on reflection not all my subclasses
are consistent about __len__ meaning the length of their binary form;
I'm probably going to have to fix that - some subclasses are actually
namedtuples where __len__ would be the field count. Ugh.

Still, thoughts? I'm interested in any approaches that would have let me
make list() fast while keeping __len__==binary_length.

I'm accepting that __len__ != len(__iter__) is a bad idea now, though.


Indeed. I see how that train wreck happened -- but the weirdness is not
the list behavior.

Maybe you can capture the intended behavior of your class with two
classes, a MyIterable without length that can be converted into MyList
as needed.


--
https://mail.python.org/mailman/listinfo/python-list

[issue40059] Provide a toml module in the standard library

[Michał Górny]

Michał Górny  added the comment:

It seems that the spec has reached 1.0.0:

> 1.0.0 / 2021-01-11

I guess that there are still ~2 months left to do this before 3.10 reaches beta.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43054] What does the existence of a struct in a header file imply about the C-API

[Ronald Oussoren]

Ronald Oussoren  added the comment:

Sorry about the slow response.

I misread your initial message, the distinction between (1) and (2) in your 
list is clear.

to (3): New fields can be added while maintaining API (but not ABI) 
compatibility as long as the default value is the default value for static 
initialisers (assuming the usual way we initialise structs in CPython). That 
is, adding a new field to PyTypeObject is ok, as long as it at the end and 
defaults to NULL or 0.

Especially with category 1 it is not entirely clear which structs are in that 
category. Is PyLongObject in this category?  The struct is not documented, but 
has a name that seems to indicate that it is public.   Likewise for 
PyTupleObject, where the shape of the struct is used by documented APIs but the 
shape of the strut itself is not documented.

BTW. For my own code I do directly access structs where necessary, even if they 
aren't documented. I totally expect that this will require adjustments for new 
Python releases (such as when the unicode representation changed).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22100] Use $HOSTPYTHON when determining candidate interpreter for $PYTHON_FOR_BUILD.

[pmp-p]

Change by pmp-p :


--
nosy: +pmpp

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42129] Support resources in namespace packages

[Jason R. Coombs]

Jason R. Coombs  added the comment:

> I would harmonize towards what the concrete implementations...

For FileReader and ZipReader, both rely on TraversableResources to implement 
contents, which rely on `files().iterdir()`, which could raise 
FileNotFoundError and definitely don't return lists. Most importantly, this 
method is an abstract method, so subclasses can't rely on it (must override 
it). Searching Github, I couldn't find any other classes subclassing this ABC.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43312] Interface to select preferred "user" or "home" sysconfig scheme for an environment

[Tzu-ping Chung]

Tzu-ping Chung  added the comment:

How do we move the discussion forward? I would really want this to be included 
in 3.10. Assuming distutils is going to be removed in 3.12, pip would be left 
with an extremely short window if this has to wait for another year.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43319] A possible misleading expression in the Virtual Environment Tutorial

[cmhzc]

Change by cmhzc :


--
type: enhancement -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com