[issue41974] Remove complex.__float__, complex.__floordiv__, etc

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

The complex class has special methods which always raise a TypeError:

   __int__
   __float__
   __floordiv__
   __mod__
   __divmod__

After removing them the corresponding operations (converting to int and float, 
operators // and %, function divmod()) will still a TypeError.

Advantages of removing:

* Less code to maintain.
* More uniform error messages.
* Clearer output of help().
* Possibility to implement a type with __rfloordiv__, __rmod__ and __rdivmod__ 
which support complex numbers.

--
components: Interpreter Core
messages: 378218
nosy: gvanrossum, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Remove complex.__float__, complex.__floordiv__, etc
type: enhancement
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41973] Docs: TypedDict is now of type function instead of class

[Bernat Gabor]

Bernat Gabor  added the comment:

This seem to apply to typing.NamedTuple too.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41973] Docs: TypedDict is now of type function instead of class

[Bernat Gabor]

Bernat Gabor  added the comment:

Static checkers/linters (see e.g. https://github.com/PyCQA/pylint/issues/3884) 
are caught of guard by being able to inherit from functions. They'll need to 
adapt, but let's update the documentation to reflect this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41376] site.getusersitepackages() incorrectly claims that PYTHONNOUSERSITE is respected

[miss-islington]

Change by miss-islington :


--
pull_requests: +21581
pull_request: https://github.com/python/cpython/pull/22592

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41376] site.getusersitepackages() incorrectly claims that PYTHONNOUSERSITE is respected

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 3.0 -> 4.0
pull_requests: +21580
pull_request: https://github.com/python/cpython/pull/22591

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41376] site.getusersitepackages() incorrectly claims that PYTHONNOUSERSITE is respected

[Inada Naoki]

Inada Naoki  added the comment:


New changeset 35f041dd0171f575fc3adce1709b31fdf45a5ff6 by Phil Elson in branch 
'master':
bpo-41376: Fix the documentation of `site.getusersitepackages()` (GH-21602)
https://github.com/python/cpython/commit/35f041dd0171f575fc3adce1709b31fdf45a5ff6


--
nosy: +methane

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41966] datetime.time issue with pickling in PyPy

[Dean]

Dean  added the comment:

Code demonstrating the issue; the CTimeFoo class is pickled correctly, but 
TimeFoo isn't.


import builtins
from _datetime import time as ctime

original_importer = builtins.__import__

def my_importer(name, globals, locals, fromlist, level):
if name == '_datetime':
raise ImportError

return original_importer(name, globals, locals, fromlist, level)

builtins.__import__ = my_importer

import datetime

builtins.__import__ = original_importer

import pickle


class CTimeFoo(ctime): pass
class TimeFoo(datetime.time): pass
class DateFoo(datetime.date): pass


if __name__ == "__main__":
t = DateFoo(2001, 2, 3) 
d = pickle.dumps(t)  # OK
print(d)
# 
b'\x80\x04\x95#\x00\x00\x00\x00\x00\x00\x00\x8c\x08__main__\x94\x8c\x07DateFoo\x94\x93\x94C\x04\x07\xd1\x02\x03\x94\x85\x94R\x94.'
t = pickle.loads(d)

t = CTimeFoo(1, 2, 3)  
d = pickle.dumps(t)  # OK
print(d)
# 
b'\x80\x04\x95&\x00\x00\x00\x00\x00\x00\x00\x8c\x08__main__\x94\x8c\x08CTimeFoo\x94\x93\x94C\x06\x01\x02\x03\x00\x00\x00\x94\x85\x94R\x94.'
t = pickle.loads(d)

t = TimeFoo(1, 2, 3)
d = pickle.dumps(t)  # :(
print(d)
# 
b'\x80\x04\x95"\x00\x00\x00\x00\x00\x00\x00\x8c\x08datetime\x94\x8c\x04time\x94\x93\x94C\x06\x01\x02\x03\x00\x00\x00\x94\x85\x94R\x94.'
t = pickle.loads(d)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41973] Docs: TypedDict is now of type function instead of class

[Anthony Sottile]

Anthony Sottile  added the comment:

Seems to be due to https://bugs.python.org/issue40187

--
nosy: +Anthony Sottile, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41973] Docs: TypedDict is now of type function instead of class

[Bernat Gabor]

New submission from Bernat Gabor :

❯ py -3.8 -c 'from typing import TypedDict; print(type(TypedDict))'


❯ py -3.9 -c 'from typing import TypedDict; print(type(TypedDict))'


Python 3.9 changed the type of the TypedDict but the documentation still says 
it's a class - see 
https://docs.python.org/3.9/library/typing.html#typing.TypedDict. I must also 
say I'm suprised you can inherit from a function, but we should update the 
documentation to reflect that the type of the element in question is no longer 
class.

--
messages: 378212
nosy: gaborjbernat
priority: normal
severity: normal
status: open
title: Docs: TypedDict is now of type function instead of class
versions: Python 3.10, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41894] UnicodeDecodeError during load failure in non-UTF-8 locale

[Inada Naoki]

Inada Naoki  added the comment:

> I have since changed the PR to use PyUnicode_DecodeFSDefault based on review 
> feedback. I was going to say that you will have to fight it out with @methane 
> on GH, but I see that that's you. :D Would have been nice if you would have 
> left the updated feedback there as well so people who aren't familiar would 
> know it's one person adjusting their recommendation vs two different people 
> with conflicting recommendations.

OK, I changd my b.p.o username.


> The only issue I see with using backslashreplace is that users of non-UTF-8 
> locales would see message text that contains non-ASCII characters only as 
> escape codes. eg, the message above would show "Il modulo dipendente 
> libbz2.so non \xe8 stato caricato." instead of "Il modulo dipendente 
> libbz2.so non è stato caricato."

The issue is not caused by backslashreplace, but by UTF-8 instead of locale. I 
notice it of course, but:

* Using UTF-8 is status quo. UTF-8:backslashreplace is the simplest fix 
approach.
* There is no guarantee that the error message can be decoded by locale 
encoding. Backslash escape is much better than "ignore" or "surrogateescape".


> By using PyUnicode_DecodeFSDefault instead, the message should be properly 
> decoded but any encoding errors (such as utf-8 paths, etc) would be handled 
> by surrogateescape.
> 

There is no guranatee that the message is properly decoded with fsencoding.
And surrogateescape is for round-tripping bytes path, not for human readable.
Error message should be human readable. So backslashreplace is better than 
surrogateescape.

Additionally, non-UTF-8 locale is quite rare on Unix systems, and users of such 
systems would be able to handle backslash escaped message, because they might 
see such message often.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41972] bytes.find consistently hangs in a particular scenario

[pmp-p]

Change by pmp-p :


--
nosy: +pmpp

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41972] bytes.find consistently hangs in a particular scenario

[Tim Peters]

Tim Peters  added the comment:

Also reproduced on 64-bit Win10 with just-released 3.9.0.

Note that string search tries to incorporate a number of tricks (pieces of 
Boyer-Moore, Sunday, etc) to speed searches.  The "skip tables" here are 
probably computing a 0 by mistake. The file here contains only 2 distinct byte 
values, and the relatively huge string to search for has only 1, so there's 
plenty of opportunity for those tricks to get confused ;-)

--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41972] bytes.find consistently hangs in a particular scenario

[Josh Rosenberg]

Change by Josh Rosenberg :


--
type: performance -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41972] bytes.find consistently hangs in a particular scenario

[Josh Rosenberg]

Josh Rosenberg  added the comment:

Can reproduce on Alpine Linux, with CPython 3.8.2 (running under WSLv2), so 
it's not just you. CPU usage is high; seems like it must be stuck in an 
infinite loop.

--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41970] test_lib2to3 fails since Python 3.9

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> I have also seen this and linked issue on running whole test suite while 
> building python myself but running individual tests show no error and 
> buildbots also never complained so I thought it's something wrong with my 
> setup. I introduced the commit to silence deprecation warning due to lib2to3 
> and cannot reproduce the same running it individually.

What I don't understand then is how the buildbots or the CI in Github that run 
all tests sequentially with the extra flags (-uall and friends) have not 
complained yet about this.

@Felix, how consistent is this failure you are experiencing?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

For instance, consider this:

x: (
   3 + 
 4
   + 5) = 3

Once the expression in the annotations is parsed, the value of parser->tok->buf 
is "   +5) = 3" and the tokenizer has thrown away the rest of the lines and the 
information there. There is no way for us to "grab" the other lines and save 
them as they are consumed unless we add custom logic into the parser, which is 
not immediately easy because is auto-generated.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Eric V. Smith]

Eric V. Smith  added the comment:

It's true that f-string expressions can't contain newlines.

f-strings are definitely easier, because the tokenizer has already tokenized 
the string from the input, so I'm just remembering pointers inside the 
tokenized string.

I was thinking that maybe you could get access to the buffer that the tokenizer 
is using. I'd have to check to see if it's guaranteed to all be in one 
contiguous buffer or not.

Anyway, since the problem is at least superficially similar (at least to me!), 
I thought I'd mention how f-strings handle it. There might not be anything of 
value here to take away.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41971] multiple tests in test_tools fail since Python 3.9

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41970] test_lib2to3 fails since Python 3.9

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

I have also seen this and linked issue on running whole test suite while 
building python myself but running individual tests show no error and buildbots 
also never complained so I thought it's something wrong with my setup. I 
introduced the commit to silence deprecation warning due to lib2to3 and cannot 
reproduce the same running it individually.

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> I remember the char* pointer where the expression starts, then I parse the 
> expression into an AST, then I note the char* pointer where the expression 
> ended. The text between those is what's output before the equal sign []. This 
> is how I preserve all of the whitespace inside the expression.

Unless I am missing something, the problem is that the tokenizer has thrown 
away the extra newlines so we cannot keep the pointer where the expression 
starts. Also, I am not sure if this is the same scenario because f-strings 
cannot contain newlines in the expression brackets, can't they?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31329] Add idlelib module entry to doc

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I closed #41968 as a duplicate of this.

--
nosy:  -louielu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41968] 3.9 IDLE documentation.

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I agree that the doc needs more, but I am closing this as a duplicate of 
#31329, which is specifically doc about starting IDLE.  You can still answer 
Paine's questions here if you want.

File association: IDLE is not Python.  It is one of many Python-oriented 
editors and IDEs.  .py files are and by default should be associated for 
running with something that runs the file with python.exe.  On Windows, this is 
done via C:/Windows/py.exe.  The default version for double clicking is 
determined by a checkmark in the installer.

The Windows installer does associate .py files with IDLE for editing: rt click, 
edit with IDLE 

idle.exe is not needed for starting idle.

I don't know what you mean by 'source format'.  IDLE is written in Python.  The 
directory structure is mostly implementation detail not relevant to using IDLE. 
 File are described in idlelib/README.txt.  This might be mentioned in the doc.

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> Add idlelib module entry to doc
type:  -> enhancement
versions: +Python 3.10 -Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Eric V. Smith]

Eric V. Smith  added the comment:

For what it's worth, here's how f-strings with the "=" feature work:

I remember the char* pointer where the expression starts, then I parse the 
expression into an AST, then I note the char* pointer where the expression 
ended. The text between those is what's output before the equal sign []. This 
is how I preserve all of the whitespace inside the expression.

In my case I keep the AST to use when the expression gets evaluated, but in the 
string annotation case you'd throw it away. I don't think it would be very 
complicated to make this approach work across newlines.

[] Actually, I keep the equal sign itself and whitespace to the right of it, 
which is how f'{ x = }' produces " x = 42", instead of "x=42".

--
nosy: +eric.smith
resolution: rejected -> 
stage: resolved -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7946] Convoy effect with I/O bound threads and New GIL

[Dima Tisnek]

Dima Tisnek  added the comment:

My 2c as Python user:

Back in 2010, I've used multithreading extensively, both for concurrency and 
performance. Others used multiprocessing or just shelled out. People talked 
about using **the other** core, or sometimes the other socket on a server.

Now in 2020, I'm using asyncio exclusively. Some colleagues occasionally still 
shell out 🙈. None talking about using all cores on a single machine, rather, 
we'd spin up dozens of identical containers, which are randomly distributed 
across N machines, and the synchronisation is offloaded to some database (e.g. 
atomic ops in redis; transactions in sql).

In my imagination, I see future Python as single-threaded (from user's point of 
view, that is without multithreading api), that features speculative 
out-of-order async task execution (using hardware threads, maybe pinned) that's 
invisible to the user.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38914] Clarify wording for warning message when checking a package

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26680] Incorporating float.is_integer into Decimal

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Applied the reversion and removed the "release blocker" designation.

When a new PR ready, we'll be more expeditious in getting reviewed.

--
priority: release blocker -> 
versions: +Python 3.10 -Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26680] Incorporating float.is_integer into Decimal

[Raymond Hettinger]

Raymond Hettinger  added the comment:


New changeset 4e0ce820586e93cfcefb16c2a3df8eaefc54cbca by Raymond Hettinger in 
branch 'master':
Revert "bpo-26680: Incorporate is_integer in all built-in and standard library 
numeric types (GH-6121)" (GH-22584)
https://github.com/python/cpython/commit/4e0ce820586e93cfcefb16c2a3df8eaefc54cbca


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41972] bytes.find consistently hangs in a particular scenario

[Kevin Mills]

New submission from Kevin Mills :

Sorry for the vague title. I'm not sure how to succinctly describe this issue.

The following code:

```
with open("data.bin", "rb") as f:
data = f.read()

base = 15403807 * b'\xff'
longer = base + b'\xff'

print(data.find(base))
print(data.find(longer))
```

Always hangs on the second call to find.

It might complete eventually, but I've left it running and never seen it do so. 
Because of the structure of data.bin, it should find the same position as the 
first call to find.

The first call to find completes and prints near instantly, which makes the 
pathological performance of the second (which is only searching for one b"\xff" 
more than the first) even more mystifying.

I attempted to upload the data.bin file I was working with as an attachment 
here, but it failed multiple times. I assume it's too large for an attachment; 
it's a 32MiB file consisting only of 00 bytes and FF bytes.

Since I couldn't attach it, I uploaded it to a gist. I hope that's okay.

https://gist.github.com/Zeturic/7d0480a94352968c1fe92aa62e8adeaf

I wasn't able to trigger the pathological runtime behavior with other sequences 
of bytes, which is why I uploaded it in the first place. For example, if it is 
randomly generated, it doesn't trigger it.

I've verified that this happens on multiple versions of CPython (as well as 
PyPy) and on multiple computers / operating systems.

--
messages: 378197
nosy: Zeturic
priority: normal
severity: normal
status: open
title: bytes.find consistently hangs in a particular scenario
type: performance
versions: Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41970] test_lib2to3 fails since Python 3.9

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Same thing as in https://bugs.python.org/issue41971. I cannot reproduce this 
with a fresh install of 3.9 and our buildbots are not complaining about this.

ownloads/Python-3.9.0
❯ ./python -m test test_lib2to3
0:00:00 load avg: 3.64 Run tests sequentially
0:00:00 load avg: 3.64 [1/1] test_lib2to3

== Tests result: SUCCESS ==

1 test OK.

Total duration: 7.5 sec
Tests result: SUCCESS

---

❯ ./lel/bin/python3 -m test test_lib2to3
0:00:00 load avg: 3.87 Run tests sequentially
0:00:00 load avg: 3.87 [1/1] test_lib2to3

== Tests result: SUCCESS ==

1 test OK.

Total duration: 7.4 sec
Tests result: SUCCESS

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41971] multiple tests in test_tools fail since Python 3.9

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

All our buildbots are running these test without any problem and I cannot 
reproduce this locally after a fresh install from the binaries in 3.9, so my 
guess is that there must be something going on on your end.

❯ ./installed_dir/bin/python3 -m test test_tools
0:00:00 load avg: 6.90 Run tests sequentially
0:00:00 load avg: 6.90 [1/1] test_tools

== Tests result: SUCCESS ==

1 test OK.

Total duration: 40 ms
Tests result: SUCCESS

~/Downloads/Python-3.9.0
❯ ./python -m test test_tools
0:00:00 load avg: 6.43 Run tests sequentially
0:00:00 load avg: 6.43 [1/1] test_tools

== Tests result: SUCCESS ==

1 test OK.

Total duration: 2.5 sec
Tests result: SUCCESS

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41557] Upgrade Windows and macOS builds to use SQLite 3.33

[Steve Dower]

Steve Dower  added the comment:

I got the source repo update, and poked the PR to get it to re-run. If it's all 
clear, I have no opposition to someone else doing the final merge.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Guido van Rossum]

Guido van Rossum  added the comment:

Too bad. :-(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39934] Fatal Python error "XXX block stack overflow" when exception stacks >10

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

PR 22395 should be backported to 3.8 and 3.9

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41557] Upgrade Windows and macOS builds to use SQLite 3.33

[Erlend Egeberg Aasland]

Change by Erlend Egeberg Aasland :


--
nosy: +steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41971] multiple tests in test_tools fail since Python 3.9

[Felix Yan]

New submission from Felix Yan :

I am packaging Python for Arch and the test suite of Python 3.8.6 passes here 
without these:

==
FAIL: test_multiple_roots 
(test.test_tools.test_c_analyzer.test_common.test_files.IterFilesTests)
--
Traceback (most recent call last):
  File 
"/build/python/src/Python-3.9.0/Lib/test/test_tools/test_c_analyzer/test_common/test_files.py",
 line 118, in test_multiple_roots
self.assertEqual(self.calls, [
AssertionError: Lists differ: [('_w[26 chars]tion walk at 0x7ff2eb562f70>)), 
('_walk', ('eg[42 chars]0>))] != [('_w[26 chars]tion _walk_tree at 
0x7ff2e5babdc0>)), ('_walk'[54 chars]0>))]

First differing element 0:
('_walk', ('spam', '.c', ))
('_walk', ('spam', '.c', ))

- [('_walk', ('spam', '.c', )),
? ^

+ [('_walk', ('spam', '.c', )),
? ++   + 

-  ('_walk', ('eggs', '.c', ))]
? ^

+  ('_walk', ('eggs', '.c', ))]
? ++   + 


==
FAIL: test_multiple_suffixes 
(test.test_tools.test_c_analyzer.test_common.test_files.IterFilesTests)
--
Traceback (most recent call last):
  File 
"/build/python/src/Python-3.9.0/Lib/test/test_tools/test_c_analyzer/test_common/test_files.py",
 line 189, in test_multiple_suffixes
self.assertEqual(self.calls, [
AssertionError: Lists differ: [('_walk', ('spam', None, ))] != [('_walk', ('spam', None, ))]

First differing element 0:
('_walk', ('spam', None, ))
('_walk', ('spam', None, ))

- [('_walk', ('spam', None, ))]
? ^

+ [('_walk', ('spam', None, ))]
? ++   + 


==
FAIL: test_no_suffix 
(test.test_tools.test_c_analyzer.test_common.test_files.IterFilesTests) [None]
--
Traceback (most recent call last):
  File 
"/build/python/src/Python-3.9.0/Lib/test/test_tools/test_c_analyzer/test_common/test_files.py",
 line 209, in test_no_suffix
self.assertEqual(self.calls, [
AssertionError: Lists differ: [('_walk', ('spam', None, ))] != [('_walk', ('spam', None, ))]

First differing element 0:
('_walk', ('spam', None, ))
('_walk', ('spam', None, ))

- [('_walk', ('spam', None, ))]
? ^

+ [('_walk', ('spam', None, ))]
? ++   + 


==
FAIL: test_no_suffix 
(test.test_tools.test_c_analyzer.test_common.test_files.IterFilesTests) []
--
Traceback (most recent call last):
  File 
"/build/python/src/Python-3.9.0/Lib/test/test_tools/test_c_analyzer/test_common/test_files.py",
 line 209, in test_no_suffix
self.assertEqual(self.calls, [
AssertionError: Lists differ: [('_walk', ('spam', '', ))] != [('_walk', ('spam', '', ))]

First differing element 0:
('_walk', ('spam', '', ))
('_walk', ('spam', '', ))

- [('_walk', ('spam', '', ))]
?   ^

+ [('_walk', ('spam', '', ))]
?   ++   + 


==
FAIL: test_no_suffix 
(test.test_tools.test_c_analyzer.test_common.test_files.IterFilesTests) [()]
--
Traceback (most recent call last):
  File 
"/build/python/src/Python-3.9.0/Lib/test/test_tools/test_c_analyzer/test_common/test_files.py",
 line 209, in test_no_suffix
self.assertEqual(self.calls, [
AssertionError: Lists differ: [('_walk', ('spam', (), ))] != [('_walk', ('spam', (), ))]

First differing element 0:
('_walk', ('spam', (), ))
('_walk', ('spam', (), ))

- [('_walk', ('spam', (), ))]
?   ^

+ [('_walk', ('spam', (), ))]
?   ++   + 


==
FAIL: test_one_root 
(test.test_tools.test_c_analyzer.test_common.test_files.IterFilesTests)
--
Traceback (most recent call last):
  File 
"/build/python/src/Python-3.9.0/Lib/test/test_tools/test_c_analyzer/test_common/test_files.py",
 line 99, in test_one_root
self.assertEqual(self.calls, [
AssertionError: Lists differ: [('_walk', ('spam', '.c', ))] != [('_walk', ('spam', '.c

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

After some extra testing, we found that is not enough with handling spaces, 
unfortunately, as things like the walrus need also the parentheses to be 
preserved. Given that multi-line strings will complicate the code considerably, 
I am going to drop this for the time being unless we have a good way to solve 
these problems in a way that does not involve several workarounds.

--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41970] test_lib2to3 fails since Python 3.9

[Felix Yan]

New submission from Felix Yan :

I am packaging Python for Arch and the tests suite of Python 3.8.6 pass here 
without this:

```
0:09:06 load avg: 0.87 [205/424] test_lib2to3   


  
test test_lib2to3 crashed -- Traceback (most recent call last): 


  
  File "/build/python/src/Python-3.9.0/Lib/test/libregrtest/runtest.py", line 
270, in _runtest_inner  


refleak = _runtest_inner2(ns, test_name)


  
  File "/build/python/src/Python-3.9.0/Lib/test/libregrtest/runtest.py", line 
221, in _runtest_inner2 


the_module = importlib.import_module(abstest)   


  
  File "/build/python/src/Python-3.9.0/Lib/importlib/__init__.py", line 127, in 
import_module   

  
return _bootstrap._gcd_import(name[level:], package, level) 


  
  File "", line 1030, in _gcd_import   


  
  File "", line 1007, in _find_and_load


  
  File "", line 986, in _find_and_load_unlocked


  
  File "", line 680, in _load_unlocked 


  
  File "", line 790, in exec_module   


  
  File "", line 228, in _call_with_frames_removed  


  
  File "/build/python/src/Python-3.9.0/Lib/test/test_lib2to3.py", line 5, in 


 
from lib2to3.tests import load_tests


  
  File "/build/python/src/Python-3

[issue41923] Add PEP 613 typing.TypeAlias to the standard library

[Guido van Rossum]

Change by Guido van Rossum :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28343] Bad encoding alias cp936 -> gbk: euro sign

[Steve Dower]

Change by Steve Dower :


--
versions: +Python 3.10 -Python 2.7, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41917] Python 3.9rc2 fails to install matplotlib

[Steve Dower]

Change by Steve Dower :


--
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41952] sys.version has double space between month and date

[Steve Dower]

Change by Steve Dower :


--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41965] distutils.spawn.find_executable() fails to find .py files on Windows

[Steve Dower]

Steve Dower  added the comment:

Yep, unless this directly impacts normal distutils usage, it's not going to be 
fixed (regardless of the formal deprecation proposal, we already have a policy 
of minimizing changes to distutils).

You could submit a patch to Bugzilla to switch them to shutil.which.

--
nosy: +steve.dower
resolution:  -> wont fix
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7946] Convoy effect with I/O bound threads and New GIL

[Gregory P. Smith]

Gregory P. Smith  added the comment:

It's a known issue and has been outlined very well and still comes up from time 
to time in real world applications, which tend to see this issue and Dave's 
presentation and just work around it in any way possible for their system and 
move on with life.

Keeping it open even if nobody is actively working on it makes sense to me as 
it is still a known issue that could be resolved should someone have the 
motivation to complete the work.

--
versions: +Python 3.10 -Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> Yeah, we are trying to modify the patch to automatically add spaces between 
> keywords and other tokens.

Once we can see how this looks, we could think if is worth the effort or is 
better to abandon the idea. One could also argue that having multi-line 
annotations produce some uglier strings is inconsistent enough to drop this, 
but I think is easier to consider once we have a fully working version.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41923] Add PEP 613 typing.TypeAlias to the standard library

[Guido van Rossum]

Guido van Rossum  added the comment:


New changeset 4f3c25043d651a13c41cffcee703f7d5cb677cc7 by Mikhail Golubev in 
branch 'master':
bpo-41923: PEP 613: Add TypeAlias to typing module (#22532)
https://github.com/python/cpython/commit/4f3c25043d651a13c41cffcee703f7d5cb677cc7


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> Okay, though aren't there cases where spaces are necessary? E.g. between 
> names or between a name or keyword and a string or number.

Yeah, we are trying to modify the patch to automatically add spaces between 
keywords and other tokens.

> Assuming this will only happen when the annotation spans multiple lines, 
> right?

Yeah, for annotations in a single line we can just grab the string from the 
code as it is.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Guido van Rossum]

Guido van Rossum  added the comment:

> the string generated for multi line annotations [...] won't have spaces 
> between characters.

Okay, though aren't there cases where spaces are necessary? E.g. between names 
or between a name or keyword and a string or number.

Assuming this will only happen when the annotation spans multiple lines, right?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> We cannot declare those illegal.

They won't be illegal, is just that the string generated for multi line 
annotations will be a bit uglier because it won't have spaces between 
characters.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Guido van Rossum]

Guido van Rossum  added the comment:

Okay, but doesn't *any* way of handling multi-line annotations spoil the
wins? We cannot declare those illegal. (Though we can switch to a different
way of representing those.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

>  I just wonder why bother
changing this unless we reproduce the spaces in the string exactly.

Maintenance reasons: it removes over 1000 lines of hand written code. 
Additionally, makes the compiler a bit faster because it does not need to 
transform the AST into a string.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Guido van Rossum]

Guido van Rossum  added the comment:

I know the final result is the same (i.e. the AST is identical and the
string differs only in the use of whitespace). I just wonder why bother
changing this unless we reproduce the spaces in the string exactly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39934] Fatal Python error "XXX block stack overflow" when exception stacks >10

[Irit Katriel]

Irit Katriel  added the comment:

After studying Mark's patch and the rest of the code, I understand the for loop 
oddity.

The else block of the for loop comes after the 
compiler_pop_fblock(c, FOR_LOOP, start);

So you can do this all day and it won't complain:

for x in '1': pass
else:
 for x in '2': pass
 else:
  for x in '3': pass
  else:
   for x in '4': pass
   else:
for x in '5': pass
else:
 for x in '6': pass
 else:
  for x in '7': pass
  else:
   for x in '8': pass
   else:
for x in '9': pass
else:
 for x in '10': pass
 else:
  for x in '11': pass
  else:
   for x in '12': pass
   else:
for x in '13': pass
else:
 for x in '14': pass
 else:
  for x in '15': pass
  else:
   for x in '16': pass
   else:
for x in '17': pass
else:
 for x in '18': pass
 else:
  for x in '19': pass
  else:
   for x in '20': pass
   else:
for x in '21': pass
else:
 for x in '22': pass
 else:
  for x in '23': pass
  else:
   for x in '24': pass
   else:
for x in '25': pass
else:
 for x in '26': pass
 else:
  for x in '27': pass
  else:
   for x in '28': pass
   else:
for x in '29': pass



I guess the same goes for while loops, and the else of a try-except.


Since If blocks were deliberately left out of this game, I'm assuming that this 
"static nesting" is actually number of frames, rather than true static nesting. 
If that is the case then there is no issue here and we can close this ticket.

But if this is something to be fixed, then I am happy to make a patch along the 
lines of Mark's (which I partially already have).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> Hm, this buffer thing is not what I had expected. I think it's a show-stopper 
> -- if we don't reproduce spaces exactly.

Hummm, I think I may have not clarified this in my previous messages: the space 
issue is an aesthetic one. The AST generated with and without spaces is the 
same. As far as I understand get_type_hints() will give the same result in all 
cases being considered.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

It is not actually much of a difference;
 $ cat t.py
def func(
arg: typing.Callable[
[int, str],
str # test
]
):
pass

import typing
print("Annotations: ", func.__annotations__)
print("get_type_hints: ", typing.get_type_hints(func))
$ ./python t.py
Annotations:  {'arg': 'typing.Callable[[int,str],str]'}
get_type_hints:  {'arg': typing.Callable[[int, str], str]}

$ ./python
>>> foo: (List[List[List[int, str, bytes]], List[(int, str)]])
>>> __annotations__
{'foo': 'List[List[List[int,str,bytes]],List[(int,str)]]'}

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Guido van Rossum]

Guido van Rossum  added the comment:

Hm, this buffer thing is not what I had expected. I think it's a show-stopper 
-- if we don't reproduce spaces exactly.

Maybe we should just buffer the entire input always?

A way to handle multi-line annotations would be to surround them with 
parentheses? That way get_type_hints() doesn't need to change. Alternatively, 
strip comments and newlines in get_type_hints().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> What's the issue exactly with multi-line annotations? 

The issue is that for grabbing the source, we are inspecting the tokenize 
buffer and using the information in the expr node of the annotation to know 
what we need to pick from it.

In multi-line annotations, the tokenize buffer does not contain the whole 
source, it just contains the last line.

To make it work, we would need to grab the individual tokens (because the 
tokenizer has all of them) and append them together to form the expression, but 
in this process we lose the non-significant whitespace.

---

Regarding loosing the whitespace for multi-line strings (or all string if we 
want to be consistent): the more I think about it the more I think is not that 
bad. This means that the string that the parser will place in the annotation 
field (and therefore the one that ends in __annotations__) will not have 
spaces, but the ast will be able to parse it without any problems. It will just 
look a bit crumbled, but that field is supposed to be used by tools, not 
humans, so I don't think is a huge problem.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41968] 3.9 IDLE documentation.

[E. Paine]

E. Paine  added the comment:

I am not touching the issue nosy and components for now but suspect this is a 
Windows installation issue. As you said, there is no `idle.exe` file and 
instead file association is done through a `idle.bat` file found at 
Lib/idlelib/idle.bat. I have a few of questions that I would appreciate if you 
could answer to give some more context:
 - What behaviour is incorrect? Is it that double clicking doesn't launch 
python.exe or there is no "edit with IDLE" entry in the right-click menu?
 - Did you install it for different users compared to the other versions (all 
users / admin vs. just for this user)
 - Do you have any other Python versions installed simultaneously with 3.9?

I agree it may be nice to give a brief overview on IDLE's "behind-the-scenes" 
though I am slightly confused why you raised this issue as a documentation 
issue.

--
nosy: +epaine

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Guido van Rossum]

Guido van Rossum  added the comment:

What's the issue exactly with multi-line annotations? Is it that they can't be 
parsed by eval()? Could that be fixed by replacing comments and newlines with 
spaces? (Either in the parser or in get_type_hints() -- actually 
ForwardRef._evaluate(), which calls eval().)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41969] ttk.RadioButtons mis-sized under Windows 10 UI Scaling, with dpiAware set true

[Athanasius]

New submission from Athanasius :

Having recently looked into UI Scaling for an application I help develop I 
became aware with an issue involving tkinter.ttk.RadioButton and Windows UI 
Scaling.

When you have some UI Scaling set along with dpiAware set true you might find 
that radio buttons are mis-rendered. The usual manifestation of this is that 
the last button will be properly sized initially, with the others being 
undersized. If you even only mouse-over the 'correct' radio button it will then 
become undersized as well.

For more detail see: https://github.com/Athanasius/tk-radio-buttons

NB: This *has* been fixed upstream in the Tcl/Tk source, but I see they've not 
made a new release since 8.6.10 in November 2019.  Thus this issue is more in 
the way of a heads up about a bug others might encounter, and will have to wait 
for newer Tcl/Tk in some version of Python for it to be fixed.

See https://core.tcl-lang.org/tk/info/3c6660b6f0bed337 for the issue and fix.

--
components: Tkinter
messages: 378171
nosy: Athanasius
priority: normal
severity: normal
status: open
title: ttk.RadioButtons mis-sized under Windows 10 UI Scaling, with dpiAware 
set true
type: behavior
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41894] UnicodeDecodeError during load failure in non-UTF-8 locale

[Kevin]

Kevin  added the comment:

Glad you were able to reproduce on Linux.

I have since changed the PR to use PyUnicode_DecodeFSDefault based on review 
feedback. I was going to say that you will have to fight it out with @methane 
on GH, but I see that that's you. :D Would have been nice if you would have 
left the updated feedback there as well so people who aren't familiar would 
know it's one person adjusting their recommendation vs two different people 
with conflicting recommendations.


The only issue I see with using backslashreplace is that users of non-UTF-8 
locales would see message text that contains non-ASCII characters only as 
escape codes. eg, the message above would show "Il modulo dipendente libbz2.so 
non \xe8 stato caricato." instead of "Il modulo dipendente libbz2.so non è 
stato caricato." By using PyUnicode_DecodeFSDefault instead, the message should 
be properly decoded but any encoding errors (such as utf-8 paths, etc) would be 
handled by surrogateescape.

I guess the question comes to: what's more important to be decoded, the message 
text or the path?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41961] Windows install failure "could not set file security"

[Eryk Sun]

Eryk Sun  added the comment:

> It might be that Python is the first/only MSI that the user 
> has tried though?

It's likely the first per-user MSI install attempted since the security of the 
"Installer" directory was modified. There's no problem with a per-machine 
install.

> did your change reproduce the "Error: 0"? 

It's the same "Could not set file security for file... Error: 0" dialog, which 
is created by the python-3.9.0.exe child process that executes from a copy in 
%TEMP%. 

In Process Monitor, I see the actual access-denied error due to the installer 
service trying to open the directory with WRITE_DAC and WRITE_OWNER access.

If I also remove the Everyone group that grants read and execute access, the 
installer service fails at an earlier step, which results in an 0x80070643 
fatal installation error.

> It sounds like just resetting the owner isn't enough on its own, 
> but the inherited ACLs should include SYSTEM and not prevent it 
> from working. 

The security descriptor that the installer service sets prevents inheritance of 
discretionary access control entries (i.e. the DACL is protected):

>>> sd = GetFileSecurity('Installer', DACL_SECURITY_INFORMATION)
>>> sd.GetSecurityDescriptorControl()[0] & SE_DACL_PROTECTED
4096

Thus the entries in the ACL for SYSTEM and Administrators groups are set 
explicitly instead of inherited from the parent directory. If something else 
rewrites the security on the directory, it can just as easily protect the DACL 
from inheritance.

In my first experiment, I had left the entry for Everyone (WD) in the DACL, 
but, as mentioned above, it turns out that it's required at an earlier step. So 
a fix has to also add it back:

>icacls "%APPDATA%\Microsoft\Installer" /grant:r *WD:(OI)(CI)(RX)

Also, in my first message, I manually re-added the SYSTEM and Administrators 
entries unnecessarily -- though it doesn't hurt to do so. It turns out that all 
the service needs is for the directory's owner to be set back to the 
Administrators group. Then it implicitly has WRITE_DAC access. It gets 
WRITE_OWNER access as well, even though the file security at the time doesn't 
grant it (the owner of an object does not implicitly have WRITE_OWNER access to 
it), so presumably the service is temporarily enabling SeTakeOwnershipPrivilege 
for the process or thread.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

s/For 516389 annotated arguments/For 516389 arguments/.

Example usages;
https://search.tree.science/?query=AnnAssign(annotation%20=%20expr(lineno%20=%20not%20~end_lineno))

https://search.tree.science/?query=arg(annotation=expr(lineno%20=%20not%20~end_lineno))

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41964] difflib SequenceMatcher get_matching_blocks returns non-matching blocks in some cases

[Tim Peters]

Tim Peters  added the comment:

I believe your testing code is in error, perhaps because it's so overly 
elaborate you've lost track of what it's doing.  Here's a straightforward test 
program:

import difflib
s1='http://local:56067/register/200930162135700";>'
s2='http://local:53813/register/20100517282450281";>'
d = difflib.SequenceMatcher(None, s1, s2)
for m in d.get_matching_blocks():
print(m, repr(s1[m.a : m.a + m.size]),
 repr(s2[m.b : m.b + m.size]))

and its output under 3.9.0:

Match(a=0, b=0, size=39) 'http://local:5' 'http://local:5'
Match(a=43, b=43, size=12) '/register/20' '/register/20'
Match(a=59, b=55, size=1) '1' '1'
Match(a=66, b=56, size=2) '00' '00'
Match(a=68, b=70, size=2) '">' '">'
Match(a=70, b=72, size=0) '' ''

Your test program is obtaining the substrings to display from these two lines:

mtch_a = dpiy_Frst[mblk.a : mblk.a + mblk.size];
mtch_b = dpiy_Frst[mblk.b : mblk.b + mblk.size];

But BOTH of those are extracting substrings from `dply_Frst`. Looks like you 
intended to use the `dply_Scnd` argument in the second line instead.

If I change my test program to use `s1` for both substrings, then it matches 
the output you gave. But that doesn't make sense ;-)

So I'm closing this as not-a-bug. If I missed something relevant, feel free to 
re-open it.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

After scanning 1552 annotated assignments in over 25k python modules (from most 
populer packages in PyPI) there are only 6 AnnAssign nodes where the 
annotations are multiline. For 516389 annotated arguments, only 20 of them are 
multiline annotations.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

On the other hand, multi-line annotations are rare enough we could consider 
falling back to token reconstruction then (the produced string will miss all 
whitespace in that case).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

After investigating with BTaskaya, seems that we cannot use the literal string 
from the tokenizer for multi-line annotations. But we could use the tokens 
(losing the literal whitespace), which honestly is less exciting because we 
will lose the literal formatting.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41961] Windows install failure "could not set file security"

[Steve Dower]

Steve Dower  added the comment:

> I was able to reproduce the error dialog by changing the owner of the 
> "Installer" folder to the current user and removing the two DACL entries that 
> grant access to Administrators and SYSTEM.

Yeah, I'd assumed the most likely cause was the ACLs having been reset on the 
machine in question, though I'd expect that to cause most installs to fail. It 
might be that Python is the first/only MSI that the user has tried though?

It sounds like just resetting the owner isn't enough on its own, but the 
inherited ACLs should include SYSTEM and not prevent it from working. So I 
wonder if something else is at play?

Also, did your change reproduce the "Error: 0"? Or did you get an actual error 
code?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41968] 3.9 IDLE documentation.

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
components: +IDLE
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36534] tarfile: handling Windows (path) illegal characters in archive member names

[Eryk Sun]

Eryk Sun  added the comment:

> extract the sanitizing function into a common module 
> (could be *pathlib*?) to avoid duplicates

I would prefer something common, cross-platform, and function-based such as 
os.path.isreservedname and os.path.sanitizename. In posixpath, it would just 
have to reserve and sanitize slash [/] and null [\0]. The real work would be in 
ntpath.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41968] 3.9 IDLE documentation.

[Velson Horie]

New submission from Velson Horie :

https://docs.python.org/3/library/idle.html
I am trying to sort out why a new installation of 3.9 onto Win10 64bit fails to 
associate a .py file type with IDLE. {hint - there is no idle.exe. installed - 
Why?}. 2.7, 3.6 and 3.8 worked fine.
The documentation gives no indication of the source format of "IDLE", where it 
might be found, what it calls etc - only how it is used.
Some indication/link to the underlying software structure should be included in 
the documentation.
PS thank you for all the other work involved.

--
assignee: docs@python
components: Documentation
messages: 378161
nosy: c.v.horie, docs@python
priority: normal
severity: normal
status: open
title: 3.9 IDLE documentation.
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41965] distutils.spawn.find_executable() fails to find .py files on Windows

[Eryk Sun]

Eryk Sun  added the comment:

The existing behavior of find_executable() is unusual compared to a native 
Windows file search via CreateProcessW or SearchPathW, for which a default 
".exe" extension is appended only if the executable name doesn't already have 
an extension. (CreateProcessW supports searching for a file that has no 
extension by adding a trailing dot to the name, which in Windows is equivalent 
to the name without the trailing dot.) 

However, I don't think it's prudent to change the default behavior of this old 
code, especially since there are plans to deprecate distutils per PEP 632. It's 
recommended to migrate existing code to use shutil.which() and 
subprocess.run(). 

There are open issues to improve the behavior of shutil.which() in Windows. For 
example, currently it's limited to PATHEXT extensions, which is a 
generalization of how find_executable() works with ".exe". shutil.which() won't 
find "rst2man.py" if ".PY" isn't in PATHEXT. The proper behavior, if the CMD 
shell is the standard of correctness, is to include the exact name as the first 
candidate to check before trying the name plus the PATHEXT extensions. To be 
more like the CMD shell, a name without an extension should be excluded, but 
effectively included anyway if PATHEXT contains a "." entry.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I have attached a draft PR for this:

https://github.com/python/cpython/pull/22587

I think this may be a considerable win as the diff is +66 −1,013

--
nosy: +gvanrossum, lukasz.langa, lys.nikolaou
stage: patch review -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
nosy: +BTaskaya

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +21579
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/22587

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41967] Handle annotations in the parser to avoid the need for roundtrip

[Pablo Galindo Salgado]

New submission from Pablo Galindo Salgado :

For Python 3.10 now that annotations are always returned as strings, we can 
drop all the unparse logic by obtaining the string in the parser directly 
(after checking that the annotation is a valid expression).

--
messages: 378158
nosy: pablogsal
priority: normal
severity: normal
status: open
title: Handle annotations in the parser to avoid the need for roundtrip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41965] distutils.spawn.find_executable() fails to find .py files on Windows

[Alexander Todorov]

Alexander Todorov  added the comment:

@Eryk Sun,
you are of course correct but still don't we need to handle this in Python? 

- find_executable() will search for rst2man.py.exe which doesn't exist so no 
good for the user
- there is also no rst2man.exe which is probably an issue with how docutils 
distributes this script - e.g. it doesn't distribute the wrapper .exe file
- lastly the caller (in my case python-bugzilla) will try to execute rst2man.py 
which will probably fail but at least they would see another failure and are 
more likely to change their own code base

The trouble for me here is that find_executable() will generally not find 
anything that doesn't have .exe suffix on Windows. This is independent of how 
the resulting file would be consumed later.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32592] Drop support of Windows Vista and 7 in Python 3.9

[Roundup Robot]

Change by Roundup Robot :


--
nosy: +python-dev
nosy_count: 10.0 -> 11.0
pull_requests: +21578
pull_request: https://github.com/python/cpython/pull/22586

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41965] distutils.spawn.find_executable() fails to find .py files on Windows

[Eryk Sun]

Eryk Sun  added the comment:

The linked code runs subprocess.check_output([rstbin, path]), which won't work 
with "rst2man.py" in Windows. Reliably running a .py script in Windows requires 
running it explicitly via py[w].exe or python[w].exe, or via sys.executable 
from an existing Python process. 

subprocess.Popen calls WinAPI CreateProcessW, which supports PE executables, 
and also CMD/BAT scripts via %ComSpec%. With shell=True the command is run via 
`cmd.exe /c`, which tries ShellExecuteExW in order to execute the file using 
the filetype's default registered action. However, the default action for .py 
files isn't dependable. It's often configured to open the script in an editor 
or IDE.

Thus script entrypoints in Windows get installed as wrapped launchers, e.g. 
"script.exe" instead of "script.py". The launcher executes the embedded 
entrypoint script via the fully-qualified path of python.exe.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41966] datetime.time issue with pickling in PyPy

[Dean]

New submission from Dean :

I've run into an issue pickling a datetime.time subclass in PyPy3. I believe it 
arises because PyPy uses the pure Python implementation of time and 
time.__reduce_ex__() returns (time, ...) on this line 
(https://github.com/python/cpython/blob/044a1048ca93d466965afc027b91a5a9eb9ce23c/Lib/datetime.py#L1551)
 rather than (self.__class__, ...) as it does for datetime.datetime (and in 
datetime.date.__reduce__()). So when pickle creates the dump it uses the time 
class rather than my subclass.

--
components: Library (Lib)
messages: 378155
nosy: belopolsky, , p-ganssle
priority: normal
severity: normal
status: open
title: datetime.time issue with pickling in PyPy
type: behavior
versions: Python 3.10, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 
3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36534] tarfile: handling Windows (path) illegal characters in archive member names

[Cristi Fati]

Cristi Fati  added the comment:

As I see things now, there are multiple things (not necessarily related to this 
issue) to deal with:

1. Update *tarfile* and add *\_sanitize\_windows\_name* (name can change), that 
uses *pathlib.\_WindowsFlavour.reserved\_names* (or some public wrapper), and 
also handles control chars (pointed out by @eriksun), so that it covers as many 
cases as possible (I'd say all, but there's almost always one that gets away)

2. Fix *pathlib.\_WindowsFlavour.reserved\_names*

3. Apply the fix to *zipfile* as well

4. (optional) extract the sanitizing function into a common module (could be 
*pathlib*?) to avoid duplicates

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27827] pathlib is_reserved fails for some reserved paths on Windows

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-37517 "Improve error messages for Windows reserved file names".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27827] pathlib is_reserved fails for some reserved paths on Windows

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-36534 "tarfile: handling Windows (path) illegal characters in 
archive member names".

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36534] tarfile: handling Windows (path) illegal characters in archive member names

[STINNER Victor]

STINNER Victor  added the comment:

> IIRC there's already an open issue for that.

Ah, I found bpo-27827 "pathlib is_reserved fails for some reserved paths on 
Windows", open since 2016 (by you ;-)).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41965] distutils.spawn.find_executable() fails to find .py files on Windows

[Alexander Todorov]

New submission from Alexander Todorov :

As part of installing python-bugzilla via pip it searches for `rst2man` or 
`rst2man.py`, see:
https://github.com/python-bugzilla/python-bugzilla/blob/master/setup.py#L81

on Windows venvs there is venv\Scripts\rst2man.py (no .exe or without suffix) 
and when you call find_executable('rst2man.py') if doesn't find it.


The trouble is in this code snippet:

```
_, ext = os.path.splitext(executable)
if (sys.platform == 'win32') and (ext != '.exe'):
executable = executable + '.exe'
```

`ext` here is `.py` and the if condition executes its body so the executable to 
search for becomes `rst2man.py.exe` which doesn't exist.

The extension check has been like that for more than 20 years:
https://github.com/python/cpython/commit/69628b0ad10f89a65902f5b911d1040ed9ae1ca2

but IMO it should be 

```
if (sys.platform == 'win32') and (ext == ''):
executable = executable + '.exe'
```

i.e. add `.exe` only if the file we're looking for doesn't already have an 
extension.


Let me know what you think? I can submit a PR for this.


Related issues:

- https://bugs.python.org/issue2200

- https://bugs.python.org/issue39260

--
components: Distutils
messages: 378150
nosy: Alexander.Todorov, dstufft, eric.araujo
priority: normal
severity: normal
status: open
title: distutils.spawn.find_executable() fails to find .py files on Windows
type: behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41964] difflib SequenceMatcher get_matching_blocks returns non-matching blocks in some cases

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41964] difflib SequenceMatcher get_matching_blocks returns non-matching blocks in some cases

[Snidhi Sofpro]

New submission from Snidhi Sofpro :

-- Demo case with unexpected results starting from matching block 3 
(result of code that follows):

sys.version_info(major=3, minor=6, micro=9, releaselevel='final', serial=0) 

Matches between:
http://local:56067/register/200930162135700";>
http://local:53813/register/20100517282450281";>

Match(a=0, b=0, size=39)
same-> http://local:5
same=> http://local:5 

Match(a=43, b=43, size=12)
same-> /register/20
same=> /register/20 

Match(a=59, b=55, size=1)
same-> 1
same=> 0 

Match(a=66, b=56, size=2)
same-> 00
same=> 93 

Match(a=68, b=70, size=2)
same-> ">
same=>  


# -- code that results in the above:

def get_mblk(dpiy_Frst, dpiy_Scnd):
import difflib;
sqmn_o = difflib.SequenceMatcher(None, dpiy_Frst, dpiy_Scnd);
mblk_ls = [ block for block in sqmn_o.get_matching_blocks()];
for mblk in mblk_ls[:-1]: #exclude the last dummy block
print(mblk);
mtch_a = dpiy_Frst[mblk.a : mblk.a + mblk.size];
mtch_b = dpiy_Frst[mblk.b : mblk.b + mblk.size];
print('same->', mtch_a);
print('same=>', mtch_b, '\n');
#endfor
#endef get_mblk

# --- main --

s1='http://local:56067/register/200930162135700";>'
s2='http://local:53813/register/20100517282450281";>'

import sys; print(sys.version_info, '\n');
print("Matches between:"); print(s1); print(s2); print('\n');
get_mblk(s1, s2);

--
messages: 378149
nosy: Snidhi
priority: normal
severity: normal
status: open
title: difflib SequenceMatcher get_matching_blocks returns non-matching blocks 
in some cases
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38914] Clarify wording for warning message when checking a package

[Jürgen Gmach]

Jürgen Gmach  added the comment:

This issue can be closed as the related PR was merged.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41963] ConfigParser: stripping of comments should be documented

[Jürgen Gmach]

Jürgen Gmach  added the comment:

This "while comments have a default value, inline comments do not" should read 
"while comment prefixes have a default value, inline comment prefixes do not"

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41963] ConfigParser: stripping of comments should be documented

[Jürgen Gmach]

New submission from Jürgen Gmach :

While working on `tox-ini-fmt`, a formatter for - you guessed it - `tox.ini` 
files, I noticed ConfigParser strips comments when reading a config file.

( https://github.com/tox-dev/tox-ini-fmt/issues/34 )

While reasonable, this behaviour is surprising, as it is neither documented at 
https://docs.python.org/3/library/configparser.html nor in the docstrings (read 
and _read) which I read at first.

The stripping of comments is only documented with inline comments

https://github.com/jugmac00/cpython/blob/610a60c601fb4380eee30e15be1cd4dcbdaeec4c/Lib/configparser.py#L1019

and

https://github.com/jugmac00/cpython/blob/610a60c601fb4380eee30e15be1cd4dcbdaeec4c/Lib/configparser.py#L1031

Once I found these comments, I was surprised once again, as in my code the 
inline comments were not stripped. After some more pdb-ing and reading the 
source of ConfigParser, I noticed that - while comments have a default value, 
inline comments do not - and that is why when you read a config file, some 
comments get removed and others not.

I'd like to work on a pull request to document this behaviour, both in the 
official documentation and in the docstrings of the read and the _read methods.

--
messages: 378146
nosy: jugmac00
priority: normal
severity: normal
status: open
title: ConfigParser: stripping of comments should be documented

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com