[issue41282] Deprecate and remove distutils

[Arfrever Frehtes Taifersar Arahesis]

Change by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43002] Exception chaining accepts exception classes

[Ram Rachum]

Ram Rachum  added the comment:

People barely know how to use the right form of this feature. Providing them 
with a wrong form that's confusingly similar to the right form and fails 
silently is an unfortunate choice. 

"Or just don't do anything. A `raise exception` inside an except block will be 
automatically chained with the current exception [...]"

With the wrong message, yes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43006] Changed behaviour of inspect.signature() in Python 3.10

[Zac Hatfield-Dodds]

New submission from Zac Hatfield-Dodds :

Consider the following snippet, which passes on Python 3.9 and earlier:


import inspect

def f(x: int = None):
pass

print(inspect.signature(constructor))
assert inspect.signature(constructor).parameters["a"].annotation == int


But under Python 3.10 (alpha 4), the annotation is instead read as 
Optional[int].  This is correct for typing.get_type_hints(), but *not* for 
inspect.signature().

I therefore suspect that this is an accidental side-effect from support for 
PEP-563 deferred evaluation of annotations.

--
components: Library (Lib)
messages: 385524
nosy: Zac Hatfield-Dodds
priority: normal
severity: normal
status: open
title: Changed behaviour of inspect.signature() in Python 3.10
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42606] Support POSIX atomicity guarantee of O_APPEND on Windows

[Eryk Sun]

Eryk Sun  added the comment:

> I've found a catch via ProcessHacker: CreateFile() with 
> GENERIC_WRITE (or FILE_GENERIC_WRITE) additionally grants 
> FILE_READ_ATTRIBUTES for some reason. 

CreateFileW always requests at least SYNCHRONIZE and FILE_READ_ATTRIBUTES 
access.

The I/O manager requires synchronize access if a file is opened in synchronous 
mode. CreateFileW goes a step further. It always requests synchronize access, 
even with asynchronous mode (overlapped). The File object gest signaled when an 
I/O request completes, but it's not very useful in the context of overlapping 
requests.

Requesting read-attributes access supports API functions that query certain 
file information. Here are some of the more common queries that require 
read-attributes access:

FileBasicInformation (GetFileInformationByHandleEx, GetFileTime)
FileAllInformation (GetFileInformationByHandle)
FileAttributeTagInformation (GetFileInformationByHandleEx)

Thus os.fstat(fd) can succeed even if the file is opened in O_WRONLY mode.

CreateFileW also implicitly requests DELETE access if FILE_FLAG_DELETE_ON_CLOSE 
is used, instead of letting the call fail with an invalid-parameter error if 
delete access isn't requested. This behavior isn't documented.

> undoing the side effect applies to O_CREAT and O_TRUNC too: we can create 
> and/or 
> truncate the file, but then fail. 

I think truncation via TRUNCATE_EXISTING (O_TRUNC, with O_WRONLY or O_RDWR) or 
overwriting with CREATE_ALWAYS (O_CREAT | O_TRUNC) is at least tolerable 
because the caller doesn't care about the existing data. When overwriting, the 
caller also wants to remove any alternate data streams and extended attributes 
in the file. Nothing important is lost. Also, since both cases retain the 
original file's security descriptor, at least failure after truncation or 
overwriting isn't a security hole.

Unless we require CREATE_NEW (O_CREAT | O_EXCL) whenever O_TEMPORARY is used 
(i.e. as the tempfile module uses it), there is a potential for an existing 
file to be deleted if all handles are closed on failure, as discussed 
previously. This is unacceptable not only because of potential unrecoverable 
data loss, but also because the security descriptor is lost. 

With OPEN_ALWAYS (O_CREAT), CREATE_ALWAYS or CREATE_NEW, there's the chance of 
leaving behind a new empty file or alternate data stream on failure, which is a 
problem, but at least nothing is lost.

> _open_osfhandle() can still fail with EMFILE. 

The CRT supports 8192 open file descriptors (128 arrays of 64 fds), so failing 
with EMFILE should be rare, in extreme cases. There's also a remote possibility 
of memory corruption that causes __acrt_lowio_set_os_handle() to fail with 
EBADF because the fd value is negative, or its handle value isn't the default 
INVALID_HANDLE_VALUE, or the CRT _nhandle count is corrupt. These aren't 
practical concerns, just as DuplicateHandle() failing isn't a practical 
concern, but failure should be handled conservatively.

> the same issue would apply even in case of direct implementation of 
> os.open()/open() via CreateFile() 

Migrating to CreateFileW() might need to be shelved until Python uses native OS 
File handles instead of CRT file descriptors. The remaining reliance on the CRT 
low I/O layer ties our hands for now.

> Truncation can simply be deferred until we have the fd and then performed 
> manually.

What if it fails after overwriting an existing file? Manually overwriting only 
after getting the new fd is complicated. To match CREATE_ALWAYS (O_CREAT | 
O_TRUNC), before overwriting it would have to query the existing file 
attributes and fail the call if FILE_ATTRIBUTE_HIDDEN or FILE_ATTRIBUTE_SYSTEM 
is set. If the file itself has to be overwritten (i.e. the default, anonymous 
data stream), as opposed to a named data stream, it would have to delete all 
named data streams and extended attributes in the file. Normally that's all 
implemented atomically in the filesystem. 

In contrast, TRUNCATE_EXISTING (O_TRUNC) is simple to emulate, since 
CreateFileW implents it non-atomically with a subsequent NtSetInformationFile: 
FileAllocationInformation system call. 

> But I still don't know how to deal with O_TEMPORARY, unless there is a 
> way to unset FILE_DELETE_ON_CLOSE on a handle.

For now, that's possible with NTFS and the Windows API in all supported 
versions of Windows by using a second kernel File with DELETE access, which is 
opened before the last handle to the first kernel File is closed. After you 
close the first open, use the second one to call SetFileInformation: 
FileDispositionInfo to undelete the file. That said, if NTFS changes the 
default for delete-on-close to use a POSIX-style delete (immediate unlink), it 
won't be possible to 'undelete' the file.

Windows 10 supports additional flags with FileDispositionInfoEx (21), or NTAPI 
FileDispositionInformationEx [1]. This provides a better way to disable or 
modify the 

[issue43004] No type variables left in collections.abc.Callable

[Tyler Yep]

Tyler Yep  added the comment:

Got it, thanks!

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43004] No type variables left in collections.abc.Callable

[Ken Jin]

Ken Jin  added the comment:

This seems to be a duplicate of Issue40494. It has already been fixed in Python 
3.10, and in Python 3.9.2 (which isn't out yet). You can see the what's new for 
it here 
https://docs.python.org/3/whatsnew/3.9.html#notable-changes-in-python-3-9-2.

The expected release date for Python 3.9.2 is Monday, 2021-02-15 according to 
PEP 596 https://www.python.org/dev/peps/pep-0596/.

For now, I guess you'll have to use the old typing.Callable, then update it in 
newer versions of Python.

I'm assuming you are currently using python 3.9.0/3.9.1, which has this bug. 
I'm unable to reproduce it on Python 3.10.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42980] argparse: GNU-style help formatter

[Will Noble]

Will Noble  added the comment:

Ya I was following the precedent of non-documentation figuring that formal 
documentation should be done for the entire class and that would be a much 
larger undertaking and maybe involve further refactoring. I could potentially 
undertake that task in the future but I think it's out-of-scope for this change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42800] Traceback objects allow accessing frame objects without triggering audit hooks

[Ryan Hileman]

Ryan Hileman  added the comment:

Just updated the PR with another much simpler attempt, using a new READ_AUDIT 
flag (aliased to READ_RESTRICTED, and newtypes documentation updated).

I re-ran timings for the new build, and in all cases they match or slightly 
beat my previous reported timings.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42800] Traceback objects allow accessing frame objects without triggering audit hooks

[Steve Dower]

Steve Dower  added the comment:

> I think it could make sense to:
> 1. Alias READ_RESTRICTED to a new READ_AUDIT flag and use the latter instead, 
> as it is more clear.
> 2. Update the newtype docs to mention READ_AUDIT and remove documentation for 
> the the unused RESTRICTED flags.
> 3. Deprecate the non-functional RESTRICTED flags if that's possible?
> 4. Only cross the setattr/delattr audit flag bridge if a future refactor 
> calls for it.

Sounds good to me. We can deprecate RESTRICTED with no intention to 
remove it, since it's documented.

Do you want to prepare a PR for this?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42800] Traceback objects allow accessing frame objects without triggering audit hooks

[Ryan Hileman]

Ryan Hileman  added the comment:

I agree that READ_RESTRICTED would work, and I'm strongly in support of 
refactoring my patch around that kind of flag, as it simplifies it quite a bit 
and the if statement is already there.

However, using the seemingly legacy RESTRICTED flag names for audit is 
confusing in my opinion:

- The audit subsystem does something entirely different from the long 
deprecated "Restricted execution" feature (removed in 3.0?)
- Nothing in the stdlib uses RESTRICTED that I can see.
- The documentation for RESTRICTED flags (Doc/extending/newtypes.rst) doesn't 
say anything about the audit system for READ_RESTRICTED, and talks about 
restricted mode as though it still exists.
- RESTRICTED only supports __getattr__ (PY_WRITE_RESTRICTED does nothing at 
all, and there is no delattr equivalent). This doesn't actually matter for this 
patch, it's just confusing in the context of audit, as there are 
`object.__setattr__` and `object.__delattr__` audit points but no corresponding 
flags.

I think it could make sense to:
1. Alias READ_RESTRICTED to a new READ_AUDIT flag and use the latter instead, 
as it is more clear.
2. Update the newtype docs to mention READ_AUDIT and remove documentation for 
the the unused RESTRICTED flags.
3. Deprecate the non-functional RESTRICTED flags if that's possible?
4. Only cross the setattr/delattr audit flag bridge if a future refactor calls 
for it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43005] Package Relative Imports - double dot doc example not working

[Guido van Rossum]

Guido van Rossum  added the comment:

This is not a bug in Python. Please ask a user forum for help understanding 
(for example, https://discuss.python.org/c/users/7).

--
nosy: +gvanrossum
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43005] Package Relative Imports - double dot doc example not working

[Ixio]

New submission from Ixio :

I've been trying to do a double dot import without success, I've tried going 
back to the official documentation in order to figure it out however even the 
example shown does not work.

Official example : 
https://docs.python.org/3/reference/import.html#package-relative-imports

The following instructions mimic pretty well the doc example :

```bash
mkdir package
cd package
touch __init__.py
mkdir subpackage1
touch subpackage1/__init__.py
mkdir subpackage2
touch subpackage2/__init__.py
echo "eggs = 42" > subpackage2/moduleZ.py
echo "from ..subpackage2.moduleZ import eggs" > subpackage1/moduleX.py
python subpackage1/moduleX.py
cd subpackage1
python moduleX.py

cd ..
echo "from subpackage1 import moduleX" > moduleA.py
python moduleA.py
```

However I get 3 times the following error :
from ..subpackage2.moduleZ import eggs
ImportError: attempted relative import beyond top-level package

Even though the docs say "In [...] subpackage1/moduleX.py [...] the following 
are valid relative imports: [...] from ..subpackage2.moduleZ import eggs".

I have no idea how to fix the doc as I would love to do "from 
..subpackage2.moduleZ import eggs" from "subpackage1/moduleX.py" since that's 
the structure I'm trying to set up for a Python project of mine but obviously 
it does not seem to work.

My system uses Python 3.9 and I've used Docker to try this scenario on Python 
3.8, 3.5 and even 3.2: it does not work either. I've also tried on another 
machine and asked a friend to try, no luck.

I'm hoping someone here can explain how double dot imports work and fix the 
documentation accordingly.

--
assignee: docs@python
components: Documentation
messages: 385515
nosy: Ixio, brett.cannon, docs@python, eric.snow, ncoghlan
priority: normal
severity: normal
status: open
title: Package Relative Imports - double dot doc example not working
type: behavior
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43004] No type variables left in collections.abc.Callable

[Guido van Rossum]

Change by Guido van Rossum :


--
nosy: +kj

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43002] Exception chaining accepts exception classes

[Steven D'Aprano]

Steven D'Aprano  added the comment:

It is intended behaviour. `raise ... from` is a general mechanism that you can 
call anywhere, it is not just limited to raising from the previous exception. 
It is designed for explicitly setting the chained exception to some arbitrary 
exception. See the PEP:

https://www.python.org/dev/peps/pep-3134/#explicit-exception-chaining

>>> raise ValueError('bad value') from IndexError('bad index')
IndexError: bad index

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "", line 1, in 
ValueError: bad value


If you wish to chain from the current exception, you have to chain from the 
current exception, and not create a new one.

This is not an interpreter bug, it's working fine. Your first instinct was 
correct: your colleague had written "bad code" and his intention was probably 
to raise from the exception he had just caught. 

Or just don't do anything. A `raise exception` inside an except block will be 
automatically chained with the current exception unless you explicitly disable 
it with `raise exception from None`.

--
nosy: +steven.daprano
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42967] [security] urllib.parse.parse_qsl(): Web cache poisoning - `; ` as a query args separator

[Éric Araujo]

Éric Araujo  added the comment:

Too bad that semicolon is not recommended nowadays, it was a nice way to avoid 
ampersand HTML escape issues!

One server software that generates links using semicolons is debbugs: 
https://bugs.debian.org/cgi-bin/pkgreport.cgi?archive=both;package=gtk3-engines-xfce;package=gtk2-engines-xfce

--
components: +Library (Lib) -C API
nosy: +eric.araujo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42800] Traceback objects allow accessing frame objects without triggering audit hooks

[Steve Dower]

Steve Dower  added the comment:

I'm fine with either approach, though adding the READ_RESTRICTED flag would 
also be fine.

The audit trailing leading to a bypass is very important, and traversing frames 
to find functions in their locals or closures is very useful.

This is nothing like a traditional sandbox, which we can't do because there are 
(few) legitimate uses and (many) existing users of our internal interpreter 
state inspection/mutation APIs. This is about tracing internal events - hence 
_audit_ events - so we want them for all equivalent ways to achieve the same 
internal inspection/mutation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43004] No type variables left in collections.abc.Callable

[Tyler Yep]

New submission from Tyler Yep :

The following code crashes when I try to run it, even though it passes when I 
use `from typing import Callable` instead.

```
from collections.abc import Callable
from typing import Any, TypeVar

V = TypeVar("V")
Function = Callable[[list[V], V, V], float]


def random_fn(fn: Function[Any]) -> Function[Any]:
return fn
```

--
messages: 385511
nosy: tyler.yep
priority: normal
severity: normal
status: open
title: No type variables left in collections.abc.Callable
type: crash
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42957] os.readlink produces wrong result on windows

[Steve Dower]

Steve Dower  added the comment:

I agree with Eryk (unsurprisingly, we discussed this change *a lot* back when 
it was made ~3 years ago).

os.readlink is the lowest-level API that gives a reliable result.

os.path.realpath is the high-level API that probably does what most users want 
most of the time (and relies on os.readlink being reliable).

We can't change either of these to more of a middle-ground without taking away 
the ability for users to implement their own behaviour. So there's no 
behavioural issue here. (The behavioural issue we fixed in 3.8 was that 
realpath didn't resolve anything at all, so we're not reverting back to that.)

If you'd like a new API added to either retrieve the display path or to read a 
link and try to partially normalise the result without resolving any more 
links, please open an issue specifically for that (targeting Python 3.10). You 
probably also want to prototype a wrapper around readlink() to do it, and be 
prepared to have many potential edge cases thrown against it.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42966] argparse: customizable help formatter

[paul j3]

paul j3  added the comment:

Years ago I proposed a `format_wrapper`, a Format class factory

https://bugs.python.org/issue12806#msg218395

also

https://bugs.python.org/issue39809

There I show that the formatter may be lambda

formatter = lambda prog: argparse.HelpFormatter(prog, width=100)

The need for improving help customization has been around for a long time:

https://bugs.python.org/issue11695
Improve argparse usage/help customization

https://bugs.python.org/issue32123
Make the API of argparse.HelpFormatter public

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42875] argparse incorrectly shows help string on a new line in case of long command string

[paul j3]

paul j3  added the comment:

This issue is discussed in:

https://bugs.python.org/issue34724
argparse subparser help indent too short

and

https://stackoverflow.com/questions/3215/max-help-position-is-not-works-in-python-argparse-library

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42980] argparse: GNU-style help formatter

[paul j3]

paul j3  added the comment:

I was thinking of a refactoring that included the ', '.join(...) loop, but on 
further thought your refactoring might be enough for the case I raised.

For example:

def _format_option_with_args(self, option_string, args_string):
if option_string.startswith('--'):
return '%s %s' % (option_string, args_string)
return '%s' % option_string

would produce a

 -f, --foo FOOsomething

and 

def _format_option_with_args(self, option_string, args_string):
return option_string
   
would produce

 -f, --foo   something

I wonder if this refactoring merits some mention in the documentation.  Or 
maybe the GnuStyleLongOptionsHelpFormatter subclass is enough of an example.  
The details of how other formatter subclasses work aren't documented, but they 
are available to developers who can read the code.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42955] Add sys.module_names: list of stdlib module names (Python and extension modules)

[Andre Roberge]

Change by Andre Roberge :


--
nosy: +aroberge

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42606] Support POSIX atomicity guarantee of O_APPEND on Windows

[Alexey Izbyshev]

Alexey Izbyshev  added the comment:

> FYI, here are the access rights applicable to files

Thanks, I checked that mapping in headers when I was writing 
_Py_wopen_noraise() as well. But I've found a catch via ProcessHacker: 
CreateFile() with GENERIC_WRITE (or FILE_GENERIC_WRITE) additionally grants 
FILE_READ_ATTRIBUTES for some reason. This is why I add FILE_READ_ATTRIBUTES to 
FILE_GENERIC_WRITE for DuplicateHandle() -- otherwise, the duplicated handle 
didn't have the same rights in my testing. So basically I have to deal with (a) 
_wopen() not guaranteeing contractually any specific access rights and (b) 
CreateFile() not respecting the specified access rights exactly. This is where 
my distrust and my original question about "default" access rights come from.

> I overlooked a case that's a complication. For O_TEMPORARY, the open uses 
> FILE_FLAG_DELETE_ON_CLOSE; adds DELETE to the requested access; and adds 
> FILE_SHARE_DELETE to the share mode. 
> _Py_wopen_noraise() can easily keep the required DELETE access. The 
> complication is that you have to be careful not to close the original file 
> descriptor until after you've successfully created the duplicate file 
> descriptor. If it fails, you have to return the original file descriptor from 
> _wopen(). If you close all handles for the kernel File and fail the call, the 
> side effect of deleting the file is unacceptable. 

Good catch! But now I realize that the problem with undoing the side effect 
applies to O_CREAT and O_TRUNC too: we can create and/or truncate the file, but 
then fail. Even if we could assume that DuplicateHandle() can't fail, 
_open_osfhandle() can still fail with EMFILE. And since there is no public 
function in MSVCRT to replace an underlying handle of an existing fd, we can't 
preallocate an fd to avoid this. There would be no problem if we could just 
reduce access rights of an existing handle, but it seems there is no such API.

I don't like the idea of silently dropping the atomic append guarantee in case 
of a failure, so I'm not sure how to proceed with the current approach.

Moreover, the same issue would apply even in case of direct implementation of 
os.open()/open() via CreateFile() because we still need to wrap the handle with 
an fd, and this may fail with EMFILE. For O_CREAT/O_TRUNC, it seems like it 
could be reasonably avoided:

* Truncation can simply be deferred until we have the fd and then performed 
manually.

* To undo the file creation, we could use GetLastError() to learn whether 
CreateFile() with OPEN_ALWAYS actually created the file, and then delete it on 
failure (still non-atomic, and deletion can fail, but probably better than 
nothing).

But I still don't know how to deal with O_TEMPORARY, unless there is a way to 
unset FILE_DELETE_ON_CLOSE on a handle.

As an aside, it's also very surprising to me that O_TEMPORARY is allowed for 
existing files at all. If not for that, there would be no issue on failure 
apart from non-atomicity.

Maybe I should forgo the idea of supporting O_APPEND for os.open(), and instead 
just support it in FileIO via WriteFile()-with-OVERLAPPED approach...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42997] Improve error message for missing : before suites

[Andre Roberge]

Change by Andre Roberge :


--
nosy: +aroberge

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42973] argparse: mixing optional and positional arguments... not again

[Glenn Linderman]

Change by Glenn Linderman :


--
nosy: +v+python

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

[Guido van Rossum]

Change by Guido van Rossum :


--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41282] Deprecate and remove distutils

[Steve Dower]

Change by Steve Dower :


--
keywords: +patch
pull_requests: +23122
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/23142

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42992] Tkinter bbox coordinates incorrectly drawn

[Ron Hoffmann]

Ron Hoffmann  added the comment:

Thank you for your response. I have been fighting this issue in a large piece 
of code for quite some time.
So I wrote a small test script as you asked for and the problem will not 
reproduce. All behaviour of code is as expected.
I must therefore assume there is no bug in tkinter and the problem lies in my 
code or with my development environment.
Please consider the issue closed

--
resolution:  -> not a bug

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42992] Tkinter bbox coordinates incorrectly drawn

[E. Paine]

E. Paine  added the comment:

I cannot reproduce. Taking a rectangle as an example,
canvas.bbox(canvas.create_rectangle(5, 5, 100, 100, width=1))
returns `(4, 4, 101, 101)` (rather than `(5, 5, 100, 100)`) because of rounding 
while calculating the outline.

As a result, running
canvas.create_rectangle(canvas.bbox(canvas.create_rectangle(5, 5, 100, 100, 
width=1)), width=1, outline="yellow")
we see a black outline inside a yellow one (as to be expected).

Could we please have a minimal script to help diagnose the problem?

--
nosy: +epaine, serhiy.storchaka
versions: +Python 3.10, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42988] Information disclosure via pydoc -p: /getfile?key=path allows to read arbitrary file on the filesystem

[Ken Jin]

Ken Jin  added the comment:

I have updated the PR to do the following:

- removed html_getfile
- implement a unique secret as suggested above

Now it says:
>>> python.exe -m pydoc -b
Server ready at 
http://localhost:52035/Y1YzOyEbitE9BB_dtH0YXbMgGXbcg3ytXLpvpg8P7GEM3z1hlCkTXgxaojtAordVqs2s6oHZHPMbXqq9mXq_wbJCVW8jnHrgQeYE5hFUQuI/


FWIW, it seems that Jupyter notebook server deals with the same problems in a 
similar manner: 
https://jupyter-notebook.readthedocs.io/en/stable/security.html#security-in-the-jupyter-notebook-server

I removed the warning message in the PR because I think this is secure enough.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42504] Failure to build with MACOSX_DEPLOYMENT_TARGET=11 on Big Sur

[Lonny Kapelushnik]

Lonny Kapelushnik  added the comment:

I believe this is causing issues when trying to install pyjq 
(https://github.com/doloopwhile/pyjq/issues/54).

@ronaldoussoren are you still planning on creating a patch for this? Or should 
we plan on fixing it downstream?

--
nosy: +lonnylot

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

[Mark Shannon]

Change by Mark Shannon :


--
keywords: +patch
pull_requests: +23121
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/24298

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43003] Parts of the API will be removed in Python 4.0 ?

[Michael Clerx]

New submission from Michael Clerx :

The unicode C-API documentation has a deprecation warning on this page:

  https://docs.python.org/3.9/c-api/unicode.html#deprecated-py-unicode-apis

stating some functions are

  Deprecated since version 3.3, will be removed in version 4.0.

But as far as I understand there are no current plans for a 4.0.
Should the text read "removed in version 3.10" ?

--
assignee: docs@python
components: Documentation
messages: 385501
nosy: MichaelClerx, docs@python
priority: normal
severity: normal
status: open
title: Parts of the API will be removed in Python 4.0 ?

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

[Mark Shannon]

Mark Shannon  added the comment:

Rather than:

  f = create_frame_or_gen(...);
  if () return f;
  retval = _PyEval_EvalFrame(tstate, f, 0);
  _PyObject_GC_TRACK(f);
  return retval;

I was thinking:

  f = create_frame(...);
  if () return make_gen(f);
  retval = _PyEval_EvalFrame(tstate, f, 0);
  _PyObject_GC_TRACK(f);
  return retval;

The complicated part is create_frame(...), so I want to clean that up first.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43002] Exception chaining accepts exception classes

[Ram Rachum]

New submission from Ram Rachum :

I saw this line of code today: 
https://github.com/hyperledger/sawtooth-sdk-python/commit/c27b962541c9ae68fd1e6dc691ddee883234f112#diff-eb008203eae2160c5e14c42e5fd2eee164709a93bf5136fa79cc256d4e46eaffR92

I was about to tell this guy that his code is bad since the part after the 
`from` should be a specific exception, not just an exception class. I thought I 
should double-check myself first. Lo and behold: 

```
$ cat x.py
def f():
raise TypeError

try:
f()
except TypeError:
raise ValueError from TypeError

$ python x.py
TypeError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "x.py", line 7, in 
raise ValueError from TypeError
ValueError
```

The line doesn't fail, but it silently trims away the stacktrace of the 
previous exception. (Normally the stacktrace would include what's inside the 
`f` function.)

I'm not sure whether this is intended behavior or not. The documentation does 
say "the second expression must be another exception class or instance" but (a) 
it's not clear what the use case is when it's a class and (b) I doubt that the 
person who wrote the code above, and any other person who writes such code, 
would be aware that their traceback got snipped until it's too late and they're 
stuck with a bug report with incomplete information.

--
components: Interpreter Core
messages: 385499
nosy: cool-RR
priority: normal
severity: normal
status: open
title: Exception chaining accepts exception classes
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42722] Add --debug command line option to unittest to enable post-mortem debugging

[Dominik Vilsmeier]

Dominik Vilsmeier  added the comment:

Is anybody interested in reviewing the PR? It seems like a useful enhancement 
to me.

--
versions: +Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42967] [security] urllib.parse.parse_qsl(): Web cache poisoning - `; ` as a query args separator

[Adam Goldschmidt]

Adam Goldschmidt  added the comment:

I haven't noticed, I'm sorry. I don't mind closing mine, just thought it could 
be a nice first contribution. Our PRs are different though - I feel like if we 
are to implement this, we should let the developer choose the separator and not 
limit to just `&` and `;` - but that discussion probably belongs in the PR.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42996] hashlib documentation references an obsolete RFC 2898

[Illia Volochii]

Change by Illia Volochii :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42967] [security] urllib.parse.parse_qsl(): Web cache poisoning - `; ` as a query args separator

[Senthil Kumaran]

Senthil Kumaran  added the comment:

Ken, Please don't close your PR. I will review it. It has a CLA signed
which is helpful.

On Fri, Jan 22, 2021 at 4:53 AM Ken Jin  wrote:

>
> Ken Jin  added the comment:
>
> Adam, I linked a PR 2 days ago here
> https://github.com/python/cpython/pull/24271 , it has the test suite
> passing and the appropriate changes to cgi.py. Would you like to review it?
> Or since you submitted a PR, would you prefer I close mine instead?
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--
title: [security] urllib.parse.parse_qsl(): Web cache poisoning - `;` as a 
query args separator -> [security] urllib.parse.parse_qsl(): Web cache 
poisoning - `; ` as a query args separator

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42967] [security] urllib.parse.parse_qsl(): Web cache poisoning - `; ` as a query args separator

[Ken Jin]

Ken Jin  added the comment:

Adam, I linked a PR 2 days ago here 
https://github.com/python/cpython/pull/24271 , it has the test suite passing 
and the appropriate changes to cgi.py. Would you like to review it? Or since 
you submitted a PR, would you prefer I close mine instead?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42967] [security] urllib.parse.parse_qsl(): Web cache poisoning - `; ` as a query args separator

[Adam Goldschmidt]

Change by Adam Goldschmidt :


--
pull_requests: +23120
pull_request: https://github.com/python/cpython/pull/24297

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42996] hashlib documentation references an obsolete RFC 2898

[miss-islington]

miss-islington  added the comment:


New changeset b745a6143ae79efe00aa46affe5ea31a06b0b532 by Illia Volochii in 
branch 'master':
bpo-42996: Update a reference to PKCS #5 in hashlib docs to version 2.1 
(GH-24289)
https://github.com/python/cpython/commit/b745a6143ae79efe00aa46affe5ea31a06b0b532


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42955] Add sys.module_names: list of stdlib module names (Python and extension modules)

[STINNER Victor]

STINNER Victor  added the comment:

Another potential use case: restrict pydoc web server to stdlib modules, see 
bpo-42988.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42988] Information disclosure via pydoc -p: /getfile?key=path allows to read arbitrary file on the filesystem

[STINNER Victor]

STINNER Victor  added the comment:

> Python's http.server at least warns about this in the docs:
> https://docs.python.org/3/library/http.server.html
> and limits the serving to the current dir (and subdirs).

I would be fine with a warning in the pydoc documentation, but I dislike 
warnings display on the command line. When I see such warning, I feel that the 
machine considers that I'm dumb and I have no idea of what I am doing.

If it's unsafe, can we make it safe by default?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

[STINNER Victor]

Change by STINNER Victor :


--
title: Improve the C code for calling Python code -> Improve the C code for 
calling Python code: _PyEval_EvalCode()

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42990] Improve the C code for calling Python code

[STINNER Victor]

STINNER Victor  added the comment:

PyObject *
_PyEval_EvalCode(PyThreadState *tstate,
   PyObject *_co, PyObject *globals, PyObject *locals,
   PyObject *const *args, Py_ssize_t argcount,
   PyObject *const *kwnames, PyObject *const *kwargs,
   Py_ssize_t kwcount, int kwstep,
   PyObject *const *defs, Py_ssize_t defcount,
   PyObject *kwdefs, PyObject *closure,
   PyObject *name, PyObject *qualname)

Hum no, only 16 arguments, everything is fine! :-D

More seriously, I already considered to rework this code.

The pseudo code is:

  if (...) return ;
  frame = (...);
  retval = _PyEval_EvalFrame(tstate, f, 0);
  _PyObject_GC_TRACK(f);
  return retval;

I like the idea of splitting these two parts:

  f = create_frame_or_gen(...);
  if () return f;
  retval = _PyEval_EvalFrame(tstate, f, 0);
  _PyObject_GC_TRACK(f);
  return retval;

I see one advantage: stack memory consumation, we don't have to hold the 16 
arguments on the stack, only 3 parameters (tstate, f, 0).

> Split the Eval functions into vector and tuple/dict forms, both forms taking 
> a "frame descriptor", as well as the arguments.

Hum, it seems like you have a different idea how to refactor the code.

Would it be worth it to have more specialized create_frame_or_gen() functions 
for the common cases?

--

I would also be interested by the ability to not create a frame at all, but I 
guess that it's a way larger refactoring.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43001] python3.9.1 test_embed test_tabnanny failed

[Alexei S]

New submission from Alexei S :

== Tests result: FAILURE then FAILURE ==

396 tests OK.

2 tests failed:
test_embed test_tabnanny

27 tests skipped:
test_bz2 test_curses test_dbm_gnu test_dbm_ndbm test_devpoll
test_gdb test_idle test_ioctl test_kqueue test_lzma test_msilib
test_ossaudiodev test_readline test_smtpnet test_ssl
test_startfile test_tcl test_tix test_tk test_ttk_guionly
test_ttk_textonly test_turtle test_winconsoleio test_winreg
test_winsound test_zipfile64 test_zoneinfo

2 re-run tests:
test_embed test_tabnanny

Total duration: 10 min 58 sec
Tests result: FAILURE then FAILURE
make: *** [Makefile:1199: test] Error 2

--
components: Tests
messages: 385490
nosy: asholomitskiy84
priority: normal
severity: normal
status: open
title: python3.9.1 test_embed test_tabnanny failed
type: compile error
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42988] Information disclosure via pydoc -p: /getfile?key=path allows to read arbitrary file on the filesystem

[STINNER Victor]

STINNER Victor  added the comment:

Fidget-Spinner wrote on the PR:
> AFAIK no. However, pydoc currently works by calling inspect on files it sees 
> in path, and this may reveal private code as Marc-Andre Lemburg pointed out 
> on the bpo. I will try the random url token he suggested via 
> secrets.token_urlsafe to see if it helps.

pydoc shows global constant values in the doc. So yes, if you find a 
settings.py of a Django project, you can discover secrets.

I'm working on bpo-42955 "Add sys.module_names: list of stdlib module names 
(Python and extension modules)".

One option would be to restrict pydoc to stdlib modules by defaults, and ask to 
opt-in for discovery of any module installed on the system (sys.path).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42988] Information disclosure via pydoc -p: /getfile?key=path allows to read arbitrary file on the filesystem

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Why not limit the serving to sys.path?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42985] AMD64 Arch Linux Asan 3.x fails: command timed out: 1200 seconds without output

[STINNER Victor]

STINNER Victor  added the comment:

For faulthandler.enable(), maybe we reset SIGSEGV signal handler to the default 
handler if __has_feature(address_sanitizer) is true:
https://clang.llvm.org/docs/AddressSanitizer.html#conditional-compilation-with-has-feature-address-sanitizer

But we cannot do that in faulthandler._sigsegv() since this function is used to 
test_faulthandler to check the signal handler installed by faulthandler 
previously.

Maybe we should add a function to test.support which resets the signal handler 
and then trigger a crash.

There are multiple functions which trigger crashes on purpose:

* _testcapi.crash_no_current_thread() => Py_FatalError()
* _testcapi.return_null_without_error() => Py_FatalError()
* _testcapi.return_result_wit_error() => Py_FatalError()
* _testcapi.negative_refcount() => Py_FatalError()
* _testcapi.pymem_buffer_overflow() => Py_FatalError()
* _testcapi.set_nomemory(0) is used to trigger a _PyErr_NormalizeException 
crash => Py_FatalError()
* etc.

Py_FatalError() calls abort() which raises SIGABRT signal, but ASAN doesn't 
catch this signal.

More generally, search for support.SuppressCrashReport usage in tests.

See also faulthandler_suppress_crash_report() C function.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41798] [C API] Revisit usage of the PyCapsule C API with multi-phase initialization API

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 2f12a1b7ecc9e0cf39b4c6994473e6cb9989f81b by Hai Shi in branch 
'master':
bpo-41798: Allocate the _curses._C_API on the heap memory (GH-24186)
https://github.com/python/cpython/commit/2f12a1b7ecc9e0cf39b4c6994473e6cb9989f81b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42988] Information disclosure via pydoc -p: /getfile?key=path allows to read arbitrary file on the filesystem

[Marc-Andre Lemburg]

Marc-Andre Lemburg  added the comment:

On 22.01.2021 01:28, STINNER Victor wrote:
> 
> STINNER Victor  added the comment:
> 
>> I'd suggest to print a big warning on the console, explaining that the web 
>> server will potentially make all content accessible by the user visible to 
>> anyone else on the same server.
> 
> I dislike this idea. If they are vulnerabilities, they should be fixed. Users 
> usually have no idea what to do when seeing such warning.

The problem is that neither the docs nor the help text in the command
make it clear what exactly is exposed via the web server pydoc
launches.

While the getfile API endpoint can be used to view non-Python files
as well (which is certainly not intended), the tool also makes available
all Python modules which can be found on sys.path of the user starting
pydoc -p. It shows all doc-strings, functions, the class structure and
literal values of any constants found in those modules.

In a corporate environment this can easily result in data leaks
of e.g. unreleased software, personal information, disclosure of
NDA protected code, designs, algorithms and other secrets.

Fixing just getfile or replacing those links with file:// ones will
only address one part of the problem. The other is educating the
user about possible consequences of running a server on the machine
-- just like you warn users about deleting files before going ahead
with it.

Python's http.server at least warns about this in the docs:
https://docs.python.org/3/library/http.server.html
and limits the serving to the current dir (and subdirs).

My guess is that pydoc -p really is just intended to be useful
for the current user. Rather than having it serve files under
a blanket URL, it could restrict browsing to a random URL
token generated at pydoc startup and open this in the browser
via the "b" command or the -b option, e.g.

"""
Server ready at http://localhost:8080/uLy6t87AD-ScPthd/
Server commands: [b]rowser, [q]uit
server>
"""

That would make it harder to guess the base URL and limit
exposure.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27477] IDLE: Switch search dialogs to ttk widgets, and other refinement

[Terry J. Reedy]

Terry J. Reedy  added the comment:

#33987 was reopened, 4 patches merged and backported, and reclosed.
This is only open for other possible refinements listed above.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43000] All SSL requests fail with WRONG_VERSION_NUMBER when a packet sniffer is open

[Christian Heimes]

Christian Heimes  added the comment:

I'm not familiar with Telerik Fiddler and don't have access to the tool. From 
the description on its Wikipedia page, it sounds like the tool is an active 
interceptor that uses a man-in-the-middle attack approach.

Python's ssl module wraps OpenSSL. All steps of the TLS handshake are performed 
by OpenSSL code. Your connection issue is most likely an incompatibility 
between recent OpenSSL 1.1.1 and your tool. I recommend that you contact the 
vendor of your tool.

The ssl module supports passive introspection of TLS connections with 
SSLKEYLOGFILE. The format is supported by Wireshark and other tools.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com