[issue43179] Remove 32-bit s390 Linux support (s390-linux-gnu triplet)

2021-02-15 Thread John Paul Adrian Glaubitz


John Paul Adrian Glaubitz  added the comment:

s390 is a 31-bit platform, not a 32-bit platform.

I also don't see what this change achieves other than making the use of Python 
3.10 on s390 harder.

It's not like the removal of support for non-threaded builds which actually 
saved quite some code:

> https://github.com/python/cpython/commit/a6a4dc816d68df04a7d592e0b6af8c7ecc4d4344

--

___
Python tracker 
<https://bugs.python.org/issue43179>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43179] Remove s390 support

2021-02-15 Thread John Paul Adrian Glaubitz


John Paul Adrian Glaubitz  added the comment:

> I opened this ticket after a user told me that they grepped the source code 
> of Python, found the string "s390", and concluded that s390 is still 
> supported by us.

Because one user was surprised by a few lines in configure.ac, the conclusion 
is to remove support for that architecture?

That's a very odd justification.

If it really just affects configure.ac, I don't really see a point in removing 
them. Even in OpenJDK we keep such unofficial architectures in configure.ac 
(with my OpenJDK upstream maintainer hat on).

--

___
Python tracker 
<https://bugs.python.org/issue43179>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43179] Remove s390 support

2021-02-15 Thread John Paul Adrian Glaubitz


John Paul Adrian Glaubitz  added the comment:

That's an argument I have personally never heard before and I have been dealing 
with a lot of architecture support in many packages.

FWIW, lots of upstream projects have targets which are officially supported and 
others which are there but not supported and that was never really a problem.

As long as the architectures are being maintained in the Linux kernel, GCC, 
binutils and glibc, they can be considered usable and maintained.

The m68k architecture for example has a very active and large community due to 
the popularity of retro computing. This has lead to efforts which will soon 
bring Rust support to M68k and the Amiga.

I therefore don't really understand what you gain when you make it harder for 
downstreams to use Python.

--

___
Python tracker 
<https://bugs.python.org/issue43179>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43179] Remove s390 support

2021-02-15 Thread John Paul Adrian Glaubitz


John Paul Adrian Glaubitz  added the comment:

And, FWIW, I generally don't quite understand what the problem with old triplet 
definitions in configure.ac are. I assume they don't hurt anyone, do they?

You never know what usecases your users have and as long as a code snippet 
doesn't interfere with other code, I don't see little point in removing it.

--

___
Python tracker 
<https://bugs.python.org/issue43179>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43179] Remove s390 support

2021-02-15 Thread John Paul Adrian Glaubitz


John Paul Adrian Glaubitz  added the comment:

I'm a Debian Developer and maintainer for multiple Debian Ports architectures.

Please don't remove support for Alpha, HPPA, m68k, ia64, PowerPC, SuperH, SPARC 
as we're still maintaining these in Debian.

Here are the latest build results for Python3.9 in Debian with build logs for 
all these architectures:

> https://buildd.debian.org/status/package.php?p=python3.9=sid

There are no issues reported and we heavily rely on Python.

--
nosy: +glaubitz

___
Python tracker 
<https://bugs.python.org/issue43179>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43160] argparse: add extend_const action

2021-02-08 Thread paul j3


paul j3  added the comment:

It's not clear what your patch does that the existing 'store_const' doesn't.   
Even 'append_const' does the same except for a extra level of list/tuple 
nesting.

But I'll admit that I didn't much need for  'extend' addition, but others, 
including the original developer liked it.

I suspect users of your addition will get a surprise if they aren't careful to 
provide a list or tuple 'const':

const='foobar'
const=str

--

___
Python tracker 
<https://bugs.python.org/issue43160>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43143] Allow multiple assignment (i.e. tuple on LHS) in walrus operator

2021-02-07 Thread Paul Sokolovsky


Paul Sokolovsky  added the comment:

Thanks.

I would like to put this ticket in the context of other grammar-related 
tickets/elaboration for the assignment operator:

https://bugs.python.org/issue42316 - allow foo[a:=1] instead of foo[(a:=1)]
https://bugs.python.org/issue42374 - allow (c := i for i in b) instead of ((c 
:= i) for i in b)
https://bugs.python.org/issue42381 - allow {i := 0 for i in b} instead of {(i 
:= 0) for i in b}

All of the above were implemented. Of course, allow parallel assignment, which 
was previously disabled, is somewhat a bigger change then allowing 
unparenthesized assignment usage. But from the grammar point of view, they are 
of the same nature (using walrus-aware term at right place). The initial patch 
I have on my hands is:
 
 named_expression[expr_ty]:
-| a=NAME ':=' ~ b=expression { _Py_NamedExpr(CHECK(expr_ty, 
_PyPegen_set_expr_context(p, a, Store)), b, EXTRA) }
+| a=star_targets ':=' ~ b=expression { _Py_NamedExpr(CHECK(expr_ty, 
_PyPegen_set_expr_context(p, a, Store)), b, EXTRA) }


And let's review the 'foo[(a := 1)]' case from more angles. C had assignment as 
expression from the very beginning, but in all fairness, I never saw "foo[a = 
1]" in my whole life (well, maybe in https://www.ioccc.org/ submissions). But 
we see that with Python, people are not too shy to use that. And they even 
submit issues like "hey, I don't want to write foo[(a := 1)], I want to write 
foo[a := 1] !" And they don't get replies like "you know, doing assignment in 
index would hamper readability/maintainability; and those extra parens are 
there exactly to hint you more about this fact". Instead, the case is getting 
acked and fixed.

So, under such circumstances, I don't think that writing "min((a, b) := (b, 
a))" should be considered "much worse" than "foo[a := 1]", and should be kept 
disallowed (as again, fix for both is of the same nature).

--

___
Python tracker 
<https://bugs.python.org/issue43143>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43143] Allow multiple assignment (i.e. tuple on LHS) in walrus operator

2021-02-07 Thread Paul Sokolovsky


Change by Paul Sokolovsky :


--
nosy: +lys.nikolaou

___
Python tracker 
<https://bugs.python.org/issue43143>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42269] Add ability to set __slots__ in dataclasses

2021-02-07 Thread Paul Bryan


Change by Paul Bryan :


--
nosy: +pbryan

___
Python tracker 
<https://bugs.python.org/issue42269>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43143] Allow multiple assignment (i.e. tuple on LHS) in walrus operator

2021-02-06 Thread Paul Sokolovsky


New submission from Paul Sokolovsky :

Currently (CPython 3.10.0a4) having a tuple on left-hand side of assignment 
expression/operator (aka walrus operator) is not allowed:

>>> ((a, b) := (1, 2))
  File "", line 1
((a, b) := (1, 2))
 ^
SyntaxError: cannot use assignment expressions with tuple

As far as can tell, there's no explicit consideration of this case in the 
PEP572. There closest it has is under the 
https://www.python.org/dev/peps/pep-0572/#differences-between-assignment-expressions-and-assignment-statements
 section, "Iterable packing and unpacking (both regular or extended forms) are 
not supported".

But note that the usecase presented above is better seen as "parallel 
assignment", not as "iterable unpacking".

Next issue: PEP572 puts heavy emphasis on the "named expression" usecase. I 
would like to emphasize, that "naming an expression" seems like *just one of 
usecases* for assignment operator. First and foremost assignment operator is, 
well, assignment.

With that in mind, ((a, b) := (1, 2)) is compatible with "naming expressions" 
idea, it "gives names" to multiple expressions at once.

There's a temptation to suggest that the above could be rewritten as:

((a := 1), (b := 2))

, and for the "naming" usecase, that would be true.  But as noted above, 
"naming" is just *one* of the usecases. Following can't be "sequentialized" 
like that:

((b, a) := (a, b))

And for as long as someone considers the following acceptable:

func(a := val)

There's little point to prohibit the following:

min((b, a) := (a, b))


And that's the main argument - consistency between assignment statement and 
assignment operator. For as long as this is allowed:

a, b = b, c

it makes sense to allow:

((a, b) := (b,c))

(Extra parens account for different in grammar and precedence for the operator).


This issue is posted to capture discussion initially posted to 
python-ideas/python-dev:

https://mail.python.org/archives/list/python-id...@python.org/thread/6IFDG7PAFPHVPGULANOQDAHP2X743HCE/
https://mail.python.org/archives/list/python-id...@python.org/thread/X3LNCCO6PHTLAQXHEAP6T3FFW5ZW5GR5/
https://mail.python.org/archives/list/python-...@python.org/thread/6IFDG7PAFPHVPGULANOQDAHP2X743HCE/

, to serve as a reference to whoever may be searching the bugtracker for this 
case.

--
components: Interpreter Core
messages: 386551
nosy: pfalcon
priority: normal
severity: normal
status: open
title: Allow multiple assignment (i.e. tuple on LHS) in walrus operator

___
Python tracker 
<https://bugs.python.org/issue43143>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43132] Incorrect handling of PyObject_RichCompareBool() in the _zoneinfo module

2021-02-05 Thread Paul Ganssle


Paul Ganssle  added the comment:

Re-opening because this was merged without tests.

@ZackerySpytz would you mind adding tests to hit these error cases? I've spent 
some time satisfying myself that it's indeed possible, and I've detailed a 
general outline here: 
https://github.com/python/cpython/pull/24450#issuecomment-774190609

--
nosy: +p-ganssle
resolution: fixed -> 
stage: resolved -> needs patch
status: closed -> open
type:  -> behavior

___
Python tracker 
<https://bugs.python.org/issue43132>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43131] MMAP duplicate reads and writes on ARM64 platform

2021-02-04 Thread Paul Swirhun


Paul Swirhun  added the comment:

A workaround is using memoryview (as reported here: 
https://stackoverflow.com/questions/53492716/python-writing-to-memory-in-a-single-operation).
 This results in only 1 transaction on the bus for some reason.

import os, mmap
fd = os.open("/dev/mem", os.O_RDWR | os.O_SYNC)
mapping = mmap.mmap(fd, 4096, flags=mmap.MAP_SHARED, prot=(mmap.PROT_READ | 
mmap.PROT_WRITE), offset=0x8000)
mv = memoryview(mapping)
mv_as_int32 = mv.cast('I')  # Note "I" works as intended, "L" still results in 
duplicate writes; "LL" is not an option
mv_as_int32[0x0 // 4] = 0xDEADBEEF  # this works; results in 1 write issued on 
the AXI bus

--

___
Python tracker 
<https://bugs.python.org/issue43131>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43131] MMAP duplicate reads and writes on ARM64 platform

2021-02-04 Thread Paul Swirhun


New submission from Paul Swirhun :

mmap (example, as used in python-periphery) duplicates writes, compared to 
devmem and a c-periphery on an ARM64 platform (Zynq/Petalinux/Yocto). There are 
various other reports of duplicated writes using mmap suggesting it is a bug. 
Example:

fd = os.open("/dev/mem", os.O_RDWR | os.O_SYNC)
mapping = mmap.mmap(fd, 4096, flags=mmap.MAP_SHARED, prot=(mmap.PROT_READ | 
mmap.PROT_WRITE), offset=0x8000)
mapping.write(bytearray([0x0,]* 4))  # results in 2 writes, should be 1
mapping.write(bytearray([0x0,]* 8))  # results in 4 writes, should be 1-2 
depending on bus width (32b vs 64b)
# OR:
mapping[0:4] = struct.pack("=L", 0x0)  # results in 2 writes, should be 1

--
components: Library (Lib)
messages: 386501
nosy: psswirhun
priority: normal
severity: normal
status: open
title: MMAP duplicate reads and writes on ARM64 platform
versions: Python 3.7

___
Python tracker 
<https://bugs.python.org/issue43131>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41490] Update bundled pip to 20.2.1 and setuptools to 49.2.1

2021-02-02 Thread Paul Moore


Paul Moore  added the comment:

That's specifically the ensurepip PR, which is now outdated since pip 21.0.1 is 
in master. But for this bug report, Steve said "I still consider the changes a 
release blocking regression, but if it's not blocking ensurepip then I guess 
the updated packages can go in now" so I don't know if he considers there to be 
something else that this issue should still track.

--

___
Python tracker 
<https://bugs.python.org/issue41490>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41490] Update bundled pip to 20.2.1 and setuptools to 49.2.1

2021-02-02 Thread Paul Moore


Paul Moore  added the comment:

It can be closed.

--

___
Python tracker 
<https://bugs.python.org/issue41490>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41490] Update bundled pip to 20.2.1 and setuptools to 49.2.1

2021-02-01 Thread Paul Moore


Paul Moore  added the comment:

I'm not sure why, but the PR to update the bundled pip to 21.0.1 (and 
setuptools to 52.0.0) merged cleanly, so this may be obsolete now.

--

___
Python tracker 
<https://bugs.python.org/issue41490>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43077] Update bundled pip to 21.0.1 and setuptools to 52.0.0

2021-01-30 Thread Paul Moore


Change by Paul Moore :


--
pull_requests: +23203
pull_request: https://github.com/python/cpython/pull/24388

___
Python tracker 
<https://bugs.python.org/issue43077>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43077] Update bundled pip to 21.0.1 and setuptools to 52.0.0

2021-01-30 Thread Paul Moore


Change by Paul Moore :


--
keywords: +patch
pull_requests: +23202
pull_request: https://github.com/python/cpython/pull/24387

___
Python tracker 
<https://bugs.python.org/issue43077>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43077] Update bundled pip to 21.0.1 and setuptools to 52.0.0

2021-01-30 Thread Paul Moore


Paul Moore  added the comment:


New changeset 4d11ecbb5ed78e6259ee27289c7638aad795f473 by Paul Moore in branch 
'master':
bpo-43077: Update bundled pip to 21.0.1 and setuptools to 52.0.0 (GH-24386)
https://github.com/python/cpython/commit/4d11ecbb5ed78e6259ee27289c7638aad795f473


--

___
Python tracker 
<https://bugs.python.org/issue43077>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43077] Update bundled pip to 21.0.1 and setuptools to 52.0.0

2021-01-30 Thread Paul Moore


Change by Paul Moore :


--
assignee:  -> paul.moore
nosy: +Marcus.Smith, dstufft, ncoghlan, pradyunsg
stage:  -> patch review

___
Python tracker 
<https://bugs.python.org/issue43077>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43077] Update bundled pip to 21.0.1 and setuptools to 52.0.0

2021-01-30 Thread Paul Moore


New submission from Paul Moore :

I've just released pip 21.0.1, so this updates the bundled copy (as well as 
updating setuptools).

I assume this should be backported to 3.8 and 3.9.

--
components: Library (Lib)
messages: 385986
nosy: paul.moore
priority: normal
pull_requests: 23201
severity: normal
status: open
title: Update bundled pip to 21.0.1 and setuptools to 52.0.0
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue43077>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42973] argparse: mixing optional and positional arguments... not again

2021-01-27 Thread paul j3


paul j3  added the comment:

Sometimes patches have unforeseen benefits.  My earlier patch for this issue, 
parse_intermixed_args, has been useful beyond the OP's case.

https://stackoverflow.com/questions/50916124/allow-positional-command-line-arguments-with-nargs-to-be-seperated-by-a-flag

https://bugs.python.org/issue15112
argparse: nargs='*' positional argument doesn't accept any items if preceded by 
an option and another positional

With

 usage: test.py [-h] [-a A] b [c]

and 

 test.py B -a A C

has problems because the optional '[c]' positional is used up when 'b' is 
processed.   Intermixed gets around this by first processing '-a', and then 
handling 'b [c]' together.

--

___
Python tracker 
<https://bugs.python.org/issue42973>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42973] argparse: mixing optional and positional arguments... not again

2021-01-27 Thread paul j3


paul j3  added the comment:

So in the big picture, the purpose of this change is to treat the inputs like a 
kind of state-machine.

In the bigger example, the `**` positional values are processed one by one, 
using the interspersed optionals to set state parameters.

`args.sources` then ends up as a list of dicts, each of the form:

{'name': 'input3.txt', 'frobnicate': None, 'massage_level': 5}

where 'name' is the positional value, and 'frobnicate' and 'massage_level' are 
the latest values (default or user supplied).  The optionals can be specified 
in any order or repeatedly.

While the proposed change to the core parser is (apparently) minor, it does 
occur at the center of the action.  That is not the place we want any (new) 
bugs or backward incompatibility.  And the full implementation requires a new 
Action subclass that is quite different from existing ones.

Given how different this is from the normal argparse parsing (and the POSIX 
parsing argparse seeks to emulate), I question the wisdom of adding this, in 
part or whole, to the stock distribution.  It could certainly be published as a 
pypi.  That already has a number of  parsers, some built on argparse, others 
stand alone.

I also wonder whether it would be simpler to do this kind of parsing directly 
from sys.argv.  Just step through that list, consuming the values and flags in 
sequence.  

Sorry to sound like a wet blanket, but I've seen too many seemingly innocent 
patches that caused unforeseen problems down the line.

--

___
Python tracker 
<https://bugs.python.org/issue42973>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42973] argparse: mixing optional and positional arguments... not again

2021-01-26 Thread paul j3


paul j3  added the comment:

I haven't had a chance to study your longer posts, but it seems to me that the 
AddFruitAction example could just as well be implemented with 

parser.add_argument('--color', nargs='*', action='append')

with post parsing processing to create the 'fruits' dicts from the appended 
lists.  

The basic logic of argparse is to accept optionals in any order, and 
positionals in strict positional order.  'nargs' allows us to pair any number 
of strings with each optional's flag.

While custom Action classes can implement interactions between arguments based 
on values in the namespace, it is usually easier to do this after parsing.  

But back to your core change, I wonder if adding a new nargs, such as '**' 
would be better than than the new parameter 'greedy_star=True'.  I prefer not 
to add parameters that are hard to document.  At this point in the argparse 
development, changes should be minimally invasive.

--

___
Python tracker 
<https://bugs.python.org/issue42973>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43022] Unable to dynamically load functions from python3.dll

2021-01-25 Thread Paul Moore


Paul Moore  added the comment:

> PathAllocCombine() is Windows 8+, so sample code that uses it would only be 
> for Python 3.9+.

Yeah, I'm probably going to remove that. I mainly used it because I'm *so* 
spoiled by Python, writing code in C where I have to actually implement stuff 
for myself rather than just using a stdlib function, just feels so tiresome 
these days :-)

Thanks for the other suggestions.

--

___
Python tracker 
<https://bugs.python.org/issue43022>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43022] Unable to dynamically load functions from python3.dll

2021-01-25 Thread Paul Moore


Paul Moore  added the comment:

> I think here you're in a very small minority who could get away with this, 
> and so I'd hesitate to make it sound like the recommended approach.

Well, the evidence here is that maybe even I shouldn't be doing this :-)

> What I'd actually recommend (on Windows) is to bundle a copy of Python with 
> your application.

That's actually precisely what I'm doing. But I don't exactly have an 
"application", in the sense that I suspect you mean, which is the difficulty.

I'm trying to address the problem that it's a real pain to ship Python scripts 
as "utilities" on a Windows system. You want such scripts to still be plain 
text, because you typically hack on them a lot. You can't rely on shebangs and 
the launcher, because I continually find that PATHEXT doesn't include ".py" so 
they don't run properly, and "py name_of_script.py" doesn't do path searches. 
And anyway, I don't want to run the scripts with my system Python, because I 
don't want to dump all my dependencies in there.

So what I have is a small stub, that looks for a .py file of the same name, 
alongside the executable (so myutil.exe looks for myutil.py). Rather than using 
the system Python, it looks for a copy of Python in a subdirectory next to the 
script, and uses that. I can then install dependencies in the dedicated 
interpreter.

(And yes, PEP 582 would help make things easier in this area, too, but it never 
gained enough traction.)

I use the embedded distribution as that interpreter. I may be able to use a 
virtualenv instead, but I've not tried that yet.

Multiple copies of the launcher, one per script, and you're done :-)

To be honest, it really sucks that this is the most reliable way of managing 
small utility scripts in Python on Windows. But every other solution I've tried 
has had its own issues. I'd *much* prefer something standard, but I don't know 
how to bundle this in a way that makes it straightforward for "ordinary" users, 
so I've decided just to solve my own problem and leave it at that :-)

--

___
Python tracker 
<https://bugs.python.org/issue43022>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue33129] Add kwarg-only option to dataclass

2021-01-25 Thread Paul Bryan


Change by Paul Bryan :


--
nosy: +pbryan

___
Python tracker 
<https://bugs.python.org/issue33129>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43022] Unable to dynamically load functions from python3.dll

2021-01-25 Thread Paul Moore


Paul Moore  added the comment:

Confirmed. The following code works as I want:

Py_Main_t get_pymain(wchar_t *base_dir) {
wchar_t *dll_path;
HRESULT hr = PathAllocCombine(
base_dir, L"python\\python3.dll",
PATHCCH_ALLOW_LONG_PATHS, _path
);
if (hr != S_OK) {
error(L"Could not construct Python DLL path");
}

HMODULE py_dll = LoadLibraryExW(dll_path, 0, 
LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR | LOAD_LIBRARY_SEARCH_DEFAULT_DIRS);
if (!py_dll) {
error(L"Could not load Python DLL %ls", dll_path);
}
LocalFree(dll_path);

Py_Main_t py_main = (Py_Main_t)GetProcAddress(py_dll, "Py_Main");
if (!py_main) {
error(L"Could not locate Py_Main function");
}

return py_main;
}


If people think it's worthwhile, I can put together a change to 
https://docs.python.org/3/extending/embedding.html#embedding-python-in-another-application
 (maybe a new section, "Embedding on Windows by dynamically loading the stable 
ABI"?) that uses a code snippet like this.

--

___
Python tracker 
<https://bugs.python.org/issue43022>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43022] Unable to dynamically load functions from python3.dll

2021-01-25 Thread Paul Moore


Paul Moore  added the comment:

Thanks, I'll give that a try. It sounds like I'm just using the APIs 
incorrectly, which would be good (in the sense that I can do what I wanted, I 
just didn't know how ;-))

I wonder whether it would be worth having a section in the docs somewhere 
explaining how to do this, or more generally what the "best practices" are for 
embedding? If I were to try to put something together, would that be worthwhile?

> I noted that the path to "python3.dll" must be fully qualified.

Just to note, my actual code does use an absolute path, I was over-simplifying 
here. But it's worth being clear on it, particularly if I do write something 
for the docs.

--

___
Python tracker 
<https://bugs.python.org/issue43022>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43022] Unable to dynamically load functions from python3.dll

2021-01-25 Thread Paul Moore


Paul Moore  added the comment:

Thinking about what you said, "the loader waits to load it until first accessed 
via GetProcAddress()" did you mean by that, that the following code should work:

h = LoadLibraryW(L"some/path/to/python3.dll")
py_main = GetProcAddress(h, "Py_Main")

(with some/path/to/python39.dll being loaded on the second line)? Because if 
so, then that's what doesn't work for me.

Or are you suggesting that I do

AddDllDirectoryW(L"some/path/to");
h = LoadLibraryW(L"python3.dll");
py_main = GetProcAddress(h, "Py_Main");


Because I didn't think to try that - and I'm not 100% sure how I'd have to do 
stuff like LOAD_LIBRARY_SEARCH_USER_DIRS and/or SetDefaultDllDirectories to 
make that work. I find the Microsoft docs on all of this really complex and 
hard to follow if you're a non-expert :-(

--

___
Python tracker 
<https://bugs.python.org/issue43022>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43022] Unable to dynamically load functions from python3.dll

2021-01-25 Thread Paul Moore


Paul Moore  added the comment:

So I need to dynamically load *both* python3.dll and python39.dll, but if I do 
that I can get the functions from python3.dll? What's the point in doing that? 
Surely I might as well just load python39.dll and get the functions from there, 
in that case.

I hoped that by using python3.dll, I'd be able to avoid needing to deal with 
the version-specific DLL at all, so making my code portable to any version of 
Python 3. I'm pretty sure that's how it works if I statically link to 
python3.dll...

--

___
Python tracker 
<https://bugs.python.org/issue43022>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43022] Unable to dynamically load functions from python3.dll

2021-01-25 Thread Paul Moore


New submission from Paul Moore :

I am writing a small application using the embedded distribution to run a 
script supplied by the user. The requirements are very simple, so all I need to 
do is set up argv and call Py_Main.

I'm trying to load the Py_Main function dynamically (for flexibility - see 
below) but GetProcAddress returns 0 when loading the function from python3.dll 
(the stable ABI). This seems to be because the symbols in python3.dll are 
special "fowarding" symbols, that GetProcAddress can't handle.

Is there a way to dynamically load the Python API from the stable ABI? If there 
isn't currently, could one be added?

To explain my requirements in a bit more detail, I don't want to statically 
link, because I want to put the Python distribution in a subdirectory, so that 
it isn't visible on PATH along with my executable, but I'd rather avoid the 
complexities involved in adding a dedicated SxS manifest to point the loader to 
the correct subdirectory for the python3.dll.

Furthermore, I want to provide a graceful fallback if the Python distribution 
is missing (initially, just a friendly error message, but in future maybe 
locating an alternative Python installation to use) and static linking won't 
allow that as I can't recover control if the expected Python DLL isn't present.

The reason I want to use the stable ABI is so that I can upgrade the embedded 
distribution without needing to rebuild the C code. I could search for 
python3X.dll, and just ignore the stable ABI. But that's more filesystem code 
than I really want to write in C... And honestly, I feel that dynamically 
linking is a perfect use case for the stable ABI, so it really should be 
supported.

--
components: Windows
messages: 385621
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Unable to dynamically load functions from python3.dll
type: behavior
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue43022>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42966] argparse: customizable help formatter

2021-01-22 Thread paul j3


paul j3  added the comment:

Years ago I proposed a `format_wrapper`, a Format class factory

https://bugs.python.org/issue12806#msg218395

also

https://bugs.python.org/issue39809

There I show that the formatter may be lambda

formatter = lambda prog: argparse.HelpFormatter(prog, width=100)

The need for improving help customization has been around for a long time:

https://bugs.python.org/issue11695
Improve argparse usage/help customization

https://bugs.python.org/issue32123
Make the API of argparse.HelpFormatter public

--

___
Python tracker 
<https://bugs.python.org/issue42966>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42875] argparse incorrectly shows help string on a new line in case of long command string

2021-01-22 Thread paul j3


paul j3  added the comment:

This issue is discussed in:

https://bugs.python.org/issue34724
argparse subparser help indent too short

and

https://stackoverflow.com/questions/3215/max-help-position-is-not-works-in-python-argparse-library

--

___
Python tracker 
<https://bugs.python.org/issue42875>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42980] argparse: GNU-style help formatter

2021-01-22 Thread paul j3


paul j3  added the comment:

I was thinking of a refactoring that included the ', '.join(...) loop, but on 
further thought your refactoring might be enough for the case I raised.

For example:

def _format_option_with_args(self, option_string, args_string):
if option_string.startswith('--'):
return '%s %s' % (option_string, args_string)
return '%s' % option_string

would produce a

 -f, --foo FOOsomething

and 

def _format_option_with_args(self, option_string, args_string):
return option_string
   
would produce

 -f, --foo   something

I wonder if this refactoring merits some mention in the documentation.  Or 
maybe the GnuStyleLongOptionsHelpFormatter subclass is enough of an example.  
The details of how other formatter subclasses work aren't documented, but they 
are available to developers who can read the code.

--

___
Python tracker 
<https://bugs.python.org/issue42980>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42980] argparse: GNU-style help formatter

2021-01-21 Thread paul j3


paul j3  added the comment:

The refactoring looks reasonable.

But while we are tweaking:

def _format_action_invocation(self, action):

I wonder if we also give users more control over how multiple option strings 
are formatted.  Currently if 

 parser.add_argument('-f', '--foo', help='something')

the help line will be

 -f FOO, --foo FOO something

with this alternate formatter it would be (I think)

 -f FOO, --foo=FOO something

But with longer option strings users often want

 -f, --foo FOO something

or even just

 -f, --foo something

we can almost get the last with `metavar=''`

 -f , --foosomething

which has a superfluous space before the comma.

I don't recall if there's already a bug/issue for this or not.  Maybe the fix 
is a subclass with a complete replacement of this _format_action_invocation 
method.  I'll have do more research (here and on Stackoverflow).

--

___
Python tracker 
<https://bugs.python.org/issue42980>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42973] argparse: mixing optional and positional arguments... not again

2021-01-20 Thread paul j3


paul j3  added the comment:

Your patch is incomplete, without documentation or tests.

Your example is incomplete.  That is _CustomAction?  What namespace does your 
patch produce.

It's been a while since I worked on the intermixed patch, but as I recall the 
OP was happy with the fix.  Also I don't recall any talk about 'matching 
"optionals" with "positionals" with they relate to. Was that part of the 
discussion?

On Stackoverflow I have seen questions about pairing arguments, and answered 
some.  I don't recall the best answers.  The tough version is when some of the 
grouped inputs may be missing 

In your example I'd suggest making '--tracks' a nargs=2 and 'append' option.  
'choices' would have to be replaced with a either a custom 'type', a custom 
Action, or post-parsing testing.

I haven't had time yet to test your patch.

One question I frequently ask posters who want to parse complex inputs is: 'how 
are going to explain this to your users?'  'What kind of usage and help do you 
want see?'

--

___
Python tracker 
<https://bugs.python.org/issue42973>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42874] running configure on Solaris 10 gives grep "illegal option" errors

2021-01-12 Thread Paul Ganssle


Paul Ganssle  added the comment:

This particular grep statement is used to validate the `tzpath` variable. 
Apparently it is easy enough to achieve what I was going for using vanilla grep 
with no options, so I've created GH-24200 to fix the issue.

I notice that there are other uses of `-q` and `-E` in the configure file, but 
presumably those are on more optional paths.

@martin.wheatley.home: Can you check to see if GH-24200 fixes your issue?

Also, can you clarify whether this happens with a plain `./configure` 
invocation, or are you specifying `./configure 
--with-tzpath='/usr/share/zoneinfo:/usr/lib/zoneinfo:/usr/share/lib/zoneinfo:/etc/zoneinfo'`
 yourself?

--
versions: +Python 3.10

___
Python tracker 
<https://bugs.python.org/issue42874>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42874] running configure on Solaris 10 gives grep "illegal option" errors

2021-01-12 Thread Paul Ganssle


Change by Paul Ganssle :


--
keywords: +patch
pull_requests: +23025
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24200

___
Python tracker 
<https://bugs.python.org/issue42874>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42902] a python embedded program may load "C:\Lib\os.py" on windows system

2021-01-12 Thread Paul Moore


Paul Moore  added the comment:

I thought that *all* versions of Python located the standard library by 
searching for os.py...

--

___
Python tracker 
<https://bugs.python.org/issue42902>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42904] get_type_hints does not provide localns for classes

2021-01-11 Thread Paul Bryan


Change by Paul Bryan :


--
nosy: +larry

___
Python tracker 
<https://bugs.python.org/issue42904>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42904] get_type_hints does not provide localns for classes

2021-01-11 Thread Paul Bryan


New submission from Paul Bryan :

According to PEP 563:

> The get_type_hints() function automatically resolves the correct value of 
> globalns for functions and classes. It also automatically provides the 
> correct localns for classes.

This statement about providing correct localns for classes does not appear to 
be true.

Guido suggested this should be treated as a bug.

--
components: Library (Lib)
messages: 384885
nosy: gvanrossum, pbryan
priority: normal
severity: normal
status: open
title: get_type_hints does not provide localns for classes
type: behavior
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue42904>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42863] Python venv inconsistent folder structure on windows

2021-01-08 Thread Paul Moore


Paul Moore  added the comment:

Many tools hard code the "if windows then 'Scripts' else 'bin'" logic. They 
will simply fail to work with environments that don't have that layout. Yes, 
they probably should use sysconfig, but many don't (and sometimes for arguably 
good reasons, for example tools run using one Python interpreter that 
manipulate the environment of a different one).

It's something that *could* be changed, but the cost would be significant, the 
work needed to manage the transition is non-trivial, and the benefit isn't 
particularly obvious - "consistency" is never very compelling in isolation, and 
"works better with meson" is likely to be met with the argument that Python's 
layout has been around longer than meson, so it's on meson to accommodate 
Python, not the other way around.

I'm not actively against the idea, but neither do I particularly support it 
(I'm pretty sure it would require work for pip, for no benefit to pip's 
existing users, so it's a net loss for me personally).

If you're interested in pushing for this change go for it - I suspect that it 
would probably need a PEP, and at the very least it would need a community 
discussion - but be prepared for it to meet with a certain amount of resistance.

--

___
Python tracker 
<https://bugs.python.org/issue42863>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue9694] argparse required arguments displayed under "optional arguments"

2021-01-08 Thread paul j3


paul j3  added the comment:

Since this issue is closed it might be a good idea to open a new one with this 
problem.  And if possible identify the failed tests.  

We forgot to allow for the fact that working code/tests might be checking for 
specific help messages, checks the will fail when this group label is changed.

--

___
Python tracker 
<https://bugs.python.org/issue9694>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42855] pathlib.exists on Windows raises an exception on URL like/bad input

2021-01-07 Thread Paul Moore


Paul Moore  added the comment:

So I guess the key question then is whether Path.exists() should trap 
exceptions and interpret them as "does not exist" (on all platforms, although 
it looks like the null character case in Unix has now been fixed). Which 
doesn't seem unreasonable, I guess.

--

___
Python tracker 
<https://bugs.python.org/issue42855>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42855] pathlib.exists on Windows raises an exception on URL like/bad input

2021-01-07 Thread Paul Moore


Paul Moore  added the comment:

"http:" isn't a valid drive letter, I'd imagine.

--

___
Python tracker 
<https://bugs.python.org/issue42855>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42855] pathlib.exists on Windows raises an exception on URL like/bad input

2021-01-07 Thread Paul Moore

Paul Moore  added the comment:

It's an invalid filename so it raises an exception.

You can get the same on Unix by using an invalid filename (embedded null):

>>> from pathlib import Path
>>> Path("/usr/\0").exists()
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib64/python3.7/pathlib.py", line 1356, in exists
    self.stat()
  File "/usr/lib64/python3.7/pathlib.py", line 1178, in stat
    return self._accessor.stat(self)
ValueError: embedded null byte

You need to be prepared for exceptions if you aren't sure you have a valid 
path. One thing that might be useful, I guess, is a `Path.is_valid()` function. 
But I don't know if all platforms have a way of asking the OS "is this a valid 
pathname?" So catching the exception is probably best.

--

___
Python tracker 
<https://bugs.python.org/issue42855>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue2142] difflib.unified_diff(...) produces invalid patches

2021-01-01 Thread Paul "TBBle" Hampson


Paul "TBBle" Hampson  added the comment:

I just bounced off this issue and proposed a work-around for it in Black 
(https://github.com/psf/black/pull/1897).

Since it wasn't mentioned here earlier, the `\` marker _is_ documented in the 
GNU Diffutils documentation, under "Incomplete Lines" 
(https://www.gnu.org/software/diffutils/manual/html_node/Incomplete-Lines.html#Incomplete-Lines),
 and applies to both Context and Normal diff output formats.

--
nosy: +TBBle

___
Python tracker 
<https://bugs.python.org/issue2142>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42798] pip search fails

2020-12-31 Thread Paul Watson


New submission from Paul Watson :

Fresh install of 3.9.1
Created venv
Activated venv

(py3.9) 13:12:59.52  C:\venv\py3.9\Scripts
C:>pip search astronomy
ERROR: Exception:
Traceback (most recent call last):
  File "c:\venv\py3.9\lib\site-packages\pip\_internal\cli\base_command.py", 
line 228, in _main
status = self.run(options, args)
  File "c:\venv\py3.9\lib\site-packages\pip\_internal\commands\search.py", line 
60, in run
pypi_hits = self.search(query, options)
  File "c:\venv\py3.9\lib\site-packages\pip\_internal\commands\search.py", line 
80, in search
hits = pypi.search({'name': query, 'summary': query}, 'or')
  File 
"C:\Users\mike\AppData\Local\Programs\Python\Python39\lib\xmlrpc\client.py", 
line 1116, in __call__
return self.__send(self.__name, args)
  File 
"C:\Users\mike\AppData\Local\Programs\Python\Python39\lib\xmlrpc\client.py", 
line 1456, in __request
response = self.__transport.request(
  File "c:\venv\py3.9\lib\site-packages\pip\_internal\network\xmlrpc.py", line 
45, in request
return self.parse_response(response.raw)
  File 
"C:\Users\mike\AppData\Local\Programs\Python\Python39\lib\xmlrpc\client.py", 
line 1348, in parse_response
return u.close()
  File 
"C:\Users\mike\AppData\Local\Programs\Python\Python39\lib\xmlrpc\client.py", 
line 662, in close
raise Fault(**self._stack[0])
xmlrpc.client.Fault: https://status.python.org/ for more information.">

(py3.9) 13:13:08.09  C:\venv\py3.9\Scripts
C:>python --version
Python 3.9.1

--
components: Demos and Tools
messages: 384133
nosy: Paul Watson
priority: normal
severity: normal
status: open
title: pip search fails
type: crash
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue42798>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42729] tokenize, ast: No direct way to parse tokens into AST, a gap in the language processing pipiline

2020-12-24 Thread Paul Sokolovsky


Paul Sokolovsky  added the comment:

> the idea was proposed purely to "close a gap"

That pinpoints it well. I was just writing a tutorial on implementing custom 
import hooks, with the idea to show people how easy it to do it in Python. As 
first step, I explained that it's bad idea to do any transformations on surface 
representation of a program. At the very least, it should be converted to token 
stream. But then I found that I need to explain that we need to convert it 
back, which sounds pretty weird and undermines the idea:

def xform(token_stream):
for t in token_stream:
if t[0] == tokenize.NAME and t[1] == "function":
yield (tokenize.NAME, "lambda") + t[2:]
else:
yield t

with open(filename, "rb") as f:
# Fairly speaking, tokenizing just to convert back to string form
# isn't too efficient, but CPython doesn't offer us a way to parse
# token stream so far, so we have no choice.
source = tokenize.untokenize(xform(tokenize.tokenize(f.readline)))
mod = type(imphook)("")
exec(source, vars(mod))
return mod

Having written that comment, I thought I could as well just make one more step 
and monkey-patch "ast" for parse_tokens() function - I'll need to explain that, 
but the explanation probably wouldn't sound worse than the explanation above. 
And then it was just one more step to actually submit patch for 
ast.parse_tokens(), and that's how this ticket was created!

--

___
Python tracker 
<https://bugs.python.org/issue42729>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42729] tokenize, ast: No direct way to parse tokens into AST, a gap in the language processing pipiline

2020-12-24 Thread Paul Sokolovsky


Paul Sokolovsky  added the comment:

> There is a considerable tension on exposed parts of the compiler pipeline for 
> introspection and other capabilities and our ability to do optimizations. 
> Given how painful it has been in the past to deal with this, my view is to 
> avoid exposing as much as possible anything in the compiler pipeline, so we 
> don't shoot ourselves in the foot in the future if we need to change stuff 
> around.

That's somewhat extreme outcome when a problem is known and understood, but the 
best solution is deemed not doing anything.

But the problem of "doing it wrong" definitely known and exists. One known way 
to address it is to design generic interfaces and implement them. This ticket 
is exactly about that - defining a missing interface for a parser in Python. 
It's not about *the* CPython's C parser and its peculiarities. (But even it 
fits with the generic interface proposed.)

--

___
Python tracker 
<https://bugs.python.org/issue42729>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42729] tokenize, ast: No direct way to parse tokens into AST, a gap in the language processing pipiline

2020-12-24 Thread Paul Sokolovsky


Paul Sokolovsky  added the comment:

> but the thing I don't quite get is the use case.

And if that went unanswered: the usecase, how I'd formulate it, is to not 
expose CPython historical implementation detail of "tokenize" being 
disconnected from the rest of the language processing infrastructure, and make 
them learn tricks like needing to go back to character program form if they 
ever start to use "tokenize", but let it all integrate well into single 
processing pipeline.

--

___
Python tracker 
<https://bugs.python.org/issue42729>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42729] tokenize, ast: No direct way to parse tokens into AST, a gap in the language processing pipiline

2020-12-24 Thread Paul Sokolovsky


Paul Sokolovsky  added the comment:

> What prevents you from using ast.parse(tokenize.untokenize(token_stream))?

That's exactly the implementation in the patch now submitted against this 
issue. But that's the patch for CPython, the motive of the proposal here is to 
establish a standard API call for *Python*, which different implementation can 
implement how they like/can/need.

> Also, tokens -> AST is not the only disconnected part in the underlying 
> compiler.

We should address them, one by one.

> Stuff like AST -> Symbol Table 

Kinda yes, again, based on CPython implementation history, we have only source 
-> Symbol table (https://docs.python.org/3/library/symtable.html). Would be 
nice to address that (separately of course).

> AST -> Optimized AST

Yes. PEP511 touched on that, but as it-as-a-whole was rejected, any useful 
sub-ideas from it don't seem to get further progress either (like, being able 
to disable some optimizations, and then maybe even exposing them as standalone 
passes).

> I'd also suggest moving the discussion to the Python-ideas, for a much 
> greater audience.

That's how I usually do, but I posted too much there recently. I wanted to 
submit a patch right away, but noticed that standard commit message format is 
"bpo-X: ...", so I created a ticket here to reference in the commit.

--

___
Python tracker 
<https://bugs.python.org/issue42729>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42729] tokenize, ast: No direct way to parse tokens into AST, a gap in the language processing pipiline

2020-12-24 Thread Paul Sokolovsky


Change by Paul Sokolovsky :


--
keywords: +patch
pull_requests: +22773
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/23922

___
Python tracker 
<https://bugs.python.org/issue42729>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42729] tokenize, ast: No direct way to parse tokens into AST, a gap in the language processing pipiline

2020-12-24 Thread Paul Sokolovsky


New submission from Paul Sokolovsky :

Currently, it's possible:

* To get from stream-of-characters program representation to AST representation 
(AST.parse()).
* To get from AST to code object (compile()).
* To get from a code object to first-class function to the execute the program.

Python also offers "tokenize" module, but it stands as a disconnected island: 
the only things it allows to do is to get from stream-of-characters program 
representation to stream-of-tokens, and back. At the same time, conceptually, 
tokenization is not a disconnected feature, it's the first stage of language 
processing pipeline. The fact that "tokenize" is disconnected from the rest of 
the pipeline, as listed above, is more an artifact of CPython implementation: 
both "ast" module and compile() module are backed by the underlying bytecode 
compiler implementation written in C, and that's what connects them.

On the other hand, "tokenize" module is pure-Python, while the underlying 
compiler has its own tokenizer implementation (not exposed). That's the likely 
reason of such disconnection between "tokenize" and the rest of the 
infrastructure.

I propose to close that gap, and establish an API which would allow to parse 
token stream (iterable) into an AST. An initial implementation for CPython can 
(and likely should) be naive, making a loop thru surface program 
representation. That's ok, again, the idea is to establish a standard API to be 
able to go tokens -> AST, then individual Python implementation can 
make/optimize it based on their needs.

The proposed name is ast.parse_tokens(). It follows the signature of the 
existing ast.parse(), except that first parameter is "token_stream" instead of 
"source".

Another alternative would be to overload existing ast.parse() to accept token 
iterable. I guess, at the current stage, where we try to tighten up type 
strictness of API, and have clear typing signatures for API functions, this is 
not favored solution.

--
components: Library (Lib)
messages: 383680
nosy: BTaskaya, pablogsal, pfalcon, serhiy.storchaka
priority: normal
severity: normal
status: open
title: tokenize, ast: No direct way to parse tokens into AST, a gap in the 
language processing pipiline
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue42729>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42697] 3.8.7rc1 regression: 'free(): invalid pointer' after running backports-zoneinfo test suite

2020-12-21 Thread Paul Ganssle

Paul Ganssle  added the comment:

For future reference, this bug is triggered only when `.fromutc` is called on a 
subclass of `datetime` and the resulting date is the second ambiguous time 
(e.g. if there's a DST transition from 02:00 → 01:00, and the result of the 
`.fromutc` call is the *second* 01:30, you will trigger this issue).

Here's an MWE:

```
from backports.zoneinfo import ZoneInfo
from datetime import datetime, timezone

class SubDT(datetime):
pass

LON = ZoneInfo("Europe/London")
d = SubDT(2020, 10, 25, 1, 30, tzinfo=timezone.utc)

# Each pass through the loop inappropriately reduces the reference count on the
# `1` object by 1. Since there are usually a large number of live references to
# `1`, this won't have any immediate noticeable effect unless you do it a lot.
for i in range(1):
d.astimezone(LON)
```

--

___
Python tracker 
<https://bugs.python.org/issue42697>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42650] Can people use dest=argparse.SUPPRESS in custom Action classes?

2020-12-20 Thread paul j3


paul j3  added the comment:

I'd have to study the code (and docs), but I'm not sure setting the `dest` to 
'SUPPRESS' does anything meaningful.

default=argparse.SUPPRESS

is useful, keeping the default out of the namespace.  That argument appears 
only if the user has used the option.

But with `dest` (in a default 'store'), I get a namespace like

Namespace(**{'==SUPPRESS==': 'foo'})

'help' and 'version' exit right after displaying their message, so I'm no sure 
the 'SUPPRESS' is doing anything.  The parser doesn't return a namespace.

It appears that the 'dest' is passed to the Action.  A custom Action could act 
on that 'dest'.  The default 'store' just uses it as the attribute for storing 
the value in the namespace.

Anyways, this is not the kind of thing that we'll be tweaking in the source.  I 
don't recall any bug/issue touching on this behavior (but we could do a search).

--

___
Python tracker 
<https://bugs.python.org/issue42650>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37717] argparse subcommand docs has non-existent parameter "action"

2020-12-19 Thread paul j3


paul j3  added the comment:

I'm closing the is as a duplicate of https://bugs.python.org/issue23487, which 
is already closed.

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue37717>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42683] asyncio should handle keyboard interrupt while the event loop is running

2020-12-19 Thread Paul Moore


Change by Paul Moore :


--
nosy: +njs

___
Python tracker 
<https://bugs.python.org/issue42683>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42683] asyncio should handle keyboard interrupt while the event loop is running

2020-12-19 Thread Paul Moore


New submission from Paul Moore :

See the comment on Discourse here: 
https://discuss.python.org/t/feeding-data-generated-via-asyncio-into-a-synchronous-main-loop/5436/28
 (and the thread leading up to this comment).

In the thread, @njs states that if the user hits Ctrl-C while the asyncio event 
loop is running, it's possible for internal asyncio data structures to end up 
in an inconsistent state. If that's the case, then this would make 
asyncio-based code unreliable in real-world use.

I don't have a way to reproduce this - from the Discourse thread, I had assumed 
that ctrl-C was safe to use on an asyncio-based program, but was told 
otherwise, and I can't find anything definitive either way.

At a minimum, the asyncio documentation should confirm that it is 
exception-safe (specifically against Ctrl-C, but in general I'd assume that 
asyncio is safe in the face of uncaught exceptions in user-written async code).

--
components: asyncio
messages: 383370
nosy: asvetlov, paul.moore, yselivanov
priority: normal
severity: normal
status: open
title: asyncio should handle keyboard interrupt while the event loop is running
type: behavior
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue42683>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42677] Support comments in argparse fromfile_prefix_chars files

2020-12-18 Thread paul j3


paul j3  added the comment:

https://docs.python.org/3/library/argparse.html#customizing-file-parsing

in the docs describes how to customize the @file reading.  This particular 
extension handles several strings on a line, but that's not the limit of what 
you could do.

You can also read a file before parsing and construct a custom `argv` list, as 
described in https://docs.python.org/3/library/argparse.html#beyond-sys-argv

--

___
Python tracker 
<https://bugs.python.org/issue42677>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42663] zoneinfo does not support full range of allowed transition hours in fallback string

2020-12-16 Thread Paul Ganssle

New submission from Paul Ganssle :

TZif files consist of a list of transitions followed by a POSIX TZ var-style 
string of a form like "AAA3BBB,M3.2.0/01:30,M11.1.0/02:15:45", which decodes to 
"AAA (UTC-3) is the standard time and BBB (UTC-2) is the DST time, DST applies 
starting at 02:15:45 local time on the 1st Sunday in November and ending at 
1:30:00 local time on the 2nd Sunday in March". After the last listed 
transition, the rule specified by the TZ variable applies.

POSIX says that the "hours" part of the transition rule must be in the range 
±24, but as mentioned in a TODO comment in _zoneinfo.c, RFC 8536 §3.3.1 
specifies that the hours part of transition times may range from -167 to 167 
(see: 
https://github.com/python/cpython/blob/master/Modules/_zoneinfo.c#L1844-L1847 ).

Currently, zoneinfo does not support the full range of possible transitions, 
and a TZif file with a 3-digit transition hour would likely fail to parse. This 
isn't a terribly high priority at the moment, but if the tz project ever 
releases a TZif file with one of these TZ strings on it, it will all of a 
sudden become very critical to fix it, so we should probably try to get it 
fixed before Python 3.9 is EOL, so that all versions of Python with `zoneinfo` 
can handle this properly.

--
components: Library (Lib)
messages: 383206
nosy: p-ganssle
priority: normal
severity: normal
stage: needs patch
status: open
title: zoneinfo does not support full range of allowed transition hours in 
fallback string
type: behavior
versions: Python 3.10, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue42663>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42662] Propose: Data model explict about __annotations__ key ordering.

2020-12-16 Thread Paul Bryan


Paul Bryan  added the comment:

Retracting.

--
resolution:  -> not a bug
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue42662>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42662] Propose: Data model explict about __annotations__ key ordering.

2020-12-16 Thread Paul Bryan


Change by Paul Bryan :


--
keywords: +patch
pull_requests: +22668
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/23808

___
Python tracker 
<https://bugs.python.org/issue42662>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42662] Propose: Data model explict about __annotations__ key ordering.

2020-12-16 Thread Paul Bryan


New submission from Paul Bryan :

Currently the data model documentation does not specify the order of keys in 
__annotations__ dictionary. It is currently in the order that arguments or 
attributes are declared. I propose to make this explicit.

Rationale: Having order explicitly specified in the documentation makes it a 
documented feature that code can depend on. Current code cannot safely rely on 
this behavior.

--
assignee: docs@python
components: Documentation
messages: 383204
nosy: docs@python, pbryan
priority: normal
severity: normal
status: open
title: Propose: Data model explict about __annotations__ key ordering.
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue42662>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42660] _zoneinfo.c incorrectly checks bounds of `day` variable in calenderrule_new

2020-12-16 Thread Paul Ganssle


Paul Ganssle  added the comment:

> Adding a static assertion about the signedness of uint8_t looks meaningless 
> to me.

My proposal is to add a static assertion that `day` is an unsigned type. In the 
code it would look something like it does in the backports.zoneinfo code 
(https://github.com/pganssle/zoneinfo/blob/07ec80ad5dc7e7e4b4f861ddbb61a9b71e9f27c7/lib/zoneinfo_module.c#L1247-L1255):


```
// The actual bounds of day are (day >= 0 && day <= 6), but since day is an
// unsigned variable, day >= 0 is always true. To ensure that a bug is not
// introduced in the event that someone changes day to a signed type, we
// will assert that it is an unsigned type.
Py_ASSERT_VAR_UNSIGNED(day);
if (day > 6) {
PyErr_Format(PyExc_ValueError, "Day must be in [0, 6]");
return -1;
}
```

> I propose to change types of function parameters and local variables. In any 
> case the constructor checks the range of parameters, so there is no problem 
> with packing them into compact structure.

> This will help to avoid errors of implicit conversion between different 
> integer types. Also it can help to avoid code duplication in parsing integers 
> of different size and signedness.

This is not entirely unreasonable in my opinion, though in this case everything 
is determined by the POSIX standard and an RFC, so it is unlikely that we'll 
ever see anything outside of 8 bit integers for these things. If you'd like to 
refactor the parsing code to use signed integers unconditionally and have them 
converted as part of the constructor that seems reasonable.

--

___
Python tracker 
<https://bugs.python.org/issue42660>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42660] _zoneinfo.c incorrectly checks bounds of `day` variable in calenderrule_new

2020-12-16 Thread Paul Ganssle

Paul Ganssle  added the comment:

There are at least two issues with this:

1. This is a constructor for a struct, and the struct would really 
unnecessarily balloon in size if we switched everything over to be 32-bit 
integers (which I think is your suggestion?). This is not a major concern 
because in typical operation there are not likely to be more than 500 of these 
structs in existence at any one time (due to the cache), but it's still a minor 
annoyance.

I would guess that accepting `int` in the constructor function and converting 
it to uint8_t as part of building the struct would be maybe neutral to 
negative, since it involves casting a large signed type to a small unsigned 
one. This could cause more problems with overflow, not less.

2. In the current implementation `day` is unsigned by its nature — it 
represents a value indexed at 0 for which negative indices have no meaning. If 
we leave `day` as a uint8_t, then that tells the compiler at least something 
about the valid range of the value. By turning `day` (and presumably the other 
elements of this struct) into a less-suitable type, we're depriving ourselves 
of possibly *useful* compiler warnings.

Barring a solution where we have a simple and robust way to suppress warnings, 
I think the next-best solution is to add a static assertion about the 
signedness of the type for this particular case. I'd be happy to write it in a 
relatively general way so that it can be re-used elsewhere in CPython for the 
same purpose.

--

___
Python tracker 
<https://bugs.python.org/issue42660>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42660] _zoneinfo.c incorrectly checks bounds of `day` variable in calenderrule_new

2020-12-16 Thread Paul Ganssle


Paul Ganssle  added the comment:

> Just my two cents, this isn't just "some compilers". Everything from gcc, 
> msvc, C# to the rust compiler complain about this sort of code. As they 
> should, this is effectively dead code.

They complain because most of the time it's a sign that people were intending 
to use a signed integer, and it's an indicator that they may be susceptible to 
integer overflow.

In this case, it was a deliberate choice to include the extra check knowing 
it's dead code, because it is essentially a machine-checked documentation of 
the bounds of that particular variable.


> I think the more pragmatic way to enforce and document this assumption would 
> be to have a unit test that actually checks that the constructor fails with 
> "negative" days. It'll continue to fail right now as its interpretation as an 
> unsigned int will be large and it will start failing if someone changes this 
> to a signed type.

This would be helpful, but it's not an either-or situation. This particular 
thing would be tricky to write a targeted unit test for, because it's a very 
deep implementation detail. Such a unit test would also be quite physically 
separated from the code in question, whereas if we left the code as-is, we'd 
never see this type of bug, and if we added a static assertion we'd be told at 
compile time exactly what is wrong.

The biggest problem here is the fact that we cannot disable compiler warnings 
when complying with them makes the code less readable and/or less robust. This 
makes compiler warnings less useful, since it changes the calculus around 
whether or not it's worth it to introduce a new compiler warning.

--

___
Python tracker 
<https://bugs.python.org/issue42660>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42660] _zoneinfo.c incorrectly checks bounds of `day` variable in calenderrule_new

2020-12-16 Thread Paul Ganssle

Paul Ganssle  added the comment:

As I mentioned, it's a style issue. Yes this did not introduce any 
user-observable bugs, nor should it have changed the machine code emitted by 
the assembly on any competent compiler.

The issue is that I had deliberately chosen to do a "redundant" check to 
improve the readability and to be robust against someone saying, "Why is this a 
unit8_t instead of an int? I don't think it makes anything faster..." or some 
other justification for changing all the uint8_t fields to ints.

Previous to this, we had something where the compiler would fix any bugs caused 
by that by itself by emitting an additional bounds check. In my proposed fix to 
the compiler warnings, there was a static assertion where the machine would 
alert us if the situation changed. The problem is that we now have a 
non-machine-checked comment for something that would be difficult to issue 
tests for.

Frankly, I think that even the worst case scenario here isn't that bad — TZif 
files malformed in a very specific way would be accepted as valid. As it's 
coded now, this would probably just lead to a crash later, or possibly some 
weird results in time zone math. Still, I think we need a solution to this 
problem because our blind adherence to an invalid compiler warning is leading 
us to write more fragile code, whic h is especially problematic in C, which is 
notoriously dangerous and problematic to write.

--

___
Python tracker 
<https://bugs.python.org/issue42660>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42660] _zoneinfo.c incorrectly checks bounds of `day` variable in calenderrule_new

2020-12-16 Thread Paul Ganssle

New submission from Paul Ganssle :

This is a code style issue — in https://github.com/python/cpython/pull/23614, a 
regression was deliberately introduced to satisfy an overzealous compiler. The 
`day` variable has logical bounds `0 <= day <= 6`. In the original code, both 
sides of this boundary condition were explicitly checked (since this logically 
documents the bounds of the variable).

Some compilers complain about checking `day < 0`, because `day` is an unsigned 
type. It is not an immutable fact that `day` will always be an unsigned type, 
and implicitly relying on this fact makes the code both less readable and more 
fragile. This was changed over my objections and despite the fact that I had a 
less fragile solution available that also satisfied the overzealous compiler.

In the short term, my preferred solution would be to add in a static assertion 
that `day` is an unsigned type — this does not have to work on every platform, 
it simply needs to serve as a notification to make the code less fragile and to 
document our assumptions to both readers and the compiler.

In the long term, I think we need a way to solve the problem that it is 
apparently not possible to disable any compiler warnings even if they don't 
apply to the situation!

--
components: Library (Lib)
messages: 383180
nosy: p-ganssle
priority: normal
severity: normal
stage: needs patch
status: open
title: _zoneinfo.c incorrectly checks bounds of `day` variable in 
calenderrule_new
type: behavior
versions: Python 3.10, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue42660>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42622] Add support to add existing parser to ArgumentParser.subparsers

2020-12-11 Thread paul j3


paul j3  added the comment:

An existing parser can be included in a new subparser via the 'parents' 
mechanism.  With that the _actions and groups of the parent are copied (by 
reference) to the new subparser.

There are some rough edges to the parents mechanism, but people have used it to 
add a common subset of arguments to all subparsers.

--

___
Python tracker 
<https://bugs.python.org/issue42622>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39106] Add suggestions to argparse error message output for unrecognized arguments

2020-12-11 Thread paul j3


paul j3  added the comment:

In the subparser example, it's the `build` subparser that puts '--overwritte' 
in the unrecognized list.  That is then passed back to the main parser, which 
then issues the 'unrecognized' error, along with its own usage.

The subparser is called with `parse_known_args` while the proposed patch is run 
in the `parse_args` method of the main parser.  It doesn't have access to the 
subparser's arguments.  So implementing the proposed matching will be much 
harder.

For some types of error, such as type or choices, the subparser itself raises 
the error, with the appropriate usage.  

===

https://bugs.python.org/issue42297
[argparse] Bad error message formatting when using custom usage text

is another case where error messages produced by the subparser differ from 
messages produced by the main. In this case the unrecognized error usage 
message is clearer since it is produced by the main parser.

===

I didn't close this issue, but it does feel like an enhancement that's too big 
for the bug/issues forum.  The proposed patch could be developed as a separate 
'parser.parse_args_with_hints' method, and distributed as a pypi addition.  
During development and testing, the regular 'parser.parse_args()' does not need 
to be touched.

--

___
Python tracker 
<https://bugs.python.org/issue39106>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42572] Better path handling with argparse

2020-12-08 Thread paul j3


paul j3  added the comment:

One caution - the type parameter is a callable (function) that takes one string 
as argument.  I proposed `pathlib.Path` because it does that, returning a Path 
object.  It's a class instance creator.  I believe the module has other class 
initiators.

bool() has confused many users.  While it returns a bool class instance, True 
or False, the only string that returns False is the empty one, with argparse 
can't supply. It does not convert strings like 'False' or 'no' to boolean False.

--

___
Python tracker 
<https://bugs.python.org/issue42572>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42572] Better path handling with argparse

2020-12-08 Thread paul j3


paul j3  added the comment:

The pathlib.Path is new since I paid much attention to os matters (I cut my 
teeth on py2.5).

Off hand it looks like the user could

import pathlib
parser.add_argument('-p', type=pathlib.Path)

to convert a string into a Path object.

A custom type function could call this, and apply any desired methods before 
returning the Path object. But that should be up to the user, not the 
`argparse` developers.

Importing path specific modules such as pathlib (which in turn has a lot of 
imports) goes against the attempt to reduce the number of unnecessary imports 
with modules like argparse.

https://bugs.python.org/issue30152
Reduce the number of imports for argparse

After this diet argparse still imports os, and uses:

os.path.basename

I might also note that argparse has only one custom 'type' function, the 
FileType factory.  Other common type functions like 'int' and 'float' are 
Python builtin's.  Adding a custom 'bool' (to be used instead of the builtin 
'bool') has been rejected.

https://bugs.python.org/issue37564

json, yaml, and datetime types have also been rejected
https://bugs.python.org/issue35005

--

___
Python tracker 
<https://bugs.python.org/issue42572>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42572] Better path handling with argparse

2020-12-07 Thread paul j3


paul j3  added the comment:

What exactly do you do with a path argument?

Admittedly I'm not expert with os and os.path modules, but isn't a path just a 
string passed to a function such as cwd(), or joined to another create a 
filename.

I don't know what a 'path' type or action would do.

--

___
Python tracker 
<https://bugs.python.org/issue42572>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42592] TypedDict: total=False but still key required

2020-12-07 Thread Paul Bryan


Paul Bryan  added the comment:

Your patch LGTM, Brandt.

--

___
Python tracker 
<https://bugs.python.org/issue42592>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42592] TypedDict: total=False but still key required

2020-12-07 Thread Paul Bryan


New submission from Paul Bryan :

I believe "a" below should be an optional key, not a required one.

Python 3.9.0 (default, Oct  7 2020, 23:09:01) 
[GCC 10.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import typing
>>> TD = typing.TypedDict("TD", {"a": str}, total=False)
>>> TD.__total__
False
>>> TD.__required_keys__
frozenset({'a'})
>>> TD.__optional_keys__
frozenset()
>>>

--
components: Library (Lib)
messages: 382662
nosy: gvanrossum, pbryan
priority: normal
severity: normal
status: open
title: TypedDict: total=False but still key required
type: behavior
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue42592>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42559] random.getrandbits: Should it be explicit that it returns unsigned/non-negative integer?

2020-12-07 Thread Paul Sokolovsky


Paul Sokolovsky  added the comment:

Raymond Hettinger: Thanks for acking it would be a useful change!

ZackerySpytz: Thanks for making a patch!

--

___
Python tracker 
<https://bugs.python.org/issue42559>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42559] random.getrandbits: Should it be explicit that it returns unsigned/non-negative integer?

2020-12-03 Thread Paul Sokolovsky


New submission from Paul Sokolovsky :

Current docs for random.getrandbits() ( 
https://docs.python.org/3/library/random.html#random.getrandbits ) read 
(3.9.1rc1 at the top of the page):

Returns a Python integer with k random bits. This method is supplied with the 
MersenneTwister generator and some other generators may also provide it as an 
optional part of the API. When available, getrandbits() enables randrange() to 
handle arbitrarily large ranges.

So, based that it talks about "bits", it's easy to imagine that the result is 
unsigned. But it's interesting that it mentions "a Python integer", not even 
just "integer". And Python integers are known to be signed.

So I'd say, there's enough of ambiguity in there, and would be nice to 
explicitly specify that it's "unsigned (non-negative) Python integer".


If there's interest for the background of concern, some implementations may 
have "small" and "large" integers underlyingly (which used to be exposed at the 
user-level in Python2). "Small" integers would be cheaper, but of course, they 
would be signed. So, at certain value of getrandbits() parameter, there may be 
a temptation to use up a sign bit of a small integer, instead of going straight 
to allocating expensive large integer. It should be clear this is NOT a valid 
"optimization", and result should be always non-negative.

--
messages: 382434
nosy: pfalcon
priority: normal
severity: normal
status: open
title: random.getrandbits: Should it be explicit that it returns 
unsigned/non-negative integer?
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue42559>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42547] argparse: add_argument(... nargs='+', metavar=) does not work with positional arguments

2020-12-02 Thread paul j3


paul j3  added the comment:

I'll reopen it - your patch, while not a complete resolution, does take care of 
the immediate error.

--
resolution: duplicate -> 
status: closed -> open

___
Python tracker 
<https://bugs.python.org/issue42547>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42547] argparse: add_argument(... nargs='+', metavar=) does not work with positional arguments

2020-12-02 Thread paul j3


paul j3  added the comment:

Duplicate of

https://bugs.python.org/issue14074

argparse allows nargs>1 for positional arguments but doesn't allow metavar to 
be a tuple

--
resolution:  -> duplicate
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue42547>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42518] Error Message

2020-12-01 Thread Paul Moore


Paul Moore  added the comment:

You need to learn about chained comparisons in Python. This is not a bug. 
Please stop reopening it.

--
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue42518>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42501] Improve error messages for argparse choices using enum

2020-11-29 Thread paul j3


paul j3  added the comment:

choices is fine for a few strings, but quickly becomes awkward with other types 
and large numbers.  The testing isn't an issue, since it just does a simple 
`in/contains` test.  But display, whether in usage, help or error, is 
problematic if you try anything too fancy.  metavar gets around some of those 
issues, but doesn't change the error messages.

Custom type or action is the best alternative.

I'm in favor omitting the enums mention in the docs, since it seems to be more 
confusing than helpful.

--

___
Python tracker 
<https://bugs.python.org/issue42501>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35498] Parents objects in pathlib.Path don't support slices as __getitem__ arguments

2020-11-23 Thread Paul Ganssle


Paul Ganssle  added the comment:


New changeset 79d2e62c008446fbbc6f264bb8a30e2d38b6ff58 by Yaroslav Pankovych in 
branch 'master':
Added support for negative indexes to PurePath.parents (GH-21799)
https://github.com/python/cpython/commit/79d2e62c008446fbbc6f264bb8a30e2d38b6ff58


--

___
Python tracker 
<https://bugs.python.org/issue35498>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue21041] pathlib.PurePath.parents rejects negative indexes

2020-11-23 Thread Paul Ganssle


Paul Ganssle  added the comment:


New changeset 79d2e62c008446fbbc6f264bb8a30e2d38b6ff58 by Yaroslav Pankovych in 
branch 'master':
Added support for negative indexes to PurePath.parents (GH-21799)
https://github.com/python/cpython/commit/79d2e62c008446fbbc6f264bb8a30e2d38b6ff58


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue21041>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42444] pathlib.PurePath properties annotated with .. data directive

2020-11-23 Thread Paul Ganssle

New submission from Paul Ganssle :

Currently, it seems that the pathlib module uses `.. data::` to annotate the 
properties of the PurePath type (e.g. .parts, .drive, .root, etc). See: 
https://github.com/python/cpython/blob/ff420f0e08a2443339da0df7ace95e14177bac53/Doc/library/pathlib.rst

According to the documentation 
(https://devguide.python.org/documenting/#information-units), `data` is for 
module-level constants, specifically:

> Describes global data in a module, including both variables and values used
> as “defined constants.” Class and object attributes are not documented using
> this directive.

I believe that we should switch these over to use the `.. attribute:` directive 
instead.

>From what I can tell, you can still link to these attributes using the 
>`:attr:` role. I haven't checked if you can link to `:attribute:`s using the 
>`:data:` role, though. If not, it might break some links to change these to 
>`:attribute:`.

--
assignee: docs@python
components: Documentation
messages: 381673
nosy: docs@python, eric.araujo, ezio.melotti, mdk, p-ganssle, willingc
priority: low
severity: normal
status: open
title: pathlib.PurePath properties annotated with .. data directive
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue42444>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue21041] pathlib.PurePath.parents rejects negative indexes

2020-11-23 Thread Paul Ganssle


Paul Ganssle  added the comment:

I think you may have confused my thoughts as to why this might be considered 
ambiguous with an actual suggestion for what the semantics should be.

I think that we should stick with `p.parents[x] == tuple(p.parents)[x]` for any 
valid value of `x`, which means that `p.parents[-1]` will always be `Path('.')` 
for any non-empty `p`.

--

___
Python tracker 
<https://bugs.python.org/issue21041>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35498] Parents objects in pathlib.Path don't support slices as __getitem__ arguments

2020-11-20 Thread Paul Ganssle


Change by Paul Ganssle :


--
dependencies:  -pathlib.PurePath.parents rejects negative indexes
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue35498>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35498] Parents objects in pathlib.Path don't support slices as __getitem__ arguments

2020-11-20 Thread Paul Ganssle


Paul Ganssle  added the comment:


New changeset 452058448335b39c784af9a047f9c4a1767c0b00 by Joshua Cannon in 
branch 'master':
bpo-35498: Added slice support to PathLib parents attribute. (GH-11165)
https://github.com/python/cpython/commit/452058448335b39c784af9a047f9c4a1767c0b00


--

___
Python tracker 
<https://bugs.python.org/issue35498>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35498] Parents objects in pathlib.Path don't support slices as __getitem__ arguments

2020-11-19 Thread Paul Ganssle


Paul Ganssle  added the comment:

One question I would have about this is that `.parents` is a lazily-calculated 
sequence, not a list or a tuple, so it's not immediately obvious what the 
return type of a slice would be. I don't think it makes sense to return, e.g. a 
`_PathParents` object with fewer parts, but I don't know if there's any 
traditional choice here, other than that the result of slicing Sequence is 
another Sequence.

The PR returns a list, but I'm inclined to say we should return a tuple.

--
nosy: +p-ganssle
versions: +Python 3.10 -Python 3.8

___
Python tracker 
<https://bugs.python.org/issue35498>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue21041] pathlib.PurePath.parents rejects negative indexes

2020-11-19 Thread Paul Ganssle

Paul Ganssle  added the comment:

I am not seeing any compelling reasons to avoid supporting negative indexes 
*or* slices here.

If I had to guess about the confusing semantics of negative indices, I would 
guess it's the fact that the index in the -1 position for a non-empty Path will 
always be `Path('.')`. Since that's not terribly useful, it might be reasonable 
to have negative indices start counting at `len(p)-2`.

That said, I don't think this is a big deal, and I think we have more 
speculation on why this was avoided in the first place than we have actual 
objections to changing it, so I vote for changing it.

I think our best option is to say that the semantics of indexing `.parents` 
should be the same as indexing the result of casting it to a tuple, so this 
should be true:

p = Path(x)
assert p.parents[y] == tuple(p.parents)[y]

For all values of `x` and `y`.

I've gone ahead and changed the version support matrix to 3.10 only, since I 
think that this was a deliberate choice and we should be considering this an 
enhancement rather than a bugfix. That said, I'll admit that it's on the 
borderline — the semantics of sequences are unambiguous (see, which says that 
sequences support both slices and negative indices: 
https://docs.python.org/3/library/stdtypes.html#typesseq ), and PEP 428 
explicitly says that .parents returns a "an immutable sequence of the path's 
logical ancestors": 
https://www.python.org/dev/peps/pep-0428/#sequence-like-access . So if someone 
is motivated to try and make the case that this is a bugfix that could be 
backported to earlier supported versions, I won't stand in their way.

--
nosy: +p-ganssle
versions:  -Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue21041>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42390] Other Python implementations may not expose the module name in datetime type names

2020-11-17 Thread Paul Ganssle


Paul Ganssle  added the comment:

What is an example of another Python implementation that has this property? Is 
there a concrete issue open somewhere that this is solving?

I am not unsympathetic to the idea of accommodating other implementations of 
Python, but this is very abstract and I think the assumption is probably that 
if we're explicitly testing for something and we don't say it's 
implementation-defined that it is part of the language spec.

If there's some evidence that stuff like this is intended to be 
implementation-defined always, or there's some concrete problem that we can 
solve (and possibly also add tests to avoid regressions), I'd be much more 
comfortable with something like this.

--

___
Python tracker 
<https://bugs.python.org/issue42390>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42094] isoformat() / fromisoformat() for datetime.timedelta

2020-11-17 Thread Paul Ganssle

Paul Ganssle  added the comment:

This is probably more feasible than the proposal in bpo-41254 since it's a 
well-defined spec (mostly — it includes an optional alternative format and the 
number of digits allowed is defined "by agreement", thus defeating the purpose 
of using a spec in the first place) that's not even particularly difficult to 
implement, but there are still a few problems (and one reason I've never 
implemented this, despite desperately wanting a better string representation 
for time deltas). Two minor problems first:

1. Unlike ISO 8601 datetimes, these are not especially "human-friendly" 
formats, so I don't think they're especially useful for displaying timedeltas.

2. Also unlike ISO 8601 datetimes, I don't think these are in particularly wide 
use, or widely supported. That's not a major strike against it, but if it's not 
useful as something to show to humans and it's not especially useful as 
something to show to / read from other computers, that weighs against its 
inclusion in the standard library.

The biggest problem, however, is that `timedelta` does not and cannot represent 
"Year" or "Month", which means that `P1Y` or `P1M` would always need to be 
invalid to parse. We could eliminate this format, but it means that we would 
never at any point in the future be able to implement a parser for the full 
spec. Since the concept of a year and a month are ambiguous and at least the 
2016 version of ISO 8601 doesn't seem to define what it means for a duration to 
last 1 year or 1 month, you can't even really count on such a thing as an 
interchange format, because different implementations might give you different 
results! What does `20200131T00:00:00/P1M` represent? The interval (2020-01-31, 
2020-02-29)? (2020-01-31, 2020-03-02)? Something else?

A better target for parsing ISO 8601 durations would be something like 
`dateutil.relativedelta`, which does have defined semantics for years and 
months (though as I mentioned above, those are not necessarily consistent with 
the semantics of other libraries parsing or writing out this format).

I am also not entirely clear on whether "weeks" is just an alias for "7 days" 
or if it means something related to weeks in the ISO calendar (and if that 
makes a difference for durations).

I imagine that generating these formats is a bit more forgiving, because you 
would simply never generate the forbidden formats, and we can offer 
configuration options in the formatter method to allow the user to tweak the 
various ambiguities in the spec.

--

___
Python tracker 
<https://bugs.python.org/issue42094>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42094] isoformat() / fromisoformat() for datetime.timedelta

2020-11-17 Thread Paul Ganssle


Change by Paul Ganssle :


--
nosy: +p-ganssle

___
Python tracker 
<https://bugs.python.org/issue42094>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42390] Other Python implementations may not expose the module name in datetime type names

2020-11-17 Thread Paul Ganssle


Paul Ganssle  added the comment:

Is this an actual problem for another implementation of Python?

Is there some reason to think that we intended the repr of a `datetime` object 
to be implementation-defined?

--
nosy: +p-ganssle

___
Python tracker 
<https://bugs.python.org/issue42390>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42371] datetime.fromisoformat(): Omitted colon in timezone suffix raises ValueError

2020-11-16 Thread Paul Ganssle


Paul Ganssle  added the comment:

This is the expected behavior of `.fromisoformat()`. A similar issue is 
https://bugs.python.org/issue35829, which asks for the "Z" suffix to be 
supported.

There is a note about this in the documentation: 
https://docs.python.org/3/library/datetime.html#datetime.datetime.fromisoformat

"Caution This does not support parsing arbitrary ISO 8601 strings - it is only 
intended as the inverse operation of datetime.isoformat(). A more full-featured 
ISO 8601 parser, dateutil.parser.isoparse is available in the third-party 
package dateutil."

At some point we will work out the kinks in offering as full an ISO 8601 
datetime parser as possible, but the ISO 8601 datetime spec is very complicated 
and includes many optional features. We deliberately chose to keep the scope of 
`.fromisoformat()` minimal at first, whereas `dateutil.parser.isoparse` 
attempts to be a full-featured ISO8601 parser.

Changing the version affected to 3.10, since this is a feature request.

--
type: behavior -> enhancement
versions: +Python 3.10 -Python 3.8

___
Python tracker 
<https://bugs.python.org/issue42371>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42293] Installation of pyinstaller in Windows fails with utf8 error

2020-11-14 Thread Paul Moore


Paul Moore  added the comment:

I've raised https://github.com/pypa/pip/issues/9135 for this.

--

___
Python tracker 
<https://bugs.python.org/issue42293>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42293] Installation of pyinstaller in Windows fails with utf8 error

2020-11-14 Thread Paul Moore


Paul Moore  added the comment:

Erik is correct, this is a pip bug. Please raise an issue on the pip tracker 
linking to this issue and we can fix it.

--

___
Python tracker 
<https://bugs.python.org/issue42293>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42297] [argparse] Bad error message formatting when using custom usage text

2020-11-10 Thread paul j3


paul j3  added the comment:

We could look into using a different more compact 'prog' for these error 
messages.

Currently the subparser 'prog' uses the main prog plus positionals plus the 
subparser name.  The goal is to construct a subparser usage that reflects 
required input.  

This is fine for the usage line, but could be too verbose for some errors.  In 
an error, the 'prog' just helps identify which argument has problems, and 
doesn't need the extra usage information.   Most of the time that isn't an 
issue, since we don't use positional much in the main parser (and when used 
can't have variable nargs).

But I don't have immediate ideas as to what can be conveniently (and safely) 
changed.

--

___
Python tracker 
<https://bugs.python.org/issue42297>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42297] [argparse] Bad error message formatting when using custom usage text

2020-11-09 Thread paul j3


paul j3  added the comment:

It's the subparser that's producing this error, specifically its 'prog' 
attribute.

If I use a custom usage with a simple parser:

1129:~/mypy$ python3 issue42297.py --foo=on
usage: issue42297.py
one
two
three
issue42297.py: error: argument --foo: ignored explicit argument 
'on'

Notice that the error line includes the 'prog'.  

With subparsers, the main usage is included in the subcommand prog:

print(subcmd_parser.prog)

produces:

test usage string ending in newlines

foo  --bar
foo  --bar
foo  --bar foo

That's the usage plus the subcommand name, 'foo'.

Generating the explicit error in the subcommand:

1244:~/mypy$ python3 issue42297.py foo --bar=on
test usage string ending in newlines

foo  --bar
foo  --bar
foo  --bar foo: error: argument --bar: ignored explicit 
argument 'on'

'issue42297.py: ' has been replaced by the usage+'foo', and no newline.

We don't see this in the 'unrecognized' case because that error issued by the 
main parser.

issue42297.py: error: unrecognized arguments: on

If I explicitly set the prog of the subcommand:

subcmd_parser = subparser.add_parser('foo', prog='myscript foo')

The error becomes:

1256:~/mypy$ python3 issue42297.py foo --bar=on
usage: myscript foo [-h] [--bar]
myscript foo: error: argument --bar: ignored explicit argument 'on'

I can also add 'usage=usage_string' to the add_parser.  For the most part 
add_parser takes the same parameters as ArgumentParser.

Alternatively we can specify prog in 

subparser = parser.add_subparsers(dest='subcmd', metavar='subcmd', 
prog='myscript')

resulting in:

myscript foo: error: argument --bar: ignored explicit argument 'on'

I recently explored how 'prog' is set with subparsers in

https://bugs.python.org/issue41980

I don't think anything needs to be corrected in argparse.  There are enough 
options for setting prog and usage in subcommands to get around this issue.  

In the worse case, you might want to create an alternative 

_SubParsersAction

Action subclass that defines the prog/usage differently.

--

___
Python tracker 
<https://bugs.python.org/issue42297>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



<    1   2   3   4   5   6   7   8   9   10   >