[issue46511] dataclasses: Allow typing.Annotated to wrap dataclasses-specific annotations
Gregory Beauregard added the comment: It occurred to be that we do need to add the __call__ to KW_ONLY, but for a different reason than this bpo: If you call get_type_hints on a dataclass with a KW_ONLY parameter when PEP 563 is enabled, the entire call will fail if KW_ONLY isn't callable(). This can also happen if you stringize KW_ONLY without PEP 563. I made a bpo to suggest removing the callable() check entirely, but it's waiting discussion currently: https://bugs.python.org/issue46644 My feeling is you probably wanted to wait on making changes of this kind for the 5.11 PEP 563 et al decision to play out, but on the other hand I think the KW_ONLY (and similar __call__ method for InitVar this patch already adds) change would likely be backportable so we may want to make them anyway for that purpose. Do you have an opinion on this? This patch might be backportable to mirror https://bugs.python.org/issue46491 but I don't have a strong opinion on that. -- ___ Python tracker <https://bugs.python.org/issue46511> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46643] typing.Annotated cannot wrap typing.ParamSpec args/kwargs
Gregory Beauregard added the comment: I wrote a PR that fixes the underlying issue here, but I'm leaving it as a draft while the discussion plays out. I think the stuff currently in the patch should be okay regardless of the discussion decision, because the underlying issue is that P.args and P.kwargs didn't pass typing._type_check, which is needed for reasons unrelated to Annotated (such as if they were stringified and had get_type_hints called on them). If the result of the discussion is that we need to start supporting limitations on where Annotated is used, I'll add another PR that introduces support for this pattern in whatever locations that are decided it shouldn't be allowed. -- ___ Python tracker <https://bugs.python.org/issue46643> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46643] typing.Annotated cannot wrap typing.ParamSpec args/kwargs
Change by Gregory Beauregard : -- keywords: +patch pull_requests: +29408 stage: -> patch review pull_request: https://github.com/python/cpython/pull/31238 ___ Python tracker <https://bugs.python.org/issue46643> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46643] typing.Annotated cannot wrap typing.ParamSpec args/kwargs
Gregory Beauregard added the comment: I have made a thread on typing-sig to discuss this: https://mail.python.org/archives/list/typing-...@python.org/thread/WZ4BCFO4VZ7U4CZ4FSDQNFAKPG2KOGCL/ -- ___ Python tracker <https://bugs.python.org/issue46643> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46643] typing.Annotated cannot wrap typing.ParamSpec args/kwargs
Gregory Beauregard added the comment: My general understanding has been that since Annotated exists primarily to allow interoperability between typing and non-typing uses of annotation space, and that this is quite clear in the PEP, most cases where this is not allowed (_especially_ at the top level of the annotation, as in this case) are either a bug or unintentional. We fixed Annotated wrapping Final and ClassVar a couple weeks ago for arguments sorta along these lines after discussion on typing-sig, where the concern of what constitutes a type to wrap was specifically addressed: https://bugs.python.org/issue46491 But, I am not opposed to discussing this case too, or discussing the issue more broadly. Is typing-sig the appropriate place for me to make a thread, or do we wait for comments here? Even if we decide this particular issue is okay to fix, I think you bring up a good point about documenting our decision clearly once and for all so I agree we should do this. -- ___ Python tracker <https://bugs.python.org/issue46643> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46676] ParamSpec args and kwargs are not equal to themselves.
Change by Gregory Beauregard : -- pull_requests: +29380 pull_request: https://github.com/python/cpython/pull/31210 ___ Python tracker <https://bugs.python.org/issue46676> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46676] ParamSpec args and kwargs are not equal to themselves.
Change by Gregory Beauregard : -- keywords: +patch pull_requests: +29373 stage: -> patch review pull_request: https://github.com/python/cpython/pull/31203 ___ Python tracker <https://bugs.python.org/issue46676> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46676] ParamSpec args and kwargs are not equal to themselves.
New submission from Gregory Beauregard :
from typing import ParamSpec
P = ParamSpec("P")
print(P.args == P.args) # False
print(P.kwargs == P.kwargs) # False
ParamSpec args and kwargs are not equal to themselves; this can cause problems
for unit tests and type introspection w/ e.g. `get_type_hints`.
I will fix this by adding an __eq__ method like other places in typing.py
--
components: Library (Lib)
messages: 412781
nosy: GBeauregard, Jelle Zijlstra
priority: normal
severity: normal
status: open
title: ParamSpec args and kwargs are not equal to themselves.
type: behavior
versions: Python 3.10, Python 3.11
___
Python tracker
<https://bugs.python.org/issue46676>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46655] typing.TypeAlias is not in the list of allowed plain _SpecialForm typeforms
Change by Gregory Beauregard : -- pull_requests: +29346 pull_request: https://github.com/python/cpython/pull/31175 ___ Python tracker <https://bugs.python.org/issue46655> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46644] typing: remove callable() check from typing._type_check
Gregory Beauregard added the comment: I compiled your PR to run it and was testing in 3.10 as well, but I was testing in a file with from __future__ import annotations unintentionally. I retract the comment. It turns out `list[42]` is okay though, which I suppose is more relevant going forward. My confusion here is sort of the crux of my problem with these runtime checks: they are inconsistently applied in different locations which is why callable() was causing a lot of bugs. -- ___ Python tracker <https://bugs.python.org/issue46644> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46644] typing: remove callable() check from typing._type_check
Gregory Beauregard added the comment: I'm referring to within type annotations, where this code path isn't used: try a: List[42] This code path can show up in type aliases though. -- ___ Python tracker <https://bugs.python.org/issue46644> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46644] typing: remove callable() check from typing._type_check
Gregory Beauregard added the comment: List[42] is already accepted, and your proposed patch does not change it to make it not accepted. The issue is _type_check is only called in a few particular locations; this is part of the technical reason I'm not very concerned about relaxing the _type_check requirements. >From a type checking philosophy point of view I agree with Jelle and am >negative on strict runtime requirements in typing.py. -- ___ Python tracker <https://bugs.python.org/issue46644> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46642] typing: tested TypeVar instance subclass TypeError is incidental
Gregory Beauregard added the comment: The issue in real code I had in mind was internal to cpython: if we remove the callable() check from _type_check naively, this test starts to fail. Both of our proposed changes happen to not fail this check. Given your second example, would you prefer if we remove this testcase if the issue comes up in the final proposed patches? On the other hand, there is precedent for giving this a nice error message other places in typing.py. -- ___ Python tracker <https://bugs.python.org/issue46642> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46655] typing.TypeAlias is not in the list of allowed plain _SpecialForm typeforms
Change by Gregory Beauregard : -- keywords: +patch pull_requests: +29331 stage: -> patch review pull_request: https://github.com/python/cpython/pull/31156 ___ Python tracker <https://bugs.python.org/issue46655> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46655] typing.TypeAlias is not in the list of allowed plain _SpecialForm typeforms
Change by Gregory Beauregard : -- nosy: +kj ___ Python tracker <https://bugs.python.org/issue46655> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46655] typing.TypeAlias is not in the list of allowed plain _SpecialForm typeforms
Change by Gregory Beauregard : -- nosy: +Jelle Zijlstra ___ Python tracker <https://bugs.python.org/issue46655> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46655] typing.TypeAlias is not in the list of allowed plain _SpecialForm typeforms
New submission from Gregory Beauregard : typing.TypeAlias is allowed to be bare, but it's not listed in the list of types in typing._type_check that are allowed to be bare. This means it's possible to reach the wrong error `TypeError: Plain typing.TypeAlias is not valid as type argument` at runtime. Examples offhand: from typing import TypeAlias, get_type_hints class A: a: "TypeAlias" = int get_type_hints(A) from typing import Annotated, TypeAlias b: Annotated[TypeAlias, ""] = int There's likely more and/or more realistic ways to trigger the problem. Anything that triggers typing._type_check on typing.TypeAlias will give this error (TypeError: Plain typing.TypeAlias is not valid as type argument). I will fix this by adding TypeAlias to the list of typing special forms allowed to be bare/plain. I intend to move these to their own named var to reduce the chance of types not getting added in the future. -- components: Library (Lib) messages: 412618 nosy: GBeauregard priority: normal severity: normal status: open title: typing.TypeAlias is not in the list of allowed plain _SpecialForm typeforms type: behavior ___ Python tracker <https://bugs.python.org/issue46655> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46644] typing: remove callable() check from typing._type_check
Change by Gregory Beauregard : -- keywords: +patch pull_requests: +29328 stage: -> patch review pull_request: https://github.com/python/cpython/pull/31151 ___ Python tracker <https://bugs.python.org/issue46644> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46644] typing: remove callable() check from typing._type_check
Gregory Beauregard added the comment: I made a draft pull request where I went ahead and added a check to disallow tuple literals. This is basically already disallowed for types by restrictions on `__getitem__` because Union[typeform]->type needs to be different from Union[type,type]->Union[type,type]. -- ___ Python tracker <https://bugs.python.org/issue46644> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46644] typing: remove callable() check from typing._type_check
Gregory Beauregard added the comment: Further questions: the msg argument in _type_check now wouldn't be used for anything! It was only used in the case where a type wasn't callable(). I think it should be removed. I'm also a bit negative on disallowing tuples in the case of e.g. Final and such since it complicates implementing tuple types in Python down the line if desired. -- ___ Python tracker <https://bugs.python.org/issue46644> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46644] typing: remove callable() check from typing._type_check
Gregory Beauregard added the comment: Under the same failing int test cases before there were 2 more cases next to them that fail: with self.assertRaises(TypeError): ClassVar[int, str] with self.assertRaises(TypeError): Final[int, str] These fail because tuple literals are not callable(). There is code that clearly intends for this to be the case: https://github.com/python/cpython/blob/96b344c2f15cb09251018f57f19643fe20637392/Lib/typing.py#L486 I can either remove support for this runtime check or change the implementation of Final et al to reject tuple literals. I will do the latter for now. For https://github.com/python/cpython/blob/bf95ff91f2c1fc5a57190491f9ccdc63458b089e/Lib/test/test_typing.py#L4262-L4263 I think the best approach is to just remove these tests. -- ___ Python tracker <https://bugs.python.org/issue46644> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46648] `test.test_urllib2.MiscTests.test_issue16464` started to fail
Change by Gregory Beauregard : -- nosy: +GBeauregard nosy_count: 2.0 -> 3.0 pull_requests: +29325 pull_request: https://github.com/python/cpython/pull/31148 ___ Python tracker <https://bugs.python.org/issue46648> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46642] typing: tested TypeVar instance subclass TypeError is incidental
Gregory Beauregard added the comment: Fixing this test unblocks bpo-46644 -- ___ Python tracker <https://bugs.python.org/issue46642> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46642] typing: tested TypeVar instance subclass TypeError is incidental
Change by Gregory Beauregard : -- keywords: +patch pull_requests: +29324 stage: -> patch review pull_request: https://github.com/python/cpython/pull/31148 ___ Python tracker <https://bugs.python.org/issue46642> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46642] typing: tested TypeVar instance subclass TypeError is incidental
Gregory Beauregard added the comment:
The reason this test passed before is a bit confusing. Run the following code
standalone to see where the type(TypeVar('T'))(name, bases, namespace) check is
coming from.
```
class TypeVar:
def __init__(self, name, *constraints):
# in actual TypeVar, each constraint is run through
# typing._type_check, casuing TypeError via not callable()
print(repr(constraints))
class V(TypeVar("T")):
pass
```
--
___
Python tracker
<https://bugs.python.org/issue46642>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46644] typing: remove callable() check from typing._type_check
Gregory Beauregard added the comment: In addition to the 10 tests failed in test_typing.py, one additional test fails in test_types.py with this change: https://github.com/python/cpython/blob/bf95ff91f2c1fc5a57190491f9ccdc63458b089e/Lib/test/test_types.py#L834-L838 and those are all the tests in cpython changed. This falls in category (1), checking that an int literal is not a type, but with the apparent intent to prevent index subscripting. -- ___ Python tracker <https://bugs.python.org/issue46644> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46643] typing.Annotated cannot wrap typing.ParamSpec args/kwargs
Gregory Beauregard added the comment: We can also fix this with my proposal in bpo-46644. I'm okay with either fix (that or implementing __call__), or both. -- ___ Python tracker <https://bugs.python.org/issue46643> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46644] typing: remove callable() check from typing._type_check
New submission from Gregory Beauregard :
I propose removing the callable() check[1] from typing._type_check. This
restriction is usually met in typeform instances by implementing a __call__
method that raises at runtime[2]. _type_check is called at runtime in a few
disparate locations, such as in an argument to typing.Annotated or for certain
stringified annotations in typing.get_type_hints.
Because the requirement to be callable is unexpected and shows up in situations
not easily discoverable during development or common typing usage, it is the
cause of several existing cpython bugs and will likely continue to be the cause
of bugs in typeforms outside of cpython. Known cpython bugs caused by the
callable() check are bpo-46643, bpo-44799, a substantial contributing factor to
bpo-46642, and partly bpo-46511. I discovered bpo-46643 with only a cursory
check of typing.py while writing this proposal. Moreover, it doesn't make any
particular technical sense to me why it should be required to add an awkward
__call__ method.
Removing the callable() check fails 10 tests:
7 tests: checking that an int literal is not a type
2 tests: testing that list literals are not valid types (e.g. [3] raises a
TypeError because the literal [('name', str), ('id', int)] doesn't pass
callable())
1 test: bpo-46642
The responsibility of determining these invalid typeforms (e.g. int literals)
would need to be passed to a static type checker. If it's desired to do this at
runtime it's my opinion that a different check would be more appropriate.
Have I missed any reasons for the callable() check? Can I remove the check and
adjust or remove the tests?
[1]
https://github.com/python/cpython/blob/bf95ff91f2c1fc5a57190491f9ccdc63458b089e/Lib/typing.py#L183-L184
[2]
https://github.com/python/cpython/blob/bf95ff91f2c1fc5a57190491f9ccdc63458b089e/Lib/typing.py#L392-L393
[3]
https://github.com/python/cpython/blob/bf95ff91f2c1fc5a57190491f9ccdc63458b089e/Lib/test/test_typing.py#L4262-L4263
--
components: Library (Lib)
messages: 412548
nosy: AlexWaygood, GBeauregard, Jelle Zijlstra, gvanrossum, kj, levkivskyi,
sobolevn
priority: normal
severity: normal
status: open
title: typing: remove callable() check from typing._type_check
type: enhancement
___
Python tracker
<https://bugs.python.org/issue46644>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46643] typing.Annotated cannot wrap typing.ParamSpec args/kwargs
Change by Gregory Beauregard : -- nosy: +sobolevn ___ Python tracker <https://bugs.python.org/issue46643> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46643] typing.Annotated cannot wrap typing.ParamSpec args/kwargs
New submission from Gregory Beauregard :
Consider the following.
```
import logging
from typing import Annotated, Callable, ParamSpec, TypeVar
T = TypeVar("T")
P = ParamSpec("P")
def add_logging(f: Callable[P, T]) -> Callable[P, T]:
"""A type-safe decorator to add logging to a function."""
def inner(*args: Annotated[P.args, "meta"], **kwargs: P.kwargs) -> T:
logging.info(f"{f.__name__} was called")
return f(*args, **kwargs)
return inner
@add_logging
def add_two(x: float, y: float) -> float:
"""Add two numbers together."""
return x + y
```
This raises an error at runtime because P.args/P.kwargs cannot pass the
typing._type_check called by Annotated because they are not callable(). This
prevents being able to use Annotated on these type annotations.
This can be fixed by adding __call__ methods that raise to typing.ParamSpecArgs
and typing.ParamSpecKwargs to match other typeforms.
I can write this patch given agreement
--
components: Library (Lib)
messages: 412546
nosy: AlexWaygood, GBeauregard, Jelle Zijlstra, gvanrossum, kj
priority: normal
severity: normal
status: open
title: typing.Annotated cannot wrap typing.ParamSpec args/kwargs
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9
___
Python tracker
<https://bugs.python.org/issue46643>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46642] typing: tested TypeVar instance subclass TypeError is incidental
New submission from Gregory Beauregard :
https://github.com/python/cpython/blob/bf95ff91f2c1fc5a57190491f9ccdc63458b089e/Lib/test/test_typing.py#L227-L230
typing's testcases contain the following test to ensure instances of TypeVar
cannot be subclassed:
def test_cannot_subclass_vars(self):
with self.assertRaises(TypeError):
class V(TypeVar('T')):
pass
The reason this raises a TypeError is incidental and subject to behavior
change, not because doing so is prohibited per se; what's happening is the
class creation does the equivalent of type(TypeVar('T')(name, bases,
namespace), but this calls TypeVar's __init__ function with these items as the
TypeVar constraints. TypeVar runs typing._type_check on the type constraints
passed to it, and the literals for the namespace/name do not pass the
callable() check in typing._type_check, causing it to raise a TypeError. I find
it dubious this is the behavior the testcase is intending to test and the error
it gives is confusing
I propose we add __mro_entries__ to the TypeVar class that only contains a
raise of TypeError to properly handle this case
I can write this patch
--
components: Library (Lib)
messages: 412544
nosy: AlexWaygood, GBeauregard, Jelle Zijlstra, gvanrossum, kj, sobolevn
priority: normal
severity: normal
status: open
title: typing: tested TypeVar instance subclass TypeError is incidental
type: enhancement
___
Python tracker
<https://bugs.python.org/issue46642>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46511] dataclasses: Allow typing.Annotated to wrap dataclasses-specific annotations
Gregory Beauregard added the comment: I had a few style, approach, and testing preference questions, but I decided they're probably best addressed in a code review so I went ahead and posted the PR. -- ___ Python tracker <https://bugs.python.org/issue46511> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46511] dataclasses: Allow typing.Annotated to wrap dataclasses-specific annotations
Change by Gregory Beauregard : -- keywords: +patch pull_requests: +29178 stage: -> patch review pull_request: https://github.com/python/cpython/pull/30997 ___ Python tracker <https://bugs.python.org/issue46511> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46553] typing: get_type_hints on stringified lone ClassVar raises TypeError
Change by Gregory Beauregard : -- keywords: +patch pull_requests: +29162 stage: -> patch review pull_request: https://github.com/python/cpython/pull/30983 ___ Python tracker <https://bugs.python.org/issue46553> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46553] typing: get_type_hints on stringified lone ClassVar raises TypeError
Gregory Beauregard added the comment: I think I miscommunicated my intent with sentence placement. I already posted the thoughts I referred to; they're just my concluding opinion on the technical merit of using get_type_hints in dataclasses to solve the Annotated problem: https://bugs.python.org/msg411945 In any case, I apologize for the faux pas and I'll be careful in the future. I dug up the issue where pyright was changed (to allow bare ClassVar) that has an argument for not allowing it to be bare: https://github.com/microsoft/pyright/issues/2377 -- ___ Python tracker <https://bugs.python.org/issue46553> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46511] dataclasses: Allow typing.Annotated to wrap dataclasses-specific annotations
Gregory Beauregard added the comment: Hi Eric, to follow up on https://bugs.python.org/msg411943 I'm currently a bit negative on moving to get_type_hints, even though I got it working for the test suite. I think your worries with nesting are well placed, particularly with namespaces and such. In that vein, I suggest we move forward with patching the existing implementation with the discussed regex restrictions. I'm not sure if you want to remove the test cases with leading spaces; it seems not too important. While we're there I found a bug in the test suite, a missing comma that can be fixed at the same time: https://github.com/python/cpython/blob/b1a3446f077b7d56b89f55d98dadb8018986a3e5/Lib/test/test_dataclasses.py#L3080 Do you have any other concerns before I take a stab at this? -- ___ Python tracker <https://bugs.python.org/issue46511> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46553] typing: get_type_hints on stringified lone ClassVar raises TypeError
Gregory Beauregard added the comment: It's acceptable to mypy, and pyright added support a few months ago when I made an issue and Eric Traut discovered this pattern in the wild too. Some of the other type checkers (pyre) still error I believe. My feeling is that since this has apparently become used in practice we should fix the runtime error for when it's stringified, but we don't necessarily need to prescribe that it's a legal type annotation for type checkers (?). If I was prescribing how to use ClassVar in a vacuum, I don't see any good typing reason we should prohibit this. re: moving dataclasses: I'm not sure either! I'm trying to look into the issues it brings up in practice to weigh against the difficulties of maintaining the existing bespoke introspection implementation and the problems it has (dealing with stringified annotations, supporting renaming the Annotated symbol). I made an implementation that fully moves dataclasses over to using `get_type_hints` (and got all the tests to pass), but my feeling right now is that a few issues are too serious for it to be viable: 1) the namespaces are a mess, and issues like https://github.com/python/typing/issues/508 don't have a solution at the moment 2) get_type_hints paints a wide brush on the entire class when it raises for errors, so it's not very viable to deal with non-typing non-Annotated[] annotations as a one-off There's also nontechnical issues, like the potential politics of making dataclasses always import typing. I'll be updating bpo-46511 later with thoughts. -- ___ Python tracker <https://bugs.python.org/issue46553> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46553] typing: get_type_hints on stringified lone ClassVar raises TypeError
Gregory Beauregard added the comment: I'm drafting an implementation for the purpose of investigating performance right now; I will share when ready. -- ___ Python tracker <https://bugs.python.org/issue46553> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46553] typing: get_type_hints on stringified lone ClassVar raises TypeError
Change by Gregory Beauregard : -- nosy: +GBeauregard ___ Python tracker <https://bugs.python.org/issue46553> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46553] typing: get_type_hints on stringified lone ClassVar raises TypeError
Change by Gregory Beauregard : -- type: -> behavior ___ Python tracker <https://bugs.python.org/issue46553> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46553] typing: get_type_hints on stringified lone ClassVar raises TypeError
Change by Gregory Beauregard : -- nosy: +AlexWaygood, sobolevn -GBeauregard ___ Python tracker <https://bugs.python.org/issue46553> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46553] typing: get_type_hints on stringified lone ClassVar raises TypeError
Gregory Beauregard added the comment: I think this is needed for moving dataclasses to using typing.py introspection tools to be viable. -- ___ Python tracker <https://bugs.python.org/issue46553> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46553] typing: get_type_hints on stringified lone ClassVar raises TypeError
New submission from Gregory Beauregard : ``` class C: a: "ClassVar" get_type_hints(C, globals()) # TypeError: Plain typing.ClassVar is not valid as type argument ``` A stringified lone ClassVar raises at runtime, but this pattern is tested for in dataclasses unit tests and used in the wild. The PEP is not clear that it should or should not be used with arguments, and it works fine when not stringified. The fix for this is trivial and I can submit a patch if there's agreement. -- components: Library (Lib) messages: 411923 nosy: GBeauregard, Jelle Zijlstra, eric.smith, gvanrossum, kj priority: normal severity: normal status: open title: typing: get_type_hints on stringified lone ClassVar raises TypeError versions: Python 3.10, Python 3.11, Python 3.9 ___ Python tracker <https://bugs.python.org/issue46553> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46552] typing: get_type_hints can't handle stringified annotations with leading spaces
New submission from Gregory Beauregard : ``` class C: a: " ClassVar[int]" = 3 get_type_hints(C, globals()) # SyntaxError: Forward reference must be an expression -- got ' ClassVar[int]' ``` I discovered while investigating the viability of moving dataclasses.py to using typing.py's internal type introspection tools that it can't handle stringified annotations with leading spaces. This is covered in dataclasses unit tests: https://github.com/python/cpython/blob/26b0482393a313e3bda364a35e7417e9db52c1c4/Lib/test/test_dataclasses.py#L3033 The relevant failing code in typing.py is here: https://github.com/python/cpython/blob/26b0482393a313e3bda364a35e7417e9db52c1c4/Lib/typing.py#L671 Leading spaces are treated like indention so there's a syntax error. This would be trivial to fix by adding an lstrip() to the compilation code, but it's not clear to me this should be considered a bug. Should this be left as-is, or changed? I'm happy to submit a patch if there's agreement it's a bug. -- components: Library (Lib) messages: 411914 nosy: GBeauregard, Jelle Zijlstra, eric.smith priority: normal severity: normal status: open title: typing: get_type_hints can't handle stringified annotations with leading spaces type: behavior versions: Python 3.10, Python 3.11, Python 3.9 ___ Python tracker <https://bugs.python.org/issue46552> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46539] typing: forward references don't understand special type forms
Gregory Beauregard added the comment: I did try the proposed patch in bpo-41370 and verified it didn't resolve the issue so I'm not certain they strictly overlap, but I also haven't had time to fully digest the underlying issues in bpo-41370 yet. I think it does have relevance for changes we want to make for dataclasses re: Annotated, though: https://bugs.python.org/issue46511 -- ___ Python tracker <https://bugs.python.org/issue46539> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46539] typing: forward references don't understand special type forms
Gregory Beauregard added the comment: typo: the line is `g: Annotated["ClassVar[int]", (2, 5)] = 3` in the code sample. Somehow I left it at only `(` for the annotation instead of `(2,5)` -- ___ Python tracker <https://bugs.python.org/issue46539> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46539] typing: forward references don't understand special type forms
Change by Gregory Beauregard : -- type: -> behavior ___ Python tracker <https://bugs.python.org/issue46539> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46539] typing: forward references don't understand special type forms
Change by Gregory Beauregard : -- keywords: +patch pull_requests: +29105 stage: -> patch review pull_request: https://github.com/python/cpython/pull/30926 ___ Python tracker <https://bugs.python.org/issue46539> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46539] typing: forward references don't understand special type forms
New submission from Gregory Beauregard :
Consider the following code on 3.11 main:
```
from typing import Annotated, ClassVar, get_type_hints
class DC:
a: ClassVar[int] = 3
b: ClassVar["int"] = 3
c: "ClassVar[int]" = 3
d: Annotated[ClassVar[int], (2, 5)] = 3
e: Annotated[ClassVar["int"], (2, 5)] = 3
f: "Annotated[ClassVar[int], (2, 5)]" = 3
class DC_Special_ForwardRef:
g: Annotated["ClassVar[int]", (] = 3
# OK
assert get_type_hints(DC, globals(), locals()) == {
"a": ClassVar[int],
"b": ClassVar[int],
"c": ClassVar[int],
"d": ClassVar[int],
"e": ClassVar[int],
"f": ClassVar[int],
}
# TypeError: typing.ClassVar[int] is not valid as type argument
get_type_hints(DC_Special_ForwardRef, globals(), locals())
```
Currently, the `Annotated["ClassVar[int]", (2, 5)]` annotation raises at
runtime when `get_type_hints` is called, but all the other forward reference
annotations are okay.
My understanding is this is because when typing._type_check runs on a type
where special forms are allowed it's possible for the typing._type_convert it
calls it itself run a typing._type_check on contained forward references.
However, if that forward reference was itself a special form then it's possible
to get an error because typing._type_check doesn't pass on that special forms
are allowed.
I have drafted a patch to pass on this information. This will become important
in the future as more special forms are allowed to wrap each other, such as
allowing Final and ClassVar to nest each other in dataclasses, or when Required
and NotRequired land. In the future we may also want to reconsider runtime
restrictions on special forms in `typing.py` entirely and instead choose to
leave this to type checkers.
Is my analysis/patch approach okay? Forward references can be tricky. Should we
be discussing runtime restrictions in typing.py more generally in the future?
--
components: Library (Lib)
messages: 411790
nosy: GBeauregard, Jelle Zijlstra, gvanrossum, kj
priority: normal
severity: normal
status: open
title: typing: forward references don't understand special type forms
versions: Python 3.10, Python 3.11, Python 3.9
___
Python tracker
<https://bugs.python.org/issue46539>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46511] dataclasses: Allow typing.Annotated to wrap dataclasses-specific annotations
Gregory Beauregard added the comment: Or rather, ^\s*(?:(?:\w+\s*\.)?\s*Annotated\s*\[)?(?:\s*(\w+)\s*\.)?\s*(\w+) -- ___ Python tracker <https://bugs.python.org/issue46511> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46511] dataclasses: Allow typing.Annotated to wrap dataclasses-specific annotations
Gregory Beauregard added the comment: Thanks for getting back so quickly. Annotated is set up to be 'transparent' by default to `typing.get_type_hints` so in the case of using `typing.py` it can be made straightforward by chaining with `typing.get_origin`, I think. I don't see any reasonable way to do the regex that allows `Annotated` to be renamed, so I agree your suggested restriction. I think the regex would be something like this: ^\s*(?:(?:\w+\s*\.)?\s*Annotated\[)?(?:\s*(\w+)\s*\.)?\s*(\w+) I'm a bit worried people who are into Annotated annotations might be concerned about line length and more likely to rename `Annotated`, but I don't know if this is a realistic concern. And a part of me wants to say always importing typing.py should be okay since dataclasses was designed to work with type hints after all. On the other hand, I'm also a bit worried that if we made dataclasses always import typing.py it would rub people the wrong way even if we profiled it and decided it was okay. I'm going to spend some more time digesting the code tomorrow and try to decide if there's any major speed bumps to a full `typing` approach. I'll also look at import time and such. -- ___ Python tracker <https://bugs.python.org/issue46511> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46511] dataclasses: Allow typing.Annotated to wrap dataclasses-specific annotations
New submission from Gregory Beauregard : In https://bugs.python.org/issue46491 the typing runtime behavior was changed so that `Annotated[Classvar[...]]` is now valid at runtime in order to alleviate tension between typing and non-typing annotation space uses. dataclasses.py should likely follow suit in its runtime use of `ClassVar` and `InitVar`. Reviewing the code I see two areas that would need addressed: 1) `InitVar` needs changed so `Annotated[InitVar[...]]` is no longer a runtime error. This is currently a runtime error completely by accident: typing.py expects special type forms to be `callable()`, usually by implementing a `__call__` that throws an error, but `InitVar` does not implement this. Adding an implementation like in typing.py would fix the runtime error: https://github.com/python/cpython/blob/b1a3446f077b7d56b89f55d98dadb8018986a3e5/Lib/typing.py#L391-L392 2) The dataclasses-specific typehint introspection implementation needs modified to accommodate being wrapped by an `Annotated` annotation. I see in the comments the code is performance sensitive so I'm not sure what you want to do; f.ex. the regex needs modified, but it's not clean. What are your thoughts? -- components: Library (Lib) messages: 411567 nosy: GBeauregard, Jelle Zijlstra, eric.smith priority: normal severity: normal status: open title: dataclasses: Allow typing.Annotated to wrap dataclasses-specific annotations type: behavior versions: Python 3.10, Python 3.11, Python 3.9 ___ Python tracker <https://bugs.python.org/issue46511> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46491] typing: allow Annotated in outermost scope
Change by Gregory Beauregard : -- keywords: +patch pull_requests: +29045 stage: -> patch review pull_request: https://github.com/python/cpython/pull/30864 ___ Python tracker <https://bugs.python.org/issue46491> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46491] typing: allow Annotated in outermost scope
New submission from Gregory Beauregard : Currently, `typing.Annotated` (PEP 593) cannot be used at runtime with `typing.Final` and `typing.ClassVar` with `Annotated` on the outside: ``` from typing import Annotated, Final # TypeError: typing.Final[int] is not valid as type argument var: Annotated[Final[int], "foo"] = 4 ``` The only tenuously related mention of this I can find in a PEP is in PEP 593 (Annotated) which states "The first argument to Annotated must be a valid type". I believe the runtime behavior should be changed to allow any ordering for `Annotated` with `ClassVar` and `Final`. This was discussed in the typing-sig PEP 655 thread (TypedDict `Required` and `NotRequired`) where the current plan is to allow `Required`/`NotRequired` in any nesting order with `Annotated` while suggesting the `ClassVar`/`Final` ordering restriction be lifted: https://mail.python.org/archives/list/typing-...@python.org/message/22CJ5TJGIJ563D6ZKB7R3VUZXTZQND5X/ The argument for doing so is on the mailing list: https://mail.python.org/archives/list/typing-...@python.org/message/MPMOIBX3XFXCD4ZNDC6AV4CLSI5LN544/ To summarize: adopting an overly strict view of what constitutes a valid type for `Annotated` creates difficulties for people who use runtime introspection of `Annotated` annotations by requiring them to parse additional typing or field annotations (https://bugs.python.org/msg411067). This needlessly exacerbates tension between typing and non-typing uses of annotation space. In order to be a good citizen to other annotation users, the `Annotated` runtime ordering restriction should be lifted. -- messages: 411404 nosy: GBeauregard priority: normal severity: normal status: open title: typing: allow Annotated in outermost scope type: behavior versions: Python 3.10, Python 3.11, Python 3.9 ___ Python tracker <https://bugs.python.org/issue46491> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45384] Accept Final as indicating ClassVar for dataclass
Gregory Beauregard added the comment: Hi Michael, Thanks for taking the time to look into this. I don't feel strongly enough about following the existing PEP wording to desire creating a situation where instance vars can't be marked Final if we can instead make a workaround with ClassVar[Final[...]]. Plus, I can appreciate the argument that dataclasses are their own thing that should be treated specially in the PEP. If you know what's involved in formally proposing and enacting a PEP amendment I can get behind that. -- ___ Python tracker <https://bugs.python.org/issue45384> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45443] 'ThreadPoolExecutor' object has no attribute 'map'
Gregory Beauregard added the comment: I get no attribute error with this code in 3.9 or 3.10. -- nosy: +GBeauregard ___ Python tracker <https://bugs.python.org/issue45443> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45384] Accept Final as indicating ClassVar for dataclass
Gregory Beauregard added the comment: Thanks for the feedback Carl. Your proposed nesting PEP change is not possible: ClassVar[Final[int]] isn't valid because Final[int] isn't a type. As far as I know this would need type intersections to be possible. I'm going to try sending a one-off email to the PEP authors since probably whatever happens the PEP needs a clarification anyway. -- ___ Python tracker <https://bugs.python.org/issue45384> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45384] Accept Final as indicating ClassVar for dataclass
Gregory Beauregard added the comment: Yeah, I was just discussing this with someone in IRC, and I appreciate an example of it in the wild. I agree there's some wiggle room with what "initialized in a class body" means when it comes to dataclasses. I see several interpretations there, and I would strongly prefer feedback from typing folks, particularly since they would be responsible for implementing any Final default_factory exceptions. On the implementation side this does complicate things a bit depending on specifics. Are Final default_factory fields real fields or pseudo-fields? (i.e. are they returned by dataclasses.fields()?) Depending on how this works dataclasses might need a bit more refactoring than I'd be the right person for, but I'm still willing to give it a shot. -- ___ Python tracker <https://bugs.python.org/issue45384> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45384] Accept Final as indicating ClassVar for dataclass
Gregory Beauregard added the comment: When I originally submitted the issue I hadn't finished going through all of the dataclasses code and it hadn't even occurred to me that it could be valid to use ClassVar with field(). I (wrongly) assumed this would always raise and that field() is only valid for things intended to be instance vars. I do find this behavior a little surprising, but on reflection I don't think it's explicitly wrong as long we raise for default_factory like it currently does. I think it's then appropriate to just do the exact same behavior for Final as ClassVar. I'm going to start working on a PR, thanks for your feedback. -- ___ Python tracker <https://bugs.python.org/issue45384> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45384] Accept Final as indicating ClassVar for dataclass
Gregory Beauregard added the comment: Hi Eric, I've been shopping this idea around on the mailing list and haven't received any objections. Do you have any concerns? Can we handle Final with the same checks as ClassVar? Regarding potentially merging a change, I'm not sure where this falls between a bug fix and a feature. On one hand the PEP 591 instruction to typecheckers on how to treat Final is relatively absolute and the dataclasses behavior could be considered a buggy interaction with it. On the other hand this is sorta a new behavior. Do you have any opinions? Should I not worry about it when working on patch and let core devs figure out if it would need backported? I've read through the dataclasses code and I think I can implement this myself and submit a PR, but I may need a bit of a heavy handed code review since I've only recently started getting serious with Python. Thanks for your time and work on dataclassess. -- ___ Python tracker <https://bugs.python.org/issue45384> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45384] Accept Final as indicating ClassVar for dataclass
Change by Gregory Beauregard : -- type: enhancement -> behavior ___ Python tracker <https://bugs.python.org/issue45384> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45384] Accept Final as indicating ClassVar for dataclass
New submission from Gregory Beauregard : PEP 591 for the Final Attribute states "Type checkers should infer a final attribute that is initialized in a class body as being a class variable. Variables should not be annotated with both ClassVar and Final." This is a bit of a typing conflict for dataclasses, where ClassVar is used to indicate a desired library behavior, but one may want to indicate Final. I propose accepting the Final attribute as an indicator of a ClassVar in dataclasses class bodies in order to be better compatible with the Final PEP. There is at least one edge case that would need to be handled where someone might want to explicitly mark a dataclass field Final, which could be allowed as a field: a: Final[int] = dataclasses.field(init=False, default=10) -- components: Library (Lib) messages: 403277 nosy: GBeauregard priority: normal severity: normal status: open title: Accept Final as indicating ClassVar for dataclass type: enhancement ___ Python tracker <https://bugs.python.org/issue45384> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
