[issue46644] typing: remove callable() check from typing._type_check

[Yurii Karabas]

Change by Yurii Karabas <1998uri...@gmail.com>:


--
nosy: +uriyyo
nosy_count: 8.0 -> 9.0
pull_requests: +29939
pull_request: https://github.com/python/cpython/pull/27553

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Jelle Zijlstra]

Change by Jelle Zijlstra :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Jelle Zijlstra]

Jelle Zijlstra  added the comment:


New changeset 75174371e6cac935b598a68c1113f6db1e0d6ed8 by Jelle Zijlstra in 
branch 'main':
bpo-46644: Fix test_typing test broken by GH-31151 due to a merge race 
(GH-31833)
https://github.com/python/cpython/commit/75174371e6cac935b598a68c1113f6db1e0d6ed8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Jelle Zijlstra]

Change by Jelle Zijlstra :


--
pull_requests: +29930
stage: resolved -> patch review
pull_request: https://github.com/python/cpython/pull/31833

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Jelle Zijlstra]

Jelle Zijlstra  added the comment:

Some tests are failing on main, probably due to a race. PR incoming.

--
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Jelle Zijlstra]

Jelle Zijlstra  added the comment:

Thanks for your contribution!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Jelle Zijlstra]

Jelle Zijlstra  added the comment:


New changeset 870b22b9c442d035190d2b8fb82256cd9a03da48 by Gregory Beauregard in 
branch 'main':
bpo-46644: Remove callable() requirement from typing._type_check (GH-31151)
https://github.com/python/cpython/commit/870b22b9c442d035190d2b8fb82256cd9a03da48


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Guido van Rossum]

Change by Guido van Rossum :


--
nosy:  -gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Gregory Beauregard]

Gregory Beauregard  added the comment:

I compiled your PR to run it and was testing in 3.10 as well, but I was testing 
in a file with from __future__ import annotations unintentionally. I retract 
the comment. It turns out `list[42]` is okay though, which I suppose is more 
relevant going forward. My confusion here is sort of the crux of my problem 
with these runtime checks: they are inconsistently applied in different 
locations which is why callable() was causing a lot of bugs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

But it gives the same result.

What version of Python do you test with?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Gregory Beauregard]

Gregory Beauregard  added the comment:

I'm referring to within type annotations, where this code path isn't used: try 
a: List[42]

This code path can show up in type aliases though.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

No, List[42] is not currently accepted.

>>> List[42]
Traceback (most recent call last):
  File "", line 1, in 
  File "/home/serhiy/py/cpython/Lib/typing.py", line 318, in inner
return func(*args, **kwds)
   ^^^
  File "/home/serhiy/py/cpython/Lib/typing.py", line 1127, in __getitem__
params = tuple(_type_check(p, msg) for p in params)
 ^^
  File "/home/serhiy/py/cpython/Lib/typing.py", line 1127, in 
params = tuple(_type_check(p, msg) for p in params)
   ^^^
  File "/home/serhiy/py/cpython/Lib/typing.py", line 184, in _type_check
raise TypeError(f"{msg} Got {arg!r:.100}.")
^^^
TypeError: Parameters to generic types must be types. Got 42.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Gregory Beauregard]

Gregory Beauregard  added the comment:

List[42] is already accepted, and your proposed patch does not change it to 
make it not accepted. The issue is _type_check is only called in a few 
particular locations; this is part of the technical reason I'm not very 
concerned about relaxing the _type_check requirements.

>From a type checking philosophy point of view I agree with Jelle and am 
>negative on strict runtime requirements in typing.py.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +29332
pull_request: https://github.com/python/cpython/pull/31159

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

There were two reasons of accepting arbitrary callables in _type_check():

1. NewType() returned a function.
2. xml.etree.cElementTree.Element was a function in Python 2.

Now NewType is a class, and Python 2 no longer supported. I agree that we 
should get rid of the callable() check, but I disagree with the proposed 
change. The check should be more explicit and strict, not more lenient. 
List[42] does not make sense and should not be accepted.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Gregory Beauregard]

Change by Gregory Beauregard :


--
keywords: +patch
pull_requests: +29328
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31151

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Gregory Beauregard]

Gregory Beauregard  added the comment:

I made a draft pull request where I went ahead and added a check to disallow 
tuple literals. This is basically already disallowed for types by restrictions 
on `__getitem__` because Union[typeform]->type needs to be different from 
Union[type,type]->Union[type,type].

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Gregory Beauregard]

Gregory Beauregard  added the comment:

Further questions: the msg argument in _type_check now wouldn't be used for 
anything! It was only used in the case where a type wasn't callable(). I think 
it should be removed. I'm also a bit negative on disallowing tuples in the case 
of e.g. Final and such since it complicates implementing tuple types in Python 
down the line if desired.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Gregory Beauregard]

Gregory Beauregard  added the comment:

Under the same failing int test cases before there were 2 more cases next to 
them that fail:

with self.assertRaises(TypeError):
ClassVar[int, str]
with self.assertRaises(TypeError):
Final[int, str]

These fail because tuple literals are not callable(). There is code that 
clearly intends for this to be the case: 
https://github.com/python/cpython/blob/96b344c2f15cb09251018f57f19643fe20637392/Lib/typing.py#L486

I can either remove support for this runtime check or change the implementation 
of Final et al to reject tuple literals. I will do the latter for now.

For 
https://github.com/python/cpython/blob/bf95ff91f2c1fc5a57190491f9ccdc63458b089e/Lib/test/test_typing.py#L4262-L4263
 I think the best approach is to just remove these tests.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Jelle Zijlstra]

Jelle Zijlstra  added the comment:

I agree with removing this check. I suspect it's a holdover from very early 
typing when static types were supposed to be runtime types. Now the check is a 
bug magnet and doesn't serve a useful purpose.

I think we can just remove the tests that check for ints. I don't see a 
principled reason to special-case int literals.

I wonder if we should apply this change to 3.10 and 3.9. It's arguably a 
bugfix, but it's a pretty big change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Gobot1234]

Gobot1234  added the comment:

I also support this change. I've had to write a lot of code to make 
SpecialForms able to accept my types where the code has to look like:
```py
class Something:
...
def __call__(self, *args, **kwargs):
   raise NotImplementedError
```
I also know this comes up in typing-extensions a fair bit. I think type 
checkers should be enforcing this at type-checking-time not by typing.py 
run-time.

--
nosy: +Gobot1234

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Gregory Beauregard]

Gregory Beauregard  added the comment:

In addition to the 10 tests failed in test_typing.py, one additional test fails 
in test_types.py with this change: 
https://github.com/python/cpython/blob/bf95ff91f2c1fc5a57190491f9ccdc63458b089e/Lib/test/test_types.py#L834-L838
 and those are all the tests in cpython changed.

This falls in category (1), checking that an int literal is not a type, but 
with the apparent intent to prevent index subscripting.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Gregory Beauregard]

New submission from Gregory Beauregard :

I propose removing the callable() check[1] from typing._type_check. This 
restriction is usually met in typeform instances by implementing a __call__ 
method that raises at runtime[2]. _type_check is called at runtime in a few 
disparate locations, such as in an argument to typing.Annotated or for certain 
stringified annotations in typing.get_type_hints.

Because the requirement to be callable is unexpected and shows up in situations 
not easily discoverable during development or common typing usage, it is the 
cause of several existing cpython bugs and will likely continue to be the cause 
of bugs in typeforms outside of cpython. Known cpython bugs caused by the 
callable() check are bpo-46643, bpo-44799, a substantial contributing factor to 
bpo-46642, and partly bpo-46511. I discovered bpo-46643 with only a cursory 
check of typing.py while writing this proposal. Moreover, it doesn't make any 
particular technical sense to me why it should be required to add an awkward 
__call__ method.

Removing the callable() check fails 10 tests:
7 tests: checking that an int literal is not a type
2 tests: testing that list literals are not valid types (e.g. [3] raises a 
TypeError because the literal [('name', str), ('id', int)] doesn't pass 
callable())
1 test: bpo-46642

The responsibility of determining these invalid typeforms (e.g. int literals) 
would need to be passed to a static type checker. If it's desired to do this at 
runtime it's my opinion that a different check would be more appropriate.

Have I missed any reasons for the callable() check? Can I remove the check and 
adjust or remove the tests?

[1] 
https://github.com/python/cpython/blob/bf95ff91f2c1fc5a57190491f9ccdc63458b089e/Lib/typing.py#L183-L184
[2] 
https://github.com/python/cpython/blob/bf95ff91f2c1fc5a57190491f9ccdc63458b089e/Lib/typing.py#L392-L393
[3] 
https://github.com/python/cpython/blob/bf95ff91f2c1fc5a57190491f9ccdc63458b089e/Lib/test/test_typing.py#L4262-L4263

--
components: Library (Lib)
messages: 412548
nosy: AlexWaygood, GBeauregard, Jelle Zijlstra, gvanrossum, kj, levkivskyi, 
sobolevn
priority: normal
severity: normal
status: open
title: typing: remove callable() check from typing._type_check
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com