[issue44353] PEP 604 NewType

[miss-islington]

miss-islington  added the comment:


New changeset 3f024e27c29a57dd4f805aa2431d713ed0fe57b2 by Miss Islington (bot) 
in branch '3.10':
bpo-44353: Correct docstring for `NewType` (GH-29785)
https://github.com/python/cpython/commit/3f024e27c29a57dd4f805aa2431d713ed0fe57b2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[miss-islington]

Change by miss-islington :


--
pull_requests: +28031
pull_request: https://github.com/python/cpython/pull/29796

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Guido van Rossum]

Guido van Rossum  added the comment:


New changeset 93c65df83cef71a4bc77d71afecdec8744c4f73a by Alex Waygood in 
branch 'main':
bpo-44353: Correct docstring for `NewType` (#29785)
https://github.com/python/cpython/commit/93c65df83cef71a4bc77d71afecdec8744c4f73a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Alex Waygood]

Change by Alex Waygood :


--
nosy: +AlexWaygood
nosy_count: 12.0 -> 13.0
pull_requests: +28022
pull_request: https://github.com/python/cpython/pull/29785

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Ken Jin]

Ken Jin  added the comment:

Thanks a bunch Yurii, Serhiy, Jelle, Łukasz and Pablo for working on this! I'm 
re-closing this issue. *Fingers-crossed* we won't have to open this again ;-).

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I am closing this one again, apologies for the confusion.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Bisecting points to:

fe13f0b0f696464dd6f283576668dbf57cb11399 is the first bad commit
commit fe13f0b0f696464dd6f283576668dbf57cb11399
Author: Yurii Karabas <1998uri...@gmail.com>
Date:   Fri Jul 23 12:47:00 2021 +0300

bpo-44676: Add ability to serialize types.Union (GH-27244)

 Lib/test/test_types.py | 35 +++
 Lib/typing.py  |  4 +-
 .../2021-07-19-19-53-46.bpo-44676.WgIMvh.rst   |  2 +
 Objects/unionobject.c  | 51 ++
 4 files changed, 90 insertions(+), 2 deletions(-)
 create mode 100644 Misc/NEWS.d/next/Core and 
Builtins/2021-07-19-19-53-46.bpo-44676.WgIMvh.rst
bisect run success

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Łukasz Langa]

Łukasz Langa  added the comment:

types is not typing. This is a different failure from what we've seen before. 
I'll fix that today.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Pick any of the failing refleak buildbots. For example:

https://buildbot.python.org/all/#/builders/280/builds/99

You will see:

--
Ran 106 tests in 0.074s
OK
..
test_types leaked [2, 2, 2] references, sum=6
test_types leaked [1, 1, 1] memory blocks, sum=3
1 test failed again:
test_types

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Ken Jin]

Ken Jin  added the comment:

@Pablo, I don't think this change is causing the buildbots to fail. The test 
failure on all the currently failing buildbots is:

==
ERROR: test_absolute_circular_submodule (test.test_import.CircularImportTests)
--
Traceback (most recent call last):
  File 
"/buildbot/buildarea/cpython/3.x.ware-gentoo-x86.installed/build/target/lib/python3.11/test/test_import/__init__.py",
 line 1355, in test_absolute_circular_submodule
import test.test_import.data.circular_imports.subpkg2.parent

ModuleNotFoundError: No module named 
'test.test_import.data.circular_imports.subpkg2'
--

Which seems to be GH-27299.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

The buildbots are still not green. Unfortunately we will need to revert all PRs 
if this is not fixed soon as this is already making other issues

--
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset e8c01749c02a97a2f5b01086951bb564121c5113 by Miss Islington (bot) 
in branch '3.10':
bpo-44353: Improve tests covering typing.NewType pickling (GH-27302) (GH-27328)
https://github.com/python/cpython/commit/e8c01749c02a97a2f5b01086951bb564121c5113


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[miss-islington]

Change by miss-islington :


--
pull_requests: +25872
pull_request: https://github.com/python/cpython/pull/27328

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 05f5d8e48c86c8025d3f82285666d93e52e360f9 by Łukasz Langa in 
branch '3.10':
[3.10] bpo-44353: Expand NewType tests for complex __qualname__ (GH-27311) 
(GH-27326)
https://github.com/python/cpython/commit/05f5d8e48c86c8025d3f82285666d93e52e360f9


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset a22b05da87bdfb081d6aaecfce136ae8dbb8680c by Yurii Karabas in 
branch 'main':
bpo-44353: Improve tests covering typing.NewType pickling (GH-27302)
https://github.com/python/cpython/commit/a22b05da87bdfb081d6aaecfce136ae8dbb8680c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Łukasz Langa]

Change by Łukasz Langa :


--
pull_requests: +25870
pull_request: https://github.com/python/cpython/pull/27326

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset d15949a845a6db66675bca7105ad508ba9e79639 by Miss Islington (bot) 
in branch '3.10':
bpo-44353: Document that typing.NewType is now a class (GH-27319) (GH-27321)
https://github.com/python/cpython/commit/d15949a845a6db66675bca7105ad508ba9e79639


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[miss-islington]

Change by miss-islington :


--
pull_requests: +25866
pull_request: https://github.com/python/cpython/pull/27321

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 7aac3f623610cf3dffbf548a5be5bfd4fa6790a0 by Ken Jin in branch 
'main':
bpo-44353: Document that typing.NewType is now a class (#27319)
https://github.com/python/cpython/commit/7aac3f623610cf3dffbf548a5be5bfd4fa6790a0


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Ken Jin]

Change by Ken Jin :


--
pull_requests: +25865
pull_request: https://github.com/python/cpython/pull/27319

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset e89ef0ad2a299770a88ece8f7a316f7d3eb65c9f by Serhiy Storchaka in 
branch 'main':
bpo-44353: Expand NewType tests for complex __qualname__. (#27311)
https://github.com/python/cpython/commit/e89ef0ad2a299770a88ece8f7a316f7d3eb65c9f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +25864
pull_request: https://github.com/python/cpython/pull/9951

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

PR 27311 copies tests from PR 9951, adds support of nested NewTypes (with 
__qualname__ containing multiple components) an makes them pickleable by name 
as functions and classes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +25856
pull_request: https://github.com/python/cpython/pull/27311

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Łukasz Langa]

Change by Łukasz Langa :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 8f42106b5c362495f72c6ca2fa3884538e4023db by Yurii Karabas in 
branch 'main':
bpo-44353: Fix memory leak introduced by GH-27262 (GH-27305)
https://github.com/python/cpython/commit/8f42106b5c362495f72c6ca2fa3884538e4023db


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Yurii Karabas]

Change by Yurii Karabas <1998uri...@gmail.com>:


--
pull_requests: +25849
stage: resolved -> patch review
pull_request: https://github.com/python/cpython/pull/27305

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

PR27262 introduced reference leaks and currently all refleak buildbots are red:

Example:

https://buildbot.python.org/all/#/builders/205/builds/102/steps/5/logs/stdio

Ran 367 tests in 0.198s
OK (skipped=1)
.
test_typing leaked [220, 220, 220] references, sum=660
test_typing leaked [69, 68, 68] memory blocks, sum=205
1 test failed again:
test_typing
== Tests result: FAILURE then FAILURE ==
Per our buildbot policy, if a fix is not performed in 24 h we will need to 
revert

--
nosy: +pablogsal
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Yurii Karabas]

Change by Yurii Karabas <1998uri...@gmail.com>:


--
pull_requests: +25846
pull_request: https://github.com/python/cpython/pull/27302

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Łukasz Langa]

Łukasz Langa  added the comment:

Note: we won't be backporting _typing to Python 3.10 as it is much too late for 
a new module at this point in the life cycle. Consequently, 3.10 will be a 
(temporary) performance regression in terms of typing.NewType.

Thanks everyone, most of all Josepth for reporting and Yurii for implementing 
the required changes.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 96c4cbd96c769e92869c62ba898dd9eb670baa81 by Yurii Karabas in 
branch 'main':
bpo-44353: Implement typing.NewType __call__ method in C (#27262)
https://github.com/python/cpython/commit/96c4cbd96c769e92869c62ba898dd9eb670baa81


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Denis Laxalde]

Change by Denis Laxalde :


--
nosy: +dlax

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 9ae5ba7dbf08d56a1b30d67fcde75532fe136d77 by Miss Islington (bot) 
in branch '3.10':
bpo-44353: Add test to cover __or__ of two NewType (GH-27259) (#27261)
https://github.com/python/cpython/commit/9ae5ba7dbf08d56a1b30d67fcde75532fe136d77


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Yurii Karabas]

Change by Yurii Karabas <1998uri...@gmail.com>:


--
pull_requests: +25807
pull_request: https://github.com/python/cpython/pull/27262

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[miss-islington]

Change by miss-islington :


--
pull_requests: +25806
pull_request: https://github.com/python/cpython/pull/27261

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset c2f33dfc83ab270412bf243fb21f724037effa1a by Miss Islington (bot) 
in branch '3.10':
bpo-44353: Refactor typing.NewType into callable class (GH-27250) (#27258)
https://github.com/python/cpython/commit/c2f33dfc83ab270412bf243fb21f724037effa1a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Yurii Karabas]

Yurii Karabas <1998uri...@gmail.com> added the comment:

Jelle thanks for pointing this out, I will implement helper function in C and 
use it as a __call__ method for NewType

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Jelle Zijlstra]

Jelle Zijlstra  added the comment:

I found that replacing __call__ on the NewType class with an identity function 
written in C makes things faster instead:

In [54]: %timeit ntc2(1)
79 ns ± 0.37 ns per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [55]: %timeit ntc(1)
126 ns ± 0.315 ns per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [56]: %timeit nt(1)
103 ns ± 4.23 ns per loop (mean ± std. dev. of 7 runs, 1000 loops each)

Here ntc2 has __call__ implemented in C, ntc is the previous class-based 
version and nt is the current function-based version.

So perhaps we could just stick a private `._idfunc` in some C-implemented 
module (functools? types?) and use it as the __call__ for our NewType class.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Guido van Rossum]

Guido van Rossum  added the comment:

OTOH, for 3.10 the clock is ticking. Let's go for it. (If it's deemed too slow, 
we could eventually add an accelerator just for this.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Łukasz Langa]

Łukasz Langa  added the comment:

> Hm, 20% isn't so bad, but one of the arguments for NewType was that it 
> "disappears" at runtime -- which the current version does, but the 
> class-based version doesn't quite.

NewType was described originally in https://github.com/python/typing/issues/189 
and the resulting PEP text landed in 
https://github.com/python/typing/pull/226/files.

I find the 20% performance impact non-ideal but ultimately worth it because:
- it solves the issue described in this bug;
- it allows us to implement issubclass/isinstance in the future if we so choose;
- it also potentially allows us to chain NewType.

Since we can't fully rely on `from __future__ import annotations` to side-step 
the performance cost, we don't really have a different option than to change 
it. The alternative is to leave it as is which makes it a non-composable 
pseudo-type unlike the others.

Currently adoption of NewType is relatively low, in part due to the feature's 
obscurity and partially because of its limits. Slower instantiation performance 
will at worst keep the adoption low, but it can potentially bring more users to 
NewType since it becomes less hacky.

I'm +1 to converting to a class as done by the PR.


> Right now I don't have the cycles to think about this deeply. Maybe a few 
> weeks after I'm back from vacation (~August 8) I will have more time.

Sorry, I overeagerly already merged the change to `main`. Let's hold off with 
the 3.10 backport until you had time to think about it. If we decide against 
the change after all, reverting on `main` should have no negative impact since 
we're pre-alpha 1.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 4868b94c6089d457673b1ba5b5b64c2f38c435af by Yurii Karabas in 
branch 'main':
bpo-44353: Add test to cover __or__ of two NewType (#27259)
https://github.com/python/cpython/commit/4868b94c6089d457673b1ba5b5b64c2f38c435af


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Yurii Karabas]

Change by Yurii Karabas <1998uri...@gmail.com>:


--
pull_requests: +25804
pull_request: https://github.com/python/cpython/pull/27259

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 8.0 -> 9.0
pull_requests: +25803
stage: resolved -> patch review
pull_request: https://github.com/python/cpython/pull/27258

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 965dd76e9060e27e2253ba8c8d21a142b178720d by Yurii Karabas in 
branch 'main':
bpo-44353: Refactor typing.NewType into callable class (GH-27250)
https://github.com/python/cpython/commit/965dd76e9060e27e2253ba8c8d21a142b178720d


--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Ken Jin]

Ken Jin  added the comment:

> Does that work if you try to union two NewTypes?

Oh woops I was too eager in closing this issue -- it doesn't work. Sorry, 
please disregard my previous message. I'm reopening this. Thanks for the 
reminder Jelle.

--
resolution: out of date -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Jelle Zijlstra]

Jelle Zijlstra  added the comment:

Does that work if you try to union two NewTypes?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Ken Jin]

Ken Jin  added the comment:

This issue is now out of date on. After Serhiy's refactoring, any function 
types can be unioned.

>>> NewType('x', int)
>>> int | NewType('x', int)
int | typing.NewType..new_type

The only problem now is that the repr is weird, but Serhiy also has a fix for 
that in issue34963 (PR 9951) without converting to a class.

Thus, I am closing the issue. Please go to issue44642 for a general discussion 
on whether union should support arbitrary functions at all, or issue34963 for 
the repr problem. Thanks all!

--
resolution:  -> out of date
stage: patch review -> resolved
status: open -> closed
superseder:  -> Union of a type and the typing module function

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Guido van Rossum]

Guido van Rossum  added the comment:

Hm, 20% isn't so bad, but one of the arguments for NewType was that it 
"disappears" at runtime -- which the current version does, but the class-based 
version doesn't quite.

Right now I don't have the cycles to think about this deeply. Maybe a few weeks 
after I'm back from vacation (~August 8) I will have more time.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Jelle Zijlstra]

Jelle Zijlstra  added the comment:

This is what I got last year (copied from the typing issues):

In [83]: from typing import NewType

In [84]: nt = NewType("nt", int)

In [85]: class NewTypeClass:
...: def __init__(self, name, supertype):
...: self.name = name
...: self.supertype = supertype
...: def __call__(self, obj):
...: return obj
...: def __repr__(self):
...: return f""
...: 

In [86]: ntc = NewTypeClass("ntc", int)

In [87]: ntc
Out[87]: 

In [88]: %timeit nt(3)
211 ns ± 2.27 ns per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [89]: %timeit ntc(3)
253 ns ± 5.35 ns per loop (mean ± std. dev. of 7 runs, 100 loops each)


Not sure what version of Python I used at the time. I just repeated it on 3.9.4 
MacOS and got much faster time but a similar ~20% slowdown:

In [6]: %timeit nt(3)
112 ns ± 5.11 ns per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [7]: %timeit ntc(3)
136 ns ± 2.36 ns per loop (mean ± std. dev. of 7 runs, 1000 loops each)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Guido van Rossum]

Guido van Rossum  added the comment:

Jelle, can you post more details about your benchmark?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Yurii Karabas]

Yurii Karabas <1998uri...@gmail.com> added the comment:

I opened PR with refactored `typing.NewType` into callable class.

Also I have added `__module__` attr to typing.NewType so it will become 
pickable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Yurii Karabas]

Change by Yurii Karabas <1998uri...@gmail.com>:


--
keywords: +patch
pull_requests: +25798
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/27250

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Jelle Zijlstra]

Jelle Zijlstra  added the comment:

> Using `from __future__ import annotations` it should work

Not in type aliases or generic bases. (And if Larry's co_annotations PEP is 
accepted, it would break in normal annotations too.) I'd prefer to get rid of 
this sort of subtle difference where the obvious way to write a type only works 
in some contexts, though I realize that's not possible in all cases.

That means I think we should move forward with making NewTypes into instances 
with __call__ instead of functions, though according to my benchmarks in the 
typing issue that will incur a small performance penalty.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Guido van Rossum]

Guido van Rossum  added the comment:

Do we have to solve it? Using `from __future__ import annotations` it should 
work, at least when used as an annotation. Or is that too inconvenient or 
inconsistent?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Yurii Karabas]

Yurii Karabas <1998uri...@gmail.com> added the comment:

@Ken thanks for pointing this out.

The only solution which I see is to convert NewType into callable class but it 
will lead to performance degradation.

@Ken what do you think about that?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Ken Jin]

Ken Jin  added the comment:

@Yurii, I would like to caution against a C accelerator for the typing module. 
My reasoning follows:

1. The maintenance burden is higher. typing is already somewhat complex in 
Python (lots of MRO/metaclass wizardry). A C module would require knowledge of 
the C API.

2. Following from 1., I fear it may raise the barrier for contributions. This 
is purely anecdotal but I think there are more contributors who know Python 
than Python + C API.

3. C API code is much more error-prone than Python code.

4. It's very hard to find available reviewers for typing-related C changes.

5. Backports become harder due to point 3. and 4. Also C code needs much more 
scrutiny. If we cause some obscure bug in Python, it raises an exception; in C 
it potentially segfaults and kills the interpreter.

6. Third-party monkey-patching becomes harder.

Unfortunately, I can't offer a good solution to this issue at the moment either.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Yurii Karabas]

Yurii Karabas <1998uri...@gmail.com> added the comment:

Jelle I can try to implement it in C to see if it will give better performance.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Jelle Zijlstra]

Jelle Zijlstra  added the comment:

https://github.com/python/typing/issues/746 has some previous discussion of 
implementing NewType as a class (motivated by __repr__), including benchmarks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Ken Jin]

Change by Ken Jin :


--
nosy: +Jelle Zijlstra, gvanrossum, kj, levkivskyi
versions: +Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Yurii Karabas]

Yurii Karabas <1998uri...@gmail.com> added the comment:

What about to return callable object instead of function from a 
`typing.NewType`?

It will look something like this:
```
class _NewType:
__slots__ = ('__name__', '__supertype__')

def __init__(self, name, supertype):
self.__name__ = name
self.__supertype__ = supertype

def __call__(self, val):
return val

def __or__(self, other):
return Union[self, other]

def __ror__(self, other):
return Union[other, self]



def NewType(name, tp):
"""NewType creates simple unique types with almost zero
runtime overhead. NewType(name, tp) is considered a subtype of tp
by static type checkers. At runtime, NewType(name, tp) returns
a dummy callable object that simply returns its argument. Usage::

UserId = NewType('UserId', int)

def name_by_id(user_id: UserId) -> str:
...

UserId('user')  # Fails type check

name_by_id(42)  # Fails type check
name_by_id(UserId(42))  # OK

num = UserId(5) + 1 # type: int
"""
return _NewType(name, tp)
``` 

With such implementation `__or__` will be available for a NewType and actually 
because of __slots__ size of object will be 48 bytes (with sys.getsizeof) which 
is less than current 144 bytes (if memory is important).

--
nosy: +uriyyo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Dominic Davis-Foster]

Dominic Davis-Foster  added the comment:

It is possible to implement NewType to return a class rather than a function as 
it does currently. However, the class version uses substantially more memory 
(1072 bytes vs 144 bytes with sys.getsizeof) and NewType is supposed to have 
almost no runtime overhead.

--
nosy: +domdfcoding

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[Joseph Perez]

New submission from Joseph Perez :

`typing.NewType` doesn't support PEP 604.

```
>>> import typing
>>> A = typing.NewType("A", int)
>>> B = typing.NewType("B", str)
>>> A | B
Traceback (most recent call last):
  File "", line 1, in 
TypeError: unsupported operand type(s) for |: 'function' and 'function'
```

--
messages: 395359
nosy: joperez
priority: normal
severity: normal
status: open
title: PEP 604 NewType
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com