[issue32226] Implement PEP 560: Core support for typing module and generic types

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:

OK, I will close this issue, and open separate issues for documentation, Union, 
etc.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Guido van Rossum]

Guido van Rossum  added the comment:

ISTM that those ought to be separate issues since PEP 560 has been
implemented fully here.

Also I am getting cold feet about Union simplification (esp. this late in
the release cycle). We should probably retreat on the typing tracker and
discuss why we're considering it -- is there evidence (even anecdotal) that
it's harmful?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:

https://github.com/python/typing/issues/512 and 
https://github.com/python/typing/issues/511 are first issue (they are closely 
related to each other). This is not directly related to PEP 560, but changes in 
typing because of it will allow to fix them in a simple manner.

The second one is Union simplification (was discussed in few PRs on typing 
tracker). This is not really a bug, but something we should decide on before 
3.7 final. It will be difficult to change it later. Again, with PEP 560 in 
place the corresponding refactoring for removal should be straightforward.

In addition, there is https://github.com/python/typing/issues/392, this is 
already almost fixed by the typing PR implementing PEP 560, but it doesn't work 
for dunder attributes (I think we should not relay only specific set of 
reserved names, rather than all dunders).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Guido van Rossum]

Guido van Rossum  added the comment:

Are there more specific descriptions of those bugs? Links to a tracker?
(Even if it's the typing tracker.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:

I think all the critical things have been implemented/fixed. There are 
unfortunately no docs yet (my fault), also there are two bugs related to PEP 
560, but they are not new, they also exist in Python 3.6.

I am however marking this as high priority, since these two bugs prevent 
progress on other projects (pickling generic classes), so it would be good to 
fix them before beta-2.

--
priority: release blocker -> high

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Ned Deily]

Ned Deily  added the comment:

What's the status of this?  The issue is still open with "release blocker" 
status.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:


New changeset d911e40e788fb679723d78b6ea11cabf46caed5a by Ivan Levkivskyi in 
branch 'master':
bpo-32226: PEP 560: improve typing module (#4906)
https://github.com/python/cpython/commit/d911e40e788fb679723d78b6ea11cabf46caed5a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[STINNER Victor]

STINNER Victor  added the comment:


New changeset e860089fd93728824cf586fa4d91f08eff3bab73 by Victor Stinner in 
branch 'master':
bpo-32226: Fix memory leak in generic_alias_dealloc() (#5212)
https://github.com/python/cpython/commit/e860089fd93728824cf586fa4d91f08eff3bab73


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[STINNER Victor]

STINNER Victor  added the comment:

Oh ok, I think that I found the bug: PR 5212.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +5066

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[STINNER Victor]

STINNER Victor  added the comment:

Since the commit 45700fb757591a672e9d25b8252971c2a2caeaf2, test_genericclass 
leaks references:
---
vstinner@apu$ ./python -m test -R 3:3 test_genericclass -m 
test.test_genericclass.CAPITest.test_c_class
Run tests sequentially
0:00:00 load avg: 1.27 [1/1] test_genericclass
beginning 6 repetitions
123456
..
test_genericclass leaked [2, 3, 2] memory blocks, sum=7
test_genericclass failed

1 test failed:
test_genericclass

Total duration: 107 ms
Tests result: FAILURE
---

The leak comes from _testcapi.Generic[int]. Maybe from generic_alias_new() of 
Modules/_testcapi.c.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Sorry, I know very small about this module to writing the documentation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:

Serhiy,

I am sorry for a delay, I have recently moved to another country, this is why I 
have not much time. I will try to work on this weekend. Here are some points 
where you can be helpful:

* Make a short documentation PR for `__mro_entry__` and `__base_subclass__` 
methods.
* Document the `__class_getitem__` C API calling convention (with motivation, 
many classes are implemented in C but are generic in nature, like 
numpy.ndarray) in PEP 560.

(I know this might be not very interesting, but this will save time for me to 
focus only on typing part.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Can I help?

I'm implementing the feature related to pickling classes, and want to implement 
pickling generic types as an example. But since PEP 560 will change all here, I 
need to wait this change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Please don't forgot to document this feature.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
nosy: +ned.deily
priority: normal -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset ce5b0e9db1b9698e6ffc43ae41cf3a22ca5a6ba6 by Serhiy Storchaka in 
branch 'master':
bpo-32226: Make __class_getitem__ an automatic class method. (#5098)
https://github.com/python/cpython/commit/ce5b0e9db1b9698e6ffc43ae41cf3a22ca5a6ba6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +4966

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Ivan Levkivskyi]

Change by Ivan Levkivskyi :


--
pull_requests: +4800

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[STINNER Victor]

STINNER Victor  added the comment:

It is very to get stable benchmarks with timings shorter than 1 ms, and even 
harder for timings shorter than 1 us.

I wrote documentation to implement how to get more stable results:
http://perf.readthedocs.io/en/latest/run_benchmark.html#how-to-get-reproductible-benchmark-results

On Linux, using CPU isolation helps a lot, and perf uses automatically isolated 
CPUs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Guido van Rossum]

Guido van Rossum  added the comment:

I think the best thing to do is not to panic! Also maybe PGO could help?

On Dec 14, 2017 15:43, "Ivan Levkivskyi"  wrote:

>
> Ivan Levkivskyi  added the comment:
>
> Interesting. I have noticed similar when I tried to add a fast loop
> proposed by Serhiy. It should save at least one pointer comparison for base
> class, but sometimes it actually led to slow-downs (although very small).
> How can one fight this kind of problems?
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:

Interesting. I have noticed similar when I tried to add a fast loop proposed by 
Serhiy. It should save at least one pointer comparison for base class, but 
sometimes it actually led to slow-downs (although very small). How can one 
fight this kind of problems?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:


New changeset 2b5fd1e9ca9318673989e6ccac2c8acadc3809cd by Ivan Levkivskyi in 
branch 'master':
bpo-32226: Implementation of PEP 560 (core components) (#4732)
https://github.com/python/cpython/commit/2b5fd1e9ca9318673989e6ccac2c8acadc3809cd


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I think this is a cause. The difference may be dependent on the compiler and 
platform.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Guido van Rossum]

Guido van Rossum  added the comment:

> This adds an overhead to every indexing.

I'm not doubting your results, but I'm curious how that happened. The extra
code in PyObject_GetItem is right before the point where it would otherwise
raise TypeError -- I presume you benchmark never reached that point. Could
it be that the compiler ends up rearranging the code in a way that is less
optimal for the instruction cache?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

This adds an overhead to every indexing.

$ ./python -m perf timeit --compare-to=./python0 -s 'a = [1]' --duplicate=1 
'a[0]'
python0: . 15.3 ns +- 0.5 ns
python: . 16.1 ns +- 0.3 ns

Mean +- std dev: [python0] 15.3 ns +- 0.5 ns -> [python] 16.1 ns +- 0.3 ns: 
1.05x slower (+5%)

$ ./python -m perf timeit --compare-to=./python0 -s 'd = {1: 2}' 
--duplicate=1 'd[1]'
python0: . 17.6 ns +- 0.6 ns
python: . 18.4 ns +- 0.5 ns

Mean +- std dev: [python0] 17.6 ns +- 0.6 ns -> [python] 18.4 ns +- 0.5 ns: 
1.05x slower (+5%)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Xavier G. Domingo]

Change by Xavier G. Domingo :


--
nosy: +xgdomingo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Guido van Rossum]

Guido van Rossum  added the comment:

I like the current approach (PyObject_GetItem).  I agree with both of your 
reasons to prefer it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:

Guido, what is your preference for the implementation of ``__class_getitem__``: 
``PyObject_GetItem`` (current one) or ``type.__getitem__`` (old one)? Can we 
just go ahead with the version you like and then re-consider if some objections 
will appear? I am asking because you are going on vacation soon and there is a 
second part of implementation -- changes in typing (and someone needs to review 
them :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:

> As for __class_getitem__, why not implement type.__getitem__ instead of 
> hacking PyObject_GetItem()?

This question was raised by Mark Shannon on python-dev. Actually, my initial 
implementation did exactly this, but I didn't like it for two reasons:

1. dir(type) is already very long, I don't want to make it even longer.
2. Some users may be surprised that although ``type.__getitem__`` is defined, 
``int[something]`` will rise "TypeError: type object is not subscriptable"

Mark however disappeared since then, so I don't know what is his current 
opinion. I am however not as sure about this now. If there are some arguments 
for this, then I can revert to the older approach (implementing 
``type.__getitem__``).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

For reference: PEP 560.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

As for __class_getitem__, why not implement type.__getitem__ instead of hacking 
PyObject_GetItem()?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Ivan Levkivskyi]

Change by Ivan Levkivskyi :


--
keywords: +patch
pull_requests: +4635
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Ivan Levkivskyi]

New submission from Ivan Levkivskyi :

As discussed before, there will be two PRs. One for the core components, and 
the second one (large) for typing updates. I will open the first PR shortly.

--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Ivan Levkivskyi]

Change by Ivan Levkivskyi :


--
assignee: levkivskyi
components: Interpreter Core, Library (Lib)
nosy: levkivskyi
priority: normal
severity: normal
stage: needs patch
status: open
title: Implement PEP 560: Core support for typing module and generic types
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com