[issue39168] Generic type subscription is a huge toll on Python performance

[Guido van Rossum]

Guido van Rossum  added the comment:

No worries. I tend to run each time it command at least three times before I 
trust the numbers. Professional bench markers also configure a machine without 
background tasks (email etc.).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[David Lord]

David Lord  added the comment:

I'm using Arch Linux. After your reply I tried again and now I'm seeing the 
same result as you, negligible difference from inheriting `Generic` on Python 
3.9. I can't explain it, I ran the timings repeatedly before I posted here, but 
I guess it was a weird temporary issue with my machine.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Guido van Rossum]

Guido van Rossum  added the comment:

@davidm

I don't see such a dramatic difference -- the generic version is a tad slower, 
but the difference is less than the variation between runs.

What platform are you using?  (I'm doing this on Windows.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[David Lord]

David Lord  added the comment:

Is this performance issue supposed to be fixed in 3.9? I'm still observing 
severe slowdown by inheriting from `Generic[T]`.

I'm currently adding typing to Werkzeug, where we define many custom data 
structures such as `MultiDict`. It would be ideal for these classes to be 
recognized as generic mappings. I remembered hearing about this performance 
issue somewhere, so I decided to test what happens.

Here's a minimal example without Werkzeug, the results in Werkzeug are similar 
or worse. I'd estimate each request creates about 10 of the various data 
structures, which are then accessed by user code, so I simulated that by 
creating and iterating a list of objects.

```python
class Test:
def __init__(self, value):
self.value = value

def main():
ts = [Test(x) for x in range(10)]
sum(t.value for t in ts)
```

```
$ python3.9 -m timeit -n 10 -s 'from example import main' 'main()'
10 loops, best of 5: 7.67 usec per loop
```

```python
import typing

V = typing.TypeVar("V")

class Test(typing.Generic[V]):
def __init__(self, value: V) -> None:
self.value = value

def main():
ts = [Test(x) for x in range(10)]
sum(t.value for t in ts)
```

```
$ python3.9 -m timeit -n 10 -s 'from example import main' 'main()'
10 loops, best of 5: 18.2 usec per loop
```

There is more than a 2x slowdown when using `Generic`. The timings (7 vs 18 
usec) are the same across Python 3.6, 3.7, 3.8, and 3.9. It seems that 3.9 does 
not fix the performance issue.

Since we currently support Python 3.6+, I probably won't be able to use 
generics anyway due to the performance in those versions, but I wanted to make 
sure I'm not missing something with 3.9.

--
nosy: +davidism

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Guido van Rossum]

Change by Guido van Rossum :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[miss-islington]

miss-islington  added the comment:


New changeset 5a1384935ee8996a5bd240dd29f9b5e356cfc467 by Miss Islington (bot) 
in branch '3.9':
bpo-39168: Remove the __new__ method of typing.Generic (GH-21327)
https://github.com/python/cpython/commit/5a1384935ee8996a5bd240dd29f9b5e356cfc467


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[miss-islington]

Change by miss-islington :


--
pull_requests: +20483
pull_request: https://github.com/python/cpython/pull/21335

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[miss-islington]

miss-islington  added the comment:


New changeset 7fed75597fac11f9a6c769e2b6c6548fe0e4049d by Zackery Spytz in 
branch 'master':
bpo-39168: Remove the __new__ method of typing.Generic (GH-21327)
https://github.com/python/cpython/commit/7fed75597fac11f9a6c769e2b6c6548fe0e4049d


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Guido van Rossum]

Guido van Rossum  added the comment:

Should this be backported? How far back?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Zackery Spytz]

Change by Zackery Spytz :


--
keywords: +patch
nosy: +ZackerySpytz
nosy_count: 6.0 -> 7.0
pull_requests: +20477
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21327

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Wouter De Borger]

Change by Wouter De Borger :


--
nosy: +Wouter De Borger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[calebj]

Change by calebj :


--
nosy: +calebj

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Naveen Durai]

Change by Naveen Durai :


--
nosy: +Naveen Durai

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Ruslan Dautkhanov]

Ruslan Dautkhanov  added the comment:

Didn't see your last response before submitting an update.

That's great you have a plan how to resolve this! 

Thanks again

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Ruslan Dautkhanov]

Ruslan Dautkhanov  added the comment:

Perhaps the check should only be done in some sort of Python development mode 
and off by default?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Guido van Rossum]

Guido van Rossum  added the comment:

OK let’s do it. Clearly for *some* applications the overhead is significant.
-- 
--Guido (mobile)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Ruslan Dautkhanov]

Ruslan Dautkhanov  added the comment:

Thank you Guido and Ivan

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:

OK, here is the original issue https://github.com/python/typing/issues/681. I 
asked the author to open an issue here instead, but likely they didn't open one.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Guido van Rossum]

Guido van Rossum  added the comment:

If that solves the perf issue I am fine with it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:

This issue came up few times before (although I can't find an issue here on 
b.p.o., maybe it was on typing-sig list). Although in micro-benchmarks the 
impact may seem big, in vast majority of applications it is rarely more that a 
percent or so.

On the other hand, IIRC the only reason `Generic.__new__()` exists is so that 
one can't write `Generic()` (i.e. instantiate a plain `Generic`). I would be 
totally fine if we just remove it in 3.9. Hopefully, people already learned 
what typing is for and don't need so much "protection" against not very 
meaningful things. Also, the error can be given by static type checkers, there 
is probably no need for a runtime error.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Guido van Rossum]

Guido van Rossum  added the comment:

Hm, here's what I measure in Python 3.8.1. (I don't use IPython or notebooks so 
this looks a little different.)


>>> timeit.timeit('Foo()', 'class Foo: pass')
0.3763025619934


>>> timeit.timeit('Foo()', 'class Foo:\n  def __new__(cls): return 
>>> super().__new__(cls)')
1.575319603864


>>> timeit.timeit('Foo()', 'from typing import Generic, TypeVar\nT = 
>>> TypeVar("T")\nclass Foo(Generic[T]): pass')
3.874873715068


>From this I conclude that adding a minimal __new__() method is responsible for 
>about 4x slowdown, and the functionality in typing.py for another factor 2.5.


While this isn't great I don't see an easy way to improve upon this without 
rewriting the entire typing module in C.  (Some of this may or may not happen 
for PEP 604.)

PS. I just realized my Python binary was built with debug options, so absolute 
numbers will look different (better) for you -- but relative numbers will look 
the same, and I get essentially the same factors with 3.9.0a1+.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Guido van Rossum]

Guido van Rossum  added the comment:

Sorry, you already said 3.6 and 3.8 give the same effect. But what if you add a 
minimal __new__() to Foo?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Guido van Rossum]

Guido van Rossum  added the comment:

What Python version was used for the timings? If not 3.8, please do over in 3.8.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Ruslan Dautkhanov]

Ruslan Dautkhanov  added the comment:

In [12]: cProfile.run("for _ in range(100_000): Bar()") 

   
 23 function calls in 0.136 seconds 

   


   
   Ordered by: standard name

   


   
   ncalls  tottime  percall  cumtime  percall filename:lineno(function) 

   
10.0470.0470.1360.136 :1()  

   
   100.0790.0000.0890.000 typing.py:865(__new__)

   
   100.0100.0000.0100.000 {built-in method __new__ of type 
object at 0x55ab65861ac0}   

10.0000.0000.1360.136 {built-in method builtins.exec}   

   
10.0000.0000.0000.000 {method 'disable' of 
'_lsprof.Profiler' objects} 



   


   


   
In [13]: # And typing.py:865 points to  

   


   
In [14]: inspect.getsourcelines(Generic.__new__)

   
Out[14]:

   
(['def __new__(cls, *args, **kwds):\n', 

   
  'if cls in (Generic, Protocol):\n',   

   
  'raise TypeError(f"Type {cls.__name__} cannot be instantiated; 
"\n',   
  
  '"it can be used only as a base class")\n',   

   
  'if super().__new__ is object.__new__ and cls.__init__ is not 
object.__init__:\n',
   
  'obj = super().__new__(cls)\n',   

[issue39168] Generic type subscription is a huge toll on Python performance

[Ned Deily]

Change by Ned Deily :


--
nosy: +gvanrossum, levkivskyi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Ruslan Dautkhanov]

Ruslan Dautkhanov  added the comment:

Python typing gives an order of magnitude slow down in this case

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Ruslan Dautkhanov]

New submission from Ruslan Dautkhanov :

Reported originally here - 
https://twitter.com/__zero323__/status/1210911632953692162

See details here
https://asciinema.org/a/290643

In [4]: class Foo: pass
In [5]: %timeit -n1_000_000 Foo()
88.5 ns ± 3.44 ns per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [6]: T = TypeVar("T")
In [7]: class Bar(Generic[T]): pass
In [8]: %timeit -n1_000_000 Bar()
883 ns ± 3.46 ns per loop (mean ± std. dev. of 7 runs, 100 loops each)

Same effect in Python 3.6 and 3.8

--
messages: 359049
nosy: Ruslan Dautkhanov
priority: normal
severity: normal
status: open
title: Generic type subscription is a huge toll on Python performance
type: performance
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com