[issue31333] Implement ABCMeta in C

[miss-islington]

miss-islington  added the comment:


New changeset 034a945fa723bf68ca4127bb43bfa5c5be899f17 by Miss Islington (bot) 
in branch '3.7':
bpo-31333: Fix typo in whatsnew/3.7.rst (GH-5744)
https://github.com/python/cpython/commit/034a945fa723bf68ca4127bb43bfa5c5be899f17


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[miss-islington]

Change by miss-islington :


--
pull_requests: +5522

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:


New changeset 3fb813d2c67fe28cc98ae51e53a6890294b6e423 by Ivan Levkivskyi 
(Terry Jan Reedy) in branch 'master':
bpo-31333: Fix typo in whatsnew/3.7.rst (GH-5744)
https://github.com/python/cpython/commit/3fb813d2c67fe28cc98ae51e53a6890294b6e423


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Terry J. Reedy]

Terry J. Reedy  added the comment:

PR fixes typo in What's new: rewrittent

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Terry J. Reedy]

Change by Terry J. Reedy :


--
pull_requests: +5521

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Guido van Rossum]

Guido van Rossum  added the comment:

Congratulations all on this milestone! And thanks reviewers for your thorough 
work.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:


New changeset 38928992885d8a04b7188abdba3b04f350bde32d by Ivan Levkivskyi in 
branch '3.7':
bpo-31333: Re-implement ABCMeta in C (GH-5733)
https://github.com/python/cpython/commit/38928992885d8a04b7188abdba3b04f350bde32d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Ivan Levkivskyi]

Change by Ivan Levkivskyi :


--
pull_requests: +5514

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Ivan Levkivskyi]

Change by Ivan Levkivskyi :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:


New changeset 03e3c340a0156891a036d6dbdb9e348108826255 by Ivan Levkivskyi in 
branch 'master':
bpo-31333: Re-implement ABCMeta in C (#5273)
https://github.com/python/cpython/commit/03e3c340a0156891a036d6dbdb9e348108826255


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Guido van Rossum]

Guido van Rossum  added the comment:

> Isn't 800 lines of C code too high price for speeding up ABCs creation?

I think it's well worth it. This has long felt as a sore point to me.

On Sat, Feb 17, 2018 at 10:27 AM, Ivan Levkivskyi 
wrote:

>
> Ivan Levkivskyi  added the comment:
>
> > Isn't 800 lines of C code too high price for speeding up ABCs creation?
>
> 800 lines of C code is not something hard to notice, so I suppose the
> answer is obvious for all people involved in the work on PR :-)
>
> > ...this can save just several milliseconds at start-up.
>
> The correct way to measure this is relative, not absolute. There are just
> few ABCs used by modules loaded at Python start-up, and it already allowed
> to save 10% of start-up time. My expectation is that the number will be
> similar for a typical Python app. Moreover, `isinstance` and `issubclass`
> (functions called often with ABCs) will be 1.5x faster.
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:

> Isn't 800 lines of C code too high price for speeding up ABCs creation?

800 lines of C code is not something hard to notice, so I suppose the answer is 
obvious for all people involved in the work on PR :-)

> ...this can save just several milliseconds at start-up.

The correct way to measure this is relative, not absolute. There are just few 
ABCs used by modules loaded at Python start-up, and it already allowed to save 
10% of start-up time. My expectation is that the number will be similar for a 
typical Python app. Moreover, `isinstance` and `issubclass` (functions called 
often with ABCs) will be 1.5x faster.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Isn't 800 lines of C code too high price for speeding up ABCs creation? This 
will save several microseconds per ABC creation. Even if the program creates 
1000 ABCs, this can save just several milliseconds at start-up.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Yury Selivanov]

Yury Selivanov  added the comment:

> Let's try to get it in as soon as we can, please!

Thank you, Ned!  We'll get it merged in the next few days.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Ned Deily]

Ned Deily  added the comment:

There has been a lot of work by a number of people on this feature leading up 
to the 3.7.0b1.  Thank you!  On PR 5273, Yury says that he believes the feature 
is ready to merge but would prefer an extension until 3.7.0b2 and that Guido 
has agreed.  I'll defer to Yury's judgement on this and somewhat reluctantly 
allow an extension: there's a lot going on here.  Let's try to get it in as 
soon as we can, please!

--
nosy: +ned.deily
priority: normal -> deferred blocker
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Ivan Levkivskyi]

Change by Ivan Levkivskyi :


--
keywords: +patch
pull_requests: +5117
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[INADA Naoki]

INADA Naoki added the comment:

"python_startup_nosite" benchmark imports io module,
and "python_startup" benchmark imports os too.
So no need to additional test for them.

You can see it with `python -v -S -c 'pass'` and `python -v -c 'pass'`.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Stefan Behnel]

Stefan Behnel added the comment:

Since the number of applications that get along without any file access is 
probably close to irrelevant, "os" and "io" feel like sufficiently widely used 
modules to merit being part of a "usual Python startup" benchmark. Maybe we 
should add one to the benchmark suite? Anyone up for it?

--
nosy: +scoder

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[INADA Naoki]

INADA Naoki added the comment:

> Hm, indeed, but I see that module 'abc' is in 'sys.modules', probably it is 
> then only used by 'collections.abc'/'_collections_abc'. This means that the 
> influence of 'ABCMeta' on interpreter start-up time will be smaller than I 
> thought. But still start-up times for projects that actively use ABCs will be 
> influenced by 'ABCMeta'.

Since os.environ uses _collections_abc.MutableMapping, importing 
_collections_abc is not avoidable while startup.

`io` module uses ABC and it's used for sys.std(io|err|in).  It's not avoidable 
too.

And as you know, typing module will be very common rapidly :)
Even if it doesn't affect "python_startup" microbench, it's important.

--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Guido van Rossum]

Guido van Rossum added the comment:

I've heard many anecdotal complaints about the lack of speed of ABCs. Even if 
it doesn't affect core Python startup, it does affect startup of apps, and if 
people are afraid to use them because of this, it's reasonable to try to do 
something about it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Raymond Hettinger]

Changes by Raymond Hettinger :


--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Ivan Levkivskyi]

Ivan Levkivskyi added the comment:

Eric,

> the only ABCs in importlib are in importlib.abc (and used by importlib.util).
> This does not impact interpreter startup.

Hm, indeed, but I see that module 'abc' is in 'sys.modules', probably it is 
then only used by 'collections.abc'/'_collections_abc'. This means that the 
influence of 'ABCMeta' on interpreter start-up time will be smaller than I 
thought. But still start-up times for projects that actively use ABCs will be 
influenced by 'ABCMeta'.

But what do you think about making private ABC caches read-only attributes? 
(They are not documented)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Eric Snow]

Eric Snow added the comment:

> This mostly influences the Python interpreter start-up time
> (because of extensive use of ABCs in importlib)

Just to be clear, the only ABCs in importlib are in importlib.abc (and used by 
importlib.util).  This does not impact interpreter startup.

--
nosy: +eric.snow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Aaron Hall]

Changes by Aaron Hall :


--
nosy: +Aaron Hall

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Ivan Levkivskyi]

Changes by Ivan Levkivskyi :


--
nosy: +brett.cannon, haypo, serhiy.storchaka, yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Ivan Levkivskyi]

New submission from Ivan Levkivskyi:

The idea is that creating ABCs is approximately twice slower than normal 
classes. Plus there are smaller penalties for various operations with ABCs. 
This mostly influences the Python interpreter start-up time (because of 
extensive use of ABCs in importlib), and start-up times of programs that 
extensively use ABCs.

The situation can be improved by rewriting ABCMeta in C. I have a working 
implementation, but it is far form being ready and still needs some polishing 
and optimizations (in particular _abc_cache and friends).

Already at this stage I have one question (I will add more when they appear 
while I am finishing the implementation): is it OK to make _abc_cache, 
_abc_negative_cache, _abc_negative_cache_version, and _abc_registry read-only? 
The point is that I want to prohibit something like this:

MyABC._abc_cache = "Surprise on updating the cache!"

thus avoiding many PySet_Check(...) calls. These attributes are not documented 
and start with underscore.

--
components: Extension Modules, Library (Lib)
messages: 301198
nosy: barry, levkivskyi
priority: normal
severity: normal
status: open
title: Implement ABCMeta in C
type: performance
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com