[issue28600] asyncio: Optimize loop.call_soon

[Chris Jerdonek]

Changes by Chris Jerdonek :


--
title: shutdown_asyncgens -> asyncio: Optimize loop.call_soon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28600] asyncio: Optimize loop.call_soon

[Yury Selivanov]

Yury Selivanov added the comment:

+# Wake up the loop if the finalizer was called from
+# a different thread.
+self._write_to_self()

Yeah, looks like shutdown_asyncgens somehow ended up in 3.5 branch (there's no 
harm in it being there).  I'll sync the branches.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28600] asyncio: Optimize loop.call_soon

[Guido van Rossum]

Guido van Rossum added the comment:

PS. I noticed there are a few lines different between the "upstream" repo
and the 3.5 stdlib:

+# Wake up the loop if the finalizer was called from
+# a different thread.
+self._write_to_self()

On Thu, Nov 3, 2016 at 3:12 PM, Yury Selivanov 
wrote:

>
> Changes by Yury Selivanov :
>
>
> --
> resolution:  -> fixed
> stage: commit review -> resolved
> status: open -> closed
> type:  -> performance
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28600] asyncio: Optimize loop.call_soon

[Yury Selivanov]

Yury Selivanov added the comment:

Guido, Andrew, thanks for reviews!

I've fixed some unittests before committing the patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28600] asyncio: Optimize loop.call_soon

[Yury Selivanov]

Changes by Yury Selivanov :


--
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed
type:  -> performance

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28600] asyncio: Optimize loop.call_soon

[Roundup Robot]

Roundup Robot added the comment:

New changeset 128ffe3c3eb9 by Yury Selivanov in branch '3.5':
Issue #28600: Optimize loop.call_soon().
https://hg.python.org/cpython/rev/128ffe3c3eb9

New changeset 4f570a612aec by Yury Selivanov in branch '3.6':
Merge 3.5 (issue #28600)
https://hg.python.org/cpython/rev/4f570a612aec

New changeset 46c3eede41a6 by Yury Selivanov in branch 'default':
Merge 3.6 (issue #28600)
https://hg.python.org/cpython/rev/46c3eede41a6

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28600] asyncio: Optimize loop.call_soon

[Yury Selivanov]

Yury Selivanov added the comment:

> LGTM

Will commit this soon.

> maybe make dirty hack and check hasattr(callback, 'send') ?

If you schedule a coroutine object it will fail anyways, because it's not 
callable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28600] asyncio: Optimize loop.call_soon

[Марк Коренберг]

Марк Коренберг added the comment:

> haypo added the check because people called `.call_later()` with coroutine 
> instead of callback very often

maybe make dirty hack and check hasattr(callback, 'send') ?

--
nosy: +mmarkk

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28600] asyncio: Optimize loop.call_soon

[STINNER Victor]

STINNER Victor added the comment:

call_soon3.patch: LGTM. It enhances error messages (fix the method name) and 
should make asyncio faster.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28600] asyncio: Optimize loop.call_soon

[Ned Deily]

Changes by Ned Deily :


--
stage:  -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28600] asyncio: Optimize loop.call_soon

[Guido van Rossum]

Guido van Rossum added the comment:

Patch 3 LGTM.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28600] asyncio: Optimize loop.call_soon

[STINNER Victor]

STINNER Victor added the comment:

I didn't review the patch, but I agree with the overall approach: expensive
checks can be made only in debug mode.

If people are concerned by the removal of the check by default, we should
repeat everywhere in the doc that async programming is hard and that the
debug mode saves a lot of time ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28600] asyncio: Optimize loop.call_soon

[Yury Selivanov]

Changes by Yury Selivanov :


Added file: http://bugs.python.org/file45341/call_soon3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28600] asyncio: Optimize loop.call_soon

[Yury Selivanov]

Changes by Yury Selivanov :


Added file: http://bugs.python.org/file45340/call_soon2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28600] asyncio: Optimize loop.call_soon

[Yury Selivanov]

Yury Selivanov added the comment:

> IIRC haypo added the check because people called `.call_later()` with 
> coroutine instead of callback very often.

We'll update asyncio docs in 3.6 with a tutorial to focus on coroutines (not on 
low-level event loop).  This should leave the loop API to advanced users.

> But checking in debug mode looks very reasonable to me if it is so expensive.

Exactly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28600] asyncio: Optimize loop.call_soon

[Andrew Svetlov]

Andrew Svetlov added the comment:

The patch looks good.
IIRC haypo added the check because people called `.call_later()` with coroutine 
instead of callback very often.

But checking in debug mode looks very reasonable to me if it is so expensive.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28600] asyncio: Optimize loop.call_soon

[Yury Selivanov]

Changes by Yury Selivanov :


--
keywords: +patch
Added file: http://bugs.python.org/file45339/call_soon.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28600] asyncio: Optimize loop.call_soon

[Yury Selivanov]

New submission from Yury Selivanov:

loop.call_soon is the central function of asyncio.  Everything goes through it.

Current design of asyncio.loop.call_soon makes the following checks:

1. [debug mode] check that the loop is not closed
2. [debug mode] check that we are calling call_soon from the proper thread
3. [always] check that callback is not a coroutine or a coroutine function

Check #3 is very expensive, because it uses an 'isinstance' check.  Moreover, 
isinstance checks against an ABC, which makes the call even slower.

Attached patch makes check #3 optional and run only in debug mode.  This is a 
very safe patch to merge.

This makes asyncio another 17% (sic!) faster.  In fact it becomes as fast as 
curio for realistic streams benchmarks.

--
assignee: yselivanov
components: asyncio
messages: 280002
nosy: asvetlov, gvanrossum, haypo, inada.naoki, ned.deily, yselivanov
priority: normal
severity: normal
status: open
title: asyncio: Optimize loop.call_soon
versions: Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com