[issue31970] asyncio debug mode is very slow

[Antoine Pitrou]

Antoine Pitrou  added the comment:

In any case, I think this is better discussed on async-sig (where I already 
started a discussion thread some weeks ago -- see """APIs for high-bandwidth 
large I/O?""").

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31970] asyncio debug mode is very slow

[Yury Selivanov]

Yury Selivanov  added the comment:

> IMHO performance is a good motivation to enhance the asyncio API ;-) The 
> tricky part is to keep the backward compatibility.

Let's stop the discussion in this ticket :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31970] asyncio debug mode is very slow

[STINNER Victor]

STINNER Victor  added the comment:

> The problem is the Protocol API imposes copies by default (...)

IMHO performance is a good motivation to enhance the asyncio API ;-) The tricky 
part is to keep the backward compatibility.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31970] asyncio debug mode is very slow

[Giampaolo Rodola']

Giampaolo Rodola'  added the comment:

Sorry. It looks like I completely misunderstood. =)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31970] asyncio debug mode is very slow

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Le 08/11/2017 à 14:40, Giampaolo Rodola' a écrit :
> Exposing sendfile() should be straightforward.

sendfile() is not useful for data that's read from (or written to) memory.

> As for a readinto()-like functionality: the only thing I'm aware of is 
> splice() syscall but it's Linux only.

This is not the abstraction level we are talking about.  The problem is
the Protocol API imposes copies by default (data_received() gives you an
arbitrarily-sized bytes object that doesn't match how your protocol
chunks data).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31970] asyncio debug mode is very slow

[Giampaolo Rodola']

Giampaolo Rodola'  added the comment:

>> I've a few ideas how to add 0-copy support to protocols.

> I'd be interesting to hear about them.  The main challenge IMHO is to find a 
> way to allow a readinto()-like functionality.

Exposing sendfile() should be straightforward. I started implementing it years 
ago but I gave up pretty soon because asyncio had no solid test framework for 
testing an actual data transfer between two sockets and that basically 
represented a blocker. Basically back then all recv() / send() related tests 
were mocks. Not sure about the current situation but if that has changed I 
would be happy to contribute a PR (I was the one who contributed 
socket.sendfile()).

As for a readinto()-like functionality: the only thing I'm aware of is splice() 
syscall but it's Linux only. It must be noted that also sendfile() is UNIX 
only. Windows achieve the same via TransmitFile.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31970] asyncio debug mode is very slow

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> I've a few ideas how to add 0-copy support to protocols.

I'd be interesting to hear about them.  The main challenge IMHO is to find a 
way to allow a readinto()-like functionality.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31970] asyncio debug mode is very slow

[Yury Selivanov]

Yury Selivanov  added the comment:

Thank you for helping with asyncio!  I'll try to get to that transport 
performance issues you found sometime this week.  I've a few ideas how to add 
0-copy support to protocols.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31970] asyncio debug mode is very slow

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset d8d218ffda6b7569625ff9edadbbc9a2b1055e32 by Antoine Pitrou in 
branch '3.6':
[3.6] bpo-31970: Reduce performance overhead of asyncio debug mode. (GH-4314) 
(#4322)
https://github.com/python/cpython/commit/d8d218ffda6b7569625ff9edadbbc9a2b1055e32


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31970] asyncio debug mode is very slow

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Thank you for the quick reviews!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31970] asyncio debug mode is very slow

[Antoine Pitrou]

Change by Antoine Pitrou :


--
pull_requests: +4280

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31970] asyncio debug mode is very slow

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset 921e9432a1461bbf312c9c6dcc2b916be6c05fa0 by Antoine Pitrou in 
branch 'master':
bpo-31970: Reduce performance overhead of asyncio debug mode. (#4314)
https://github.com/python/cpython/commit/921e9432a1461bbf312c9c6dcc2b916be6c05fa0


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31970] asyncio debug mode is very slow

[Antoine Pitrou]

Change by Antoine Pitrou :


--
nosy: +asvetlov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31970] asyncio debug mode is very slow

[Antoine Pitrou]

Antoine Pitrou  added the comment:

If this is accepted on the principle, it would be great to also patch 3.6.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31970] asyncio debug mode is very slow

[Antoine Pitrou]

Change by Antoine Pitrou :


--
keywords: +patch
pull_requests: +4273
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31970] asyncio debug mode is very slow

[Antoine Pitrou]

New submission from Antoine Pitrou :

Each time a new future, handle or task is created, asyncio with debug mode 
enabled will parse the whole call stack and create a StackSummary object for 
it.  Imagine a recursive coroutine: with N nested calls, you get O(N**2) 
performance.

Ideally debug mode wouldn't slow things too much, at least for development and 
testing setups.

--
components: Library (Lib), asyncio
messages: 305752
nosy: giampaolo.rodola, haypo, pitrou, yselivanov
priority: normal
severity: normal
status: open
title: asyncio debug mode is very slow
type: behavior
versions: Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com