[issue40007] An attempt to make asyncio.transport.writelines (selector) use Scatter I/O

[Andrew Svetlov]

Andrew Svetlov  added the comment:

A duplicate of #47010

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> Implement zero copy writes in SelectorSocketTransport in asyncio

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40007] An attempt to make asyncio.transport.writelines (selector) use Scatter I/O

[jakirkham]

Change by jakirkham :


--
nosy: +jakirkham

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40007] An attempt to make asyncio.transport.writelines (selector) use Scatter I/O

[tzickel]

tzickel  added the comment:

BTW, if wanted a much more simpler PR can be made, where writelines simply 
calls sendmsg on the input if no buffer exists, and if not only then concats 
and acts like the current code base.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40007] An attempt to make asyncio.transport.writelines (selector) use Scatter I/O

[tzickel]

New submission from tzickel :

I have a code that tries to be smart and prepare data to be chunked efficiently 
before sending, so I was happy to read about:

https://docs.python.org/3/library/asyncio-protocol.html#asyncio.WriteTransport.writelines

Only to see that it simply does:

self.write(b''.join(lines))

So I've attempted to write an version that uses sendmsg (scatter I/O) instead 
(will be attached in PR).

What I've learnt is:
1. It's hard to benchmark (If someone has an good example on checking if it's 
worth it, feel free to add such).
2. sendmsg has an OS limit on how many items can be done in one call. If the 
user does not call writer.drain() it might have too many items in the buffer, 
in that case I concat them (might be an expensive operation ? but that should 
not be th enormal case).
3. socket.socket.sendmsg can accept any bytes like iterable, but os.writev can 
only accept sequences, is that a bug ?
4. This is for the selector stream socket for now.

--
components: asyncio
messages: 364565
nosy: asvetlov, tzickel, yselivanov
priority: normal
pull_requests: 18416
severity: normal
status: open
title: An attempt to make asyncio.transport.writelines (selector) use Scatter 
I/O
type: enhancement
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com