Re: [Twisted-Python] reactor for Linux io_uring

[Glyph]

> On Jan 7, 2021, at 3:24 AM, Ian Haywood  wrote:
> 
> For async file I/O my plan would be to export a new IFilesystem (which is 
> closely based on conch.interfaces.ISFTPServer) apps would be have to be 
> written to use it, conch could with minimal tweaking,  and unsurprisingly the 
> SMB server will. 
> 
> Apps can fallback to thread-based or aio(7) based implementations that 
> already  exists as PoC:https://github.com/twisted/twisted/pull/1420 
>   
> 

The mess of ctypes stuff seems somewhat irrelevant to the "asynchronous 
filesystem" part of this PR - do you think you could do a smaller version of 
this which decouples it from smb and ctypes?  It would be very nice for 
twisted.web.static.File to be able to use this as well.  And it would be much 
easier to review in isolation.  (And also easier to review as concrete "clear 
asynchronous file I/O interface" functionality, rather than tying it to the 
term "VFS" which has a fairly checkered past within Twisted :)).

(It would also be nice to have an interface that acts as an IProducer to 
integrate more natively with Twisted's support for backpressure rather than 
only having a custom readChunk method.  I think readChunk is still necessary 
for completeness since you need to be able to seek and offset, though.)

-g___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] reactor for Linux io_uring

[Ian Haywood]

On 6/01/2021 1:04 am, Adi Roiban wrote:
On Tue, 5 Jan 2021 at 13:44, Jean-Paul Calderone 
mailto:exar...@twistedmatrix.com>> wrote:


On Tue, Jan 5, 2021 at 6:49 AM Barry Scott
mailto:barry.sc...@forcepoint.com>>
wrote:


What threads? Why do you call out file FDs different from
socket FDs?

If you give epoll()/select() a file FD it lies and says the FD is always 
"ready" even when, as you point out, file operations can block.


My suggestion is to find a real world / production use case for the 
new reactor so that we can run more than unit / functional tests.


--
Adi Roiban


As a "standard" reactor any twisted app would stress-test it for 
network/pipe operations, no?


For async file I/O my plan would be to export a new IFilesystem (which 
is closely based on conch.interfaces.ISFTPServer) apps would be have to 
be written to use it, conch could with minimal tweaking,  and 
unsurprisingly the SMB server will.


Apps can fallback to thread-based or aio(7) based implementations that 
already  exists as PoC: https://github.com/twisted/twisted/pull/1420


Ian

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python