hi Mark! I think your approach works for the cases which use put-bytevector* directly. Are you assuming the server-core only handles suspendable-port with put-bytevector in http-read or http-write? If so, then your approach is not enough. When the users enabled suspendable-port, every I/O operation would be non-blocking, and be managed by overridden I/O methods rebind by suspendable-port . That is to say, every I/O operation should have the ability to return the current I/O status for the scheduler. That's why I put it in read-bytes and write-bytes.
If we add a new put-bytevector* function, and if it is the only function which returns current I/O status, then we can't tell the scheduler the progress when users are trying to access database for a bigger data, or access a file on the disk, or any other potential I/O operation registered in HTTP handler by users. I think it's better to patch read-bytes and write-bytes for any I/O operations, otherwise, we have to patch all the overridden functions in suspendable-ports to make sure all I/O operations are handled correctly by suspendable-port. Hope I understand your idea correctly. Best regards. On Wed, May 15, 2019 at 4:23 AM Mark H Weaver <m...@netris.org> wrote: > > Hi Nala, > > Nala Ginrut <nalagin...@gmail.com> writes: > > > On Tue, May 14, 2019 at 7:01 AM Mark H Weaver <m...@netris.org> wrote: > >> I guess what you want is the ability to see incremental reports on the > >> progress of your large I/O operations. Is that right? If we are going > >> to add an API for this, it needs to be reliable, and always give reports > >> in terms of the high-level requests that the user gave. > > > > Yes, that's exactly what I want. We need to get the progress of I/O > > operation when it's blocking > > so that we can compute a fair priority for the tasks. > > > >> My preferred approach would be something like this: we could add a > >> 'put-bytevector-some' I/O primitive which writes some bytes from a > >> bytevector, blocking only as needed to write at least one byte. It > >> would return the number of bytes written. This can be used to implement > >> an efficient variant of 'put-bytevector' that gives you access to the > >> real-time progress information. > > > > I'm not sure if put-bytevector-some does the work, I'll list my concerns: > > > > 1. All I/O will be managed by Guile when we enabled suspendable-port. > > That is to say, from the users side, users never know their I/O > > operations are blocking or not. It's transparent to users. > > Guile will guarantee the I/O operations to be finished by managing all > > the blocking I/O mechanisms. > > Users can only interact with the task with read or write waiter, which > > are registered by users themselves. > > In this scenario, users are out of control of I/O operations. And they > > have no way to get the progress of I/O, since there's > > no way to pass this status to the waiter function except for > > parameters in my patch. > > > > 2. suspendable-port module has already provided a bunch of overridden > > bytevector-* functions. > > However, they're hidden from users. I think it's good since the > > purpose of suspendable-port is to abstract all these details from > > users. Users only consider the read-waiter and write-waiter for scheduling. > > If we provide the low-level bytevector functions to users to let them > > do the non-blocking I/O by themselves, just like most C framework > > does. Then Guile suspendable-port will lose a critical feature, > > although users can still implement asynchronous non-blocking I/O by > > themselves with a non-managed approach. Say, do the I/O, check result > > by themselves, and do the scheduling. > > Personally, I'm fine with this way, since I'm familiar with both ways. > > But managed I/O of suspendable-port is a good selling point for many > > inexperienced server-side developers, they can use it in Scheme just > > like IOCP or AIO. > > > > Of course, I may misunderstand your mind. > > Could you elaborate more about your approach? > > Here's what I had in mind. I'm not suggesting that you use asynchronous > non-blocking I/O. 'put-bytevector-some' would be like 'put-bytevector' > except that it can write less than you asked it to. Ideally it would > block only as needed to write at least one byte. It would be analogous > to POSIX write(2). > > Here's an untested draft implementation of 'put-bytevector*' which sets > the parameter 'current-io-status' to provide the information you wanted, > except that here the information will reliably refer to your own buffer. > The primitive it is built on 'put-bytevector-some' does not yet exist, > but we could add it. Note that 'put-bytevector-some' would still block, > and it would be suspendable. > > --8<---------------cut here---------------start------------->8--- > (define* (put-bytevector* port bv > #:optional > (start 0) > (count (bytevector-length bv))) > (let loop ((start start) (count count)) > (unless (zero? count) > (let ((written (parameterize ((current-io-status (cons start count))) > (put-bytevector-some port bv start count)))) > (loop (+ start written) (- count written)))))) > --8<---------------cut here---------------end--------------->8--- > > We can already do something analogous on the reader side, using either > 'get-bytevector-some' or the already-implemented-but-not-yet-pushed > 'get-bytevector-some!'. > > What do you think? > > Mark