Re: [PATCH] Add current-suspendable-io-status parameter
BTW, the patch is based on the latest stable-2.2 branch. On Mon, May 13, 2019 at 6:54 PM Nala Ginrut wrote: > > Hi folks! > Here's a patch to add current-suspendable-io-status: > Its result is a pair: (finished-bytes . rest-bytes) > > This is useful for designing a proper scheduler algorithm for > suspendable I/O operations. > For example, the server-core based on delimited-continuation can > compute a priority based on this status to decide how to schedule the > suspended task. > > Comments are welcome. > > Best regards.
Re: [PATCH] Add current-suspendable-io-status parameter
Hi Nala, Nala Ginrut writes: > Here's a patch to add current-suspendable-io-status: > Its result is a pair: (finished-bytes . rest-bytes) > > This is useful for designing a proper scheduler algorithm for > suspendable I/O operations. > For example, the server-core based on delimited-continuation can > compute a priority based on this status to decide how to schedule the > suspended task. I'm not inclined to apply this patch. It would entail adding heap allocation to a low-level I/O procedure that otherwise need not perform any allocation. Heap allocation happens both in 'cons' and in 'parameterize'. We should try hard to avoid unnecessary complications in these low-level routines. I also have doubts about the utility of the information provided. 'start' seems completely irrelevant, since it merely indicates the position within the read/write buffer. It's easier to make a case that 'count' might be relevant, but it's not a reliable indicator of the overall amount of pending I/O on that port. With a few exceptions, 'count' is normally limited to the port buffer size, and does not necessarily reflect the amount of I/O pending in the higher-level code. If you'd like to use parameters to communicate pending I/O size to your scheduler, I think it would be better to use 'parameterize' in your own higher-level code, where you have the best knowledge of how much I/O will need to be done. If you still believe that your proposed patch is the best approach, and that the benefit of your scheduling algorithm outweighs the cost of adding these complications and heap allocations to our low-level I/O procedures, can you help me to understand why? Can you explain the theory behind your scheduling algorithm? Regards, Mark
Re: [PATCH] Add current-suspendable-io-status parameter
Hello again, Previously I wrote: > I also have doubts about the utility of the information provided. > 'start' seems completely irrelevant, since it merely indicates the > position within the read/write buffer. It's easier to make a case that > 'count' might be relevant, but it's not a reliable indicator of the > overall amount of pending I/O on that port. With a few exceptions, > 'count' is normally limited to the port buffer size, and does not > necessarily reflect the amount of I/O pending in the higher-level code. I gave this some more thought, and I think I now understand why you want these values. When performing large binary I/O operations, larger than the port buffers, Guile bypasses the port buffers and passes the user's bytevector directly to {read,write}-bytes. In that case, 'start' and 'count' correspond to values that are meaningful to the user. I'd like to help you get the information you need, but I don't want to use the mechanism you proposed. In addition to the efficiency problem regarding heap allocation that I already mentioned, another problem is that it's unreliable. The 'start' and 'count' provided *might* correspond to the user's buffer, or it might not, and there's no reliable way to find out. It would encourage users to write code that depends on undocumented details of Guile's internal ports implementation. I guess what you want is the ability to see incremental reports on the progress of your large I/O operations. Is that right? If we are going to add an API for this, it needs to be reliable, and always give reports in terms of the high-level requests that the user gave. My preferred approach would be something like this: we could add a 'put-bytevector-some' I/O primitive which writes some bytes from a bytevector, blocking only as needed to write at least one byte. It would return the number of bytes written. This can be used to implement an efficient variant of 'put-bytevector' that gives you access to the real-time progress information. On the read side, we already have 'get-bytevector-some', and I plan to add 'get-bytevector-some!' soon as well. This could be used to implement progress-tracking read operations. What do you think? Regards, Mark
Re: [PATCH] Add current-suspendable-io-status parameter
Hi Mark! Thanks for so patient reply! On Tue, May 14, 2019 at 7:01 AM Mark H Weaver wrote: > I guess what you want is the ability to see incremental reports on the > progress of your large I/O operations. Is that right? If we are going > to add an API for this, it needs to be reliable, and always give reports > in terms of the high-level requests that the user gave. Yes, that's exactly what I want. We need to get the progress of I/O operation when it's blocking so that we can compute a fair priority for the tasks. > My preferred approach would be something like this: we could add a > 'put-bytevector-some' I/O primitive which writes some bytes from a > bytevector, blocking only as needed to write at least one byte. It > would return the number of bytes written. This can be used to implement > an efficient variant of 'put-bytevector' that gives you access to the > real-time progress information. I'm not sure if put-bytevector-some does the work, I'll list my concerns: 1. All I/O will be managed by Guile when we enabled suspendable-port. That is to say, from the users side, users never know their I/O operations are blocking or not. It's transparent to users. Guile will guarantee the I/O operations to be finished by managing all the blocking I/O mechanisms. Users can only interact with the task with read or write waiter, which are registered by users themselves. In this scenario, users are out of control of I/O operations. And they have no way to get the progress of I/O, since there's no way to pass this status to the waiter function except for parameters in my patch. 2. suspendable-port module has already provided a bunch of overridden bytevector-* functions. However, they're hidden from users. I think it's good since the purpose of suspendable-port is to abstract all these details from users. Users only consider the read-waiter and write-waiter for scheduling. If we provide the low-level bytevector functions to users to let them do the non-blocking I/O by themselves, just like most C framework does. Then Guile suspendable-port will lose a critical feature, although users can still implement asynchronous non-blocking I/O by themselves with a non-managed approach. Say, do the I/O, check result by themselves, and do the scheduling. Personally, I'm fine with this way, since I'm familiar with both ways. But managed I/O of suspendable-port is a good selling point for many inexperienced server-side developers, they can use it in Scheme just like IOCP or AIO. Of course, I may misunderstand your mind. Could you elaborate more about your approach? Best regards.
Re: [PATCH] Add current-suspendable-io-status parameter
Hi Nala, Nala Ginrut writes: > On Tue, May 14, 2019 at 7:01 AM Mark H Weaver wrote: >> I guess what you want is the ability to see incremental reports on the >> progress of your large I/O operations. Is that right? If we are going >> to add an API for this, it needs to be reliable, and always give reports >> in terms of the high-level requests that the user gave. > > Yes, that's exactly what I want. We need to get the progress of I/O > operation when it's blocking > so that we can compute a fair priority for the tasks. > >> My preferred approach would be something like this: we could add a >> 'put-bytevector-some' I/O primitive which writes some bytes from a >> bytevector, blocking only as needed to write at least one byte. It >> would return the number of bytes written. This can be used to implement >> an efficient variant of 'put-bytevector' that gives you access to the >> real-time progress information. > > I'm not sure if put-bytevector-some does the work, I'll list my concerns: > > 1. All I/O will be managed by Guile when we enabled suspendable-port. > That is to say, from the users side, users never know their I/O > operations are blocking or not. It's transparent to users. > Guile will guarantee the I/O operations to be finished by managing all > the blocking I/O mechanisms. > Users can only interact with the task with read or write waiter, which > are registered by users themselves. > In this scenario, users are out of control of I/O operations. And they > have no way to get the progress of I/O, since there's > no way to pass this status to the waiter function except for > parameters in my patch. > > 2. suspendable-port module has already provided a bunch of overridden > bytevector-* functions. > However, they're hidden from users. I think it's good since the > purpose of suspendable-port is to abstract all these details from > users. Users only consider the read-waiter and write-waiter for scheduling. > If we provide the low-level bytevector functions to users to let them > do the non-blocking I/O by themselves, just like most C framework > does. Then Guile suspendable-port will lose a critical feature, > although users can still implement asynchronous non-blocking I/O by > themselves with a non-managed approach. Say, do the I/O, check result > by themselves, and do the scheduling. > Personally, I'm fine with this way, since I'm familiar with both ways. > But managed I/O of suspendable-port is a good selling point for many > inexperienced server-side developers, they can use it in Scheme just > like IOCP or AIO. > > Of course, I may misunderstand your mind. > Could you elaborate more about your approach? Here's what I had in mind. I'm not suggesting that you use asynchronous non-blocking I/O. 'put-bytevector-some' would be like 'put-bytevector' except that it can write less than you asked it to. Ideally it would block only as needed to write at least one byte. It would be analogous to POSIX write(2). Here's an untested draft implementation of 'put-bytevector*' which sets the parameter 'current-io-status' to provide the information you wanted, except that here the information will reliably refer to your own buffer. The primitive it is built on 'put-bytevector-some' does not yet exist, but we could add it. Note that 'put-bytevector-some' would still block, and it would be suspendable. --8<---cut here---start->8--- (define* (put-bytevector* port bv #:optional (start 0) (count (bytevector-length bv))) (let loop ((start start) (count count)) (unless (zero? count) (let ((written (parameterize ((current-io-status (cons start count))) (put-bytevector-some port bv start count (loop (+ start written) (- count written)) --8<---cut here---end--->8--- We can already do something analogous on the reader side, using either 'get-bytevector-some' or the already-implemented-but-not-yet-pushed 'get-bytevector-some!'. What do you think? Mark
Re: [PATCH] Add current-suspendable-io-status parameter
hi Mark! I think your approach works for the cases which use put-bytevector* directly. Are you assuming the server-core only handles suspendable-port with put-bytevector in http-read or http-write? If so, then your approach is not enough. When the users enabled suspendable-port, every I/O operation would be non-blocking, and be managed by overridden I/O methods rebind by suspendable-port . That is to say, every I/O operation should have the ability to return the current I/O status for the scheduler. That's why I put it in read-bytes and write-bytes. If we add a new put-bytevector* function, and if it is the only function which returns current I/O status, then we can't tell the scheduler the progress when users are trying to access database for a bigger data, or access a file on the disk, or any other potential I/O operation registered in HTTP handler by users. I think it's better to patch read-bytes and write-bytes for any I/O operations, otherwise, we have to patch all the overridden functions in suspendable-ports to make sure all I/O operations are handled correctly by suspendable-port. Hope I understand your idea correctly. Best regards. On Wed, May 15, 2019 at 4:23 AM Mark H Weaver wrote: > > Hi Nala, > > Nala Ginrut writes: > > > On Tue, May 14, 2019 at 7:01 AM Mark H Weaver wrote: > >> I guess what you want is the ability to see incremental reports on the > >> progress of your large I/O operations. Is that right? If we are going > >> to add an API for this, it needs to be reliable, and always give reports > >> in terms of the high-level requests that the user gave. > > > > Yes, that's exactly what I want. We need to get the progress of I/O > > operation when it's blocking > > so that we can compute a fair priority for the tasks. > > > >> My preferred approach would be something like this: we could add a > >> 'put-bytevector-some' I/O primitive which writes some bytes from a > >> bytevector, blocking only as needed to write at least one byte. It > >> would return the number of bytes written. This can be used to implement > >> an efficient variant of 'put-bytevector' that gives you access to the > >> real-time progress information. > > > > I'm not sure if put-bytevector-some does the work, I'll list my concerns: > > > > 1. All I/O will be managed by Guile when we enabled suspendable-port. > > That is to say, from the users side, users never know their I/O > > operations are blocking or not. It's transparent to users. > > Guile will guarantee the I/O operations to be finished by managing all > > the blocking I/O mechanisms. > > Users can only interact with the task with read or write waiter, which > > are registered by users themselves. > > In this scenario, users are out of control of I/O operations. And they > > have no way to get the progress of I/O, since there's > > no way to pass this status to the waiter function except for > > parameters in my patch. > > > > 2. suspendable-port module has already provided a bunch of overridden > > bytevector-* functions. > > However, they're hidden from users. I think it's good since the > > purpose of suspendable-port is to abstract all these details from > > users. Users only consider the read-waiter and write-waiter for scheduling. > > If we provide the low-level bytevector functions to users to let them > > do the non-blocking I/O by themselves, just like most C framework > > does. Then Guile suspendable-port will lose a critical feature, > > although users can still implement asynchronous non-blocking I/O by > > themselves with a non-managed approach. Say, do the I/O, check result > > by themselves, and do the scheduling. > > Personally, I'm fine with this way, since I'm familiar with both ways. > > But managed I/O of suspendable-port is a good selling point for many > > inexperienced server-side developers, they can use it in Scheme just > > like IOCP or AIO. > > > > Of course, I may misunderstand your mind. > > Could you elaborate more about your approach? > > Here's what I had in mind. I'm not suggesting that you use asynchronous > non-blocking I/O. 'put-bytevector-some' would be like 'put-bytevector' > except that it can write less than you asked it to. Ideally it would > block only as needed to write at least one byte. It would be analogous > to POSIX write(2). > > Here's an untested draft implementation of 'put-bytevector*' which sets > the parameter 'current-io-status' to provide the information you wanted, > except that here the information will reliably refer to your own buffer. > The primitive it is built on 'put-bytevector-some' does not yet exist, > but we could add it. Note that 'put-bytevector-some' would still block, > and it would be suspendable. > > --8<---cut here---start->8--- > (define* (put-bytevector* port bv > #:optional > (start 0) > (count (bytevector-length bv))) > (let loop ((start start) (count count
Re: [PATCH] Add current-suspendable-io-status parameter
On Mon, May 13, 2019 at 06:54:38PM +0800, Nala Ginrut wrote: > Hi folks! > Here's a patch to add current-suspendable-io-status: > Its result is a pair: (finished-bytes . rest-bytes) Sorry for this possibly dumb question, but... is there a way to be non-blocking even if there are no readable/writable bytes at all? Or would one have to do multi-threading (and let the single threads [1] block) for that? Thanks [1] not necessarily "operating system" threads -- tomás signature.asc Description: Digital signature
Re: [PATCH] Add current-suspendable-io-status parameter
On Wed, 15 May 2019 12:09:19 +0200 wrote: > On Mon, May 13, 2019 at 06:54:38PM +0800, Nala Ginrut wrote: > > Hi folks! > > Here's a patch to add current-suspendable-io-status: > > Its result is a pair: (finished-bytes . rest-bytes) > > Sorry for this possibly dumb question, but... is there a way > to be non-blocking even if there are no readable/writable > bytes at all? Or would one have to do multi-threading (and > let the single threads [1] block) for that? > > Thanks With guile-2.2/3.0, you install suspendable ports and parameterize current-read-waiter and/or current-write-waiter. There is an example here: https://github.com/ChrisVine/guile-a-sync2/blob/master/a-sync/await-ports.scm fibers does something similar. See also https://www.gnu.org/software/guile/docs/master/guile.html/Non_002dBlocking-I_002fO.html#Non_002dBlocking-I_002fO Chris
Re: [PATCH] Add current-suspendable-io-status parameter
hi Tomas! For Guile, if you enabled suspendable-port, you may schedule the blocking task captured by delimited continuation. And use I/O multiplex mechanism (say, select or epoll) for monitoring the file descriptor (or port). If there's no available byte at all, then you would never be mentioned by I/O multiplexer. That is to say, you can do other works before it mentions you some time. On Wed, May 15, 2019 at 6:09 PM wrote: > > On Mon, May 13, 2019 at 06:54:38PM +0800, Nala Ginrut wrote: > > Hi folks! > > Here's a patch to add current-suspendable-io-status: > > Its result is a pair: (finished-bytes . rest-bytes) > > Sorry for this possibly dumb question, but... is there a way > to be non-blocking even if there are no readable/writable > bytes at all? Or would one have to do multi-threading (and > let the single threads [1] block) for that? > > Thanks > > [1] not necessarily "operating system" threads > > -- tomás
Re: [PATCH] Add current-suspendable-io-status parameter
On Wed, May 15, 2019 at 12:25:05PM +0100, Chris Vine wrote: > On Wed, 15 May 2019 12:09:19 +0200 > wrote: > > Sorry for this possibly dumb question [...] > With guile-2.2/3.0, you install suspendable ports and parameterize > current-read-waiter and/or current-write-waiter. There is an example > here [...] Thanks :) Cheers -- t signature.asc Description: Digital signature
Re: [PATCH] Add current-suspendable-io-status parameter
On Wed, May 15, 2019 at 07:25:37PM +0800, Nala Ginrut wrote: > hi Tomas! > For Guile, if you enabled suspendable-port, you may schedule the > blocking task captured by delimited continuation. > And use I/O multiplex mechanism (say, select or epoll) for monitoring > the file descriptor (or port). So basically it's like C -- you call read() or write() when select/epoll tells you that this socket wants to play with you. Thanks! Thanks to Chris qnd you Cheers -- tomás signature.asc Description: Digital signature
Re: [PATCH] Add current-suspendable-io-status parameter
Similar, but in Guile, you don't have to care about if it's blocking and schedule it by yourself, while you have to care about it explicitly in C. 于 2019年5月15日周三 20:10写道: > On Wed, May 15, 2019 at 07:25:37PM +0800, Nala Ginrut wrote: > > hi Tomas! > > For Guile, if you enabled suspendable-port, you may schedule the > > blocking task captured by delimited continuation. > > And use I/O multiplex mechanism (say, select or epoll) for monitoring > > the file descriptor (or port). > > So basically it's like C -- you call read() or write() when select/epoll > tells you that this socket wants to play with you. Thanks! > > Thanks to Chris qnd you > > Cheers > -- tomás >
Re: [PATCH] Add current-suspendable-io-status parameter
Hi Nala, Nala Ginrut writes: > I think your approach works for the cases which use put-bytevector* directly. > Are you assuming the server-core only handles suspendable-port with > put-bytevector in http-read or http-write? > If so, then your approach is not enough. If you're not using 'put-bytevector', then the 'start' and 'count' variables in {read,write}-bytes will not relate to your buffer, and therefore will not be meaningful. For example, suppose you use 'put-string' to write 20 thousand characters to a socket. While that operation is pending, presumably you would like information on how many of those characters have been written, i.e. you want a number in the range 0 to 2. Am I right? 'start' and 'count' from {read,write}-bytes will not tell you how much of that string has been written. What will happen is this: the initial characters of the string will be converted into the port encoding, as much as will fit in the port "auxiliary write buffer", which is 256 bytes long. Those ~256 bytes will be written using 'write-bytes', so 'start' will typically be 0 and 'count' will be a number <= 256. That will happen repeatedly until the entire string has been written. Do you see now why your approach will not work? In general, the I/O operations visible to the user may be broken up into smaller operations within Guile. If we are to provide I/O progress information, we must provide information that relates to the operations that the user knows about. The fundamental problem with your proposed patch is that it does not do that. > That is to say, every I/O operation should have the ability to return > the current I/O status for the scheduler. It's a reasonable wish. You'd like to provide progress information for all I/O operations in a single stroke, without having to write something like 'put-bytevector*' for each I/O primitive. It would be nice if 'start' and 'count' from {read,write}-bytes were able to provide universally-applicable progress information for any I/O operation. I've explained above why it doesn't. I thought a bit about what a universally-applicable progress indicator would look like, and I can't think of a good way to do it. It might be better to start by thinking about some specific examples of other I/O operations. I've already given the example of writing a string, but let's think a bit more about that case. For textual I/O, arguably the progress information should be given as a number of *characters* written. We could also in theory provide the number of bytes written, but we can't say how many bytes remain to be written, or what the total number of bytes will be. We can only say how many *characters* remain to be written. What about when we write something more structured, such as writing a large S-expression using 'write', or serializing XML from SXML. In those cases, it's even more obvious that we have no idea how many bytes, nor how many characters will be written, until we're finished. Both of these operations result in a large number of small write operations, and clearly, status information for each of those small write operations is not useful. In your opinion, how should I/O progress information be represented in these cases? Can you see now why your approach doesn't work for most I/O operations? Regards, Mark
Re: [PATCH] Add current-suspendable-io-status parameter
Hi Mark! I need some time to think about all the situations in front of us. And here's a quick reply to one of your questions: On Thu, May 16, 2019 at 8:59 AM Mark H Weaver wrote: > What about when we write something more structured, such as writing a > large S-expression using 'write', or serializing XML from SXML. In > those cases, it's even more obvious that we have no idea how many bytes, > nor how many characters will be written until we're finished. Both of > these operations result in a large number of small write operations, and > clearly, status information for each of those small write operations is > not useful. At least in Artanis server-core (Ragnarok), every blocking I/O operation from users (the downstream developer of Artanis) will be scheduled by delimited-continuations with suspendable-port. I've made a lot of works to make sure users don't have to care about any blocking I/O jobs by themselves, this would be a great selling point of both Artanis and Guile. So even if they're trying to load/write very small bytes, the task continuation will be scheduled by suspendable-port, and it can not be changed because Guile manages all asynchronous I/O. Although this looks unnecessary for small bytes, we can make a fair scheduler to smooth the situation. That's why we still need to care for the small bytes I/O.
Re: [PATCH] Add current-suspendable-io-status parameter
Hi Nala, Nala Ginrut writes: > On Thu, May 16, 2019 at 8:59 AM Mark H Weaver wrote: > >> What about when we write something more structured, such as writing a >> large S-expression using 'write', or serializing XML from SXML. In >> those cases, it's even more obvious that we have no idea how many bytes, >> nor how many characters will be written until we're finished. Both of >> these operations result in a large number of small write operations, and >> clearly, status information for each of those small write operations is >> not useful. > > At least in Artanis server-core (Ragnarok), every blocking I/O > operation from users > (the downstream developer of Artanis) will be scheduled by > delimited-continuations > with suspendable-port. I've made a lot of works to make sure users > don't have to care about > any blocking I/O jobs by themselves, this would be a great selling > point of both Artanis and Guile. > > So even if they're trying to load/write very small bytes, the task > continuation will be scheduled by > suspendable-port, and it can not be changed because Guile manages all > asynchronous I/O. > > Although this looks unnecessary for small bytes, we can make a fair > scheduler to smooth the situation. > That's why we still need to care for the small bytes I/O. Sorry, I failed to communicate clearly. I agree that we need to care about small I/O operations. When I wrote that "status information for each of those small write operations is not useful", I meant, for example, that if we are writing a large S-expression containing nested list structure with 20 thousand symbols, it is not useful to find out that "we have written 5 out of 18 bytes [in the current symbol]", if we do not also know which symbol is the "current symbol". Similarly, if we write a string containing 20 thousand characters, it is not useful to find out that "we have written 30 out of 254 bytes [from the auxiliary write buffer]", if we do not also know how much of the string we have written. That's the kind of information that your proposed mechanism would provide, and that I said is "not useful". If you disagree, please try the following exercise: show me code that uses your proposed mechanism and suspendable ports to provide real time reports of what percentage of a string has been written, e.g. when using 'put-string'. Verify that it works by writing a large string (~100 million characters) over a bandwidth-limited port with UTF-8 encoding, such that it takes at least 10 seconds to write the entire string. Regards, Mark