Re: High Performance I/O (more)
>Events could also (I haven't thought this out, so please forgive me if >there's an obvious bugaboo here) return additional information about the >descriptor/whatever. One nice possibility would be outgoing buffer space >on sockets. This may or may not be worth the coding effort. > First, If you are using POSIX real time signals, you can get extra information by using sigwaitinfo(). In a somewhat nonportable fashion, the Linux people (Stephen Tweedie, in particular) added two things: (1) An fcntl() value called F_SETSIG that delivers a real time signal whenever data to the file descriptor is ready for read or write, (2) when the signal is delivered the si_band of the signal is filled in with the poll() values. >If implemented in a multiprocessor aware manner, events could also allow >for more parallelism that we have now. > Two of the strong benefits of asynchronous I/O over the nonblocking/select() is that (1) there is no context switch to send the data, since the kernle takes care (as opposed to the select() philosophy of the program taking care of it) and (2) if you are running on an MP machine, the kernel can be using one processor to take care of asynchronous data while the application uses the other. >-Chris > -jason To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: High Performance I/O (more)
>Events could also (I haven't thought this out, so please forgive me if >there's an obvious bugaboo here) return additional information about the >descriptor/whatever. One nice possibility would be outgoing buffer space >on sockets. This may or may not be worth the coding effort. > First, If you are using POSIX real time signals, you can get extra information by using sigwaitinfo(). In a somewhat nonportable fashion, the Linux people (Stephen Tweedie, in particular) added two things: (1) An fcntl() value called F_SETSIG that delivers a real time signal whenever data to the file descriptor is ready for read or write, (2) when the signal is delivered the si_band of the signal is filled in with the poll() values. >If implemented in a multiprocessor aware manner, events could also allow >for more parallelism that we have now. > Two of the strong benefits of asynchronous I/O over the nonblocking/select() is that (1) there is no context switch to send the data, since the kernle takes care (as opposed to the select() philosophy of the program taking care of it) and (2) if you are running on an MP machine, the kernel can be using one processor to take care of asynchronous data while the application uses the other. >-Chris > -jason To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
High Performance I/O (more)
I did research this weekend on high performance I/O. I looked at differerent approaches and to me they all appear the same (I know that I will get some flamage for this). The two most prominent models that I saw were IO Completion Ports and Synchronous Events (such as the Gaurav http://www.cs.rice.edu/~gaurav/papers/usenix99.ps). I think that both of these models are basically the same. They both have an event queue that you pick up events from. The only way that they differ is in what they call an event. Completion ports take asynchronous opperations and queue an event when the opperation completes (hence the name). Synchronous events do the opposite: they queue an event when an opperation is possible and then the synchronous (usually, non-blocking) opperation is performed. From this, you can decouple and event queue from what you call an event.
High Performance I/O (more)
I did research this weekend on high performance I/O. I looked at differerent
approaches and to me they all appear the same (I know that I will get some
flamage for this). The two most prominent models that I saw were IO
Completion Ports and Synchronous Events (such as the Gaurav
http://www.cs.rice.edu/~gaurav/papers/usenix99.ps).
I think that both of these models are basically the same. They both have
an event queue that you pick up events from. The only way that they differ
is in what they call an event. Completion ports take asynchronous opperations
and queue an event when the opperation completes (hence the name). Synchronous
events do the opposite: they queue an event when an opperation is possible
and then the synchronous (usually, non-blocking) opperation is performed.
>From this, you can decouple and event queue from what you call an event.
>From what I can see either model will give roughly the same performance,
as they both do roughly the same amount of work. The one benefit that seems
to exist for the Completion Ports model is that there are fewer contex
switches.
Now, looking at POSIX.1b signals and signal queues and getting some
information from Stephen Tweedie it looks like completion ports are doable
without anything new, I think that I have decided.
If you find an available signal, set the handler for it, the block it,
this signal number now effectively becomes the completion port. You then
can fcntl() a file descriptor with F_SETSIG and the signal number. Then to
fetch the blocked signals, use sigwaitinfo(). I guess you could also use
aio_{read,write}() and set sigevent appropriately. This actually seems
preferable since you can then use aio_return() to find the return value
out and use aio_cancel() to cancel the request if wanted.
The one drawback that I see to this is that it can only really handle
aio_{read,write}() and {read,write}()/fcntl(). Any other events such as
thread/child deaths cannot really be worked into this scheme unless you
could set the signal they deliver on termination.
If you really wanted to, you could have signals delivered for the ability
to read/write to a file descriptor and then you would have Gaurav's model.
Basically, unless anybody can see anything wrong with this get to work
implementing!
-jason
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
aio_*
While reading through (at least trying to... I wish there was some sort of kernel documentation available, the entry fee is very high) the aio_* calls, I had a few questions to clear up my understanding: 1) Do they only work on files? The only implementation I see is in the VFS layer. 2) It is my understanding that it uses an aio daemon running as a kernel thread (the aio_daemon() call kind of give that one away). It seems as if this can be almost entirely done in user space. More important to what I am trying to do, it seems as if aio_* does not give peak latency or throughput performace, since the aio_daemon has to compete for resources along with all other processes. Should aio_* be used for applications that have high performance requirements? What does aio_* get you above having a seperate thread pumping in/out data? -jason To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Just mailed authors of USENIX99 event i/o paper
I just mailed drusc...@cs.rice.edu to ask about extra information, work, or references that he may have on the event queues paper that he was a coauthor of. Just saying this so other people dont just have about a dozen messages about the same thing. Please notice that this message is going to both freebsd-hackers and linux-kernel, so if you reply, please remember to change the To line appropriately. -jason To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Just mailed authors of USENIX99 event i/o paper
I just mailed [EMAIL PROTECTED] to ask about extra information, work, or references that he may have on the event queues paper that he was a coauthor of. Just saying this so other people dont just have about a dozen messages about the same thing. Please notice that this message is going to both freebsd-hackers and linux-kernel, so if you reply, please remember to change the To line appropriately. -jason To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: message queues for I/O (usenix paper)
thank for the pointers to the mailing list... very enlightening. I am going to try to write up an API for it this weekend with some cooperation with others. I will then give it a first pass around the freebsd-hackers and linux-kernl mailing lists. There are a few unanswered questions as to what an event it and how to handle them between threads and processes, but the general structure of an event queue seems to be wanted by almost everybody. I am not convinced by the usenix paper that this is the right way to do things, yet. state management is a bitch and taken too lightly most of the time. -jason To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: message queues for I/O (usenix paper)
thank for the pointers to the mailing list... very enlightening. I am going to try to write up an API for it this weekend with some cooperation with others. I will then give it a first pass around the freebsd-hackers and linux-kernl mailing lists. There are a few unanswered questions as to what an event it and how to handle them between threads and processes, but the general structure of an event queue seems to be wanted by almost everybody. I am not convinced by the usenix paper that this is the right way to do things, yet. state management is a bitch and taken too lightly most of the time. -jason To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: message queues for I/O (usenix paper)
>Yes. I don't particularly like some of the things in the paper, >although it does have several good concepts. I have an implementation >that does exactly this, and have a line on two other implementations >that do the same thing (but in a different fashion). Unfortunately, >all of these are somewhat problem-specific and are not a general >solution. > >I've spent some time working on a generic implementation that draws >its ideas from several places. I hope to be in a position where I >can work on this almost full time within a month. >-- >Jonathan > Would you like to share your implementation and the line you have on two other implementation that do the same thing (but in a different fassion)? I (and others I know) would be very interested to know what you have come across and what these other ideas are. -jason To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: message queues for I/O (usenix paper)
>Yes. I don't particularly like some of the things in the paper, >although it does have several good concepts. I have an implementation >that does exactly this, and have a line on two other implementations >that do the same thing (but in a different fashion). Unfortunately, >all of these are somewhat problem-specific and are not a general >solution. > >I've spent some time working on a generic implementation that draws >its ideas from several places. I hope to be in a position where I >can work on this almost full time within a month. >-- >Jonathan > Would you like to share your implementation and the line you have on two other implementation that do the same thing (but in a different fassion)? I (and others I know) would be very interested to know what you have come across and what these other ideas are. -jason To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
message queues for I/O (usenix paper)
There is alot of talk going on over at the linux-kernel mailing list about implementing synchronous messaging for I/O. They are talking about a paper that was presented at USENIX: http://www.cs.rice.edu/~gaurav/papers/usenix99.ps The general idea is that select() and poll() fall over with large numbers of file descriptors for two reasons. First, scanning the interest list begins to consume more time. Second, the stateless nature between calls means that alot of redundant processing occurs. The solution these guys (the authors) say is to have a way of registering interest in descriptors, then you can call a procedure to find out what has changed since last time. I personally think that select() is just fine and can be implemented more efficiently than currently, but I would be willing to give it a shot at both cooperating with the Linux people to get a good Linux/FreeBSD API layed down and then implementing it. I know some of you heard this paper presented so does anybody have any ideas about it? Does anybody care? -jason To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
message queues for I/O (usenix paper)
There is alot of talk going on over at the linux-kernel mailing list about implementing synchronous messaging for I/O. They are talking about a paper that was presented at USENIX: http://www.cs.rice.edu/~gaurav/papers/usenix99.ps The general idea is that select() and poll() fall over with large numbers of file descriptors for two reasons. First, scanning the interest list begins to consume more time. Second, the stateless nature between calls means that alot of redundant processing occurs. The solution these guys (the authors) say is to have a way of registering interest in descriptors, then you can call a procedure to find out what has changed since last time. I personally think that select() is just fine and can be implemented more efficiently than currently, but I would be willing to give it a shot at both cooperating with the Linux people to get a good Linux/FreeBSD API layed down and then implementing it. I know some of you heard this paper presented so does anybody have any ideas about it? Does anybody care? -jason To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
