re: disk-based fds in select/poll
Pierre Phaneuf <[EMAIL PROTECTED]> wrote: > It's fairly widely-known that select/poll returns immediately when > testing a filesystem-based file descriptor for writability or > readability. > > On top of this, even when in non-blocking mode, read() could block if > the pages needed aren't in core. sendfile() behaves in a similar way. > > What would be needed to alleviate this? > ... > I remember seeing a suggestion by Linus for an event-based I/O > interface, similar to kqueue on FreeBSD but much simpler. I'd just say > "I want it too!", ok? :-) > > SGI's AIO might be a solution here, does it use threads? I'm trying to > avoid context switching as much as possible, to keep the CPU cache as > warm as possible. IMHO, you want AIO. SGI's is fine for now. I hear rumors that there will be something even better coming in 2.5, though I have no details. Or you could use explicit userspace threads... say, divide up your network connections among 8 or so threads. Then if one thread blocks, the others are there to usefully soak up the CPU time. Readiness events for readahead completion on disk files used to seem like a neat idea to me, but now AIO seems more appealing in the long run, since they handle random access properly. - Dan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: disk-based fds in select/poll
> Ok, so while knowing about select "lying" about readability of a file > fd, if I would stick a file fd in my select-based loop anyway, but would You could fix select to return when the page was cachied and return EWOULDBLOCK on reads if the page was not present to be honest. I don't think that would actually break any apps, and the specs seem to allow it > only try to read a bit at a time (say, 4K or 8K) would trigger > readahead, yet finish quickly enough that I can get back to processing > other fds in my select loop? Probably > Wouldn't that cause too many syscalls to be done? Or if this is actually > the way to go without an actual thread, how should I go determining an > optimal block size? fs block size I suspect or small multiple thereof - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: disk-based fds in select/poll
Alan Cox wrote: > > I am thinking that a read() (or sendfile()) that would block because the > > pages aren't in core should instead post a request for the pages to be > > loaded (some kind of readahead mecanism?) and return immediately (maybe > > having given some data that *was* in core). A subsequent read() could > > reads posts a readahead anyway so streaming reads tend not to block much Ok, so while knowing about select "lying" about readability of a file fd, if I would stick a file fd in my select-based loop anyway, but would only try to read a bit at a time (say, 4K or 8K) would trigger readahead, yet finish quickly enough that I can get back to processing other fds in my select loop? Wouldn't that cause too many syscalls to be done? Or if this is actually the way to go without an actual thread, how should I go determining an optimal block size? Was there anything new on the bind_event/get_events API idea that Linus proposed a while ago? That one had got me foaming at the mouth... :-) -- Pierre Phaneuf - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: disk-based fds in select/poll
> I am thinking that a read() (or sendfile()) that would block because the > pages aren't in core should instead post a request for the pages to be > loaded (some kind of readahead mecanism?) and return immediately (maybe > having given some data that *was* in core). A subsequent read() could reads posts a readahead anyway so streaming reads tend not to block much > SGI's AIO might be a solution here, does it use threads? I'm trying to > avoid context switching as much as possible, to keep the CPU cache as > warm as possible. glibc 2.2 does thread based aio_ and it will tend to avoid cache damage as the thread share the mm but on SMP its quite possible the read wil occur on the other CPU. Of course kernel based I/O might do the same too.. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
