Having a thread block to page in data that isn't currently in the cache
is fine, as long as other threads are able to handle all the other
requests. What I would really like to see is support for zero copy
sends without using sendfile(). Right now I believe that apache is
calling sendfile() on small chunks of data at a time, so that it can
make sure the transfer is not stalled. This tends to let the pipeline
stall for short periods between each sendfile() call. Also sendfile()
can not be used at all to send data that is not in a file on disk. For
instance, the output of a cgi script, or data that has been encrypted to
send over SSL.
On NT you can direct the kernel to set the send buffer size to zero (
setsockopt I think it was ) and that will prevent it from making the
user->kernel copy when you send. To keep the pipeline full you just
have to overlap multiple async send requests. This allows you to get
sendfile() performance without its drawbacks. I hope to one day see
linux and apache support this.
I have wondered though, what it would be like to be able to
asynchronously page fault in the cache pages to send, rather than
relying on the send() call to synchronously fault in the pages if
needed. That would save threads from blocking on disk IO, which might
be nice.
Brian Pane wrote:
The async write completion stuff that's now in the trunk does nonblocking
network writes, and it uses sendfile to do zero-copy transfer from the
filesystem
cache to the network on platforms that support it, but it doesn't try to
do file
I/O asynchronously. Thus it's possible for the current code to block on a
disk read while trying to do a nonblocking socket write. (The Event MPM
compensates for this by allowing multiple threads to do sendfile calls on
different sockets concurrently; if one thread happens to block on a
physical disk read, it doesn't delay other connections.) Having a means
of doing nonblocking filesystem I/O, via either aio or something like
mincore checks, might be a nice addition.
Brian
