[ feel free to use any of this in the mod_perl guide... ]
I've seen in previous threads about sockets and buffering that there was
some confusion in the area, and that it was not clear in anyone's mind
just which socket operations in the server (write(), close()) can block
when the client is on a slow link, and just what help the kernel
buffering gives. So I decided to do some tests, with little perl
scripts talking to each other (via IO::Socket's), and using strace -tt
-T for timing. This is all under Linux btw, with kernel 2.2.15.
This is pretty important, because it lets us decide whether a front-end
server is needed; if things can be setup so that the backend generates
the page, leaves the kernel to spoonfeed it to the slow client, and is
immediately free to go on, then a front-end reverse proxy shouldn't be
needed.
The tests consisted of a client process that connects to a server,
sleeps for a few seconds, then reads from the network. The server
write()s a configurable amount of data, and strace tells us how long the
write() calls took (I never saw close() block).
The results, depending on the size (n) of the written data, are:
n < 57900 : less than 1.3ms
n = 60000 : 6ms
n = 70000 : 25ms
n > 118000 : as long as the client's sleep()
This shows quite clearly that the server's kernel buffer is about 58k,
and the client's kernel read buffer is another 58k. This is a 10baseT
lan; to simulate a slow client, I added some sleep()s on both sides just
after the connection, and used them to pull the ethernet plug on the
client for a few seconds, so the first write() happens while the client
is disconnected. The results (measuring the write() blocking time
again) confirmed this:
n < 57900 : less than 1.3ms
n > 58000 : for as long as the client is disconnected, plus a few
tenths of second (interestingly enough, the delay here
is always an integer number of seconds; Linux must be
retrying its sends once a second).
These 58k are related to the socket's SO_SNDBUF, which is set by default
to 64k. This is good to know too: under Linux 2.2 there's a 6k
discrepancy between the SO_SNDBUF value and how much you can actually
write() without blocking.
This is all nicer than I was expecting: the kernel actually lets the
server do a write() followed by a close(), both in no time (less than
1ms), as long as the write length is less than the socket's SO_SNDBUF.
(I was sort of expecting close() to block until the data was sent).
Since SO_SNDBUFs are configurable (SendBufferSize in Apache, and
/proc/sys/net/core/wmem_{default,max} under Linux), then the obvious
conclusion would be, as long as your documents are of reasonable length
(say, less than 50k) and you tune /proc/sys/net/core/wmem_* and
SendBufferSize high enough, there shouldn't be any need for a front-end
server.
BUT, there's more!
Doing the same tests again, but with an actual Apache as the server, and
with keep-alives turned off (as they should generally be with mod_perl),
it turns out Apache does a lingering close, which is a 2-second select
for read on the client socket. On a fast lan, the client gets the
"Connection: close", reads the data, closes the socket, in a few tenths
of seconds at most. On a slow connection, you can't be sure. My guess
is that the client won't close its socket until it has read the data,
which can easily take a few seconds, so Apache's lingering close would
time out and Apache would wait the 2 seconds, then close() the socket
and go on to the next client.
So the "less obvious" conclusion seems to be that a proxy server is
worth it, if and only if lingering closes are actually lingering for a
significant amount of time in average. 2 seconds is significantly
longer than the time it takes to process a request, after all.
Has anyone looked into that? If not, I can try to find a moment to log
the lingering_close times on a real server...
--
Roger Espel Llima, [EMAIL PROTECTED]
http://www.iagora.com/~espel/index.html
