On Tue, 2 May 2006, Graham Leggett wrote:
If it's:
* Link to latest GNOME Live CD gets published on Slashdot.
* A gazillion users click the link to download it.
* mod_disk_cache starts a new instance of caching the file for each
request, until someone has completed caching the file.
Then this is the thundering herd problem :)
OK :)
Either a site is slashdotted (as in your case), or a cached entry expires,
and suddenly the backend gets nailed until at least one request "wins",
then we are back to normal serving from the cache.
In your case, the "backend" is the disk, while in the bug from 1998, the
backend was another webserver. Either way, same problem.
OK.
Then this patch solves the problem regardless of whether it's a static
file or dynamically generated content since it only allows one
instance to cache the file (OK, there's a small hole so there can be
multiple instances but it's waaaay smaller than now), all other
instances delivers data as the caching process is writing it.
Additionally, if it's a static file that's allowed to be cached in
the background it solves:
* Reduce chance of user getting bored since the data is delivered
while being cached.
* The user got bored and closed the connection so the painfully cached
file gets deleted.
Hmmm - thinking about this we try to cache the brigade (all X GB of it)
first, then we try write it to the network, thus the delay.
Does your patch solve all of these already, or are they planned?
It solves everything I've mentioned. The solution is probably not
perfect for the not-static-file case since it falls back to the old
behaviour of caching the whole file, but it should be a lot better
than the current mod_disk_cache since the rest of the threads get
reply-while-caching. There are issues here with the fact that the
result is discarded if the connection is aborted, but I'm not familiar
enough with apache filter internals to state that you can keep the
result even though the connection is aborted.
/Nikke
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se | [EMAIL PROTECTED]
---------------------------------------------------------------------------
Anything is edible if it's chopped finely enough
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
