On Tue, 2 May 2006 11:22:31 +0200 (MEST) Niklas Edmundsson <[EMAIL PROTECTED]> wrote:
> On Mon, 1 May 2006, Davi Arnaut wrote: > > > More important, if we stick with the key/data concept it's possible to > > implement the header/body relationship under single or multiple keys. > > I've been hacking on mod_disk_cache to make it: > * Only store one set of data when one uncached item is accessed > simultaneously (currently all requests cache the file and the last > finished cache process is "wins"). > * Don't wait until the whole item is cached, reply while caching > (currently it stalls). > * Don't block the requesting thread when requestng a large uncached > item, cache in the background and reply while caching (currently it > stalls). > > This is mostly aimed at serving huge static files from a slow disk > backend (typically an NFS export from a server holding all the disk), > such as http://ftp.acc.umu.se/ and http://ftp.heanet.ie/ . > > Doing this with the current mod_disk_cache disk layout was not > possible, doing the above without unneccessary locking means: > > * More or less atomic operations, so caching headers and data in > separate files gets very messy if you want to keep consistency. > * You can't use tempfiles since you want to be able to figure out > where the data is to be able to reply while caching. > * You want to know the size of the data in order to tell when you're > done (ie the current size of a file isn't necessarily the real size > of the body since it might be caching while we're reading it). > > In the light of our experiences, I really think that you want to have > a concept that allows you to keep the bond between header and data. > Yes, you can patch up a missing bond by require locking and stuff, but > I really prefer not having to lock cache files when doing read access. > When it comes to "make the common case fast" a lockless design is very > much preferred. I will repeat once again: there is no locking involved, unless your format of storing the header/data is really wrong. _The data format is up to the module using it_, while the storage backend is a completely different issue. > However, if all those issues are sorted out in the layer above disk > cache then the above observations becomes more or less moot. Yes, that's the point. > In any case the patch is more or less finished, independent testing > and auditing haven't been done yet but I can submit a preliminary > jumbo-patch if people are interested in having a look at it now. -- Davi Arnaut
