On Fri, 17 Sep 2010, Graham Leggett wrote:
On 17 Sep 2010, at 1:41 PM, Niklas Edmundsson wrote:
I personally favor designs needs at most O_EXCL style write locking.
Having been bitten by various lock-related issues over the years I'm in
favor of a explicit-lock-free design if it can be done cleanly and with
good performance.
If going this route, I'd suggest to put the entire path to the data file in
the header and not just a uniqifying string (to make it easier to split
hashing of header and data in the future).
The problem with this is that if you bake the location of the file into the
cache, you would never be able to move the cache around.
Just store the path relative to the cache root.
Is there a benefit to keeping headers and bodies separate?
As we cache files from an nfs mount, we hash on device:inode as a
simple method of reducing duplicates of files (say a dozen URL:s all
resolving to the same DVD image). We see a huge benefit of being able
to do this as we get a grotesque amount of data duplication otherwise.
So we usually have multiple header files all pointing to the same data
file.
For the more generic cache it might also be useful provided that you
have a mechanism to identify duplicated data, the only thing I can
think of is hashing on the data block but that isn't really feasible
for large files. I suspect there might be cases where there exists
usecases with a backend that can provide hints for this though.
/Nikke
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se | [email protected]
---------------------------------------------------------------------------
"Come on, higher now! A watcher scoffs at gravity!" - Giles
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
