Neil Gunton wrote:
Well, the du just finished, it took 214 minutes to complete. I just took
a look at one of the directories in the cache. Now, I have it set for a
depth of 3, so I looked at d/d/d just randomly selected. Then I did a du
there. Here's the output:
server:/var/cache/www/d/d/d# du -h
4.0K ./2BykLs49Xm7cnV6MrWA.header.vary/Y/z/m
8.0K ./2BykLs49Xm7cnV6MrWA.header.vary/Y/z
12K ./2BykLs49Xm7cnV6MrWA.header.vary/Y
16K ./2BykLs49Xm7cnV6MrWA.header.vary
4.0K ./YFPZLpyo_NRtEUoJQQA.header.vary/k/a/y
8.0K ./YFPZLpyo_NRtEUoJQQA.header.vary/k/a
12K ./YFPZLpyo_NRtEUoJQQA.header.vary/k
16K ./YFPZLpyo_NRtEUoJQQA.header.vary
16K ./[EMAIL PROTECTED]@QqLWnrA.header.vary/F/O/b
20K ./[EMAIL PROTECTED]@QqLWnrA.header.vary/F/O
24K ./[EMAIL PROTECTED]@QqLWnrA.header.vary/F
28K ./[EMAIL PROTECTED]@QqLWnrA.header.vary
4.0K ./FrakgI6EKDUjb4dgMXQ.header.vary/G/N/n
8.0K ./FrakgI6EKDUjb4dgMXQ.header.vary/G/N
12K ./FrakgI6EKDUjb4dgMXQ.header.vary/G
16K ./FrakgI6EKDUjb4dgMXQ.header.vary
80K .
So you see, there are actually a lot more directories there than you
might assume based on a 3-level tree! I didn't know it was doing all
this as well, it makes more sense now that it would take a long time to
traverse - we're talking about a huge number of directories after you do
3 levels, one for each letter (large and small case) at each level, then
throw in those additional sub-levels... for EVERY leaf of the 3-level
tree, that's staggering. I need to look into the documentation for
mod_cache to see if there is something I need to tweak with this "vary"
stuff - maybe it's doing more than it has to, but I just don't know.
It seems like this might have something to do with mod_deflate, which I
am using in combination with mod_disk_cache. This page gives a clue that
there might be a problem with the way files are cached when these
modules are both enabled:
http://www.digitalsanctuary.com/tech-blog/general/apache-mod_deflate-and-mod_cache-issues.html
Seems like a very recent post (Nov 18th).
Any ideas? Seems like a big problem, if you're trying to use a reverse
proxy on a large dynamic site, and also optimize bandwidth by using
mod_deflate too.
Neil
