Chris Woodfield wrote:
Just tested this - -F appears to work for aufs rebuilds but not COSS
rebuilds.
To reproduce:
- Config squid with an aufs and a coss store, like so:
cache_dir aufs /usr/squidcache 5000 16 256 min-size=500000
cache_dir coss /usr/squidcache/coss1.dat 500 block-size=4096
max-size=500000 membufs=100
cache_swap_log /usr/squidcache/%s.swap
- Start squid, send it a bunch of queries (I use a script that grabs
random URLs from recent access.log files)
- Stop squid, delete /usr/squidcache/usr.squidcache.swap
- Start sending squid requests (again, the random URL script above)
- Restart squid with -F
What I'm seeing when I do the above is that as soon as the AUFS stores
finish rebuilding, squid starts answering queries - but the COSS isn't
rebuilt yet, and until that completes, all objects < 500K are cache
misses (SO_FAIL in store.log).
Will get a bugzilla entry in place for this.
Aha thank you. Methinks this has more to do with the global flag
indicating store rebuild in progress is being unset by the first store
type to finish. :(
Squid needs a set of per-store flags and a function to test whether
any stores are still pending (temporary until the per-store rework is
done). That would solve several other rebuild related bugs at the same time.
Amos
-C
On Apr 22, 2009, at 10:00 AM, Chris Woodfield wrote:
...and sure enough, it's right there in -h output...
cache$ /usr/local/squid/sbin/squid -h
...
-F Don't serve any requests until store is rebuilt.
...
/me goes to write "I will RTFM Before Posting To squid-users" 100
times on the whiteboard... :)
-C
On Apr 22, 2009, at 9:56 AM, Adrian Chadd wrote:
Well, I killed the swaplog writing entirely in Lusca - the COSS
rebuild code doesn't read from it (it was broken for various reasons
revolving mostly around code bitrot IIRC.)
There's a flag you can pass Squid to not handle requests until the
store is rebuilt - its the "-F" flag.
I'm fixing the store rebuild times in Lusca-HEAD at the moment and
this includes writing some new COSS rebuild-from-index, rebuild-from-log
and rebuild-from-rawdevice tools.
Adrian
On Wed, Apr 22, 2009, Chris Woodfield wrote:
On Apr 22, 2009, at 4:56 AM, Amos Jeffries wrote:
Chris Woodfield wrote:
So I'm running with COSS under 2.7STABLE6, we've noticed (as I can
see others have, teh Googles tell me so) that the COSS rebuild a.
happens every time squid is restarted, and b. takes quite a while
if the COSS stripes are large. However, I've noticed that while the
stripes are being rebuilt, squid still listens for and handles
requests - it just SO_FAILs on every object that would normally get
saved to a COSS stripe. So much for that hit ratio.
SO - the questions are:
1. Is there *any* way to prevent the COSS rebuild if squid is
terminated normally?
The indexes are stored in swap.state. Check that it is being done
properly by your Squid.
This could be the issue - when I exit squid, I notice that my
$coss_file.dat and $coss_file.dat.last-clean files all have zero size.
Any idea why this might be happening?
The relevant section of our squid.conf reads as follows:
cache_dir aufs /usr/squidcache.0/cache/ 750000 16 256 min-size=1000000
cache_dir coss /usr/squidcache.0/cache/coss1.dat 30000 block-size=4096
max-size=1000000 membufs=100
cache_dir coss /usr/squidcache.0/cache/coss2.dat 30000 block-size=4096
max-size=1000000 membufs=100
cache_dir coss /usr/squidcache.0/cache/coss3.dat 30000 block-size=4096
max-size=1000000 membufs=100
cache_swap_log /usr/squidcache.0/cache/%s
Thanks,
-C
2. Is there a way to prevent squid from handling requests until the
COSS stripe is fully rebuilt (this is obviously not good if you
don't have redundant squids, but that's not a problem for us) ?
I believe its possible. If its not a local failure to find
swap.state for the COSS dir then it will be a code fix.
Unfortunately we developers are no longer actively working on
Squid-2 without a paid support contract. Also Adrian our storage
expert who would be the best to ask has retired from active
alterations.
Amos
--
Please be using
Current Stable Squid 2.7.STABLE6 or 3.0.STABLE14
Current Beta Squid 3.1.0.7
--
- Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid
Support -
- $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -
--
Please be using
Current Stable Squid 2.7.STABLE6 or 3.0.STABLE14
Current Beta Squid 3.1.0.7
