Andreev Nikita wrote:
Hi.
What are the reasons of squid rebuilding it's cache? Sometimes after
startup sometimes during work it starts to rebuild cache. I see it in
cahce.log:
Squid rebuilds the swap.state file when a logrotate (and I think
reconfigure) is called for. Usually it's not a big deal. My main cache
has about 13 million objects and rebuilding the cache takes just over 10
seconds:
2009/01/15 04:02:27| storeDirWriteCleanLogs: Starting...
...
2009/01/15 04:02:39| Finished. Wrote 12918887 entries.
2009/01/15 04:02:39| Took 12.4 seconds (1044380.1 entries/sec).
and
2009/01/16 04:02:29| storeDirWriteCleanLogs: Starting...
...
2009/01/16 04:02:40| Finished. Wrote 12906114 entries.
2009/01/16 04:02:40| Took 11.3 seconds (1138371.9 entries/sec).
2009/01/16 09:30:24| Starting Squid Cache version 2.6.STABLE6 for
x86_64-redhat-linux-gnu...
2009/01/16 09:30:24| Process ID 2625
2009/01/16 09:30:24| With 1024 file descriptors available
2009/01/16 09:30:24| Using epoll for the IO loop
2009/01/16 09:30:24| DNS Socket created at 0.0.0.0, port 60302, FD 5
2009/01/16 09:30:24| Adding domain mydomain.com from /etc/resolv.conf
2009/01/16 09:30:24| Adding nameserver 22.179.1.11 from /etc/resolv.conf
2009/01/16 09:30:24| Adding nameserver 22.179.1.15 from /etc/resolv.conf
2009/01/16 09:30:24| User-Agent logging is disabled.
2009/01/16 09:30:24| Referer logging is disabled.
2009/01/16 09:30:24| Unlinkd pipe opened on FD 10
2009/01/16 09:30:24| Swap maxSize 16777216 KB, estimated 1290555 objects
2009/01/16 09:30:24| Target number of buckets: 64527
2009/01/16 09:30:24| Using 65536 Store buckets
2009/01/16 09:30:24| Max Mem size: 393216 KB
2009/01/16 09:30:24| Max Swap size: 16777216 KB
2009/01/16 09:30:24| Local cache digest enabled; rebuild/rewrite every
3600/3600 sec
2009/01/16 09:30:24| Store logging disabled
2009/01/16 09:30:24| Rebuilding storage in /var/spool/squid (CLEAN)
2009/01/16 09:30:24| Using Least Load store dir selection
2009/01/16 09:30:24| Set Current Directory to /var/spool/squid
2009/01/16 09:30:24| Loaded Icons.
2009/01/16 09:30:26| Accepting proxy HTTP connections at 0.0.0.0, port 8080, FD
11.
2009/01/16 09:30:26| Accepting ICP messages at 0.0.0.0, port 3130, FD 12.
2009/01/16 09:30:26| WCCP Disabled.
2009/01/16 09:30:26| Ready to serve requests.
2009/01/16 09:30:35| Store rebuilding is 0.4% complete
2009/01/16 09:30:53| Store rebuilding is 1.1% complete
2009/01/16 09:31:08| Store rebuilding is 1.5% complete
2009/01/16 09:31:24| Store rebuilding is 2.3% complete
2009/01/16 09:31:44| Store rebuilding is 3.1% complete
2009/01/16 09:32:01| Store rebuilding is 3.4% complete
...
2009/01/16 10:32:41| Store rebuilding is 98.1% complete
2009/01/16 10:33:06| Store rebuilding is 98.5% complete
2009/01/16 10:33:26| Store rebuilding is 98.9% complete
2009/01/16 10:33:53| Store rebuilding is 99.6% complete
2009/01/16 10:34:13| Done reading /var/spool/squid swaplog (1072895 entries)
2009/01/16 10:34:13| Finished rebuilding storage from disk.
2009/01/16 10:34:13| 1072895 Entries scanned
2009/01/16 10:34:13| 0 Invalid entries.
2009/01/16 10:34:13| 0 With invalid flags.
2009/01/16 10:34:13| 1044791 Objects loaded.
2009/01/16 10:34:13| 0 Objects expired.
2009/01/16 10:34:13| 0 Objects cancelled.
2009/01/16 10:34:13| 7253 Duplicate URLs purged.
2009/01/16 10:34:13| 20851 Swapfile clashes avoided.
2009/01/16 10:34:13| Took 3829.6 seconds ( 272.8 objects/sec).
Yikes. That's slow...
2009/01/16 10:34:13| Beginning Validation Procedure
2009/01/16 10:35:04| 262144 Entries Validated so far.
2009/01/16 10:35:17| 524288 Entries Validated so far.
2009/01/16 10:35:25| 786432 Entries Validated so far.
2009/01/16 10:35:43| Completed Validation Procedure
2009/01/16 10:35:43| Validated 1044747 Entries
2009/01/16 10:35:43| store_swap_size = 15131260k
2009/01/16 10:37:58| storeLateRelease: released 4184 objects
I never stop squid abnormally and I never restart server abnormally.
So I don't understand why does squid do it. It takes a lot of CPU
resources and about an hour of time.
What kind of CPU resources? Userspace, system, or wait?
It never happened before. But now
it happens several times a day.
Could somebody help whith that?
P.S. I have 18G partition with 16G granted for squid cache and 2G free
for temporary squid files. So I hope it's enough space for squid to
perform it's everyday tasks.
Ah. A lot of file systems start to perform poorly as they approach
capacity. I'd suggest setting your cache_dir to 13G and let squid clear
out the excess or grow the partition to 30G (if you have the space).
Regards,
LPIC-1, EMCPA
Nikita Andreev
Chris
