Just to add some more datapoints to the discussion...
Our webservers are HP DL360G5's, 14GB RAM. Pair of 36GB 15K 2.5" SAS
drives in RAID-1. /usr/vice/cache is ext2 with noatime,nodiratime. These
machines run dovecot IMAP, apache with lots of php applications, RT, and
vsftpd serving anon and private ftp accounts.
Serving content that isn't in the cache yet, we can get about 70-80MB/s
depending on which fileserver it's coming from, and after it's cached, the
gigabit network becomes the bottleneck. The cache partition is ~34GB in
size, and we're running with these options:
-dynroot -dynroot-sparse -fakestat -afsdb -nosettime -daemons 20
-stat 48000 -volumes 2048 -chunksize 19 -rxpck 2048
With those cache manager settings, cache partition utilization is sitting
at about 92%. I can get even better numbers with memcache, and indeed most
of our other machines are running with 2GB of memcache, I like seeing read
performance in GB/s, and when most of your machines have 32GB or more (we
have 3 with 256GB), a couple GB here and there won't have a noticeable
impact.
Jason: do you know in particular what kind of workload is causing issues
for you? You mentioned your wait times are on the order of seconds, are
you sure that's caused by the underlying disk? At the very least, I would
try mounting your cache partition as ext2, as has already been suggested.
Turning off atime and diratime shouldn't hurt, and if your disks are
having issues with seaks, this should help some.
Also, you really want to run 1.6.0pre7, or 1.6.0 when it shows up. Nothing
wrong with 1.4, but if you're trying to get the most performance out of
afs on modern hardware, switching to 1.6 gives you some real cheap gains.
There are huge performance improvements on Linux going from 1.4 to 1.6,
and all of my new installations are 1.6.0pre7 for that reason. Especially
with disk-based caches, as Simon mentioned. 1.6.0pre7 gets write
performance for disk caches almost on par with memcache, though read
performance is still lacking, as memory will almost always be faster than
disk, but disk will always be 'cheaper' than memory.
Worth a try at least, and pre7 has been very stable in our environment.
--andy
_______________________________________________
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info
