On Thu, May 12, 2011 at 7:48 AM, Anton Lundin <[email protected]> wrote: > Hi. > > I saw a serous performance problem with our afs fileservers and have done > some "tuning" that I would like to share and discuss. > > We are currently running OpenAFS 1.4.14, on Ubuntu 10.04 amd64.
if you'd like something higher-performing, try a 1.6.0 prerelease. you probably want pre5 but i'm not sure there are .debs yet; there are no rpms yet, and that's why it hasn't been announced. > We run some hpc-clusters with our software-repos in afs. When we > reinstall a node, some of that software is copied to local filesystem on > that node. Not much of that, about 1 to 4 GB. > > It all went very slow so I started to look at our afs fileservers and > saw that they where maxing-out at about 9 MB/s. On modern hardware thats > very slow, so I checked if it did matter how many clients that were > accessing the fileserver. As it turned out it didn't matter if it were 1 > or 10 clients copying data from afs, the fileserver maxed out at 9 MB/s. > > I tested to turn of encryption on the clients, and woha, I got a wopping > 50 MB/s, but it still didn't matter if I ran one or 10 clients. > > 50 MB/s is much better than 9 MB/s, but still not even close to what > modern hardware should handle, and when looking at the fileserver, it > didn't bottom out anything obvious. > > Then on a hunch I tested to offline cores in that quad-core server. > 4 cores 50 MB/s, 3 cores 60 MB/s, 2 cores 80 MB/s, 1 core 110 MB/s. > > > Is des-crypto that hard? Should it impact performance that much? > The smp-scaling in the fileserver is really bad. Have anyone done any > profiling on what is causing this? Is any work getting done on this? fcrypt, not des. there's nothing optimized for fcrypt, so, not surprisingly, it's not that fast. GSS prf-based crypto is being integrated with the next release series. -- Derrick _______________________________________________ OpenAFS-devel mailing list [email protected] https://lists.openafs.org/mailman/listinfo/openafs-devel
