Hi. I saw a serous performance problem with our afs fileservers and have done some "tuning" that I would like to share and discuss.
We are currently running OpenAFS 1.4.14, on Ubuntu 10.04 amd64. We run some hpc-clusters with our software-repos in afs. When we reinstall a node, some of that software is copied to local filesystem on that node. Not much of that, about 1 to 4 GB. It all went very slow so I started to look at our afs fileservers and saw that they where maxing-out at about 9 MB/s. On modern hardware thats very slow, so I checked if it did matter how many clients that were accessing the fileserver. As it turned out it didn't matter if it were 1 or 10 clients copying data from afs, the fileserver maxed out at 9 MB/s. I tested to turn of encryption on the clients, and woha, I got a wopping 50 MB/s, but it still didn't matter if I ran one or 10 clients. 50 MB/s is much better than 9 MB/s, but still not even close to what modern hardware should handle, and when looking at the fileserver, it didn't bottom out anything obvious. Then on a hunch I tested to offline cores in that quad-core server. 4 cores 50 MB/s, 3 cores 60 MB/s, 2 cores 80 MB/s, 1 core 110 MB/s. Is des-crypto that hard? Should it impact performance that much? The smp-scaling in the fileserver is really bad. Have anyone done any profiling on what is causing this? Is any work getting done on this? //Anton -- Anton Lundin +46702-161604 _______________________________________________ OpenAFS-devel mailing list [email protected] https://lists.openafs.org/mailman/listinfo/openafs-devel
