On 06/12/2014 03:09 PM, Jeffrey Layton wrote:
Tom,
Without digging into the details too much, can you describe the Lustre
setup? As Bernd alluded to, it might be something in the configuration or
version that is hampering better performance. But then again, you may not
want to upgrade to a newer because of disruption. But maybe there are some
things that can be done to improve Lustre performance.
The problem with millions of files are the hash directories Lustre is
creating files in - it is a hard job for the kernel to cash the random
access there.
Here's a (not accepted) patch to improve this issue:
http://lists.openwall.net/linux-ext4/2011/06/20/31
Some patches landed in kernel version 3.11, that improved the situation
a bit if you set low /proc/sys/vm/vfs_cache_pressure values
(see http://lwn.net/Articles/551016/ ).
But then if you have a very high number of files, there is simply no way
to cache dentries/inodes properly. That is way we decided to re-work our
entire storage layout in FhGFS and to avoid hash directories at all.
I don't claim that Tom has suffers from that issue, but with a high
number of files that is just a common issue.
Cheers,
Bernd
_______________________________________________
Beowulf mailing list, [email protected] sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
