Tom, Without digging into the details too much, can you describe the Lustre setup? As Bernd alluded to, it might be something in the configuration or version that is hampering better performance. But then again, you may not want to upgrade to a newer because of disruption. But maybe there are some things that can be done to improve Lustre performance.
Thanks! Jeff On Thu, Jun 12, 2014 at 7:42 AM, Bernd Schubert <[email protected]> wrote: > Hello Tom, > > this is rather self advertising, but you might want to try out > FhGFS/BeeGFS. We spent lots of development resources to have good handling > of zillions of small files. > > > On 06/11/2014 09:03 PM, Tom Harvill wrote: > >> >> Hello, >> >> This is my first time posting to this list, thanks in advance for any >> time you spend >> replying. >> >> We've found that a large majority of our files (~40MM of ~50MM) are less >> than 10KB. >> We believe our filesystem (lustre) is bottlenecked with IOPs and locking >> related to >> > > It doesn't even need to be locking, just disk seeking for directory > entries and inodes might be the bottleneck. Try to monitor your OSTs with > 'iostat -x'. If you see a high "%util" field and and around 8 blocks (512 > byte) in the avgrq-sz field your disk head is probably seeking around most > of the time (avgrq-sz field gets mixed up with other IOs, so you would need > to use blktrace to the for the number of meta requests, but blktrace output > is not that easy to parse...). > > If you only have a single meta server that also might be the bottleneck, > of course. > > > > Cheers, > Bernd > > _______________________________________________ > Beowulf mailing list, [email protected] sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf >
_______________________________________________ Beowulf mailing list, [email protected] sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
