> On May 19, 2016, at 12:46 PM, Nathan Dauchy - NOAA Affiliate > <nathan.dau...@noaa.gov> wrote: > > Thanks for pointing out the approach of trying to keep a single file from > using too much space on an OST. It looks like the Log2(size_in_GB) method I > proposed works well up to a point, but breaks down in the capacity balancing > department at some large file size. If we take your rule of thumb that no > file should use more than 1% of an OST, and a typical new-ish target made up > of an 8+2 RAID10 group of 6TB disks, we should divide by 480 GB rather than > 150. (are your targets a couple years old? or am I making a bad assumption > here?)
Yes, my targets are several years old. The 1% number I used is just something I thought was reasonable. It is arbitrary, and you can choose whatever makes the most sense for your site. > Since the size of OSTs is a moving target (pun intended), it would be good to > "future proof" the size selection method. An automated tool like > "lfs_migrate -A" could actually calculate the changeover point from log2() > your method on the fly. stripe_count=MAX( Log2(size_in_GB)+1, size_in_GB/X ) > where "X" is ~1% of the capacity of the smallest OST. This strays away from > simplicity for the user, but the tool hides it for restriping, and initial > writes are (hopefully) tuned more for concurrent access pattern than size > anyway. That’s true. If your restriping will mostly be done by a script, then you don’t necessarily need a simple formula. -- Rick Mohr Senior HPC System Administrator National Institute for Computational Sciences http://www.nics.tennessee.edu _______________________________________________ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
