On 2010-03-02, at 13:45, Ms. Megan Larko wrote: > I logged directly into the OSS (OSS4) and just ran a df (along with a > periodic check of the log files). I last looked about two weeks ago > (I know it was after 17 Feb). Anyway, the OST0007 is more full than > any of the other OSTs. The default lustre stripe (I believe that is > set to 1) is used. Can just one file shift the size used of one OST > that significantly?
Sure, this is easy if the size of a single file is be a large fraction of the OST size. This is one reason why we recommend people use larger OSTs (up to 16TB in 1.8.2 with RHEL5.4) instead of e.g. 1TB or less that is sometimes reported here. > What other reasonable explanation for a difference on one OST in > comparison with the others? Could this cause a lustre performance > hit at this point? It is possible, if the filesystem is getting very full and it causes more seeking to do IO. At the 84% you report below it is starting to get into that range - I wouldn't recommend running the filesystem beyond 90% full unless you are more concerned with space usage than performance. You can find the file(s) that are abnormally large on that particular OST by running (preferably on a client mountpoint on the MDS): lfs find --obd crew8-OST0006_UUID -size +10G /mnt/lustre > [r...@oss4 ~]# df -h > Filesystem Size Used Avail Use% Mounted on > > /dev/sdb1 6.3T 3.6T 2.5T 60% /srv/lustre/OST/crew8- > OST0000 > /dev/sdb2 6.3T 4.1T 1.9T 69% /srv/lustre/OST/crew8- > OST0001 > /dev/sdc1 6.3T 3.3T 2.8T 55% /srv/lustre/OST/crew8- > OST0002 > /dev/sdc2 6.3T 3.3T 2.7T 56% /srv/lustre/OST/crew8- > OST0003 > /dev/sdd1 6.3T 3.5T 2.6T 58% /srv/lustre/OST/crew8- > OST0004 > /dev/sdd2 6.3T 4.1T 1.9T 69% /srv/lustre/OST/crew8- > OST0005 > /dev/sdi1 6.3T 3.9T 2.2T 65% /srv/lustre/OST/crew8- > OST0006 > /dev/sdi2 6.3T 5.0T 1015G 84% > /srv/lustre/OST/crew8-OST0007 <---- > /dev/sdj1 6.3T 3.4T 2.7T 56% /srv/lustre/OST/crew8- > OST0008 > /dev/sdj2 6.3T 3.3T 2.7T 56% /srv/lustre/OST/crew8- > OST0009 > /dev/sdk1 6.3T 3.4T 2.7T 56% /srv/lustre/OST/crew8- > OST0010 > /dev/sdk2 6.3T 3.8T 2.2T 64% /srv/lustre/OST/crew8- > OST0011 Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc. _______________________________________________ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss