not sure I understand Ur setup which one is the raid6 lun? which are the individual HD?
Rafael David Tinoco wrote: > 216MB/s using 8*128 (1024k) as bs. Too low for 8 active disks.. right ? > Arround 27MB/s.. from 50MB/s in the "real" disk. > > -----Original Message----- > From: lustre-discuss-boun...@lists.lustre.org > [mailto:lustre-discuss-boun...@lists.lustre.org] On Behalf Of Dr. Hung-Sheng > Tsao > (LaoTsao) > Sent: Thursday, September 10, 2009 5:50 PM > To: Rafael David Tinoco > Cc: lustre-discuss@lists.lustre.org > Subject: Re: [Lustre-discuss] OST - low MB/s > > raid 6 chunk size=128k the full strip size will be 128k*8 (for 10 disks > 8+2 raid 6) > in Ur dd test one should use bs=128k*8, then each 8 HDD will be busy > regards > > > Rafael David Tinoco wrote: > >> With this RAID5 configuration Im getting: >> >> r...@a02n00:~# dd if=/dev/zero of=/dev/md20 bs=128k count=10000 >> >> 10000+0 records in >> >> 10000+0 records out >> >> 1310720000 bytes (1.3 GB) copied, 5.20774 seconds, 252 MB/s >> >> r...@a02n00:~# dd if=/dev/zero of=/dev/md20 bs=128k count=10000 >> >> 10000+0 records in >> >> 10000+0 records out >> >> 1310720000 bytes (1.3 GB) copied, 5.12 seconds, 256 MB/s >> >> So, 80MB/s using these md20 as OSTs isnt quite right . >> >> *From:* lustre-discuss-boun...@lists.lustre.org >> [mailto:lustre-discuss-boun...@lists.lustre.org] *On Behalf Of *Rafael >> David Tinoco >> *Sent:* Thursday, September 10, 2009 4:26 PM >> *To:* lustre-discuss@lists.lustre.org >> *Subject:* [Lustre-discuss] OST - low MB/s >> >> Hello, >> >> I'm having problems now with my "OSTs" throughput. >> >> I have 4 OSS each one with 2 OSTs. These OSTs are RAID6 with 10 disks, >> chunk size of 128k. >> >> These disks are from J4400 (JBOD) connected in multipath using multipathd. >> >> Each disk speed is giving me 50MB/s with dd. >> >> With lustre, using IOR or DD I can get only arround 80MB/s. I was >> expecting for 8 active disks in raid 8*50 = something between 300 and >> 400MB/s. >> >> avg-cpu: %user %nice %system %iowait %steal %idle >> >> 0.00 0.00 6.00 9.06 0.00 84.94 >> >> Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await >> svctm %util >> >> md10 0.00 0.00 0.00 398.00 0.00 1.55 8.00 0.00 0.00 0.00 0.00 >> >> md11 0.00 0.00 0.00 380.00 0.00 1.48 8.00 0.00 0.00 0.00 0.00 >> >> md20 0.00 0.00 0.00 158.00 0.00 79.00 1024.00 0.00 0.00 0.00 0.00 >> >> md21 0.00 0.00 0.00 159.00 0.00 79.50 1024.00 0.00 0.00 0.00 0.00 >> >> avg-cpu: %user %nice %system %iowait %steal %idle >> >> 0.00 0.00 5.94 9.32 0.00 84.74 >> >> Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await >> svctm %util >> >> md10 0.00 0.00 0.00 407.50 0.00 1.59 8.00 0.00 0.00 0.00 0.00 >> >> md11 0.00 0.00 0.00 394.00 0.00 1.54 8.00 0.00 0.00 0.00 0.00 >> >> md20 0.00 0.00 0.00 159.00 0.00 79.50 1024.00 0.00 0.00 0.00 0.00 >> >> md21 0.00 0.00 0.00 158.00 0.00 79.00 1024.00 0.00 0.00 0.00 0.00 >> >> avg-cpu: %user %nice %system %iowait %steal %idle >> >> 0.00 0.00 6.37 9.43 0.00 84.21 >> >> Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await >> svctm %util >> >> md10 0.00 0.00 0.00 410.50 0.00 1.60 8.00 0.00 0.00 0.00 0.00 >> >> md11 0.00 0.00 0.00 376.00 0.00 1.47 8.00 0.00 0.00 0.00 0.00 >> >> md20 0.00 0.00 0.00 165.00 0.00 82.50 1024.00 0.00 0.00 0.00 0.00 >> >> md21 0.00 0.00 0.00 165.00 0.00 82.50 1024.00 0.00 0.00 0.00 0.00 >> >> Any clues ? >> >> Rafael David Tinoco - Sun Microsystems >> >> Systems Engineer - High Performance Computing >> >> rafael.tin...@sun.com - 55.11.5187.2194 >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss@lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss >> >> > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss@lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss > > _______________________________________________ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss