Andreas, if we are using 4kb blocks I understand we only transfer 1 page per RPC call, so are we limited to 10-15K RPC per second or what's the same, 10-15.000 IOPS?
jab -----Original Message----- From: andreas.dil...@sun.com [mailto:andreas.dil...@sun.com] On Behalf Of Andreas Dilger Sent: Friday, March 05, 2010 2:05 AM To: Jeffrey Bennett Cc: oleg.dro...@sun.com; lustre-discuss@lists.lustre.org Subject: Re: [Lustre-discuss] One or two OSS, no difference? On 2010-03-04, at 14:18, Jeffrey Bennett wrote: > I just noticed the sequential performance is ok, but the random IO > (which is what I am measuring) is not. Is there any way to increase > random IO performance on Lustre? We have LUNs that can provide > around 250.000 random read 4kb IOPS but we are only seeing 3.000 to > 10.000 on Lustre. There is work currently underway to improve the SMP scaling performance for the RPC handling layer in Lustre. Currently that limits the delivered RPC rate to 10-15k/sec or so. > -----Original Message----- > From: oleg.dro...@sun.com [mailto:oleg.dro...@sun.com] > Sent: Thursday, March 04, 2010 12:49 PM > To: Jeffrey Bennett > Cc: lustre-discuss@lists.lustre.org > Subject: Re: [Lustre-discuss] One or two OSS, no difference? > > Hello! > > This is pretty strange. Are there any differences in network > topology that can explain this? > If you remove the first client, does the second one shows > performance > at the level of of the first, but as soon as you start the load on > the first again, the second > client performance drops? > > Bye, > Oleg > On Mar 4, 2010, at 1:45 PM, Jeffrey Bennett wrote: > >> Hi Oleg, thanks for your reply >> >> I was actually testing with only one client. When adding a second >> client using a different file, one client gets all the performance >> and the other one gets very low performance, any recommendation? >> >> Thanks in advance >> >> jab >> >> >> -----Original Message----- >> From: oleg.dro...@sun.com [mailto:oleg.dro...@sun.com] >> Sent: Wednesday, March 03, 2010 5:20 PM >> To: Jeffrey Bennett >> Cc: lustre-discuss@lists.lustre.org >> Subject: Re: [Lustre-discuss] One or two OSS, no difference? >> >> Hello! >> >> On Mar 3, 2010, at 6:35 PM, Jeffrey Bennett wrote: >>> We are building a very small Lustre cluster with 32 clients >>> (patchless) and two OSS servers. Each OSS server has 1 OST with 1 >>> TB of Solid State Drives. All is connected using dual-port DDR IB. >>> >>> For testing purposes, I am enabling/disabling one of the OSS/OST >>> by using the "lfs setstripe" command. I am running XDD and vdbench >>> benchmarks. >>> >>> Does anybody have an idea why there is no difference in MB/sec or >>> random IOPS when using one OSS or two OSS? A quick test with "dd" >>> also shows the same MB/sec when using one or two OSTs. >> >> I wonder if you just don't saturate even one OST (both backend SSD >> and IB interconnect) with this number of clients? Does the total >> throughput decreases as you decrease >> number of active clients and increases as you increase it even >> further? >> Increasing maximum number of in-flight rpcs might help in that case. >> Also are all of your clients writing to the same file or each >> client does io to a separate file (I hope)? >> >> Bye, >> Oleg > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss@lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc. _______________________________________________ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss