> -----Original Message----- > From: Shirley Ma [mailto:shirley...@oracle.com] > Sent: Friday, November 21, 2014 00:00 > To: Cheng, Wendy; Charles EDWARD Lever; anna.schuma...@netapp.com; > devesh.sha...@emulex.com; dledf...@redhat.com; > dominique.marti...@cea.fr; jeffrey.c.bec...@nasa.gov; rsdance@soft- > forge.com; s...@lanl.gov; spra...@redhat.com; ste...@redhat.com; > sw...@opengridcomputing.com; Yan Burman; linux-rdma; Linux NFS Mailing > List > Subject: Re: NFSoRDMA developers bi-weekly meeting minutes (11/20) > > > On 11/20/2014 12:15 PM, Cheng, Wendy wrote: > >> -----Original Message----- > >> From: Shirley Ma [mailto:shirley...@oracle.com] > >> Sent: Thursday, November 20, 2014 10:24 AM > >> > >> .... > >> iser 8K could reach 4.5GB/s in 56Gb/s link speed, 1.5 million IOPS. > >> 32K could reach 1.8 million IOPS > >> > > > > How did the ISER data get measured ? Was the measure done on ISER layer, > block layer, or filesystem layer ? > > Here is the link on iser how to set up and measure performance: > http://community.mellanox.com/docs/DOC-1483
Actual numbers are (there seems to be some misunderstanding in the meeting minutes): For single LUN/session in iSER on ConnectX-3 FDR link with 8 core 2.6GHz Xeon are: 8K block size reaches 2.5GB/s Somewhere between 16K and 32K block size iSER reaches 5.5GB/s which is almost line rate 256K block size gives 5.7GB/s With 16 sessions, it is possible to reach 1.7M IOPS with 1K block size and about 600K IOPS with 8K block size. Note these numbers are for SCST iSER implementation and there are some more tunings and enhancements that can be applied to further improve performance. Another issue that came up is the benefit of RDMA vs send. In order to check that you can use ib_send_lat vs ib_write_lat and see the latencies of different block sizes. >From past experience RDMA starts to get more efficient somewhere around 8K. Yan -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
