Tzachi, You are correct.. there are 2 SG lists.. the IB scatter gather and the StorPort scatter gather list.
The issue we are seeing is with the StorPort scatter gather list size which seems to be limited to 16 entries (16 * 4096 page size = 64KB IO). The reason we want this size increased is to be able to scale single stream bandwidth by feeding the storage backend larger IOs. Basically pusing the latency down to the backend, keeping the disks busy while freeing up the host. This will give us greater bandwidth. The StorPort scatter gather list size is controlled by setting PORT_CONFIGURATION_INFORMATION::NumberOfPhysicalBreaks. On entry to srp_find_adapter() StorPort has defaulted this to 16. I was told by Microsoft that I should be able to set this to 254. (StorPort adds 1). When I set this value it does not stick. A matter of fact NT is not even reading the location. This was confirmed by setting a break on access to NumberOfPhysicalBreaks just before return from FindAdapter() and hitting GO in WinDbg. What I did find out is that the number of map registers allocated to the adpater is being limited to 16 by an internal call to IoGetDmaAdapter(). This function is called by NT during adapter initialization right after we return from FindAdapter(). According to Microsoft this is what is capping the scatter gather list size. As it stands right now they have yet to tell me why IoGetDmaAdapter() is limiting the number of map registers. I will let you know once I find out more. Daniel Cummins SeaChange International 32 Mill Street, Greenville, NH 03048 Phone (603) 878-6414 mailto:[EMAIL PROTECTED] This e-mail and any attachments may contain information which is SeaChange International confidential. The information enclosed is intended only for the addressees herein and may not be copied or forwarded without permission from SeaChange International. Hi Steve, There are two questions that I need to understand: 1) What SG lists are you talking about? The IB ones (post send/recive)? 2) why is 64kb not enough. Does the code send out one request and only once it is finished it is sending the next request? Assuming you want to reach to a BW of 1000MB/sec than this means that you should have a request once every 64us. Do you know if the time that it takes to handle a request is bigger that 64us? Thanks Tzachi > -----Original Message----- > From: ofw-bounces at lists.openfabrics.org > [mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of Steve Valimaki > Sent: Monday, June 25, 2007 10:16 PM > To: ofw at lists.openfabrics.org > Subject: [ofw] WinIB SRP driver and 64k IO's? > > Doing some basic testing (iometer, xdd) of the windows WinIB (Mellanox > 1.3) SRP driver, its seems that I've not been able to get > larger the 64K IO's to my DataDirect Networks Storage array. > The host is Windows server > 2003 SP2. I suspect it may be the need to increase the size > of the SG list, is there a known limitation? > > Thanks, > Steve > > -- > Steve Valimaki > Field Applications Engineer > DataDirect Networks Federal LLC > Cell Phone: (443) 415-4947 > Office Phone: (410) 309-9300 x152 > > http://www.datadirectnet.com > > DDN Support Announcement mailing list: > http://www.ddnsupport.com/announce > > > > _______________________________________________ > ofw mailing list > ofw at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw >
_______________________________________________ ofw mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
