we spent some time doing a little benchmarking on windows last week and discovered some things about the behavior of the srp miniport (storport) driver and windows.
windows server 2003 (sp2) 32bit running on 64-bit hardware uses 16 NumberOfPhysicalBreaks regardless of what the srp miniport tries to force. if you go one step further and try to emulate a 32-bit system as much as possible by disabled PAE and DEP, the NumberOfPhysicalBreaks is then limited to 8. This applies to the 32-bit version of windows server 2008 as well and likely applies to any of the 32-bit versions of windows running on 64-bit hardware. we were unable to test a 32-bit version of windows running on an actual 32-bit processor. the story changes for 64-bit windows. windows xp, windows server 2003 and windows server 2008 (the 64-bit versions) will respect the srp miniport's increase in NumberOfPhysicalBreaks. using any of these operating systems we were able get 1024k i/o's. however, if you try to stripe several disks together, the i/o drops to 64k again. this is likely because the default stripe size for windows is 64k and the window's i/o scheduler doesnt seem to aggregate pending i/o's. we were unable to test larger stripe sizes since we dont have a utility to create anything other than the default stripe size. it is difficult to understand why the 32-bit version of windows is using the 16 limit. i am guessing it has something to do with the IOGetDmaAdapter() for the virtual/internal device associated with the srp miniport. apparently 32-bit windows seems to have a hard limit for the available map registers in this case. additionally, if you dont allow the srp miniport to upgrade NumberOfPhysicalBreaks on 64-bit windows, the i/o rates are still about 60% faster on 64-bit windows. this makes me think that the 32-bit version of windows might be using a double/bounce buffer? at the end of this testing, we were able to get 4 threads on iometer running across 4 luns (one per controller) for a total of 1.5GB/s reads on our ddn s2a9500. the windows host had a single ddr adapter and the srp miniport was tuned to force 1024k i/o sizes. In message <[EMAIL PROTECTED]>,"Eleanor Witiak" writes: >This is a multi-part message in MIME format. > >--===============1373268742== >Content-class: urn:content-classes:message >Content-Type: multipart/alternative; > boundary="----_=_NextPart_001_01C869C2.4507D9FA" > >This is a multi-part message in MIME format. > >------_=_NextPart_001_01C869C2.4507D9FA >Content-Type: text/plain; > charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >Hi Fab, >=20 >What would you like me to try? >=20 >Eleanor > >________________________________ > >From: [EMAIL PROTECTED] on behalf of Fab Tillier >Sent: Thu 2/7/2008 2:29 PM >To: Usha Srinivasan; Jan Bottorff; [email protected] >Subject: RE: [ofw] SRP problems with WinOF 1.0.1 > > > >Hi Folks, > >I'm trying to find things out on my end. I don't have any hardware to = >test things, and would like to have a volunteer for trying things out. > >-Fab > >-----Original Message----- >From: [EMAIL PROTECTED] = >[mailto:[EMAIL PROTECTED] On Behalf Of Usha Srinivasan >Sent: Thursday, February 07, 2008 5:50 AM >To: Jan Bottorff; [email protected] >Subject: RE: [ofw] SRP problems with WinOF 1.0.1 > >Can someone at Microsoft confirm that StorPort driver in SP2 indeed >ignores a miniport's attempt to increase the NumberOfPhysicalBreaks >above 17? > >We are running WinOF SRP with SP2 and are trying to get buffer sizes >above 64K bytes, and it is just not happening. We cannot switch back to >SP1. > >If this is simply not doable in SP2 with a Storport Miniport, we would >sure like to know, so we can drop this and move on. > >Fab, can you possibly get a confirmation on this???? > >Thanks. >Usha > >-----Original Message----- >From: [EMAIL PROTECTED] >[mailto:[EMAIL PROTECTED] On Behalf Of Jan Bottorff >Sent: Wednesday, February 06, 2008 7:30 PM >To: [email protected] >Subject: RE: [ofw] SRP problems with WinOF 1.0.1 > >> Seems like MS patched the StorPort in SP2. > >SP2 has an updated storport driver. There is also a post SP2 update to >the storport driver (from about April/May 2007), available from I >believe the Microsoft support website. My unconfirmed guess is we may >eventually see an updated W2K3 storport with some of the W2K8 changes. > >There are changes in both the SP2 and post-SP2 storport that can affect >storage miniport drivers, like the SRP driver. > >Jan > > >_______________________________________________ >ofw mailing list >[email protected] >http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw >_______________________________________________ >ofw mailing list >[email protected] >http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw >_______________________________________________ >ofw mailing list >[email protected] >http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw > > > >------_=_NextPart_001_01C869C2.4507D9FA >Content-Type: text/html; > charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > ><HTML dir=3Dltr><HEAD><TITLE>https://qlm.qlogic.com - RE: [ofw] SRP = >problems with WinOF 1.0.1</TITLE>=0A= ><META http-equiv=3DContent-Type content=3D"text/html; charset=3Dunicode">=0A= ><META content=3D"MSHTML 6.00.3790.186" name=3DGENERATOR></HEAD>=0A= ><BODY>=0A= ><DIV id=3DidOWAReplyText1703 dir=3Dltr>=0A= ><DIV dir=3Dltr><FONT face=3DArial color=3D#000000 size=3D2>Hi = >Fab,</FONT></DIV>=0A= ><DIV dir=3Dltr><FONT face=3DArial size=3D2></FONT> </DIV>=0A= ><DIV dir=3Dltr><FONT face=3DArial size=3D2>What would you like me to = >try?</FONT></DIV>=0A= ><DIV dir=3Dltr><FONT face=3DArial size=3D2></FONT> </DIV>=0A= ><DIV dir=3Dltr><FONT face=3DArial size=3D2>Eleanor</FONT></DIV></DIV>=0A= ><DIV dir=3Dltr><BR>=0A= ><HR tabIndex=3D-1>=0A= ><FONT face=3DTahoma size=3D2><B>From:</B> = >[EMAIL PROTECTED] on behalf of Fab = >Tillier<BR><B>Sent:</B> Thu 2/7/2008 2:29 PM<BR><B>To:</B> Usha = >Srinivasan; Jan Bottorff; [email protected]<BR><B>Subject:</B> = >RE: [ofw] SRP problems with WinOF 1.0.1<BR></FONT><BR></DIV>=0A= ><DIV>=0A= ><P><FONT size=3D2>Hi Folks,<BR><BR>I'm trying to find things out on my = >end. I don't have any hardware to test things, and would like to = >have a volunteer for trying things out.<BR><BR>-Fab<BR><BR>-----Original = >Message-----<BR>From: [EMAIL PROTECTED] [<A = >href=3D"mailto:[EMAIL PROTECTED]">mailto:[EMAIL PROTECTED] >s.openfabrics.org</A>] On Behalf Of Usha Srinivasan<BR>Sent: Thursday, = >February 07, 2008 5:50 AM<BR>To: Jan Bottorff; = >[email protected]<BR>Subject: RE: [ofw] SRP problems with WinOF = >1.0.1<BR><BR>Can someone at Microsoft confirm that StorPort driver in = >SP2 indeed<BR>ignores a miniport's attempt to increase the = >NumberOfPhysicalBreaks<BR>above 17?<BR><BR>We are running WinOF SRP with = >SP2 and are trying to get buffer sizes<BR>above 64K bytes, and it is = >just not happening. We cannot switch back to<BR>SP1.<BR><BR>If this is = >simply not doable in SP2 with a Storport Miniport, we would<BR>sure like = >to know, so we can drop this and move on.<BR><BR>Fab, can you possibly = >get a confirmation on = >this????<BR><BR>Thanks.<BR>Usha<BR><BR>-----Original = >Message-----<BR>From: [EMAIL PROTECTED]<BR>[<A = >href=3D"mailto:[EMAIL PROTECTED]">mailto:[EMAIL PROTECTED] >s.openfabrics.org</A>] On Behalf Of Jan Bottorff<BR>Sent: Wednesday, = >February 06, 2008 7:30 PM<BR>To: [email protected]<BR>Subject: = >RE: [ofw] SRP problems with WinOF 1.0.1<BR><BR>> Seems like MS = >patched the StorPort in SP2.<BR><BR>SP2 has an updated storport driver. = >There is also a post SP2 update to<BR>the storport driver (from about = >April/May 2007), available from I<BR>believe the Microsoft support = >website. My unconfirmed guess is we may<BR>eventually see an updated = >W2K3 storport with some of the W2K8 changes.<BR><BR>There are changes in = >both the SP2 and post-SP2 storport that can affect<BR>storage miniport = >drivers, like the SRP = >driver.<BR><BR>Jan<BR><BR><BR>___________________________________________= >____<BR>ofw mailing list<BR>[email protected]<BR><A = >href=3D"http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw";>http:/= >/lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw</A><BR>______________= >_________________________________<BR>ofw mailing = >list<BR>[email protected]<BR><A = >href=3D"http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw";>http:/= >/lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw</A><BR>______________= >_________________________________<BR>ofw mailing = >list<BR>[email protected]<BR><A = >href=3D"http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw";>http:/= >/lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw</A><BR></FONT></P></D= >IV></BODY></HTML> >------_=_NextPart_001_01C869C2.4507D9FA-- > >--===============1373268742== >Content-Type: text/plain; charset="us-ascii" >MIME-Version: 1.0 >Content-Transfer-Encoding: 7bit >Content-Disposition: inline > >_______________________________________________ >ofw mailing list >[email protected] >http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw >--===============1373268742==-- > _______________________________________________ ofw mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
