Mellanox...? On Mar 2, 2011, at 12:51 PM, Michael Shuey wrote:
> Honestly, I don't know - I haven't looked into the source. OFED 1.5.2 > has a version of ibv_rc_pingpong that's been modified to work with > RoCE; you can pass the gid_index and SL as command-line arguments. > I'm not sure how that's handled at the IB layer, but the source may be > a good place to start. > > -- > Mike Shuey > > > > On Tue, Mar 1, 2011 at 9:14 AM, Jeff Squyres <jsquy...@cisco.com> wrote: >> I thought you mentioned in a prior email that you had gotten one or two >> other OFED sample applications to work properly. How are they setting the >> SL? Are they not using the RDMA CM? >> >> >> On Mar 1, 2011, at 7:35 AM, Michael Shuey wrote: >> >>> So, since RoCE has no SM, and setting an SL is required to get >>> lossless ethernet on Cisco switches (and possibly others), does this >>> mean that RoCE will never work correctly with OpenMPI on Cisco >>> hardware? >>> >>> -- >>> Mike Shuey >>> >>> >>> >>> On Tue, Mar 1, 2011 at 3:42 AM, Doron Shoham <dor...@mellanox.com> wrote: >>>> Hi, >>>> >>>> Regarding to using a specific SL with RDMA CM, I've checked in the code >>>> and it seems that RDMA_CM uses the SL from the SA. >>>> So if you want to configure a specific SL, you need to do it via the SM. >>>> >>>> Doron >>>> >>>> -----Original Message----- >>>> From: Jeff Squyres [mailto:jsquy...@cisco.com] >>>> Sent: Thursday, February 24, 2011 3:45 PM >>>> To: Michael Shuey >>>> Cc: Open MPI Users , Mike Dubman >>>> Subject: Re: [OMPI users] RoCE (IBoE) & OpenMPI >>>> >>>> On Feb 24, 2011, at 8:00 AM, Michael Shuey wrote: >>>> >>>>> Late yesterday I did have a chance to test the patch Jeff provided >>>>> (against 1.4.3 - testing 1.5.x is on the docket for today). While it >>>>> works, in that I can specify a gid_index, >>>> >>>> Great! I'll commit that to the trunk and start the process of moving it >>>> to the v1.5.x series (I know you haven't tested it yet, but it's >>>> essentially the same patch, just slightly adjusted for each of the 3 >>>> branches). >>>> >>>>> it doesn't do everything >>>>> required - my traffic won't match a lossless CoS on the ethernet >>>>> switch. Specifying a GID is only half of it; I really need to also >>>>> specify a service level. >>>> >>>> RoCE requires the use of the RDMA CM (I think?), and I didn't think there >>>> was a way to request a specific SL via the RDMA CM...? (I could certainly >>>> be wrong here) >>>> >>>> I think Mellanox will need to follow up with these questions... >>>> >>>> -- >>>> Jeff Squyres >>>> jsquy...@cisco.com >>>> For corporate legal information go to: >>>> http://www.cisco.com/web/about/doing_business/legal/cri/ >>>> >>>> >>>> _______________________________________________ >>>> users mailing list >>>> us...@open-mpi.org >>>> http://www.open-mpi.org/mailman/listinfo.cgi/users >>>> >>> >>> _______________________________________________ >>> users mailing list >>> us...@open-mpi.org >>> http://www.open-mpi.org/mailman/listinfo.cgi/users >> >> >> -- >> Jeff Squyres >> jsquy...@cisco.com >> For corporate legal information go to: >> http://www.cisco.com/web/about/doing_business/legal/cri/ >> >> >> _______________________________________________ >> users mailing list >> us...@open-mpi.org >> http://www.open-mpi.org/mailman/listinfo.cgi/users >> -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
