I'm told that it is not required; it *could* use OMPI's oob CPC if we wanted it to. But that's second-hand information; I don't know that for sure.
On Mar 6, 2011, at 8:52 AM, Doron Shoham wrote: > ibv_rc_pingpong runs over verbs and not RDMA_CM. > As far as I understood, RoCE requires the use of the RDMA CM (in open mpi), > and it is not possible to set explicitly the SL in RDMA_CM. > We should check if RDMA_CM is must for RoCE. > > Thanks, > Doron > > On Wed, Mar 2, 2011 at 8:01 PM, Jeff Squyres <jsquy...@cisco.com> wrote: > Mellanox...? > > On Mar 2, 2011, at 12:51 PM, Michael Shuey wrote: > > > Honestly, I don't know - I haven't looked into the source. OFED 1.5.2 > > has a version of ibv_rc_pingpong that's been modified to work with > > RoCE; you can pass the gid_index and SL as command-line arguments. > > I'm not sure how that's handled at the IB layer, but the source may be > > a good place to start. > > > > -- > > Mike Shuey > > > > > > > > On Tue, Mar 1, 2011 at 9:14 AM, Jeff Squyres <jsquy...@cisco.com> wrote: > >> I thought you mentioned in a prior email that you had gotten one or two > >> other OFED sample applications to work properly. How are they setting the > >> SL? Are they not using the RDMA CM? > >> > >> > >> On Mar 1, 2011, at 7:35 AM, Michael Shuey wrote: > >> > >>> So, since RoCE has no SM, and setting an SL is required to get > >>> lossless ethernet on Cisco switches (and possibly others), does this > >>> mean that RoCE will never work correctly with OpenMPI on Cisco > >>> hardware? > >>> > >>> -- > >>> Mike Shuey > >>> > >>> > >>> > >>> On Tue, Mar 1, 2011 at 3:42 AM, Doron Shoham <dor...@mellanox.com> wrote: > >>>> Hi, > >>>> > >>>> Regarding to using a specific SL with RDMA CM, I've checked in the code > >>>> and it seems that RDMA_CM uses the SL from the SA. > >>>> So if you want to configure a specific SL, you need to do it via the SM. > >>>> > >>>> Doron > >>>> > >>>> -----Original Message----- > >>>> From: Jeff Squyres [mailto:jsquy...@cisco.com] > >>>> Sent: Thursday, February 24, 2011 3:45 PM > >>>> To: Michael Shuey > >>>> Cc: Open MPI Users , Mike Dubman > >>>> Subject: Re: [OMPI users] RoCE (IBoE) & OpenMPI > >>>> > >>>> On Feb 24, 2011, at 8:00 AM, Michael Shuey wrote: > >>>> > >>>>> Late yesterday I did have a chance to test the patch Jeff provided > >>>>> (against 1.4.3 - testing 1.5.x is on the docket for today). While it > >>>>> works, in that I can specify a gid_index, > >>>> > >>>> Great! I'll commit that to the trunk and start the process of moving it > >>>> to the v1.5.x series (I know you haven't tested it yet, but it's > >>>> essentially the same patch, just slightly adjusted for each of the 3 > >>>> branches). > >>>> > >>>>> it doesn't do everything > >>>>> required - my traffic won't match a lossless CoS on the ethernet > >>>>> switch. Specifying a GID is only half of it; I really need to also > >>>>> specify a service level. > >>>> > >>>> RoCE requires the use of the RDMA CM (I think?), and I didn't think > >>>> there was a way to request a specific SL via the RDMA CM...? (I could > >>>> certainly be wrong here) > >>>> > >>>> I think Mellanox will need to follow up with these questions... > >>>> > >>>> -- > >>>> Jeff Squyres > >>>> jsquy...@cisco.com > >>>> For corporate legal information go to: > >>>> http://www.cisco.com/web/about/doing_business/legal/cri/ > >>>> > >>>> > >>>> _______________________________________________ > >>>> users mailing list > >>>> us...@open-mpi.org > >>>> http://www.open-mpi.org/mailman/listinfo.cgi/users > >>>> > >>> > >>> _______________________________________________ > >>> users mailing list > >>> us...@open-mpi.org > >>> http://www.open-mpi.org/mailman/listinfo.cgi/users > >> > >> > >> -- > >> Jeff Squyres > >> jsquy...@cisco.com > >> For corporate legal information go to: > >> http://www.cisco.com/web/about/doing_business/legal/cri/ > >> > >> > >> _______________________________________________ > >> users mailing list > >> us...@open-mpi.org > >> http://www.open-mpi.org/mailman/listinfo.cgi/users > >> > > > -- > Jeff Squyres > jsquy...@cisco.com > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/ > > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users > -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
