ed in the code
> >>>> and it seems that RDMA_CM uses the SL from the SA.
> >>>> So if you want to configure a specific SL, you need to do it via the SM.
> >>>>
> >>>> Doron
> >>>>
> >>>> -Original Message
MA CM, I've checked in the
> code and it seems that RDMA_CM uses the SL from the SA.
> >>>> So if you want to configure a specific SL, you need to do it via the
> SM.
> >>>>
> >>>> Doron
> >>>>
> >>>> -Original Mess
c SL, you need to do it via the SM.
>>>>
>>>> Doron
>>>>
>>>> -Original Message-
>>>> From: Jeff Squyres [mailto:jsquy...@cisco.com]
>>>> Sent: Thursday, February 24, 2011 3:45 PM
>>>> To: Michael Shuey
&g
RDMA_CM uses the SL from the SA.
>>> So if you want to configure a specific SL, you need to do it via the SM.
>>>
>>> Doron
>>>
>>> -----Original Message-
>>> From: Jeff Squyres [mailto:jsquy...@cisco.com]
>>> Sent: Thursday, February 24, 20
t;> From: Jeff Squyres [mailto:jsquy...@cisco.com]
>> Sent: Thursday, February 24, 2011 3:45 PM
>> To: Michael Shuey
>> Cc: Open MPI Users , Mike Dubman
>> Subject: Re: [OMPI users] RoCE (IBoE) & OpenMPI
>>
>> On Feb 24, 2011, at 8:00 AM, Michael Shuey wrote
squy...@cisco.com]
> Sent: Thursday, February 24, 2011 3:45 PM
> To: Michael Shuey
> Cc: Open MPI Users , Mike Dubman
> Subject: Re: [OMPI users] RoCE (IBoE) & OpenMPI
>
> On Feb 24, 2011, at 8:00 AM, Michael Shuey wrote:
>
>> Late yesterday I did have a chance to test t
On Feb 24, 2011, at 8:00 AM, Michael Shuey wrote:
> Late yesterday I did have a chance to test the patch Jeff provided
> (against 1.4.3 - testing 1.5.x is on the docket for today). While it
> works, in that I can specify a gid_index,
Great! I'll commit that to the trunk and start the process of
Late yesterday I did have a chance to test the patch Jeff provided
(against 1.4.3 - testing 1.5.x is on the docket for today). While it
works, in that I can specify a gid_index, it doesn't do everything
required - my traffic won't match a lossless CoS on the ethernet
switch. Specifying a GID is o
On Feb 23, 2011, at 3:54 PM, Shamis, Pavel wrote:
> I remember that I updated the trunk to select by default RDMACM connection
> manager for RoCE ports - https://svn.open-mpi.org/trac/ompi/changeset/22311
>
> I'm not sure it the change made his way to any production version. I don't
> work on t
Here is what OFA says:
http://www.google.com/url?sa=t&source=web&cd=1&ved=0CBMQFjAA&url=http%3A%2F%2Fwww.openfabrics.org%2Farchives%2Fspring2010sonoma%2FWednesday%2FLiran%2520Liss%2520RoCE%2520in%2520OFED%2Frocee_update_liss.ppt&ei=QW9lTfO-L8HYgQf2tdHhBw&usg=AFQjCNEPltfVwWeZ2d4uvaj1wMpumcxrEw&sig2=
Could you re-enable the SL param (btl_openib_ib_service_level) for
RoCE? Jeff was kind enough to provide a patch to let me specify the
gid_index, but that doesn't seem to be working. To get RoCE to work
correctly (at least, on Nexus switches) I'll need to specify both a
gid_index and an IB servic
Random thought: is there a check to ensure that the SL MCA param is not set in
a RoCE environment? If not, we should probably add a show_help warning if the
SL MCA param is set when using RoCE (i.e., that its value will be ignored).
On Feb 19, 2011, at 12:22 AM, Shamis, Pavel wrote:
> As far
As far as I remember we don't allow to user to specify SL for RoCE. RoCE
considered kinda ethernet device and RDMACM connection manager is used to setup
the connections. it means that in order to select network X or Y, you may use
ip/netmask (btl_openib_ipaddr_include) .
Pavel (Pasha) Shamis
-
Per-node GID & SL settings == bad. Site-wide GID & SL settings == good.
If this could be an MCA param (like btl_openib_ib_service_level)
that'd be great - we already have a global config file of similar
params. We'd definitely want the same N everywhere.
--
Mike Shuey
On Fri, Feb 18, 2011 at
On Feb 18, 2011, at 1:39 PM, Michael Shuey wrote:
> RoCE HCAs keep a GID table, like normal HCAs. Every time you bring up
> a vlan interface, another entry gets automatically added to the table.
> If I select one of these other GIDs, packets get a VLAN tag, and that
> contains the necessary prior
It's a little different in RoCE. There's no subnet manager, so (as
near as I can tell) you don't really have a subnet ID. Instead, the
GID = GUID + VLAN tag (more or less). gid[0] has special bits in the
VLAN tag section, to indicate that packets relating to this GID don't
get a VLAN tag. Unfor
Greetings Mike. I'll answer today because Fri-Sat is the weekend in Israel
(i.e., the MPI team at Mellanox won't see this until Sunday).
I don't have a lot of experience with RoCE; do you need a different GUID or a
different subnet ID? At least in IB, the GID = GUID + Subnet ID. The GUID
sho
I've been looking into OpenMPI's support for RoCE (Mellanox's recent
Infiniband-over-Ethernet) lately. While it's promising, I've hit a
snag: RoCE requires lossless ethernet, and on my switches the only way
to guarantee this is with CoS. RoCE adapters cannot emit CoS priority
tags unless the clie
18 matches
Mail list logo
