Re: RDMA/CM and multiple QPs

What I'm more interested in is a way to tell the CM that I only want routes that are using this ib_device that I got from the first lookup as all others are useless for me. I'm not sure I understand what you are aiming for? if you connect to a single address multiple times you will get the

Re: RDMA/CM and multiple QPs

On Thu, Sep 10, 2015 at 12:50:33PM +0300, Sagi Grimberg wrote: > >What I'm more interested in is a way to tell the CM that I only > >want routes that are using this ib_device that I got from the first > >lookup as all others are useless for me. > > > > I'm not sure I understand what you are

Re: RDMA/CM and multiple QPs

On 9/10/2015 4:29 PM, Christoph Hellwig wrote: On Thu, Sep 10, 2015 at 12:50:33PM +0300, Sagi Grimberg wrote: What I'm more interested in is a way to tell the CM that I only want routes that are using this ib_device that I got from the first lookup as all others are useless for me. I'm not

RE: RDMA/CM and multiple QPs

> right now RDMA/CM works on a QP basis, but seems very awakward if you > want multiple QPs as part of a single logical device, which will be > useful for a lot of modern protocols. For example we will need to check > in the CM handler that we're not getting a different ib_device if we > want to

Re: RDMA/CM and multiple QPs

On 9/6/2015 9:45 AM, Christoph Hellwig wrote: Hi All, right now RDMA/CM works on a QP basis, but seems very awakward if you want multiple QPs as part of a single logical device, which will be useful for a lot of modern protocols. For example we will need to check in the CM handler that we're

Re: RDMA/CM and multiple QPs

On 9/6/2015 11:15 AM, Bart Van Assche wrote: On 09/06/15 00:50, Christoph Hellwig wrote: Note that the SRP driver already in tree is a good example for this, although it doesn't use RDMA/CM and thus already operates on a per-ib_device level. The challenges with regard to adding RDMA/CM

Re: RDMA/CM and multiple QPs

On Tue, Sep 08, 2015 at 03:32:11PM +0300, Sagi Grimberg wrote: > The CM is responsible of establishing an RDMA channel. What you are > referring to is a concept of a session. I'm not entirely sure how we can > fit a model where the CM establishes a multi-channel session as the > CM request

Re: RDMA/CM and multiple QPs

On 09/08/2015 06:57 AM, Tom Talpey wrote: On 9/6/2015 11:15 AM, Bart Van Assche wrote: On 09/06/15 00:50, Christoph Hellwig wrote: Note that the SRP driver already in tree is a good example for this, although it doesn't use RDMA/CM and thus already operates on a per-ib_device level. The

Re: RDMA/CM and multiple QPs

On Sun, Sep 06, 2015 at 01:24:52PM +0530, Parav Pandit wrote: > Yes. SRP is good example. The point I am trying to make is, SRP > implements failover and request spreading where one QP fails it > delivers to other QP. But SRP doesn't implement that. There are no fail over capabilities in a

RDMA/CM and multiple QPs

Hi All, right now RDMA/CM works on a QP basis, but seems very awakward if you want multiple QPs as part of a single logical device, which will be useful for a lot of modern protocols. For example we will need to check in the CM handler that we're not getting a different ib_device if we want to

Re: RDMA/CM and multiple QPs

On Sun, Sep 06, 2015 at 01:12:56PM +0530, Parav Pandit wrote: > Hi Christoph, > > Establishing multiple QP is just one part of it. > Bigger challenge is how do we distribute the work request among > multiple QPs For my case I simply rely on the blk-mq layer to have cpu-local queues, so that's a

Re: RDMA/CM and multiple QPs

Hi Christoph, Establishing multiple QP is just one part of it. Bigger challenge is how do we distribute the work request among multiple QPs specially when STAG advertisements, their invalidation is agnostic at Verbs layer (which is not part of the IB spec and every ULP has their own method

Re: RDMA/CM and multiple QPs

On Sun, Sep 6, 2015 at 1:20 PM, Christoph Hellwig wrote: > On Sun, Sep 06, 2015 at 01:12:56PM +0530, Parav Pandit wrote: >> Hi Christoph, >> >> Establishing multiple QP is just one part of it. >> Bigger challenge is how do we distribute the work request among >> multiple QPs >

Re: RDMA/CM and multiple QPs

On 09/06/15 00:50, Christoph Hellwig wrote: Note that the SRP driver already in tree is a good example for this, although it doesn't use RDMA/CM and thus already operates on a per-ib_device level. The challenges with regard to adding RDMA/CM support to the SRP initiator and target drivers