Re: [PATCH] IB/mlx5: Fix binary compatibility with libmlx5

2014-01-29 Thread Eli Cohen 
On Wed, Jan 29, 2014 at 09:48:43PM +0100, Yann Droneaud wrote:
Yann,
thanks for reviewing, your comments are helpful :-)

> 
> 12 digits identifier are the norm for kernel. Please update your git
> configuration:
> 
>  git config --global core.abbrev 12
> 
> See http://lwn.net/Articles/571980/
> http://blog.cuviper.com/2013/11/10/how-short-can-git-abbreviate/
> 
> > libmlx5 and mlx5_ib since it defines a different value to the number of 
> > micro
> > UARs per page, leading to wrong calculation in libmlx5. This patch defines
> > struct mlx5_ib_alloc_ucontext_req_v2 as an extension to struct
> > mlx5_ib_alloc_ucontext_req.  The extended size is determined in
> > mlx5_ib_alloc_ucontext() and in case of old library we use uuarn 0 which 
> > works
> > fine. For new libraries we use the more sophisticated allocation algorithm.
> > 
> > Fixes: c1be523 ('Fix micro UAR allocator')
> ^^^
> Likewise

Will fix.
> 
> I'm not sure how this could work without subtracting sizeof(struct
> ib_uverbs_cmd_hdr).

struct ib_uverbs_get_context happens to have the same size as struct
ib_uverbs_cmd_hdr so it passed all my sanity tests. Correct, will fix
that too.
> 
> As I explained in "Re: [PATCHv4 for-3.13 00/10] create_flow/destroy_flow
> fixes for v3.13" [1] ib_uverbs_write() does not decrement input length:
> it gives hdr.in_words * 4 to the uverbs function, here 
> ib_uverbs_get_context(). Then, the function built struct ib_udata 
> without taking care of the extra bytes count in in_len:
> 
> struct ib_uverbs_get_context cmd;
> ...
> INIT_UDATA(&udata, buf + sizeof cmd,
>(unsigned long) cmd.response + sizeof resp,
>in_len - sizeof cmd, out_len - sizeof resp);

So this just seems broken and the fix is to do the subtraction here so
the hardware driver gets the correct size without needing to subtract
the extra bytes, like this:
INIT_UDATA(&udata, buf + sizeof cmd,
   (unsigned long) cmd.response + sizeof resp,
   in_len - sizeof cmd - sizeof (struct ib_uverbs_cmd_hdr),
   out_len - sizeof resp);

And the hardware driver gets the correct size for its struct.
> 
> Driver mthca does some handling which look like to what's proposed in
> your patch, but takes care of subtracting the header size from the input
> length, see mthca_reg_user_mr()[2].
> 
> [1]
> 
> 
> [2]
> 
> 
> > +   if (reqlen == sizeof(struct mlx5_ib_alloc_ucontext_req))
> > +   ver = 0;
> > +   else if (reqlen == sizeof(struct mlx5_ib_alloc_ucontext_req_v2))
> > +   ver = 2;
> > +   else
> > +   return ERR_PTR(-EINVAL);
> > +
> 
> Doing so introduce a subtle regression: there was no check on the length
> before, so it was legal to pass a input buffer far larger than needed,
> aka. trailing garbage. 
> 
> With such new test in place, it's no more allowed, and this is a
> regression. It's not a big issue, but a little departure from current
> behavor.
Well it's a regression if there was out there another library for mlx5
out there which misbehaves and there is any as far as I know. So I
think it's safe to add this strict check.
> 
> BTW, this is the correct way to handle the request, every other uverbs
> functions should behave like this, eg. being strict on its accepted
> input.
> 
> > +   err = ib_copy_from_udata(&req, udata, reqlen);
> > if (err)
> > return ERR_PTR(err);
> >  
> > +   if (req.flags || req.reserved)
> > +   return ERR_PTR(-EINVAL);
> > +
> 
> Just like this :)
> 
> > if (req.total_num_uuars > MLX5_MAX_UUARS)
> > return ERR_PTR(-ENOMEM);
> >  
> > @@ -626,6 +640,7 @@ static struct ib_ucontext 
> > *mlx5_ib_alloc_ucontext(struct ib_device *ibdev,
> > if (err)
> > goto out_uars;
> >  
> > +   uuari->ver = ver;
> > uuari->num_low_latency_uuars = req.num_low_latency_uuars;
> > uuari->uars = uars;
> > uuari->num_uars = num_uars;
> > diff --git a/drivers/infiniband/hw/mlx5/qp.c 
> > b/drivers/infiniband/hw/mlx5/qp.c
> > index 492dc33..300475c 100644
> > --- a/drivers/infiniband/hw/mlx5/qp.c
> > +++ b/drivers/infiniband/hw/mlx5/qp.c
> > @@ -430,11 +430,17 @@ static int alloc_uuar(struct mlx5_uuar_info *uuari,
> > break;
> >  
> > case MLX5_IB_LATENCY_CLASS_MEDIUM:
> > -   uuarn = alloc_med_class_uuar(uuari);
> > +   if (uuari->ver < 2)
> > +   uuarn = -ENOMEM;
> 
> In the commit message, you specified that uuarn is set to 0 when v1 is
> used. But here it's set to -ENOMEM.
> 
If you look at qp.c you can see that the code falls back from high to
medium to low class. Low class always succeeds and gives 0.

> > +   else
> > +   uuarn = alloc_med_class_uuar

Re: [PATCH] IB/mlx5: Fix binary compatibility with libmlx5

2014-01-29 Thread Yann Droneaud 
Hi,

Le mercredi 29 janvier 2014 à 16:27 +0200, Eli Cohen a écrit :
> Commit c1be523 "Fix micro UAR allocator" broke binary compatibility between
^^^

12 digits identifier are the norm for kernel. Please update your git
configuration:

 git config --global core.abbrev 12

See http://lwn.net/Articles/571980/
http://blog.cuviper.com/2013/11/10/how-short-can-git-abbreviate/

> libmlx5 and mlx5_ib since it defines a different value to the number of micro
> UARs per page, leading to wrong calculation in libmlx5. This patch defines
> struct mlx5_ib_alloc_ucontext_req_v2 as an extension to struct
> mlx5_ib_alloc_ucontext_req.  The extended size is determined in
> mlx5_ib_alloc_ucontext() and in case of old library we use uuarn 0 which works
> fine. For new libraries we use the more sophisticated allocation algorithm.
> 
> Fixes: c1be523 ('Fix micro UAR allocator')
^^^
Likewise

> Signed-off-by: Eli Cohen 
> ---
> 
> Hi Roland, 
> 
> This fixes a regression introduced during the 3.14 merge window, and must
> be applied for 3.14, thanks
> 
> Eli
> 
>  drivers/infiniband/hw/mlx5/main.c | 19 +--
>  drivers/infiniband/hw/mlx5/qp.c   | 11 +--
>  drivers/infiniband/hw/mlx5/user.h |  7 +++
>  include/linux/mlx5/driver.h   |  1 +
>  4 files changed, 34 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/infiniband/hw/mlx5/main.c 
> b/drivers/infiniband/hw/mlx5/main.c
> index 9660d09..e24d912 100644
> --- a/drivers/infiniband/hw/mlx5/main.c
> +++ b/drivers/infiniband/hw/mlx5/main.c
> @@ -536,24 +536,38 @@ static struct ib_ucontext 
> *mlx5_ib_alloc_ucontext(struct ib_device *ibdev,
> struct ib_udata *udata)
>  {
>   struct mlx5_ib_dev *dev = to_mdev(ibdev);
> - struct mlx5_ib_alloc_ucontext_req req;
> + struct mlx5_ib_alloc_ucontext_req_v2 req;
>   struct mlx5_ib_alloc_ucontext_resp resp;
>   struct mlx5_ib_ucontext *context;
>   struct mlx5_uuar_info *uuari;
>   struct mlx5_uar *uars;
>   int gross_uuars;
>   int num_uars;
> + int ver;
>   int uuarn;
>   int err;
>   int i;
> + int reqlen;
>  
>   if (!dev->ib_active)
>   return ERR_PTR(-EAGAIN);
>  
> - err = ib_copy_from_udata(&req, udata, sizeof(req));
> + memset(&req, 0, sizeof(req));
> + reqlen = udata->inlen - sizeof(struct ib_uverbs_get_context);

I'm not sure how this could work without subtracting sizeof(struct
ib_uverbs_cmd_hdr).

As I explained in "Re: [PATCHv4 for-3.13 00/10] create_flow/destroy_flow
fixes for v3.13" [1] ib_uverbs_write() does not decrement input length:
it gives hdr.in_words * 4 to the uverbs function, here 
ib_uverbs_get_context(). Then, the function built struct ib_udata 
without taking care of the extra bytes count in in_len:

struct ib_uverbs_get_context cmd;
...
INIT_UDATA(&udata, buf + sizeof cmd,
   (unsigned long) cmd.response + sizeof resp,
   in_len - sizeof cmd, out_len - sizeof resp);

Driver mthca does some handling which look like to what's proposed in
your patch, but takes care of subtracting the header size from the input
length, see mthca_reg_user_mr()[2].

[1]


[2]


> + if (reqlen == sizeof(struct mlx5_ib_alloc_ucontext_req))
> + ver = 0;
> + else if (reqlen == sizeof(struct mlx5_ib_alloc_ucontext_req_v2))
> + ver = 2;
> + else
> + return ERR_PTR(-EINVAL);
> +

Doing so introduce a subtle regression: there was no check on the length
before, so it was legal to pass a input buffer far larger than needed,
aka. trailing garbage. 

With such new test in place, it's no more allowed, and this is a
regression. It's not a big issue, but a little departure from current
behavor.

BTW, this is the correct way to handle the request, every other uverbs
functions should behave like this, eg. being strict on its accepted
input.

> + err = ib_copy_from_udata(&req, udata, reqlen);
>   if (err)
>   return ERR_PTR(err);
>  
> + if (req.flags || req.reserved)
> + return ERR_PTR(-EINVAL);
> +

Just like this :)

>   if (req.total_num_uuars > MLX5_MAX_UUARS)
>   return ERR_PTR(-ENOMEM);
>  
> @@ -626,6 +640,7 @@ static struct ib_ucontext *mlx5_ib_alloc_ucontext(struct 
> ib_device *ibdev,
>   if (err)
>   goto out_uars;
>  
> + uuari->ver = ver;
>   uuari->num_low_latency_uuars = req.num_low_latency_uuars;
>   uuari->uars = uars;
>   uuari->num_uars = num_uars;
> diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
> index 492dc33..300475c 100644
> --- a/drivers/infiniband/hw/mlx5/qp.c
> +++ b/drivers/infiniband/hw/mlx5/qp.c
> @@ -430

Re: linux rdma 3.14 merge plans

2014-01-29 Thread Or Gerlitz 

On 29/01/2014 17:06, Sagi Grimberg wrote:
Non-contiguous buffers are a well known problem/challenge for RDMA 
transports, each handles them differently: iSER uses bounce buffers, 
SRP registers multiple contiguous regions.


To be precise, we do it only on very special cases, normally the payload 
**each** SCSI command of size > PAGE_SIZE is scattered between random 
set of pages allocated by the kernel buffer cache and this set is mapped 
through fast-reg or fmr mechanism to one RDMA virtual address.


Or.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: linux rdma 3.14 merge plans

2014-01-29 Thread Sagi Grimberg 

On 1/29/2014 4:13 PM, Bart Van Assche wrote:

On 01/28/14 22:02, Or Gerlitz wrote:

Roland, ping! the signature patches were posted > three months ago.

I have a question about RDMA and T10-DIF support. The Linux block layer,
the Linux SCSI core, Linux filesystems and block drivers all support
discontiguous buffers. These are buffers that cannot be mapped onto a
single contiguous range of virtual memory addresses. I think the RDMA
FMR and FR memory registration mechanisms only support mapping a range
of addresses that can be mapped onto a contiguous region of virtual
addresses. This means that either multiple RDMA memory regions are
needed to map a discontiguous buffer or that the buffer contents has to
be copied before associating it with a memory region. What I understood
from the mlx5 T10-DIF patch series is that the API introduced in that
patch series is such that only a single memory region per data transfer
operation is supported. This made me wonder how to support T10-DIF for
discontiguous buffers.


Hey Bart, Thanks for the observation,

You are correct, the T10-PI RDMA offload API requires a single memory 
region that describes the data

and a single memory region that describes the protection information.

Non-contiguous buffers are a well known problem/challenge for RDMA 
transports, each handles them
differently: iSER uses bounce buffers, SRP registers multiple contiguous 
regions. The T10-PI offload API does
not impose any other constraint on this situation, each ULP will act the 
same: iSER will copy to contiguous
bounce buffers and SRP will register multiple "Signature" memory regions 
(each for a contiguous chunk of data)

and send the rkeys to the target.


Would one of the options below perhaps be an
appropriate solution ?
- Add a flag in struct request_queue that tells the block layer to use a
   bounce buffer (data copying) for I/O requests with a discontiguous
   data buffer. The block layer already supports copying data buffers for
   which e.g. DMA alignment requirements are not met. Rework patch
   "IB/iser: Introduce fast memory registration model" (commit
   5587856c9659ac2d6ab201141aa8a5c2ff3be4cd) such that it takes advantage
   of that new flag. See also blk_rq_map_user_iov() for an example of
   how the block layer decides when copying a data buffer is necessary.


Didn't understand why should it matter where the copy is done (iser/block)?


- Modify the patches that add T10-DIF such that discontiguous buffers
   are supported.


I thought about adding some kind of logic to T10-PI RDMA API to try and 
solve the alignment
issue at the same time, but the fact is that the two are unrelated, and 
I figured it is not
such a good idea. Non-contiguous memory registration is separate problem 
(hint: under
development) and should be addressed separately, T10-PI RDMA offload 
should play well

with it.


Sorry that I had not realized this before.

Bart.



Hops this helps,

Sagi.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] IB/mlx5: Fix binary compatibility with libmlx5

2014-01-29 Thread Eli Cohen 
Commit c1be523 "Fix micro UAR allocator" broke binary compatibility between
libmlx5 and mlx5_ib since it defines a different value to the number of micro
UARs per page, leading to wrong calculation in libmlx5. This patch defines
struct mlx5_ib_alloc_ucontext_req_v2 as an extension to struct
mlx5_ib_alloc_ucontext_req.  The extended size is determined in
mlx5_ib_alloc_ucontext() and in case of old library we use uuarn 0 which works
fine. For new libraries we use the more sophisticated allocation algorithm.

Fixes: c1be523 ('Fix micro UAR allocator')
Signed-off-by: Eli Cohen 
---

Hi Roland, 

This fixes a regression introduced during the 3.14 merge window, and must
be applied for 3.14, thanks

Eli

 drivers/infiniband/hw/mlx5/main.c | 19 +--
 drivers/infiniband/hw/mlx5/qp.c   | 11 +--
 drivers/infiniband/hw/mlx5/user.h |  7 +++
 include/linux/mlx5/driver.h   |  1 +
 4 files changed, 34 insertions(+), 4 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/main.c 
b/drivers/infiniband/hw/mlx5/main.c
index 9660d09..e24d912 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -536,24 +536,38 @@ static struct ib_ucontext *mlx5_ib_alloc_ucontext(struct 
ib_device *ibdev,
  struct ib_udata *udata)
 {
struct mlx5_ib_dev *dev = to_mdev(ibdev);
-   struct mlx5_ib_alloc_ucontext_req req;
+   struct mlx5_ib_alloc_ucontext_req_v2 req;
struct mlx5_ib_alloc_ucontext_resp resp;
struct mlx5_ib_ucontext *context;
struct mlx5_uuar_info *uuari;
struct mlx5_uar *uars;
int gross_uuars;
int num_uars;
+   int ver;
int uuarn;
int err;
int i;
+   int reqlen;
 
if (!dev->ib_active)
return ERR_PTR(-EAGAIN);
 
-   err = ib_copy_from_udata(&req, udata, sizeof(req));
+   memset(&req, 0, sizeof(req));
+   reqlen = udata->inlen - sizeof(struct ib_uverbs_get_context);
+   if (reqlen == sizeof(struct mlx5_ib_alloc_ucontext_req))
+   ver = 0;
+   else if (reqlen == sizeof(struct mlx5_ib_alloc_ucontext_req_v2))
+   ver = 2;
+   else
+   return ERR_PTR(-EINVAL);
+
+   err = ib_copy_from_udata(&req, udata, reqlen);
if (err)
return ERR_PTR(err);
 
+   if (req.flags || req.reserved)
+   return ERR_PTR(-EINVAL);
+
if (req.total_num_uuars > MLX5_MAX_UUARS)
return ERR_PTR(-ENOMEM);
 
@@ -626,6 +640,7 @@ static struct ib_ucontext *mlx5_ib_alloc_ucontext(struct 
ib_device *ibdev,
if (err)
goto out_uars;
 
+   uuari->ver = ver;
uuari->num_low_latency_uuars = req.num_low_latency_uuars;
uuari->uars = uars;
uuari->num_uars = num_uars;
diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index 492dc33..300475c 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -430,11 +430,17 @@ static int alloc_uuar(struct mlx5_uuar_info *uuari,
break;
 
case MLX5_IB_LATENCY_CLASS_MEDIUM:
-   uuarn = alloc_med_class_uuar(uuari);
+   if (uuari->ver < 2)
+   uuarn = -ENOMEM;
+   else
+   uuarn = alloc_med_class_uuar(uuari);
break;
 
case MLX5_IB_LATENCY_CLASS_HIGH:
-   uuarn = alloc_high_class_uuar(uuari);
+   if (uuari->ver < 2)
+   uuarn = -ENOMEM;
+   else
+   uuarn = alloc_high_class_uuar(uuari);
break;
 
case MLX5_IB_LATENCY_CLASS_FAST_PATH:
@@ -559,6 +565,7 @@ static int create_user_qp(struct mlx5_ib_dev *dev, struct 
ib_pd *pd,
}
}
 
+
uar_index = uuarn_to_uar_index(&context->uuari, uuarn);
mlx5_ib_dbg(dev, "uuarn 0x%x, uar_index 0x%x\n", uuarn, uar_index);
 
diff --git a/drivers/infiniband/hw/mlx5/user.h 
b/drivers/infiniband/hw/mlx5/user.h
index 32a2a5d..0f4f8e4 100644
--- a/drivers/infiniband/hw/mlx5/user.h
+++ b/drivers/infiniband/hw/mlx5/user.h
@@ -62,6 +62,13 @@ struct mlx5_ib_alloc_ucontext_req {
__u32   num_low_latency_uuars;
 };
 
+struct mlx5_ib_alloc_ucontext_req_v2 {
+   __u32   total_num_uuars;
+   __u32   num_low_latency_uuars;
+   __u32   flags;
+   __u32   reserved;
+};
+
 struct mlx5_ib_alloc_ucontext_resp {
__u32   qp_tab_size;
__u32   bf_reg_size;
diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h
index 554548c..32cb18c 100644
--- a/include/linux/mlx5/driver.h
+++ b/include/linux/mlx5/driver.h
@@ -227,6 +227,7 @@ struct mlx5_uuar_info {
 * protect uuar allocation data structs
 */
struct mutexlock;
+   u32 ver;
 };
 
 struct mlx5_bf {
-- 
1.8.5.2

--
To unsubscribe from this list: send the line "unsubscr

[patch] IB/iser: use after free in iser_snd_completion()

2014-01-29 Thread Dan Carpenter 
We use "tx_desc" again after we free it.

Signed-off-by: Dan Carpenter 

diff --git a/drivers/infiniband/ulp/iser/iser_initiator.c 
b/drivers/infiniband/ulp/iser/iser_initiator.c
index 538822684d5b..334f34b1cd46 100644
--- a/drivers/infiniband/ulp/iser/iser_initiator.c
+++ b/drivers/infiniband/ulp/iser/iser_initiator.c
@@ -610,11 +610,12 @@ void iser_snd_completion(struct iser_tx_desc *tx_desc,
ib_dma_unmap_single(device->ib_device, tx_desc->dma_addr,
ISER_HEADERS_LEN, DMA_TO_DEVICE);
kmem_cache_free(ig.desc_cache, tx_desc);
+   tx_desc = NULL;
}
 
atomic_dec(&ib_conn->post_send_buf_count);
 
-   if (tx_desc->type == ISCSI_TX_CONTROL) {
+   if (tx_desc && tx_desc->type == ISCSI_TX_CONTROL) {
/* this arithmetic is legal by libiscsi dd_data allocation */
task = (void *) ((long)(void *)tx_desc -
  sizeof(struct iscsi_task));
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH 1/1] RDMA/ocrdma: read ASIC_ID register to select asic_gen

2014-01-29 Thread Devesh Sharma 
Hi Roland,

Please discard this patch. 
Due to IP based GID changes, this won't apply clean, We are planning to send 
you a new series of patches on for-next tree please use that instead.

-Regards
 Devesh

-Original Message-
From: linux-rdma-ow...@vger.kernel.org 
[mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of devesh.sha...@emulex.com
Sent: Saturday, December 14, 2013 12:53 PM
To: linux-rdma@vger.kernel.org
Cc: rol...@kernel.org; Devesh Sharma
Subject: [PATCH 1/1] RDMA/ocrdma: read ASIC_ID register to select asic_gen

From: Devesh Sharma 

ocrdma driver selects execution path based on sli_family and asic generation 
number.
this introduces reading asic gen number from pci register instead of obtining 
from emulex NIC driver.

Signed-off-by: Devesh Sharma 
---
 drivers/infiniband/hw/ocrdma/ocrdma.h   |   15 +++
 drivers/infiniband/hw/ocrdma/ocrdma_hw.c|6 +++---
 drivers/infiniband/hw/ocrdma/ocrdma_main.c  |2 +-
 drivers/infiniband/hw/ocrdma/ocrdma_sli.h   |   17 +++--
 drivers/infiniband/hw/ocrdma/ocrdma_verbs.c |6 +++---
 5 files changed, 37 insertions(+), 9 deletions(-)

diff --git a/drivers/infiniband/hw/ocrdma/ocrdma.h 
b/drivers/infiniband/hw/ocrdma/ocrdma.h
index 294dd27..24157f0 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma.h
+++ b/drivers/infiniband/hw/ocrdma/ocrdma.h
@@ -424,4 +424,19 @@ static inline int is_cqe_wr_imm(struct ocrdma_cqe *cqe)  }
 
 
+static inline u8 ocrdma_get_asic_type(struct ocrdma_dev *dev) {
+   u32 asic_id;
+   u8 asic_gen = 0xF;
+   if (dev->nic_info.dev_family == 0xF) {
+   pci_read_config_dword(dev->nic_info.pdev,
+   OCRDMA_SLI_ASIC_ID_OFFSET,
+   &asic_id);
+   asic_gen = (asic_id & OCRDMA_SLI_ASIC_GEN_NUM_MASK) >>
+   OCRDMA_SLI_ASIC_GEN_NUM_SHIFT;
+   }
+
+   return asic_gen;
+}
+
 #endif
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_hw.c 
b/drivers/infiniband/hw/ocrdma/ocrdma_hw.c
index 56bf32f..ccb7245 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_hw.c
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_hw.c
@@ -1036,7 +1036,7 @@ static void ocrdma_get_attr(struct ocrdma_dev *dev,
attr->max_inline_data =
attr->wqe_size - (sizeof(struct ocrdma_hdr_wqe) +
  sizeof(struct ocrdma_sge));
-   if (dev->nic_info.dev_family == OCRDMA_GEN2_FAMILY) {
+   if (ocrdma_get_asic_type(dev) == OCRDMA_ASIC_GEN_SKH_R) {
attr->ird = 1;
attr->ird_page_size = OCRDMA_MIN_Q_PAGE_SIZE;
attr->num_ird_pages = MAX_OCRDMA_IRD_PAGES; @@ -1380,7 +1380,7 
@@ int ocrdma_mbx_create_cq(struct ocrdma_dev *dev, struct ocrdma_cq *cq,
   __func__, dev->id, dev->attr.max_cqe, entries);
return -EINVAL;
}
-   if (dpp_cq && (dev->nic_info.dev_family != OCRDMA_GEN2_FAMILY))
+   if (dpp_cq && (ocrdma_get_asic_type(dev) != OCRDMA_ASIC_GEN_SKH_R))
return -EINVAL;
 
if (dpp_cq) {
@@ -1439,7 +1439,7 @@ int ocrdma_mbx_create_cq(struct ocrdma_dev *dev, struct 
ocrdma_cq *cq,
}
/* shared eq between all the consumer cqs. */
cmd->cmd.eqn = cq->eqn;
-   if (dev->nic_info.dev_family == OCRDMA_GEN2_FAMILY) {
+   if (ocrdma_get_asic_type(dev) == OCRDMA_ASIC_GEN_SKH_R) {
if (dpp_cq)
cmd->cmd.pgsz_pgcnt |= OCRDMA_CREATE_CQ_DPP <<
OCRDMA_CREATE_CQ_TYPE_SHIFT;
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_main.c 
b/drivers/infiniband/hw/ocrdma/ocrdma_main.c
index 91443bc..fc25c5a 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_main.c
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_main.c
@@ -344,7 +344,7 @@ static int ocrdma_register_device(struct ocrdma_dev *dev)
 
dev->ibdev.process_mad = ocrdma_process_mad;
 
-   if (dev->nic_info.dev_family == OCRDMA_GEN2_FAMILY) {
+   if (ocrdma_get_asic_type(dev) == OCRDMA_ASIC_GEN_SKH_R) {
dev->ibdev.uverbs_cmd_mask |=
 OCRDMA_UVERBS(CREATE_SRQ) |
 OCRDMA_UVERBS(MODIFY_SRQ) |
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_sli.h 
b/drivers/infiniband/hw/ocrdma/ocrdma_sli.h
index 9f9570e..7d70107 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_sli.h
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_sli.h
@@ -30,8 +30,16 @@
 
 #define Bit(_b) (1 << (_b))
 
-#define OCRDMA_GEN1_FAMILY 0xB
-#define OCRDMA_GEN2_FAMILY 0x2
+enum {
+   OCRDMA_ASIC_GEN_SKH_R = 0x04,
+   OCRDMA_ASIC_GEN_LANCER = 0x0B
+};
+
+enum {
+   OCRDMA_ASIC_REV_A0 = 0x00,
+   OCRDMA_ASIC_REV_B0 = 0x10,
+   OCRDMA_ASIC_REV_C0 = 0x20
+};
 
 #define OCRDMA_SUBSYS_ROCE 10
 enum {
@@ -138,6 +146,11 @@ enum {
 #define OCRDMA_MIN_Q_PAGE_SIZE (4096)
 #define OCRDMA_MAX_Q_PAGES (8)
 
+#define OCRDMA_SLI_ASIC_ID_OFFSET  0x9C
+#define OCRDMA_S

RE: [PATCH V1 2/2] RDMA/ocrdma: read ASIC_ID register to select asic_gen

2014-01-29 Thread Devesh Sharma 
Hi Roland,

Please discard this patch. 
Due to IP based GID changes, this won't apply clean, We are planning to send 
you a new series of patches on for-next tree please use that instead.

-Regards
 Devesh

-Original Message-
From: linux-rdma-ow...@vger.kernel.org 
[mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of devesh.sha...@emulex.com
Sent: Saturday, December 14, 2013 2:35 PM
To: linux-rdma@vger.kernel.org
Cc: rol...@kernel.org; Devesh Sharma
Subject: [PATCH V1 2/2] RDMA/ocrdma: read ASIC_ID register to select asic_gen

From: Devesh Sharma 

compilation error fix

Signed-off-by: Devesh Sharma 
---
 drivers/infiniband/hw/ocrdma/ocrdma.h   |   17 +++--
 drivers/infiniband/hw/ocrdma/ocrdma_verbs.c |2 +-
 2 files changed, 8 insertions(+), 11 deletions(-)

diff --git a/drivers/infiniband/hw/ocrdma/ocrdma.h 
b/drivers/infiniband/hw/ocrdma/ocrdma.h
index 24157f0..0c9ec79 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma.h
+++ b/drivers/infiniband/hw/ocrdma/ocrdma.h
@@ -385,13 +385,6 @@ static inline struct ocrdma_srq *get_ocrdma_srq(struct 
ib_srq *ibsrq)
return container_of(ibsrq, struct ocrdma_srq, ibsrq);  }
 
-
-static inline int ocrdma_get_num_posted_shift(struct ocrdma_qp *qp) -{
-   return ((qp->dev->nic_info.dev_family == OCRDMA_GEN2_FAMILY &&
-qp->id < 128) ? 24 : 16);
-}
-
 static inline int is_cqe_valid(struct ocrdma_cq *cq, struct ocrdma_cqe *cqe)  {
int cqe_valid;
@@ -423,15 +416,13 @@ static inline int is_cqe_wr_imm(struct ocrdma_cqe *cqe)
OCRDMA_CQE_WRITE_IMM) ? 1 : 0;
 }
 
-
 static inline u8 ocrdma_get_asic_type(struct ocrdma_dev *dev)  {
u32 asic_id;
u8 asic_gen = 0xF;
if (dev->nic_info.dev_family == 0xF) {
pci_read_config_dword(dev->nic_info.pdev,
-   OCRDMA_SLI_ASIC_ID_OFFSET,
-   &asic_id);
+   OCRDMA_SLI_ASIC_ID_OFFSET, &asic_id);
asic_gen = (asic_id & OCRDMA_SLI_ASIC_GEN_NUM_MASK) >>
OCRDMA_SLI_ASIC_GEN_NUM_SHIFT;
}
@@ -439,4 +430,10 @@ static inline u8 ocrdma_get_asic_type(struct ocrdma_dev 
*dev)
return asic_gen;
 }
 
+static inline int ocrdma_get_num_posted_shift(struct ocrdma_qp *qp) {
+   return ((ocrdma_get_asic_type(qp->dev) == OCRDMA_ASIC_GEN_SKH_R &&
+qp->id < 128) ? 24 : 16);
+}
+
 #endif
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c 
b/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
index 2edbda6..a8f7ccd 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
@@ -1092,7 +1092,7 @@ static int ocrdma_copy_qp_uresp(struct ocrdma_qp *qp,
}
uresp.db_page_addr = usr_db;
uresp.db_page_size = dev->nic_info.db_page_size;
-   if (dev->nic_info.dev_family == OCRDMA_GEN2_FAMILY) {
+   if (ocrdma_get_asic_type(qp->dev) == OCRDMA_ASIC_GEN_SKH_R) {
uresp.db_sq_offset = OCRDMA_DB_GEN2_SQ_OFFSET;
uresp.db_rq_offset = OCRDMA_DB_GEN2_RQ_OFFSET;
uresp.db_shift = 24;
--
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the 
body of a message to majord...@vger.kernel.org More majordomo info at  
http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH V1 1/2] RDMA/ocrdma: read ASIC_ID register to select asic_gen

2014-01-29 Thread Devesh Sharma 
Hi Roland,

Please discard this patch. 
Due to IP based GID changes, this won't apply clean, We are planning to send 
you a new series of patches on for-next tree please use that instead.

-Regards
 Devesh

-Original Message-
From: linux-rdma-ow...@vger.kernel.org 
[mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of devesh.sha...@emulex.com
Sent: Saturday, December 14, 2013 2:35 PM
To: linux-rdma@vger.kernel.org
Cc: rol...@kernel.org; Devesh Sharma
Subject: [PATCH V1 1/2] RDMA/ocrdma: read ASIC_ID register to select asic_gen

From: Devesh Sharma 

ocrdma driver selects execution path based on sli_family and asic generation 
number.
this introduces reading asic gen number from pci register instead of obtining 
from emulex NIC driver.

Signed-off-by: Devesh Sharma 
---
 drivers/infiniband/hw/ocrdma/ocrdma.h   |   15 +++
 drivers/infiniband/hw/ocrdma/ocrdma_hw.c|6 +++---
 drivers/infiniband/hw/ocrdma/ocrdma_main.c  |2 +-
 drivers/infiniband/hw/ocrdma/ocrdma_sli.h   |   17 +++--
 drivers/infiniband/hw/ocrdma/ocrdma_verbs.c |6 +++---
 5 files changed, 37 insertions(+), 9 deletions(-)

diff --git a/drivers/infiniband/hw/ocrdma/ocrdma.h 
b/drivers/infiniband/hw/ocrdma/ocrdma.h
index 294dd27..24157f0 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma.h
+++ b/drivers/infiniband/hw/ocrdma/ocrdma.h
@@ -424,4 +424,19 @@ static inline int is_cqe_wr_imm(struct ocrdma_cqe *cqe)  }
 
 
+static inline u8 ocrdma_get_asic_type(struct ocrdma_dev *dev) {
+   u32 asic_id;
+   u8 asic_gen = 0xF;
+   if (dev->nic_info.dev_family == 0xF) {
+   pci_read_config_dword(dev->nic_info.pdev,
+   OCRDMA_SLI_ASIC_ID_OFFSET,
+   &asic_id);
+   asic_gen = (asic_id & OCRDMA_SLI_ASIC_GEN_NUM_MASK) >>
+   OCRDMA_SLI_ASIC_GEN_NUM_SHIFT;
+   }
+
+   return asic_gen;
+}
+
 #endif
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_hw.c 
b/drivers/infiniband/hw/ocrdma/ocrdma_hw.c
index 56bf32f..ccb7245 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_hw.c
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_hw.c
@@ -1036,7 +1036,7 @@ static void ocrdma_get_attr(struct ocrdma_dev *dev,
attr->max_inline_data =
attr->wqe_size - (sizeof(struct ocrdma_hdr_wqe) +
  sizeof(struct ocrdma_sge));
-   if (dev->nic_info.dev_family == OCRDMA_GEN2_FAMILY) {
+   if (ocrdma_get_asic_type(dev) == OCRDMA_ASIC_GEN_SKH_R) {
attr->ird = 1;
attr->ird_page_size = OCRDMA_MIN_Q_PAGE_SIZE;
attr->num_ird_pages = MAX_OCRDMA_IRD_PAGES; @@ -1380,7 +1380,7 
@@ int ocrdma_mbx_create_cq(struct ocrdma_dev *dev, struct ocrdma_cq *cq,
   __func__, dev->id, dev->attr.max_cqe, entries);
return -EINVAL;
}
-   if (dpp_cq && (dev->nic_info.dev_family != OCRDMA_GEN2_FAMILY))
+   if (dpp_cq && (ocrdma_get_asic_type(dev) != OCRDMA_ASIC_GEN_SKH_R))
return -EINVAL;
 
if (dpp_cq) {
@@ -1439,7 +1439,7 @@ int ocrdma_mbx_create_cq(struct ocrdma_dev *dev, struct 
ocrdma_cq *cq,
}
/* shared eq between all the consumer cqs. */
cmd->cmd.eqn = cq->eqn;
-   if (dev->nic_info.dev_family == OCRDMA_GEN2_FAMILY) {
+   if (ocrdma_get_asic_type(dev) == OCRDMA_ASIC_GEN_SKH_R) {
if (dpp_cq)
cmd->cmd.pgsz_pgcnt |= OCRDMA_CREATE_CQ_DPP <<
OCRDMA_CREATE_CQ_TYPE_SHIFT;
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_main.c 
b/drivers/infiniband/hw/ocrdma/ocrdma_main.c
index 91443bc..fc25c5a 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_main.c
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_main.c
@@ -344,7 +344,7 @@ static int ocrdma_register_device(struct ocrdma_dev *dev)
 
dev->ibdev.process_mad = ocrdma_process_mad;
 
-   if (dev->nic_info.dev_family == OCRDMA_GEN2_FAMILY) {
+   if (ocrdma_get_asic_type(dev) == OCRDMA_ASIC_GEN_SKH_R) {
dev->ibdev.uverbs_cmd_mask |=
 OCRDMA_UVERBS(CREATE_SRQ) |
 OCRDMA_UVERBS(MODIFY_SRQ) |
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_sli.h 
b/drivers/infiniband/hw/ocrdma/ocrdma_sli.h
index 9f9570e..7d70107 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_sli.h
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_sli.h
@@ -30,8 +30,16 @@
 
 #define Bit(_b) (1 << (_b))
 
-#define OCRDMA_GEN1_FAMILY 0xB
-#define OCRDMA_GEN2_FAMILY 0x2
+enum {
+   OCRDMA_ASIC_GEN_SKH_R = 0x04,
+   OCRDMA_ASIC_GEN_LANCER = 0x0B
+};
+
+enum {
+   OCRDMA_ASIC_REV_A0 = 0x00,
+   OCRDMA_ASIC_REV_B0 = 0x10,
+   OCRDMA_ASIC_REV_C0 = 0x20
+};
 
 #define OCRDMA_SUBSYS_ROCE 10
 enum {
@@ -138,6 +146,11 @@ enum {
 #define OCRDMA_MIN_Q_PAGE_SIZE (4096)
 #define OCRDMA_MAX_Q_PAGES (8)
 
+#define OCRDMA_SLI_ASIC_ID_OFFSET  0x9C
+#define OCRDMA

RE: [PATCH] RDMA/ocrdma: Eq full catastrophe avoidance Signed-off-by: Devesh Sharma

2014-01-29 Thread Devesh Sharma 
Hi Roland,

Please discard this patch. 
Due to IP based GID changes, this won't apply clean, We are planning to send 
you a new series of patches on for-next tree please use that instead.

-Regards
 Devesh

-Original Message-
From: linux-rdma-ow...@vger.kernel.org 
[mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of devesh.sha...@emulex.com
Sent: Saturday, December 14, 2013 12:07 PM
To: linux-rdma@vger.kernel.org
Cc: rol...@kernel.org; Devesh Sharma
Subject: [PATCH] RDMA/ocrdma: Eq full catastrophe avoidance Signed-off-by: 
Devesh Sharma 

From: Devesh Sharma 

Stale entries in the CQ being destroyed causes hardware to generate EQEs 
indefinetly for a given CQ.
Thus causing uncontrolled execution of irq_handler. This patch fixes this using 
following sementics:

* irq_handler will ring EQ doorbell atleast once and implement budgeting 
scheme.
* cq_destroy will count number of valid entires during destroy and ring
  cq-db so that hadrware does not generate uncontrolled EQE.
* cq_destroy will synchronize with last running irq_handler instance.
---
 drivers/infiniband/hw/ocrdma/ocrdma.h   |   15 -
 drivers/infiniband/hw/ocrdma/ocrdma_hw.c|   58 ++-
 drivers/infiniband/hw/ocrdma/ocrdma_verbs.c |   80 +++
 3 files changed, 111 insertions(+), 42 deletions(-)

diff --git a/drivers/infiniband/hw/ocrdma/ocrdma.h 
b/drivers/infiniband/hw/ocrdma/ocrdma.h
index adc11d1..5747fde 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma.h
+++ b/drivers/infiniband/hw/ocrdma/ocrdma.h
@@ -183,8 +183,7 @@ struct ocrdma_cq {
 */
u32 max_hw_cqe;
bool phase_change;
-   bool armed, solicited;
-   bool arm_needed;
+   bool deferred_arm, deferred_sol;
 
spinlock_t cq_lock cacheline_aligned; /* provide synchronization
   * to cq polling
@@ -197,6 +196,7 @@ struct ocrdma_cq {
struct ocrdma_ucontext *ucontext;
dma_addr_t pa;
u32 len;
+   u32 cqe_cnt;
 
/* head of all qp's sq and rq for which cqes need to be flushed
 * by the software.
@@ -422,5 +422,16 @@ static inline int is_cqe_wr_imm(struct ocrdma_cqe *cqe)
OCRDMA_CQE_WRITE_IMM) ? 1 : 0;
 }
 
+static inline int ocrdma_get_eq_table_index(struct ocrdma_dev *dev, int 
+eqid) {
+   int indx;
+
+   for (indx = 0; indx < dev->eq_cnt; indx++) {
+   if (dev->eq_tbl[indx].q.id == eqid)
+   return indx;
+   }
+   return -EINVAL;
+}
 
+int ocrdma_get_irq(struct ocrdma_dev *dev, struct ocrdma_eq *eq);
 #endif
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_hw.c 
b/drivers/infiniband/hw/ocrdma/ocrdma_hw.c
index 50219ab..9490b41 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_hw.c
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_hw.c
@@ -444,7 +444,7 @@ mbx_err:
return status;
 }
 
-static int ocrdma_get_irq(struct ocrdma_dev *dev, struct ocrdma_eq *eq)
+int ocrdma_get_irq(struct ocrdma_dev *dev, struct ocrdma_eq *eq)
 {
int irq;
 
@@ -526,6 +526,7 @@ static u32 ocrdma_encoded_q_len(int q_len)
return len_encoded;
 }
 
+
 static int ocrdma_mbx_create_mq(struct ocrdma_dev *dev,
struct ocrdma_queue_info *mq,
struct ocrdma_queue_info *cq)
@@ -574,6 +575,7 @@ static int ocrdma_create_mq(struct ocrdma_dev *dev)
if (status)
goto alloc_err;
 
+   dev->eq_tbl[0].cq_cnt++;
status = ocrdma_mbx_mq_cq_create(dev, &dev->mq.cq, &dev->eq_tbl[0].q);
if (status)
goto mbx_cq_free;
@@ -858,16 +860,8 @@ static void ocrdma_qp_cq_handler(struct ocrdma_dev *dev, 
u16 cq_idx)
BUG();
 
cq = dev->cq_tbl[cq_idx];
-   if (cq == NULL) {
-   pr_err("%s%d invalid id=0x%x\n", __func__, dev->id, cq_idx);
+   if (cq == NULL)
return;
-   }
-   spin_lock_irqsave(&cq->cq_lock, flags);
-   cq->armed = false;
-   cq->solicited = false;
-   spin_unlock_irqrestore(&cq->cq_lock, flags);
-
-   ocrdma_ring_cq_db(dev, cq->id, false, false, 0);
 
if (cq->ibcq.comp_handler) {
spin_lock_irqsave(&cq->comp_handler_lock, flags); @@ -892,27 
+886,35 @@ static irqreturn_t ocrdma_irq_handler(int irq, void *handle)
struct ocrdma_dev *dev = eq->dev;
struct ocrdma_eqe eqe;
struct ocrdma_eqe *ptr;
-   u16 eqe_popped = 0;
u16 cq_id;
-   while (1) {
+   int budget = eq->cq_cnt;
+
+   do {
ptr = ocrdma_get_eqe(eq);
eqe = *ptr;
ocrdma_le32_to_cpu(&eqe, sizeof(eqe));
if ((eqe.id_valid & OCRDMA_EQE_VALID_MASK) == 0)
break;
-   eqe_popped += 1;
+
ptr->id_valid = 0;
+   /* ring eq doorbell as soon as its consumed. */
+   ocrdma_ring_eq_db(dev,

RE: [PATCH 1/2] RDMA/ocrdma: ABI versioning between ocrdma and be2net

2014-01-29 Thread Devesh Sharma 
Hi Roland,

Please discard this patch. 
Due to IP based GID changes, this won't apply clean, We are planning to send 
you a new series of patches on for-next tree please use that instead.

-Regards
 Devesh
-Original Message-
From: linux-rdma-ow...@vger.kernel.org 
[mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of devesh.sha...@emulex.com
Sent: Saturday, December 14, 2013 11:44 AM
To: linux-rdma@vger.kernel.org
Cc: rol...@kernel.org; Devesh Sharma
Subject: [PATCH 1/2] RDMA/ocrdma: ABI versioning between ocrdma and be2net

From: Devesh Sharma 

While loading RoCE driver be2net driver should check for ABI version to catch 
functional incompatibilities.

Signed-off-by: Devesh Sharma 
---
 drivers/infiniband/hw/ocrdma/ocrdma_abi.h  |1 +
 drivers/infiniband/hw/ocrdma/ocrdma_main.c |1 +
 2 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_abi.h 
b/drivers/infiniband/hw/ocrdma/ocrdma_abi.h
index fbac8eb..2a14d4a 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_abi.h
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_abi.h
@@ -29,6 +29,7 @@
 #define __OCRDMA_ABI_H__
 
 #define OCRDMA_ABI_VERSION 1
+#define OCRDMA_BE_ROCE_ABI_VERSION 1
 /* user kernel communication data structures. */
 
 struct ocrdma_alloc_ucontext_resp {
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_main.c 
b/drivers/infiniband/hw/ocrdma/ocrdma_main.c
index 91443bc..bdecfcc 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_main.c
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_main.c
@@ -540,6 +540,7 @@ static struct ocrdma_driver ocrdma_drv = {
.add= ocrdma_add,
.remove = ocrdma_remove,
.state_change_handler   = ocrdma_event_handler,
+   .be_abi_version = OCRDMA_BE_ROCE_ABI_VERSION,
 };
 
 static void ocrdma_unregister_inet6addr_notifier(void)
--
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the 
body of a message to majord...@vger.kernel.org More majordomo info at  
http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH 2/2] RDMA/ocrdma: emulex NIC driver ABI version support

2014-01-29 Thread Devesh Sharma 
Hi Roland,

Please discard this patch. 
Due to IP based GID changes, this won't apply clean, We are planning to send 
you a new series of patches on for-next tree please use that instead.

-Regards
 Devesh

-Original Message-
From: linux-rdma-ow...@vger.kernel.org 
[mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of devesh.sha...@emulex.com
Sent: Saturday, December 14, 2013 11:44 AM
To: linux-rdma@vger.kernel.org
Cc: rol...@kernel.org; Devesh Sharma
Subject: [PATCH 2/2] RDMA/ocrdma: emulex NIC driver ABI version support

From: Devesh Sharma 

Emulex NIC driver support for ABI version support in be_roce.c

Signed-off-by: Devesh Sharma 
---
 drivers/net/ethernet/emulex/benet/be_roce.c |6 ++
 drivers/net/ethernet/emulex/benet/be_roce.h |3 +++
 2 files changed, 9 insertions(+), 0 deletions(-)

diff --git a/drivers/net/ethernet/emulex/benet/be_roce.c 
b/drivers/net/ethernet/emulex/benet/be_roce.c
index 9cd5415..aa7f943 100644
--- a/drivers/net/ethernet/emulex/benet/be_roce.c
+++ b/drivers/net/ethernet/emulex/benet/be_roce.c
@@ -35,6 +35,12 @@ static void _be_roce_dev_add(struct be_adapter *adapter)
 
if (!ocrdma_drv)
return;
+
+   if (ocrdma_drv->be_abi_version != BE_ROCE_ABI_VERSION) {
+   dev_warn(&pdev->dev, "Cannot initialize RoCE due to ocrdma ABI 
mismatch\n");
+   return;
+   }
+
if (pdev->device == OC_DEVICE_ID5) {
/* only msix is supported on these devices */
if (!msix_enabled(adapter))
diff --git a/drivers/net/ethernet/emulex/benet/be_roce.h 
b/drivers/net/ethernet/emulex/benet/be_roce.h
index 2cd1129..1bfb161 100644
--- a/drivers/net/ethernet/emulex/benet/be_roce.h
+++ b/drivers/net/ethernet/emulex/benet/be_roce.h
@@ -21,6 +21,8 @@
 #include 
 #include 
 
+#define BE_ROCE_ABI_VERSION1
+
 struct ocrdma_dev;
 
 enum be_interrupt_mode {
@@ -52,6 +54,7 @@ struct be_dev_info {
 /* ocrdma driver register's the callback functions with nic driver. */  struct 
ocrdma_driver {
unsigned char name[32];
+   u32 be_abi_version;
struct ocrdma_dev *(*add) (struct be_dev_info *dev_info);
void (*remove) (struct ocrdma_dev *);
void (*state_change_handler) (struct ocrdma_dev *, u32 new_state);
--
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the 
body of a message to majord...@vger.kernel.org More majordomo info at  
http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH] RDMA/ocrdma: SQ and RQ doorbell offset clean up

2014-01-29 Thread Devesh Sharma 
Hi Roland,

Please discard this patch. 
Due to IP based GID changes, this won't apply clean, We are planning to send 
you a new series of patches on for-next tree please use that instead.

-Regards
 Devesh
-Original Message-
From: linux-rdma-ow...@vger.kernel.org 
[mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of devesh.sha...@emulex.com
Sent: Monday, November 18, 2013 4:58 PM
To: linux-rdma@vger.kernel.org
Subject: [PATCH] RDMA/ocrdma: SQ and RQ doorbell offset clean up

From: Devesh Sharma

Introducing new macros to define SQ and RQ doorbell offset.

Signed-off-by: Devesh Sharma 
---
 drivers/infiniband/hw/ocrdma/ocrdma.h   |7 ---
 drivers/infiniband/hw/ocrdma/ocrdma_sli.h   |5 -
 drivers/infiniband/hw/ocrdma/ocrdma_verbs.c |   18 ++
 3 files changed, 10 insertions(+), 20 deletions(-)

diff --git a/drivers/infiniband/hw/ocrdma/ocrdma.h 
b/drivers/infiniband/hw/ocrdma/ocrdma.h
index adc11d1..07d156f 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma.h
+++ b/drivers/infiniband/hw/ocrdma/ocrdma.h
@@ -384,13 +384,6 @@ static inline struct ocrdma_srq *get_ocrdma_srq(struct 
ib_srq *ibsrq)
return container_of(ibsrq, struct ocrdma_srq, ibsrq);  }
 
-
-static inline int ocrdma_get_num_posted_shift(struct ocrdma_qp *qp) -{
-   return ((qp->dev->nic_info.dev_family == OCRDMA_GEN2_FAMILY &&
-qp->id < 128) ? 24 : 16);
-}
-
 static inline int is_cqe_valid(struct ocrdma_cq *cq, struct ocrdma_cqe *cqe)  {
int cqe_valid;
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_sli.h 
b/drivers/infiniband/hw/ocrdma/ocrdma_sli.h
index 9f9570e..38df269 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_sli.h
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_sli.h
@@ -103,7 +103,10 @@ enum {
OCRDMA_DB_GEN2_SRQ_OFFSET   = OCRDMA_DB_GEN2_RQ_OFFSET,
OCRDMA_DB_CQ_OFFSET = 0x120,
OCRDMA_DB_EQ_OFFSET = OCRDMA_DB_CQ_OFFSET,
-   OCRDMA_DB_MQ_OFFSET = 0x140
+   OCRDMA_DB_MQ_OFFSET = 0x140,
+
+   OCRDMA_DB_SQ_SHIFT  = 16,
+   OCRDMA_DB_RQ_SHIFT  = 24
 };
 
 #define OCRDMA_DB_CQ_RING_ID_MASK   0x3FF  /* bits 0 - 9 */
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c 
b/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
index 69f1d12..c80ba6e 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
@@ -1092,15 +1092,9 @@ static int ocrdma_copy_qp_uresp(struct ocrdma_qp *qp,
}
uresp.db_page_addr = usr_db;
uresp.db_page_size = dev->nic_info.db_page_size;
-   if (dev->nic_info.dev_family == OCRDMA_GEN2_FAMILY) {
-   uresp.db_sq_offset = OCRDMA_DB_GEN2_SQ_OFFSET;
-   uresp.db_rq_offset = OCRDMA_DB_GEN2_RQ_OFFSET;
-   uresp.db_shift = 24;
-   } else {
-   uresp.db_sq_offset = OCRDMA_DB_SQ_OFFSET;
-   uresp.db_rq_offset = OCRDMA_DB_RQ_OFFSET;
-   uresp.db_shift = 16;
-   }
+   uresp.db_sq_offset = OCRDMA_DB_GEN2_SQ_OFFSET;
+   uresp.db_rq_offset = OCRDMA_DB_GEN2_RQ_OFFSET;
+   uresp.db_shift = OCRDMA_DB_RQ_SHIFT;
 
if (qp->dpp_enabled) {
uresp.dpp_credit = dpp_credit_lmt;
@@ -1273,7 +1267,7 @@ static void ocrdma_flush_rq_db(struct ocrdma_qp *qp)  {
if (qp->db_cache) {
u32 val = qp->rq.dbid | (qp->db_cache <<
-   ocrdma_get_num_posted_shift(qp));
+   OCRDMA_DB_RQ_SHIFT);
iowrite32(val, qp->rq_db);
qp->db_cache = 0;
}
@@ -2018,7 +2012,7 @@ static int ocrdma_build_fr(struct ocrdma_qp *qp, struct 
ocrdma_hdr_wqe *hdr,
 
 static void ocrdma_ring_sq_db(struct ocrdma_qp *qp)  {
-   u32 val = qp->sq.dbid | (1 << 16);
+   u32 val = qp->sq.dbid | (1 << OCRDMA_DB_SQ_SHIFT);
 
iowrite32(val, qp->sq_db);
 }
@@ -2123,7 +2117,7 @@ int ocrdma_post_send(struct ib_qp *ibqp, struct 
ib_send_wr *wr,
 
 static void ocrdma_ring_rq_db(struct ocrdma_qp *qp)  {
-   u32 val = qp->rq.dbid | (1 << ocrdma_get_num_posted_shift(qp));
+   u32 val = qp->rq.dbid | (1 << OCRDMA_DB_RQ_SHIFT);
 
if (qp->state != OCRDMA_QPS_INIT)
iowrite32(val, qp->rq_db);
--
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the 
body of a message to majord...@vger.kernel.org More majordomo info at  
http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

mlx4 qp allocation

2014-01-29 Thread Bob Biloxi 
Hi,

I was going through the linux/drivers/net/ethernet/mellanox/mlx4/qp.c

Got a few questions. Would really appreciate if someone can clarify:

In the function, mlx4_qp_alloc_icm,

To allocate a QP, there are 2 paths taken:

using the ALLOC_RES virtual command

using the MAP_ICM

These paths are taken based on the return value of mlx4_is_func(dev).
This is true for MASTER or SLAVE which I believe is Physical Function
Driver/Virtual Function Driver. So for SRIOV, it covers all cases.

The MAP_ICM portion which gets executed as part of __mlx4_qp_alloc_icm
never gets called??

Am I understanding it properly? Because as per my understanding ICM
needs to be allocated for all the QPs.

Please help me in understanding this.

Thanks so much.

Best Regards,

Bob
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html