Re: [PATCH] IB: merge struct ib_device_attr into struct ib_device

2015-10-11 Thread Sagi Grimberg
On 9/24/2015 4:35 PM, Christoph Hellwig wrote: On Thu, Sep 24, 2015 at 08:41:17AM +0300, Or Gerlitz wrote: We had a smaller volume move to cache the device attributes on the IB device structure, and I just realized it was dropped on the floor. Ira, that was a reviewer comment you got when worke

Re: Seeing WARN_ON in ib_dealloc_pd from ipoib in kernel 4.3-rc1-debug

2015-10-11 Thread Sagi Grimberg
Yes. I'm seeing this too. The last time this popped up I fixed it by adding the code for reaping ahs. I suspect that the new code to timeout sendonly multicast joins combined with us now creating and joining what used to be sendonly groups is the likely culprit here. Is someone looking at

Re: [PATCH v1 2/2] IB/iser: Enable SG clustering

2015-10-11 Thread Sagi Grimberg
On 10/11/2015 6:35 PM, Sagi Grimberg wrote: iser can handle it and it can dramatically reduce the number of SG elements. It doesn't make much of a difference at the moment, but with arbitrary SG list support it will benefit greatly. Bahh, I updated the change-log in the previous rev

[PATCH v1 0/2] iser bounce buffering cleanup

2015-10-11 Thread Sagi Grimberg
h set. Changes from v0: - Added Reviewed-by tags - Fixed patch (1) title Sagi Grimberg (2): iser: set block queue_virt_boundary IB/iser: Enable SG clustering drivers/infiniband/ulp/iser/iscsi_iser.c | 14 +- drivers/infiniband/ulp/iser/iscsi_iser.h | 7 +- drivers/infiniband/ulp

[PATCH v1 1/2] IB/iser: set block queue_virt_boundary

2015-10-11 Thread Sagi Grimberg
By limiting the sg lists that we are allowing to meet and 4k virtual boundary, we can completely remove the entire bounce buffering logic. Signed-off-by: Sagi Grimberg Reviewed-by: Christoph Hellwig Cc: Jens Axboe Cc: Martin K. Petersen --- drivers/infiniband/ulp/iser/iscsi_iser.c | 12

[PATCH v1 2/2] IB/iser: Enable SG clustering

2015-10-11 Thread Sagi Grimberg
iser can handle it and it can dramatically reduce the number of SG elements. It doesn't make much of a difference at the moment, but with arbitrary SG list support it will benefit greatly. Signed-off-by: Sagi Grimberg --- drivers/infiniband/ulp/iser/iscsi_iser.c | 2 +- 1 file chang

Re: [PATCH 2/2] IB/iser: Enable SG clustering

2015-10-11 Thread Sagi Grimberg
but with arbitrary SG list support it will benefit greatly. Y? please put some wording here to clarify. Well, because when a device will accept true SG lists then it would benefit the HCA caching/fetching to have less SG elements. I'll add a sentence to the change log. -- To unsubscribe from

Re: [PATCH 1/2] iser: Set block queue_virt_boundary and remove bounce buffering code

2015-10-11 Thread Sagi Grimberg
On 10/10/2015 10:33 PM, Or Gerlitz wrote: On Sat, Oct 10, 2015 at 3:27 AM, Sagi Grimberg wrote: The block layer reliably enforces no SG lists with gaps to a 4k virtual boundary so we can completely remove the entire bounce buffering logic. Hi Sagi, Hey Or, (Very) nice cleanup -- so

Re: [PATCH v3 12/26] xprtrdma: Port to new memory registration API

2015-10-09 Thread Sagi Grimberg
out_list_err: - rc = PTR_ERR(f->fr_pgl); + rc = -ENOMEM; dprintk("RPC: %s: ib_alloc_fast_reg_page_list status %i\n”, Should you update this debugging message? Yes I will. + n = ib_map_mr_sg(mr, frmr->sg, frmr->sg_nents, PAGE_SIZE); + if (unlikely(n !=

[PATCH 0/2] iser bounce buffering cleanup

2015-10-09 Thread Sagi Grimberg
h set. Sagi Grimberg (2): iser: set block queue_virt_boundary IB/iser: Enable SG clustering drivers/infiniband/ulp/iser/iscsi_iser.c | 14 +- drivers/infiniband/ulp/iser/iscsi_iser.h | 7 +- drivers/infiniband/ulp/iser/iser_initiator.c | 51 + drivers/infiniband/ulp/iser/iser_mem

[PATCH 2/2] IB/iser: Enable SG clustering

2015-10-09 Thread Sagi Grimberg
iser can handle it and it can dramatically reduce the number of SG elements. It doesn't make much of a difference at the moment, but with arbitrary SG list support it will benefit greatly. Signed-off-by: Sagi Grimberg --- drivers/infiniband/ulp/iser/iscsi_iser.c | 2 +- 1 file chang

[PATCH 1/2] iser: Set block queue_virt_boundary and remove bounce buffering code

2015-10-09 Thread Sagi Grimberg
The block layer reliably enforces no SG lists with gaps to a 4k virtual boundary so we can completely remove the entire bounce buffering logic. Signed-off-by: Sagi Grimberg --- drivers/infiniband/ulp/iser/iscsi_iser.c | 12 +- drivers/infiniband/ulp/iser/iscsi_iser.h | 7 +- drivers

Re: [PATCH v3 00/26] New fast registration API

2015-10-08 Thread Sagi Grimberg
On 10/8/2015 11:01 AM, Christoph Hellwig wrote: On Thu, Oct 08, 2015 at 10:50:43AM +0300, Sagi Grimberg wrote: - Rebased against Doug's for-4.4 tree (4.3.0-rc1) + 4.3-rc fixes Does this means its not on top of the send_wr changes now? I'm fine with that as I think your series is

[PATCH v3 22/26] RDMA/cxgb3: Remove old FRWR API

2015-10-08 Thread Sagi Grimberg
No ULP uses it anymore, go ahead and remove it. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/cxgb3/iwch_cq.c | 2 +- drivers/infiniband/hw/cxgb3/iwch_provider.c | 24 --- drivers/infiniband/hw/cxgb3/iwch_qp.c | 47

[PATCH v3 19/26] IB/mlx5: Remove old FRWR API support

2015-10-08 Thread Sagi Grimberg
No ULP uses it anymore, go ahead and remove it. Keep only the local invalidate part of the handlers. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/mlx5/cq.c | 3 -- drivers/infiniband/hw/mlx5/main.c| 2 - drivers/infiniband/hw/mlx5/mlx5_ib.h | 14

[PATCH v3 20/26] IB/mlx4: Remove old FRWR API support

2015-10-08 Thread Sagi Grimberg
No ULP uses it anymore, go ahead and remove it. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/mlx4/cq.c | 3 +-- drivers/infiniband/hw/mlx4/main.c| 2 -- drivers/infiniband/hw/mlx4/mlx4_ib.h | 15 --- drivers/infiniband/hw/mlx4/mr.c

[PATCH v3 14/26] RDS/IW: Convert to new memory registration API

2015-10-08 Thread Sagi Grimberg
Get rid of fast_reg page list and its construction. Instead, just pass the RDS sg list to ib_map_mr_sg and post the new ib_reg_wr. This is done both for server IW RDMA_READ registration and the client remote key registration. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig Acked-by

[PATCH v3 08/26] IB/qib: Support the new memory registration API

2015-10-08 Thread Sagi Grimberg
(qib_mr) - key, access flags (ib_reg_wr) The IB_WR_FAST_REG_MR handlers will be removed later when all the ULPs will be converted. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/qib/qib_keys.c | 56 +++ drivers/infiniband/hw

[PATCH v3 24/26] IB/qib: Remove old FRWR API

2015-10-08 Thread Sagi Grimberg
No ULP uses it anymore, go ahead and remove it. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/qib/qib_keys.c | 56 --- drivers/infiniband/hw/qib/qib_mr.c| 32 +--- drivers/infiniband/hw/qib/qib_verbs.c

[PATCH v3 16/26] IB/srp: Convert to new registration API

2015-10-08 Thread Sagi Grimberg
Instead of constructing a page list, call ib_map_mr_sg and post a new ib_reg_wr. srp_map_finish_fr now returns the number of sg elements registered. Remove srp_finish_mapping since no one is calling it. Signed-off-by: Sagi Grimberg Tested-by: Bart Van Assche --- drivers/infiniband/ulp/srp

[PATCH v3 15/26] IB/srp: Split srp_map_sg

2015-10-08 Thread Sagi Grimberg
This is a preparation patch for the new registration API conversion. It splits srp_map_sg per registration strategy (srp_map_sg[fmr|fr|dma]. On its own it adds some code duplication, but it makes the API switch easier to comprehend. Signed-off-by: Sagi Grimberg Tested-by: Bart Van Assche

[PATCH v3 18/26] IB/srp: Dont allocate a page vector when using fast_reg

2015-10-08 Thread Sagi Grimberg
The new fast registration API does not reuqire a page vector so we can't avoid allocating it. Signed-off-by: Sagi Grimberg Tested-by: Bart Van Assche --- drivers/infiniband/ulp/srp/ib_srp.c | 20 +++- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/dr

[PATCH v3 21/26] RDMA/ocrdma: Remove old FRWR API

2015-10-08 Thread Sagi Grimberg
No ULP uses it anymore, go ahead and remove it. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/ocrdma/ocrdma_main.c | 2 - drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 104 +--- drivers/infiniband/hw/ocrdma/ocrdma_verbs.h | 4

[PATCH v3 17/26] IB/srp: Remove srp_finish_mapping

2015-10-08 Thread Sagi Grimberg
No callers left, remove it. Signed-off-by: Sagi Grimberg Tested-by: Bart Van Assche --- drivers/infiniband/ulp/srp/ib_srp.c | 10 -- 1 file changed, 10 deletions(-) diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c index 6d399378928d..d4a5a9b86390

[PATCH v3 23/26] iw_cxgb4: Remove old FRWR API

2015-10-08 Thread Sagi Grimberg
No ULP uses it anymore, go ahead and remove it. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/cxgb4/cq.c | 2 +- drivers/infiniband/hw/cxgb4/iw_cxgb4.h | 18 drivers/infiniband/hw/cxgb4/mem.c | 45 drivers

[PATCH v3 26/26] IB/core: Remove old fast registration API

2015-10-08 Thread Sagi Grimberg
No callers and no providers left, go ahead and remove it. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/core/verbs.c | 25 --- include/rdma/ib_verbs.h | 54 - 2 files changed, 79 deletions

[PATCH v3 25/26] RDMA/nes: Remove old FRWR API

2015-10-08 Thread Sagi Grimberg
No ULP uses it anymore, go ahead and remove it. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/nes/nes_hw.h| 6 -- drivers/infiniband/hw/nes/nes_verbs.c | 162 +- 2 files changed, 1 insertion(+), 167 deletions(-) diff

[PATCH v3 04/26] IB/mlx4: Support the new memory registration API

2015-10-08 Thread Sagi Grimberg
, length, access flags (ib_mr) - page array (mlx4_ib_mr) - key (ib_reg_wr) The IB_WR_FAST_REG_MR handlers will be removed later when all the ULPs will be converted. Signed-off-by: Sagi Grimberg Tested-by: Christoph Hellwig --- drivers/infiniband/hw/mlx4/cq.c | 1 + drivers/infiniband/hw/mlx4

[PATCH v3 07/26] iw_cxgb4: Support the new memory registration API

2015-10-08 Thread Sagi Grimberg
(c4iw_mr) - key, access flags (ib_reg_wr) The IB_WR_FAST_REG_MR handlers will be removed later when all the ULPs will be converted. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig Tested-by: Steve Wise --- drivers/infiniband/hw/cxgb4/iw_cxgb4.h | 7 drivers/infiniband/hw/cxgb4

[PATCH v3 13/26] svcrdma: Port to new memory registration API

2015-10-08 Thread Sagi Grimberg
Instead of maintaining a fastreg page list, keep an sg table and convert an array of pages to a sg list. Then call ib_map_mr_sg and construct ib_reg_wr. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig Tested-by: Steve Wise Tested-by: Selvin Xavier --- include/linux/sunrpc/svc_rdma.h

[PATCH v3 09/26] RDMA/nes: Support the new memory registration API

2015-10-08 Thread Sagi Grimberg
) - page array (nes_mr) - key, access flags (ib_reg_wr) The IB_WR_FAST_REG_MR handlers will be removed later when all the ULPs will be converted. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/nes/nes_verbs.c | 117 +- drivers

[PATCH v3 03/26] IB/mlx5: Support the new memory registration API

2015-10-08 Thread Sagi Grimberg
, length, access flags (ib_mr) - page array (mlx5_ib_mr) - key (ib_reg_wr) The IB_WR_FAST_REG_MR handlers will be removed later when all the ULPs will be converted. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/mlx5/cq.c | 3 ++ drivers/infiniband/hw/mlx5

[PATCH v3 10/26] IB/iser: Port to new fast registration API

2015-10-08 Thread Sagi Grimberg
Remove fastreg page list allocation as the page vector is now private to the provider. Instead of constructing the page list and fast_req work request, call ib_map_mr_sg and construct ib_reg_wr. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/ulp/iser

[PATCH v3 01/26] IB/core: Introduce new fast registration API

2015-10-08 Thread Sagi Grimberg
. This API will allow ULPs to remove the duplicated code of constructing a page vector from a given sg list. The send work request ib_reg_wr also shrinks as it will contain only mr, key and access flags in addition. Signed-off-by: Sagi Grimberg Tested-by: Christoph Hellwig --- drivers/infiniband

[PATCH v3 11/26] iser-target: Port to new memory registration API

2015-10-08 Thread Sagi Grimberg
Remove fastreg page list allocation as the page vector is now private to the provider. Instead of constructing the page list and fast_req work request, call ib_map_mr_sg and construct ib_reg_wr. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/ulp/isert

[PATCH v3 05/26] RDMA/ocrdma: Support the new memory registration API

2015-10-08 Thread Sagi Grimberg
(ib_mr) - page array (ocrdma_mr) - key (ib_reg_wr) The IB_WR_FAST_REG_MR handlers will be removed later when all the ULPs will be converted. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/ocrdma/ocrdma.h | 2 + drivers/infiniband/hw/ocrdma

[PATCH v3 00/26] New fast registration API

2015-10-08 Thread Sagi Grimberg
ck would be in a shared SW library (hfi1, qib, rxe). - Updated the change logs The code is available at: https://github.com/sagigrimberg/linux/tree/reg_api.6 Sagi Grimberg (26): IB/core: Introduce new fast registration API IB/mlx5: Remove dead fmr code IB/mlx5: Support the new memory registra

[PATCH v3 12/26] xprtrdma: Port to new memory registration API

2015-10-08 Thread Sagi Grimberg
&m=143677002622296&w=2). Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig Tested-by: Steve Wise Tested-by: Selvin Xavier --- net/sunrpc/xprtrdma/frwr_ops.c | 112 +++- net/sunrpc/xprtrdma/xprt_rdma.h | 3 +- 2 files changed, 67 insertion

[PATCH v3 02/26] IB/mlx5: Remove dead fmr code

2015-10-08 Thread Sagi Grimberg
Just function declarations - no need for those laying arround. If for some reason someone will want FMR support in mlx5, it should be easy enough to restore a few structs. Signed-off-by: Sagi Grimberg Reviewed-by: Bart Van Assche Acked-by: Christoph Hellwig --- drivers/infiniband/hw/mlx5

[PATCH v3 06/26] RDMA/cxgb3: Support the new memory registration API

2015-10-08 Thread Sagi Grimberg
(iwch_mr) - key, access flags (ib_reg_wr) The IB_WR_FAST_REG_MR handlers will be removed later when all the ULPs will be converted. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/cxgb3/iwch_provider.c | 33 drivers/infiniband/hw/cxgb3

Seeing WARN_ON in ib_dealloc_pd from ipoib in kernel 4.3-rc1-debug

2015-10-07 Thread Sagi Grimberg
This started popping up (not sure if it's new to 4.3-rc1). Happens when unloading the provider driver (mlx4/mlx5 in my case). Has anyone seen this? kernel: [ cut here ] kernel: WARNING: CPU: 2 PID: 6012 at drivers/infiniband/core/verbs.c:283 ib_dealloc_pd+0x5b/0xa0 [ib_

Re: [PATCH v1 00/24] New fast registration API

2015-10-07 Thread Sagi Grimberg
On 10/7/2015 6:46 PM, Bart Van Assche wrote: On 10/06/2015 11:42 PM, Sagi Grimberg wrote: On 10/6/2015 9:49 PM, Bart Van Assche wrote: On 10/06/2015 01:37 AM, Sagi Grimberg wrote: I see now the error you are referring to. The issue is that the device requires the MR page array to have an

Re: [PATCH v2 06/16] xprtrdma: Use workqueue to process RPC/RDMA replies

2015-10-07 Thread Sagi Grimberg
On 10/7/2015 5:48 PM, Chuck Lever wrote: On Oct 7, 2015, at 10:39 AM, Sagi Grimberg wrote: On 10/6/2015 5:59 PM, Chuck Lever wrote: The reply tasklet is fast, but it's single threaded. After reply traffic saturates a single CPU, there's no more reply processing capacity. R

Re: [PATCH v2 06/16] xprtrdma: Use workqueue to process RPC/RDMA replies

2015-10-07 Thread Sagi Grimberg
On 10/6/2015 5:59 PM, Chuck Lever wrote: The reply tasklet is fast, but it's single threaded. After reply traffic saturates a single CPU, there's no more reply processing capacity. Replace the tasklet with a workqueue to spread reply handling across all CPUs. This also moves RPC/RDMA reply hand

Re: [PATCH] IB/core: avoid 32-bit warning

2015-10-07 Thread Sagi Grimberg
On 10/7/2015 3:29 PM, Arnd Bergmann wrote: The INIT_UDATA() macro requires a pointer or unsigned long argument for both input and output buffer, and all callers had a cast from when the code was merged until a recent restructuring, so now we get core/uverbs_cmd.c: In function 'ib_uverbs_create_c

Re: [PATCH v1 00/24] New fast registration API

2015-10-07 Thread Sagi Grimberg
I don't really care either way, it just seemed like an odd change hiding in here that I missed when reviewing earlier. OK, so I'm sticking with it until someone suggests otherwise. Sagi. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord.

Re: [PATCH v1 00/24] New fast registration API

2015-10-07 Thread Sagi Grimberg
On 10/7/2015 12:20 PM, Christoph Hellwig wrote: On Tue, Oct 06, 2015 at 11:37:40AM +0300, Sagi Grimberg wrote: The issue is that the device requires the MR page array to have an alignment (0x40 for mlx4 and 0x400 for mlx5). When I modified the page array allocation to be non-coherent I didn&#

Re: [PATCH v1 00/24] New fast registration API

2015-10-06 Thread Sagi Grimberg
On 10/6/2015 9:49 PM, Bart Van Assche wrote: On 10/06/2015 01:37 AM, Sagi Grimberg wrote: I see now the error you are referring to. The issue is that the device requires the MR page array to have an alignment (0x40 for mlx4 and 0x400 for mlx5). When I modified the page array allocation to be

Re: [PATCH v1 for-next 0/7] Add support for multicast loopback prevention to mlx4

2015-10-06 Thread Sagi Grimberg
Then I have to see if any of the currently posted fixes for 4.3rc that I haven't grabbed yet resolve the iSER issue I'm seeing, then I'll move on to for-next processing. Anything I can help with Doug? -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a mess

[PATCH rdma-rc v1] xprtrdma: Don't require LOCAL_DMA_LKEY support for fasterg

2015-10-06 Thread Sagi Grimberg
place global lkey with lkey local to PD") Signed-off-by: Sagi Grimberg Reviewed-by: Chuck Lever --- Changes from v0: - Added error path rc assignment. net/sunrpc/xprtrdma/verbs.c |8 +++- 1 files changed, 3 insertions(+), 5 deletions(-) diff --git a/net/sunrpc/xprtrdma/verbs.c b/

Re: [PATCH rdma-rc] xprtrdma: Don't require LOCAL_DMA_LKEY support for fasterg

2015-10-06 Thread Sagi Grimberg
On 10/6/2015 6:05 PM, Chuck Lever wrote: Hi Sagi- On Oct 6, 2015, at 5:48 AM, Sagi Grimberg wrote: There is no need to require LOCAL_DMA_LKEY support in order to use fast registration as the PD allocation makes sure that there is a local_dma_lkey. In other words, all devices now have a

[PATCH rdma-rc] xprtrdma: Don't require LOCAL_DMA_LKEY support for fasterg

2015-10-06 Thread Sagi Grimberg
lkey with lkey local to PD") Signed-off-by: Sagi Grimberg --- net/sunrpc/xprtrdma/verbs.c |7 ++- 1 files changed, 2 insertions(+), 5 deletions(-) diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index eb081ad..7efd9ef 100644 --- a/net/sunrpc/xprtrdma/verbs.c

Re: [PATCH v1 00/24] New fast registration API

2015-10-06 Thread Sagi Grimberg
On 10/2/2015 6:37 PM, Bart Van Assche wrote: On 10/01/2015 11:14 PM, Sagi Grimberg wrote: Would you mind sending me your .config? Hello Sagi, Hi Bart, I just sent this .config file to you off-list. I see now the error you are referring to. The issue is that the device requires the MR

Re: [PATCH] svcrdma: Fix NFS server crash triggered by 1MB NFS WRITE

2015-10-05 Thread Sagi Grimberg
-off-by: Chuck Lever --- Looks correct, Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] IB/iser: fix a comment typo

2015-10-05 Thread Sagi Grimberg
On 10/4/2015 12:00 PM, Geliang Tang wrote: Just fix a typo in the code comment. Signed-off-by: Geliang Tang Acked-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo inf

Re: [PATCH v1 00/24] New fast registration API

2015-10-01 Thread Sagi Grimberg
Just this morning (my morning) I tested the v2 set on iser, srp, nfs. I placed that in branch reg_api.5. Would you mind running reg_api.5 and see if this issue persist (I would be surprised because I haven't seen any sign of it)? Sorry but I still see these messages with the reg_api.5 branch.

Re: [PATCH v1 00/24] New fast registration API

2015-09-29 Thread Sagi Grimberg
Hi Bart, How has the converted SRP initiator driver been tested ? With the kernel tree that is available on branch reg_api.4 That's odd. Although I haven't formally submitted reg_api.4 yet, I did test ib_srp initiator against upstream srpt over CX3 (mlx4) and CX4 (mlx5). I ran connect, disconn

Re: [PATCH v1 00/24] New fast registration API

2015-09-29 Thread Sagi Grimberg
On 9/29/2015 10:03 PM, Bart Van Assche wrote: On 09/17/2015 02:42 AM, Sagi Grimberg wrote: - Converted SRP initiator and RDS iwarp ULPs to the new API Hello Sagi, Hi Bart, How has the converted SRP initiator driver been tested ? With the kernel tree that is available on branch reg_api.4

Re: [PATCH v1 01/24] IB/core: Introduce new fast registration API

2015-09-29 Thread Sagi Grimberg
Shouldn't higher layers take care of this? Trying to implement the same coalescing algorithm at various layers isn't very optimal, although we need to decide and document which one is responsible. The block layer can take care of it, but I'm not sure about NFS/RDS at the moment (IIRC Steve spec

Re: [PATCH v1 01/24] IB/core: Introduce new fast registration API

2015-09-28 Thread Sagi Grimberg
On 9/28/2015 11:57 PM, Bart Van Assche wrote: On 09/24/2015 12:37 AM, Sagi Grimberg wrote: On 9/23/2015 12:21 AM, Bart Van Assche wrote: On 09/17/2015 02:42 AM, Sagi Grimberg wrote: +} else if (last_page_off + dma_len < mr->page_size) { +/* chunk this fragmen

Re: [PATCH v1 01/24] IB/core: Introduce new fast registration API

2015-09-28 Thread Sagi Grimberg
On 9/29/2015 9:47 AM, Sagi Grimberg wrote: Shouldn't higher layers take care of this? Trying to implement the same coalescing algorithm at various layers isn't very optimal, although we need to decide and document which one is responsible. The block layer can take care of it, but I&

Re: [PATCH v2 12/26] xprtrdma: Port to new memory registration API

2015-09-27 Thread Sagi Grimberg
On 9/25/2015 7:51 PM, Chuck Lever wrote: On Sep 24, 2015, at 10:34 AM, Sagi Grimberg wrote: Instead of maintaining a fastreg page list, keep an sg table and convert an array of pages to a sg list. Then call ib_map_mr_sg and construct ib_reg_wr. Note that the next step would be to have NFS

Re: [PATCH v2 16/26] IB/srp: Convert to new registration API

2015-09-27 Thread Sagi Grimberg
On 9/25/2015 7:34 PM, Bart Van Assche wrote: On 09/24/2015 10:35 AM, Sagi Grimberg wrote: Remove srp_finish_mapping since no one is calling it. Please move the srp_finish_mapping() removal into a separate patch. Moved, thanks. -- To unsubscribe from this list: send the line "unsubs

Re: [PATCH v2 15/26] IB/srp: Split srp_map_sg

2015-09-27 Thread Sagi Grimberg
On 9/25/2015 7:15 PM, Bart Van Assche wrote: On 09/24/2015 10:35 AM, Sagi Grimberg wrote: This is a perparation patch for the new registration API ^ preparation ? Fixed, thanks. +WARN_ON_ONCE(!dev->use_fmr); + +if (state->npages == 0) +return 0; + +

[PATCH RFC 2/3] mlx5: Add arbitrary sg list support

2015-09-24 Thread Sagi Grimberg
according to the device id (until we have a FW bit that correctly exposes it). Signed-off-by: Sagi Grimberg --- drivers/infiniband/hw/mlx5/main.c| 4 +++ drivers/infiniband/hw/mlx5/mlx5_ib.h | 1 + drivers/infiniband/hw/mlx5/mr.c | 50 +++- drivers/infiniband/hw

[PATCH RFC 1/3] IB/core: Add arbitrary sg_list support

2015-09-24 Thread Sagi Grimberg
memory region which is capable of handling SG lists with gaps. Signed-off-by: Sagi Grimberg --- drivers/infiniband/core/verbs.c | 2 ++ include/rdma/ib_verbs.h | 10 ++ 2 files changed, 12 insertions(+) diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c

[PATCH RFC 0/3] Arbitrary SG list registration support

2015-09-24 Thread Sagi Grimberg
type which is capable of handling SG lists with gaps (pending on the device capability). The code is available at: https://github.com/sagigrimberg/linux/tree/arb_sg.1 Sagi Grimberg (3): IB/core: Add arbitrary sg_list support mlx5: Add arbitrary sg list support iser: Accept arbitrary sg lists

Re: [PATCH v2 01/26] IB/core: Introduce new fast registration API

2015-09-24 Thread Sagi Grimberg
Not sure how a second set glitched in, my mail server flipped I guess Sorry for the double send. Sagi. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.

[PATCH RFC 3/3] iser: Accept arbitrary sg lists mapping if the device supports it

2015-09-24 Thread Sagi Grimberg
If the device support arbitrary sg list mapping (device cap IB_DEVICE_SG_GAPS_REG set) we allocate the memory regions with IB_MR_MAP_SG_GAPS_REG and allow the block layer to pass us gaps by skip setting the queue virt_boundary. Signed-off-by: Sagi Grimberg --- drivers/infiniband/ulp/iser

[PATCH v2 20/26] RDMA/ocrdma: Remove old FRWR API

2015-09-24 Thread Sagi Grimberg
No ULP uses it anymore, go ahead and remove it. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/ocrdma/ocrdma_main.c | 2 - drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 104 +--- drivers/infiniband/hw/ocrdma/ocrdma_verbs.h | 4

[PATCH v2 21/26] RDMA/cxgb3: Remove old FRWR API

2015-09-24 Thread Sagi Grimberg
No ULP uses it anymore, go ahead and remove it. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/cxgb3/iwch_cq.c | 2 +- drivers/infiniband/hw/cxgb3/iwch_provider.c | 24 --- drivers/infiniband/hw/cxgb3/iwch_qp.c | 47

[PATCH v2 20/26] RDMA/ocrdma: Remove old FRWR API

2015-09-24 Thread Sagi Grimberg
No ULP uses it anymore, go ahead and remove it. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/ocrdma/ocrdma_main.c | 2 - drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 104 +--- drivers/infiniband/hw/ocrdma/ocrdma_verbs.h | 4

[PATCH v2 18/26] IB/mlx5: Remove old FRWR API support

2015-09-24 Thread Sagi Grimberg
No ULP uses it anymore, go ahead and remove it. Keep only the local invalidate part of the handlers. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/mlx5/cq.c | 3 -- drivers/infiniband/hw/mlx5/main.c| 2 - drivers/infiniband/hw/mlx5/mlx5_ib.h | 14

[PATCH v2 17/26] IB/srp: Dont allocate a page vector when using fast_reg

2015-09-24 Thread Sagi Grimberg
The new fast registration API does not reuqire a page vector so we can't avoid allocating it. Signed-off-by: Sagi Grimberg --- drivers/infiniband/ulp/srp/ib_srp.c | 20 +++- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/drivers/infiniband/ulp/srp/ib_srp

[PATCH v2 22/26] iw_cxgb4: Remove old FRWR API

2015-09-24 Thread Sagi Grimberg
No ULP uses it anymore, go ahead and remove it. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/cxgb4/cq.c | 2 +- drivers/infiniband/hw/cxgb4/iw_cxgb4.h | 18 drivers/infiniband/hw/cxgb4/mem.c | 45 drivers

[PATCH v2 26/26] IB/core: Remove old fast registration API

2015-09-24 Thread Sagi Grimberg
No callers and no providers left, go ahead and remove it. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/core/verbs.c | 25 --- include/rdma/ib_verbs.h | 54 - 2 files changed, 79 deletions

[PATCH v2 17/26] IB/srp: Dont allocate a page vector when using fast_reg

2015-09-24 Thread Sagi Grimberg
The new fast registration API does not reuqire a page vector so we can't avoid allocating it. Signed-off-by: Sagi Grimberg --- drivers/infiniband/ulp/srp/ib_srp.c | 20 +++- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/drivers/infiniband/ulp/srp/ib_srp

[PATCH v2 12/26] xprtrdma: Port to new memory registration API

2015-09-24 Thread Sagi Grimberg
&m=143677002622296&w=2). Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- net/sunrpc/xprtrdma/frwr_ops.c | 113 +++- net/sunrpc/xprtrdma/xprt_rdma.h | 3 +- 2 files changed, 68 insertions(+), 48 deletions(-) diff --git a/net/sunrpc/

[PATCH v2 13/26] svcrdma: Port to new memory registration API

2015-09-24 Thread Sagi Grimberg
Instead of maintaining a fastreg page list, keep an sg table and convert an array of pages to a sg list. Then call ib_map_mr_sg and construct ib_reg_wr. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- include/linux/sunrpc/svc_rdma.h | 6 +-- net/sunrpc/xprtrdma

[PATCH v2 25/26] IB/hfi1: Remove Old fast registraion API support

2015-09-24 Thread Sagi Grimberg
It wasn't supported before as well (post_send returned -EINVAL for IB_WR_FAST_REG_MR). Perhaps the new API adoption will be joint for all IB SW implementations. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/staging/hfi1/keys.c

[PATCH v2 24/26] RDMA/nes: Remove old FRWR API

2015-09-24 Thread Sagi Grimberg
No ULP uses it anymore, go ahead and remove it. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/nes/nes_hw.h| 6 -- drivers/infiniband/hw/nes/nes_verbs.c | 162 +- 2 files changed, 1 insertion(+), 167 deletions(-) diff

[PATCH v2 15/26] IB/srp: Split srp_map_sg

2015-09-24 Thread Sagi Grimberg
This is a perparation patch for the new registration API conversion. It splits srp_map_sg per registration strategy (srp_map_sg[fmr|fr|dma]. On its own it adds some code duplication, but it makes the API switch easier to comprehend. Signed-off-by: Sagi Grimberg --- drivers/infiniband/ulp/srp

[PATCH v2 23/26] IB/qib: Remove old FRWR API

2015-09-24 Thread Sagi Grimberg
No ULP uses it anymore, go ahead and remove it. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/qib/qib_keys.c | 56 --- drivers/infiniband/hw/qib/qib_mr.c| 32 +--- drivers/infiniband/hw/qib/qib_verbs.c

[PATCH v2 09/26] RDMA/nes: Support the new memory registration API

2015-09-24 Thread Sagi Grimberg
) - page array (nes_mr) - key, access flags (ib_reg_wr) The IB_WR_FAST_REG_MR handlers will be removed later when all the ULPs will be converted. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/nes/nes_verbs.c | 117 +- drivers

[PATCH v2 19/26] IB/mlx4: Remove old FRWR API support

2015-09-24 Thread Sagi Grimberg
No ULP uses it anymore, go ahead and remove it. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/mlx4/cq.c | 3 +-- drivers/infiniband/hw/mlx4/main.c| 2 -- drivers/infiniband/hw/mlx4/mlx4_ib.h | 15 --- drivers/infiniband/hw/mlx4/mr.c

[PATCH v2 25/26] IB/hfi1: Remove Old fast registraion API support

2015-09-24 Thread Sagi Grimberg
It wasn't supported before as well (post_send returned -EINVAL for IB_WR_FAST_REG_MR). Perhaps the new API adoption will be joint for all IB SW implementations. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/staging/hfi1/keys.c

[PATCH v2 14/26] RDS/IW: Convert to new memory registration API

2015-09-24 Thread Sagi Grimberg
Get rid of fast_reg page list and its construction. Instead, just pass the RDS sg list to ib_map_mr_sg and post the new ib_reg_wr. This is done both for server IW RDMA_READ registration and the client remote key registration. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- net

[PATCH v2 01/26] IB/core: Introduce new fast registration API

2015-09-24 Thread Sagi Grimberg
. This API will allow ULPs to remove the duplicated code of constructing a page vector from a given sg list. The send work request ib_reg_wr also shrinks as it will contain only mr, key and access flags in addition. Signed-off-by: Sagi Grimberg Tested-by: Christoph Hellwig Tested-by: Signed-off

[PATCH v2 19/26] IB/mlx4: Remove old FRWR API support

2015-09-24 Thread Sagi Grimberg
No ULP uses it anymore, go ahead and remove it. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/mlx4/cq.c | 3 +-- drivers/infiniband/hw/mlx4/main.c| 2 -- drivers/infiniband/hw/mlx4/mlx4_ib.h | 15 --- drivers/infiniband/hw/mlx4/mr.c

[PATCH v2 16/26] IB/srp: Convert to new registration API

2015-09-24 Thread Sagi Grimberg
Instead of constructing a page list, call ib_map_mr_sg and post a new ib_reg_wr. srp_map_finish_fr now returns the number of sg elements registered. Remove srp_finish_mapping since no one is calling it. Signed-off-by: Sagi Grimberg --- drivers/infiniband/ulp/srp/ib_srp.c | 135

[PATCH v2 11/26] iser-target: Port to new memory registration API

2015-09-24 Thread Sagi Grimberg
Remove fastreg page list allocation as the page vector is now private to the provider. Instead of constructing the page list and fast_req work request, call ib_map_mr_sg and construct ib_reg_wr. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/ulp/isert

[PATCH v2 05/26] RDMA/ocrdma: Support the new memory registration API

2015-09-24 Thread Sagi Grimberg
(ib_mr) - page array (ocrdma_mr) - key (ib_reg_wr) The IB_WR_FAST_REG_MR handlers will be removed later when all the ULPs will be converted. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/ocrdma/ocrdma.h | 2 + drivers/infiniband/hw/ocrdma

[PATCH v2 00/26] New fast registration API

2015-09-24 Thread Sagi Grimberg
o other HW devices (qib, nes) nor iwarp devices so RDS is compile tested only. Santosh reports I'm targeting this to 4.4 so I'll appreciate more feedback and a bigger testing coverage. The code is available at: https://github.com/sagigrimberg/linux/tree/reg_api.4 Sagi Grimberg (26):

[PATCH v2 03/26] IB/mlx5: Support the new memory registration API

2015-09-24 Thread Sagi Grimberg
, length, access flags (ib_mr) - page array (mlx5_ib_mr) - key (ib_reg_wr) The IB_WR_FAST_REG_MR handlers will be removed later when all the ULPs will be converted. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/mlx5/cq.c | 3 ++ drivers/infiniband/hw/mlx5

[PATCH v2 14/26] RDS/IW: Convert to new memory registration API

2015-09-24 Thread Sagi Grimberg
Get rid of fast_reg page list and its construction. Instead, just pass the RDS sg list to ib_map_mr_sg and post the new ib_reg_wr. This is done both for server IW RDMA_READ registration and the client remote key registration. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- net

[PATCH v2 21/26] RDMA/cxgb3: Remove old FRWR API

2015-09-24 Thread Sagi Grimberg
No ULP uses it anymore, go ahead and remove it. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/cxgb3/iwch_cq.c | 2 +- drivers/infiniband/hw/cxgb3/iwch_provider.c | 24 --- drivers/infiniband/hw/cxgb3/iwch_qp.c | 47

[PATCH v2 08/26] IB/qib: Support the new memory registration API

2015-09-24 Thread Sagi Grimberg
(qib_mr) - key, access flags (ib_reg_wr) The IB_WR_FAST_REG_MR handlers will be removed later when all the ULPs will be converted. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/qib/qib_keys.c | 56 +++ drivers/infiniband/hw

[PATCH v2 26/26] IB/core: Remove old fast registration API

2015-09-24 Thread Sagi Grimberg
No callers and no providers left, go ahead and remove it. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/core/verbs.c | 25 --- include/rdma/ib_verbs.h | 54 - 2 files changed, 79 deletions

[PATCH v2 02/26] IB/mlx5: Remove dead fmr code

2015-09-24 Thread Sagi Grimberg
Just function declarations - no need for those laying arround. If for some reason someone will want FMR support in mlx5, it should be easy enough to restore a few structs. Signed-off-by: Sagi Grimberg Reviewed-by: Bart Van Assche Acked-by: Christoph Hellwig --- drivers/infiniband/hw/mlx5

[PATCH v2 03/26] IB/mlx5: Support the new memory registration API

2015-09-24 Thread Sagi Grimberg
, length, access flags (ib_mr) - page array (mlx5_ib_mr) - key (ib_reg_wr) The IB_WR_FAST_REG_MR handlers will be removed later when all the ULPs will be converted. Signed-off-by: Sagi Grimberg Acked-by: Christoph Hellwig --- drivers/infiniband/hw/mlx5/cq.c | 3 ++ drivers/infiniband/hw/mlx5

<    1   2   3   4   5   6   7   8   9   10   >