Re: [PATCH] RXE: A Soft RoCE back-end for the RVT

2016-01-07 Thread Sagi Grimberg
This patch introduces an implementation of a back-end that works with RVT to make RoCE Verbs transport over any Ethernet network device. Example: After loading ib_rxe_net.ko echo eth1 > /sys/module/ib_rxe_net/parameters/add will create rvt0 IB device in RVT with Ethernet link layer ---

Re: [PATCH v2 00/10] iSER support for remote invalidate

2016-01-07 Thread Sagi Grimberg
FYI, the only conflict as reported in linux-next last week is with nfsd tree and "IB: merge struct ib_device_attr into struct ib_device" here: http://marc.info/?l=linux-next&m=145155049101826&w=2 It looks like there is ongoing discussion this morning wrt rdma + nfsd trees.. Do you still want

Re: [PATCH 10/11] IB: only keep a single key in struct ib_mr

2016-01-06 Thread Sagi Grimberg
ULPs are *already* using the same registrations for both local and remote access. Where? Out of tree? I haven't found anything in-tree for sure. We have that in iSER. iSCSI allows a FirstBurst functionality and iSER as an iSCSI transport is required to support that. The FirstBurst is div

Re: [PATCH 09/15] IB/srpt: Fix srpt_close_session()

2016-01-06 Thread Sagi Grimberg
On 06/01/2016 16:34, Bart Van Assche wrote: On 01/06/2016 03:21 PM, Sagi Grimberg wrote: Avoid that srpt_close_session() waits if it doesn't have to wait. Can you explain when it doesn't have to wait? is it possible that srpt_release_channel_work() was already triggered? is

Re: [PATCH 15/15] IB/srpt: Fix a rare crash in srpt_close_session()

2016-01-06 Thread Sagi Grimberg
Keep the ib_srpt session as long as srpt_close_session() may access it. Makes sense, Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.

Re: [PATCH 14/15] IB/srpt: Fix srpt_write_pending()

2016-01-06 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 13/15] IB/srpt: Detect session shutdown reliably

2016-01-06 Thread Sagi Grimberg
On 05/01/2016 16:26, Bart Van Assche wrote: The Last WQE Reached event is only generated after one or more work requests have been queued on the QP associated with a session. Since session shutdown can start before any work requests have been queued, use a zero-length RDMA write to wait until a

Re: [PATCH 12/15] IB/srpt: Eliminate srpt_find_channel()

2016-01-06 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 10/15] IB/srpt: Fix srpt_handle_cmd() error paths

2016-01-06 Thread Sagi Grimberg
On 05/01/2016 16:25, Bart Van Assche wrote: The target core function that should be called if target_submit_cmd() fails is target_put_sess_cmd(). Additionally, change the return type of srpt_handle_cmd() from int into void. Signed-off-by: Bart Van Assche Cc: Christoph Hellwig --- drivers/i

Re: [PATCH 09/15] IB/srpt: Fix srpt_close_session()

2016-01-06 Thread Sagi Grimberg
Avoid that srpt_close_session() waits if it doesn't have to wait. Can you explain when it doesn't have to wait? is it possible that srpt_release_channel_work() was already triggered? isn't that a problem? -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a

Re: [PATCH 07/15] IB/srpt: Simplify channel state management

2016-01-06 Thread Sagi Grimberg
suggestion though, otherwise Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 08/15] IB/srpt: Simplify srpt_shutdown_session()

2016-01-06 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 06/15] IB/srpt: Simplify srpt_handle_tsk_mgmt()

2016-01-06 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 05/15] IB/srpt: Use scsilun_to_int()

2016-01-06 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 04/15] IB/srpt: Introduce target_reverse_dma_direction()

2016-01-06 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 02/15] IB/srpt: Inline srpt_sdev_name()

2016-01-06 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 01/15] IB/srpt: Add parentheses around sizeof argument

2016-01-06 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 03/15] IB/srpt: Inline srpt_get_ch_state()

2016-01-06 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/2] IB/mad: pass ib_mad_send_buf explicitly to the recv_handler

2016-01-06 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/2 v2] IB/mad: use CQ abstraction

2016-01-06 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/3] IB/srpt: Fix a race condition related to SRP login

2016-01-03 Thread Sagi Grimberg
But now the first I/O(s) could be lost if no other I/O comes in, right? I suspect that we need to keep this loop to protect against such corner cases. It can happen theoretically, but why do we even bother? Why not just post the recv buffer after we moved in to CH_LIVE? This why we let the RC

Re: [PATCH 3/3] IB/srpt: Fix a race condition related to SRP login

2016-01-03 Thread Sagi Grimberg
But now the first I/O(s) could be lost if no other I/O comes in, right? I suspect that we need to keep this loop to protect against such corner cases. It can happen theoretically, but why do we even bother? Why not just post the recv buffer after we moved in to CH_LIVE? This why we let the R

Re: [PATCH 2/3] IB/srpt: Fix the RDMA completion handlers

2016-01-03 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/3] irq_poll: Fix irq_poll_sched()

2016-01-03 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH V2] IB/mlx5: Unify CQ create flags check

2015-12-29 Thread Sagi Grimberg
From: Leon Romanovsky The create_cq() can receive creation flags which were used differently by two following commits [1] and [2]. The current code caused to not accept any flags at all. We can skip referencing the linux-rdma mailing list. This patch unifies the check into one function and

Re: [PATCH 4/6] IB/uapi: expose uverbs WC flags

2015-12-29 Thread Sagi Grimberg
+enum ib_uverbs_wc_flags { + IB_UVERBS_WC_GRH= (1 << 0), + IB_UVERBS_WC_WITH_IMM = (1 << 1), + IB_UVERBS_WC_WITH_INVALIDATE= (1 << 2), + IB_UVERBS_WC_IP_CSUM_OK = (1 << 3), + IB_UVERBS_WC_WITH_SMA

Re: [PATCH 3/6] IB/uapi: expose uverbs WC opcodes

2015-12-29 Thread Sagi Grimberg
+ IB_WC_SEND = IB_UVERBS_WC_SEND, + IB_WC_RDMA_WRITE= IB_UVERBS_WC_RDMA_WRITE, + IB_WC_RDMA_READ = IB_UVERBS_WC_RDMA_READ, + IB_WC_COMP_SWAP = IB_UVERBS_WC_COMP_SWAP, + IB_WC_FETCH_ADD = IB_UVERBS_WC_FETCH_ADD, + IB

Re: [PATCH] IB/mlx5: Unify CQ create flags check

2015-12-29 Thread Sagi Grimberg
Does this deserve a Fixes tag? -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] IB/core: Remove a set-but-not-used variable from ib_sg_to_pages()

2015-12-29 Thread Sagi Grimberg
Thanks Bart, Acked-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] IB/mad: Ensure fairness in ib_mad_completion_handler

2015-12-29 Thread Sagi Grimberg
Please just convert the mad handler to the new CQ API in drivers/infiniband/core/cq.c. If you have any question about it I'd be glad to help you. +1 on this suggestion. We had these sorts of questions in our ULPs as well. The CQ API should take care of all that for you and leaves you to just

Re: [PATCH 0/2] InfiniBand-iSER: Refactoring for two function implementations

2015-12-27 Thread Sagi Grimberg
Along with Leon's prefix comment: Acked-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 for-next 5/7] IB/mlx4: Add IB counters table

2015-12-24 Thread Sagi Grimberg
This patch seems to generate a list corruption [1] when I test with Doug's for-4.5 tree. Eran, care to take a look at this? This patch is part from a series that was introduced in 4.3-rc1 [1], Then something else broke it. Can people check their patches on doug's tree? At the moment it's unu

Re: [PATCH v2 for-next 5/7] IB/mlx4: Add IB counters table

2015-12-24 Thread Sagi Grimberg
Doug, I'm also can't load mlx5 drivers in your tree [1] but I don't know where it's from, it can come from pretty much everything... Now I'm left with no useable HW to test with :( [1]: mlx5_core :06:00.0: firmware version: 12.14.74 mlx5_core :06:00.1: firmware version: 12.14.74 mlx5_i

Re: [PATCH v2 00/10] iSER support for remote invalidate

2015-12-24 Thread Sagi Grimberg
Applied to target-pending/for-next as v4.5-rc1 material, along with Reviewed-by tags from HCH. So this is both in your and Dougs now it seems. Given the non-trivial merge with the other RDMA updates I'd suggest to drop it from the target tree as Doug already sorted out the merge. Yea, this

Re: [PATCH v2 for-next 5/7] IB/mlx4: Add IB counters table

2015-12-24 Thread Sagi Grimberg
This patch seems to generate a list corruption [1] when I test with Doug's for-4.5 tree. Eran, care to take a look at this? [1]: mlx4_core: Mellanox ConnectX core driver v2.2-1 (Feb, 2014) mlx4_core: Initializing :04:00.0 mlx4_core :04:00.0: PCIe link speed is 8.0GT/s, device support

Re: [PATCH for-next V3 01/11] IB/core: Add gid_type to gid attribute

2015-12-23 Thread Sagi Grimberg
+static const char * const gid_type_str[] = { ^^ ^^ IMHO, The white spaces can be a little bit confusing to understand. Pay attention to the double const - I think it's more clear that way. I agree. + [IB_GID_TYPE_IB]= "IB/RoCE v1", +}; + +const char *ib

Re: [PATCH] IB/cma: cma_match_net_dev needs to take into account port_num

2015-12-23 Thread Sagi Grimberg
FWIW, looks good to me, Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 10/11] IB: only keep a single key in struct ib_mr

2015-12-22 Thread Sagi Grimberg
The ULP decides if this MR is going to be used as a lkey or rkey by passing IB_REG_LKEY or IB_REG_RKEY. The HCA driver will then fill mr->key by the lkey or rkey based on that and everything will work fine. But the ULP *can* register a memory buffer with local and remote access permissions. O

Re: [PATCH 10/11] IB: only keep a single key in struct ib_mr

2015-12-22 Thread Sagi Grimberg
On 22/12/2015 15:13, Christoph Hellwig wrote: On Tue, Dec 22, 2015 at 11:17:54AM +0200, Sagi Grimberg wrote: What makes me worried here is that the IB/RoCE specification really defines different keys for local and remote access. I'm less concerned about our consumers but more abou

Re: [PATCH 10/11] IB: only keep a single key in struct ib_mr

2015-12-22 Thread Sagi Grimberg
Hi Christoph, While IB supports the notion of returning separate local and remote keys from a memory registration, the iWarp spec doesn't and neither does any of our in-tree HCA drivers [1] nor consumers. Consolidate the in-kernel API to provide only a single key and make everyones life easier.

Re: [PATCH v2 00/10] iSER support for remote invalidate

2015-12-21 Thread Sagi Grimberg
Sagi, are you going to spin an increment in the initiator version? I don't know if it's worth a driver version update? In case we do increment, I can send an incremental patch instead of re-spinning the entire series. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in

Re: [PATCH v2 00/10] iSER support for remote invalidate

2015-12-21 Thread Sagi Grimberg
Hey Nic, Applied to target-pending/for-next as v4.5-rc1 material, along with Reviewed-by tags from HCH. Thanks for picking this up! Note that I expect this patchset will conflict with Doug's pull request in case he's taking Christoph's device_attr and CQ abstraction patches for 4.5. Doug, wo

Re: [PATCH 10/10] IB: remove the unused usecnt field from struct ib_mr

2015-12-20 Thread Sagi Grimberg
the use_cnt field. I completely agree. we don't have any way to maintain mw<->mr reference so let's drop it. Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 10/10] IB: remove the unused usecnt field from struct ib_mr

2015-12-20 Thread Sagi Grimberg
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 284916d..e45776e 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -1306,7 +1306,6 @@ struct ib_mr { u64 iova; u32 length; unsigned int page_size; -atomic_t

Re: [PATCH] libmlx5: Add gitignore file to the project

2015-12-20 Thread Sagi Grimberg
Add gitignore file to the libmlx5 project. I think cscope files will be useful here too, Otherwise looks good, Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordo

Re: RoCE passive side failures on 4.4-rc5

2015-12-17 Thread Sagi Grimberg
Guys, Using 4.4-rc5+ [1] and **not** applying any of the patches I sent today, I noted that RoCE passive side isn't working (rdma-cm, ibv_rc_pingpong works). I'm using 4.4-rc2+ and I have RoCE working. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a m

Re: [PATCH rdma-next 1/6] IB/core: Save the device attributes on the device structure

2015-12-17 Thread Sagi Grimberg
+ ret = ib_query_device(device, &device->attrs); + if (ret) { + printk(KERN_WARNING "Couldn't query the device attributes\n"); + goto out; + } + I thought we're all for removing the call altogether aren't we? I'd say just call device->query_device

Re: [PATCH 2/2] IB/mlx4: Convert kmalloc to be kmalloc_array to fix checkpatch warnings

2015-12-17 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/2] IB/mlx4: Suppress memory allocations warnings in kmalloc->__vmalloc flows

2015-12-17 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] svc_rdma: use local_dma_lkey

2015-12-16 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH for-next V2 5/5] IB/mlx5: Mmap the HCA's core clock register to user-space

2015-12-16 Thread Sagi Grimberg
enum mlx5_ib_mmap_cmd { MLX5_IB_MMAP_REGULAR_PAGE = 0, - MLX5_IB_MMAP_GET_CONTIGUOUS_PAGES = 1, /* always last */ + MLX5_IB_MMAP_GET_CONTIGUOUS_PAGES = 1, + /* 5 is chosen in order to be compatible with old versions of libmlx5 */ + MLX

Re: [PATCH for-next V2 3/5] IB/mlx5: Add support for hca_core_clock and timestamp_mask

2015-12-16 Thread Sagi Grimberg
Reporting the hca_core_clock (in kHZ) and the timestamp_mask in query_device extended verb. timestamp_mask is used by users in order to know what is the valid range of the raw timestamps, while hca_core_clock reports the clock frequency that is used for timestamps. Hi Matan, Shouldn't this pa

Re: [PATCH 37/37] IB/rdmavt: Add support for new memory registration API

2015-12-16 Thread Sagi Grimberg
This patch exists to provide parity for what is in qib. Should we not have it? If not, why do we have: commit 38071a461f0a ("IB/qib: Support the new memory registration API") That was done by me because I saw this in qib and assumed that it was supported. Now that I found out that it isn't,

Re: [PATCH v3 04/11] xprtrdma: Move struct ib_send_wr off the stack

2015-12-16 Thread Sagi Grimberg
diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h index 4197191..e60d817 100644 --- a/net/sunrpc/xprtrdma/xprt_rdma.h +++ b/net/sunrpc/xprtrdma/xprt_rdma.h @@ -206,6 +206,8 @@ struct rpcrdma_frmr { enum rpcrdma_frmr_state fr_state; struct work

Re: [PATCH v3 06/11] xprtrdma: Add ro_unmap_sync method for FRWR

2015-12-16 Thread Sagi Grimberg
+static void +__frwr_dma_unmap(struct rpcrdma_xprt *r_xprt, struct rpcrdma_mr_seg *seg, +int rc) +{ + struct ib_device *device = r_xprt->rx_ia.ri_device; + struct rpcrdma_mw *mw = seg->rl_mw; + int nsegs = seg->mr_nsegs; + + seg->rl_mw = NULL; + + wh

Re: device attr cleanup

2015-12-16 Thread Sagi Grimberg
Hi Doug, Lets stop beating, both horses and people. I do understand that 1. you don't link the removal of the attr 2. you do like the removal of all the query calls I am proposing to take the path of a patch that does exactly #2 while avoiding #1. I really don't have a strong preference o

Re: [PATCH 37/37] IB/rdmavt: Add support for new memory registration API

2015-12-16 Thread Sagi Grimberg
This question is not directly related to this patch, but given that this is a copy-paste from the qib driver I'll go ahead and take it anyway. How does qib (and rvt now) do memory key invalidation? I didn't see any reference to IB_WR_LOCAL_INV anywhere in the qib driver... What am I missing?

Re: [PATCH 37/37] IB/rdmavt: Add support for new memory registration API

2015-12-14 Thread Sagi Grimberg
Hi Dennis and Ira, This question is not directly related to this patch, but given that this is a copy-paste from the qib driver I'll go ahead and take it anyway. How does qib (and rvt now) do memory key invalidation? I didn't see any reference to IB_WR_LOCAL_INV anywhere in the qib driver...

Re: [PATCH 10/13] IB/srp: use the new CQ API

2015-12-13 Thread Sagi Grimberg
Allright. How do you want to proceed? The current rdma-cq branch has all kinds of dependencies, but I've also prepared a new rdma-cq.2 branch that could go straight on top of your current queue: http://git.infradead.org/users/hch/rdma.git/shortlog/refs/heads/rdma-cq.2 If you're ready to star

Re: completion queue abstraction V2

2015-12-13 Thread Sagi Grimberg
Needless to say, on the entire series you can add my: Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 37/37] IB/rdmavt: Add support for new memory registration API

2015-12-10 Thread Sagi Grimberg
commit 38071a461f0a ("IB/qib: Support the new memory registration API")" added support for map_mr_sg to qib. This patch brings that support into rdmavt, it also adds a register mr function that will be called from the post send routine which is inline with the qib patch. Hi Dennis and Ira, Th

Re: [PATCH 26/37] IB/rdmavt: Move memory registration into rdmavt

2015-12-10 Thread Sagi Grimberg
/** @@ -83,7 +357,31 @@ struct ib_mr *rvt_reg_phys_mr(struct ib_pd *pd, int num_phys_buf, int acc, u64 *iova_start) { Why do we even bother with reg_phys_mr? Let's drop it entirely. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in t

Re: [PATCH v2 02/10] IB/iser: Reuse ib_sg_to_pages

2015-12-10 Thread Sagi Grimberg
On 09/12/2015 21:28, Christoph Hellwig wrote: On Wed, Dec 09, 2015 at 02:12:00PM +0200, Sagi Grimberg wrote: We have in iser iser_sg_to_page_vec which has exactly the same role as ib_sg_to_pages. Customize the page_vec to hold a fake MR so we can reuse ib_sg_to_pages. Looks good. In the

Re: [PATCH v2 07/10] iser-target: Declare correct flags when accepting a connection

2015-12-10 Thread Sagi Grimberg
Only mrginally related, but can someone explain what zero based virtual addresses means in this context? Does this means it uses the old RFC5046-style header without the read/write_va fields? Or does it mean those fields exist but must always be zero? That's correct, negotiating this bit mea

Re: [PATCH v2 03/10] IB/iser: Don't register memory for all immediate data writes

2015-12-10 Thread Sagi Grimberg
On 09/12/2015 21:30, Christoph Hellwig wrote: The iser_reg_rdma_mem calling conventions seem rather confusing, and this patch doesn't help that. But I think that's something to be addressed in bigger cleanup later on. I do plan to rework this area. It would be nice to have it simpler, But I

Re: device attr cleanup (was: Handle mlx4 max_sge_rd correctly)

2015-12-10 Thread Sagi Grimberg
Doug this is going to conflict with the rdmavt work. So if you take this could you respond on the list. It will also conflict with the iser remote invalidate series. Doug it would help if you share your plans so people can rebase accordingly. -- To unsubscribe from this list: send the line

[PATCH v2 09/10] IB/iser: Increment the rkey when registering and not when invalidating

2015-12-09 Thread Sagi Grimberg
With remote invalidate we won't local invalidate but we still want to increment the rkey. Signed-off-by: Sagi Grimberg Signed-off-by: Jenny Derzhavetz --- drivers/infiniband/ulp/iser/iser_memory.c | 20 ++-- 1 file changed, 10 insertions(+), 10 deletions(-) diff --

[PATCH v2 05/10] iser: Have initiator and target to share protocol structures and definitions

2015-12-09 Thread Sagi Grimberg
The iser RDMA_CM negotiation protocol is shared by the initiator and the target, so have a shared header for the defines and structure. Move relevant items from the initiator and target headers. Signed-off-by: Sagi Grimberg Signed-off-by: Jenny Derzhavetz --- drivers/infiniband/ulp/iser

[PATCH v2 06/10] iser-target: Remove unused file iser_proto.h

2015-12-09 Thread Sagi Grimberg
We don't need iser_proto.h anymore, remove it and move (non-protocol) declarations to ib_isert.h Signed-off-by: Sagi Grimberg Signed-off-by: Jenny Derzhavetz Reviewed-by: Christoph Hellwig --- drivers/infiniband/ulp/isert/ib_isert.c| 1 - drivers/infiniband/ulp/isert/ib_isert.h

[PATCH v2 07/10] iser-target: Declare correct flags when accepting a connection

2015-12-09 Thread Sagi Grimberg
From: Jenny Derzhavetz iser target does not support zero based virtual addresses and send with invalidate, so it should declare that it doesn't. Signed-off-by: Jenny Derzhavetz Signed-off-by: Sagi Grimberg --- drivers/infiniband/ulp/isert/ib_isert.c | 6 ++ 1 file changed, 6 inser

[PATCH v2 08/10] iser-target: Support the remote invalidation exception

2015-12-09 Thread Sagi Grimberg
H_INV with the correct rkey for the response. Signed-off-by: Jenny Derzhavetz Signed-off-by: Sagi Grimberg --- drivers/infiniband/ulp/isert/ib_isert.c | 46 - drivers/infiniband/ulp/isert/ib_isert.h | 2 ++ 2 files changed, 41 insertions(+), 7 deletions(-) diff --

[PATCH v2 04/10] IB/iser: set intuitive values for mr_valid

2015-12-09 Thread Sagi Grimberg
Derzhavetz Signed-off-by: Sagi Grimberg --- drivers/infiniband/ulp/iser/iser_memory.c | 8 drivers/infiniband/ulp/iser/iser_verbs.c | 4 ++-- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/ulp/iser/iser_memory.c b/drivers/infiniband/ulp/iser/iser_memo

[PATCH v2 10/10] IB/iser: Support the remote invalidation exception

2015-12-09 Thread Sagi Grimberg
emote invalidate our rkey so we must check it upon a receive (scsi response) completion. Signed-off-by: Jenny Derzhavetz Signed-off-by: Sagi Grimberg --- drivers/infiniband/ulp/iser/iscsi_iser.h | 5 ++- drivers/infiniband/ulp/iser/iser_initiator.c | 59 +++- drivers/infin

[PATCH v2 01/10] IB/iser: Fix module init not cleaning up on error flow

2015-12-09 Thread Sagi Grimberg
From: Roi Dayan Destroy workqueue on transport register error, also release kmem cache on workqueue allocation error. Signed-off-by: Roi Dayan Signed-off-by: Sagi Grimberg Reviewed-by: Christoph Hellwig --- drivers/infiniband/ulp/iser/iscsi_iser.c | 9 ++--- 1 file changed, 6 insertions

[PATCH v2 00/10] iSER support for remote invalidate

2015-12-09 Thread Sagi Grimberg
t: Declare correct flags when accepting a connection iser-target: Support the remote invalidation exception IB/iser: Support the remote invalidation exception Roi Dayan (1): IB/iser: Fix module init not cleaning up on error flow Sagi Grimberg (4): IB/iser: Reuse ib_sg_to_pages iser: Have

[PATCH v2 02/10] IB/iser: Reuse ib_sg_to_pages

2015-12-09 Thread Sagi Grimberg
We have in iser iser_sg_to_page_vec which has exactly the same role as ib_sg_to_pages. Customize the page_vec to hold a fake MR so we can reuse ib_sg_to_pages. Signed-off-by: Sagi Grimberg --- drivers/infiniband/ulp/iser/iscsi_iser.h | 5 +- drivers/infiniband/ulp/iser/iser_memory.c | 105

[PATCH v2 03/10] IB/iser: Don't register memory for all immediate data writes

2015-12-09 Thread Sagi Grimberg
From: Jenny Derzhavetz When all the task data is sent as immediate data, we are allowed to use the local_dma_lkey as it is not sent to the wire. Signed-off-by: Jenny Derzhavetz Signed-off-by: Sagi Grimberg --- drivers/infiniband/ulp/iser/iscsi_iser.h | 3 ++- drivers/infiniband/ulp/iser

Re: [PATCH v2 0/2] Handle mlx4 max_sge_rd correctly

2015-12-09 Thread Sagi Grimberg
BTW, it's already pushed out, you can look at it now. Looks fine. Thanks -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 0/2] Handle mlx4 max_sge_rd correctly

2015-12-08 Thread Sagi Grimberg
I've pulled these in. Both of them failed to apply, and the first one was just flat baffling in terms of how Sagi is working from a source base so different than the one in the upstream kernel. They should be double checked just to make sure I'm not missing something since they were so far off

Re: [PATCH 5/6] IB core: Fix ib_sg_to_pages()

2015-12-06 Thread Sagi Grimberg
Looks fine, Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 0/2] Handle mlx4 max_sge_rd correctly

2015-12-03 Thread Sagi Grimberg
Did we ever make progress on this? Just up to Doug to pull it in. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 5/6] IB core: Fix ib_sg_to_pages()

2015-12-03 Thread Sagi Grimberg
Hello Sagi, Hmm ... why would it be unacceptable to return 0 if sg_nents == 0 ? Regarding which component to modify if mapping the first page fails: for almost every kernel function I know a negative return value means failure and a return value >= 0 means success. Hence my proposal to change

Re: [PATCH 6/6] IB/srp: Fix srp_map_sg_fr()

2015-12-03 Thread Sagi Grimberg
Replying to my own email, dma_map_sg returns the actual number of entries to iterate. At least historically some IOMMU implementations would do strange tricks like: If entries 2 and 3 could be merged dma_len for 2 would span 2 and 3, and then entry 3 would actually have the dma addr and

Re: [PATCH 6/6] IB/srp: Fix srp_map_sg_fr()

2015-12-02 Thread Sagi Grimberg
x27;m not sure anyone still does that, but the first spot to check would be the Parisc IOMMU drivers. Oh I see, I didn't know that. Thanks for clarifying this for me. AFAICT this patch is fine. Reviewed-by: Sagi Grimberg iser, nfs needs fixing too. -- To unsubscribe from this list: sen

Re: [PATCH 6/6] IB/srp: Fix srp_map_sg_fr()

2015-12-02 Thread Sagi Grimberg
On 01/12/2015 20:39, Bart Van Assche wrote: On 12/01/2015 10:35 AM, Sagi Grimberg wrote: After dma_map_sg() has been called the return value of that function must be used as the number of elements in the scatterlist instead of scsi_sg_count(). Umm, but ib_map_mr_sg iterates on the sg list

Re: [PATCH 5/6] IB core: Fix ib_sg_to_pages()

2015-12-02 Thread Sagi Grimberg
On 01/12/2015 21:10, Bart Van Assche wrote: On 12/01/2015 10:32 AM, Sagi Grimberg wrote: Fix the code for detecting gaps and disable the code for chunking. A gap occurs not only if the second or later scatterlist element is not aligned but also if any scatterlist element other than the last

Re: [PATCH 4/6] IB/srp: Fix indirect data buffer rkey endianness

2015-12-02 Thread Sagi Grimberg
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c index 72fac20..fac1423 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.c +++ b/drivers/infiniband/ulp/srp/ib_srp.c @@ -1662,7 +1662,7 @@ static int srp_map_data(struct scsi_cmnd *scmnd, struct srp_rdma_ch *ch,

Re: [PATCH 4/6] IB/srp: Fix indirect data buffer rkey endianness

2015-12-01 Thread Sagi Grimberg
On 01/12/2015 20:18, Bart Van Assche wrote: Detected by sparse. Fixes: commit 330179f2fa93 ("IB/srp: Register the indirect data buffer descriptor") Signed-off-by: Bart Van Assche Cc: stable # v4.3+ Cc: Sagi Grimberg Cc: Christoph Hellwig Cc: Sebastian Parschauer --- drivers/

Re: [PATCH 6/6] IB/srp: Fix srp_map_sg_fr()

2015-12-01 Thread Sagi Grimberg
After dma_map_sg() has been called the return value of that function must be used as the number of elements in the scatterlist instead of scsi_sg_count(). Umm, but ib_map_mr_sg iterates on the sg list. Say you have sg_nents=3 and after mapping you got dma_nents=2 (2 entries are contig) and yo

Re: [PATCH 3/6] IB/srp: Initialize dma_length in srp_map_idb

2015-12-01 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 5/6] IB core: Fix ib_sg_to_pages()

2015-12-01 Thread Sagi Grimberg
Hi Bart, Fix the code for detecting gaps and disable the code for chunking. A gap occurs not only if the second or later scatterlist element is not aligned but also if any scatterlist element other than the last does not end at a page boundary. Disable the code for chunking. So can you explain

Re: [PATCH v1 10/10] IB/iser: Support the remote invalidation exception

2015-12-01 Thread Sagi Grimberg
On 30/11/2015 23:36, Or Gerlitz wrote: On Tue, Nov 24, 2015 at 6:23 PM, Sagi Grimberg wrote: From: Jenny Derzhavetz Declare that we support remote invalidation in case we are: 1. using Fastreg method 2. always registering memory. decide if you want or don't want to use periods @ th

Re: [PATCH] IB/iser: constify iser_reg_ops structure

2015-11-29 Thread Sagi Grimberg
On 28/11/2015 17:52, Julia Lawall wrote: The iser_reg_ops structures are never modified, so declare them as const. Done with the help of Coccinelle. Signed-off-by: Julia Lawall Thanks, Acked-by: Sagi Grimberg -- To unsubscribe from this list: send the line "unsubscribe linux-rdm

Re: Future of FMR support, was: Re: [PATCH v1 5/9] xprtrdma: Add ro_unmap_sync method for FMR

2015-11-26 Thread Sagi Grimberg
On 25/11/2015 21:28, Jason Gunthorpe wrote: On Wed, Nov 25, 2015 at 09:09:20AM -0800, santosh shilimkar wrote: I'd say drop the current iWarp transport if it's not testable. The only real difference between IB and iWarp is the needed to create a MR for the RDMA READ sink, and we're much bette

Re: [PATCH v1 08/10] iser-target: Support the remote invalidation exception

2015-11-26 Thread Sagi Grimberg
yep, should be probably OK. You didn't respond to my comment re adding bools vs bit-fields vs bit-flags This is outside the scope of this patchset. I'm willing to look into this incrementally to this. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a m

Re: [PATCH v1 08/10] iser-target: Support the remote invalidation exception

2015-11-25 Thread Sagi Grimberg
On 25/11/2015 10:41, Or Gerlitz wrote: On Wed, Nov 25, 2015 at 9:55 AM, Sagi Grimberg wrote: For NO_DATA commands the iser specification explicitly states that read_stag and write_stag should be 0 which means that inv_rkey is 0 too. We don't do remote invalidate in case inv_rkey is 0

Re: [PATCH v1 08/10] iser-target: Support the remote invalidation exception

2015-11-24 Thread Sagi Grimberg
On 24/11/2015 19:42, Or Gerlitz wrote: On Tue, Nov 24, 2015 at 6:23 PM, Sagi Grimberg wrote: @@ -1100,7 +1122,14 @@ isert_init_send_wr(struct isert_conn *isert_conn, struct isert_cmd *isert_cmd, isert_cmd->rdma_wr.iser_ib_op = ISER_IB_SEND; send_wr->wr_id = (uin

[PATCH v1 07/10] iser-target: Declare correct flags when accepting a connection

2015-11-24 Thread Sagi Grimberg
From: Jenny Derzhavetz iser target does not support zero based virtual addresses and send with invalidate, so it should declare that it doesn't. Signed-off-by: Jenny Derzhavetz Signed-off-by: Sagi Grimberg --- drivers/infiniband/ulp/isert/ib_isert.c | 6 ++ 1 file changed, 6 inser

[PATCH v1 10/10] IB/iser: Support the remote invalidation exception

2015-11-24 Thread Sagi Grimberg
emote invalidate our rkey so we must check it upon a receive (scsi response) completion. Signed-off-by: Jenny Derzhavetz Signed-off-by: Sagi Grimberg --- drivers/infiniband/ulp/iser/iscsi_iser.h | 5 ++- drivers/infiniband/ulp/iser/iser_initiator.c | 59 +++- drivers/infin

[PATCH v1 08/10] iser-target: Support the remote invalidation exception

2015-11-24 Thread Sagi Grimberg
H_INV with the correct rkey for the response. Signed-off-by: Jenny Derzhavetz Signed-off-by: Sagi Grimberg --- drivers/infiniband/ulp/isert/ib_isert.c | 46 - drivers/infiniband/ulp/isert/ib_isert.h | 2 ++ 2 files changed, 41 insertions(+), 7 deletions(-) diff --

  1   2   3   4   5   6   7   8   9   10   >