Re: [PATCH libibverbs V4 1/5] Add ibv_port_cap_flags
On 18/5/2014 12:38 PM, Or Gerlitz wrote:
From: Matan Barak mat...@mellanox.com
Add an enum that describes ibv_port_cap_flags that complies
with the respective kernel enum.
This value could be fetched when using ibv_query_port in
port_cap_flags.
Signed-off-by: Matan Barak mat...@mellanox.com
Signed-off-by: Or Gerlitz ogerl...@mellanox.com
---
include/infiniband/verbs.h | 26 ++
1 files changed, 26 insertions(+), 0 deletions(-)
diff --git a/include/infiniband/verbs.h b/include/infiniband/verbs.h
index 9826b72..71adf2a 100644
--- a/include/infiniband/verbs.h
+++ b/include/infiniband/verbs.h
@@ -191,6 +191,32 @@ enum {
IBV_LINK_LAYER_ETHERNET,
};
+enum ibv_port_cap_flags {
+ IBV_PORT_SM = 1 1,
+ IBV_PORT_NOTICE_SUP = 1 2,
+ IBV_PORT_TRAP_SUP = 1 3,
+ IBV_PORT_OPT_IPD_SUP= 1 4,
+ IBV_PORT_AUTO_MIGR_SUP = 1 5,
+ IBV_PORT_SL_MAP_SUP = 1 6,
+ IBV_PORT_MKEY_NVRAM = 1 7,
+ IBV_PORT_PKEY_NVRAM = 1 8,
+ IBV_PORT_LED_INFO_SUP = 1 9,
+ IBV_PORT_SYS_IMAGE_GUID_SUP = 1 11,
+ IBV_PORT_PKEY_SW_EXT_PORT_TRAP_SUP = 1 12,
+ IBV_PORT_EXTENDED_SPEEDS_SUP= 1 14,
+ IBV_PORT_CM_SUP = 1 16,
+ IBV_PORT_SNMP_TUNNEL_SUP= 1 17,
+ IBV_PORT_REINIT_SUP = 1 18,
+ IBV_PORT_DEVICE_MGMT_SUP= 1 19,
+ IB_PORT_VENDOR_CLASS_SUP= 1 20,
+ IB_PORT_DR_NOTICE_SUP = 1 21,
+ IB_PORT_CAP_MASK_NOTICE_SUP = 1 22,
+ IB_PORT_BOOT_MGMT_SUP = 1 23,
+ IB_PORT_LINK_LATENCY_SUP= 1 24,
+ IB_PORT_CLIENT_REG_SUP = 1 25,
+ IBV_PORT_IP_BASED_GIDS = 1 26
+};
+
The last few values miss the IBV_ prefix.
We'll issue a V5 to fix that soon.
Matan
struct ibv_port_attr {
enum ibv_port_state state;
enum ibv_mtumax_mtu;
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH libibverbs V1] Align Flow Steering API to the extension verbs scheme
On 20/05/2014 04:05, Roland Dreier wrote: On Thu, May 15, 2014 at 6:16 AM, Or Gerlitz ogerl...@mellanox.com wrote: Hi Roland -- this addresses all the comments by Jason who gave a ack/reviewed-by note for both the libibverbs and libmlx4 fixes. So you can move ahead and conduct point releases for both libs. OK... so I should do one more libmlx4 release before you take over? YES, please. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH libibverbs V4 0/5] Add support for UD QPs under RoCE IP addressing
On 18/05/2014 12:38, Or Gerlitz wrote: [...] changes from V3 -- addressed feedback from Jason: - replaced the char* that used to hold the MAC address provided to the provider library (e.g libmlx4) with sockaddr_storage and use ARPHRD for the af_family (similar to the practice used in the kernel) - Dropped mask1 from ibv_query_port_ex and put all fields in comp_mask - Added a man page for ibv_query_port_ex - Made the code c99 friendly by dropping the anonymous union and struct - Splitted the patches to patch that adds the interface and patch that uses it - Removed the usage of drv_ and lib_ schemas - Removed the VERBS_CONTEXT_XXX Hi Jason, so Matan has addressed you comments from V3, could you please take a look and ack? we find tiny little bug in patch #1 but want to get your feedback before re-spinning V5. Or. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/3] RDMA/core: iWARP Port Mapper Overview
On 27/03/2014 00:08, Tatyana Nikolova wrote: This patch series adds iWARP Port Mapper (IWPM) Version 2 support in RDMA/core, RDMA/nes driver and RDMA/cxgb4 driver. Guys, the link here http://vger.kernel.org/netconf2011_slides/pj_netconf2011.ppt seems to point to a damaged PPT, can you send me updated copy -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3.1] IB/umad: Fix error handling
From: Bart Van Assche bvanass...@acm.org
Avoid leaking a kref count in ib_umad_open() if port-ib_dev == NULL
or if nonseekable_open() fails. Avoid leaking a kref count, that
sm_sem is kept down and also that the IB_PORT_SM capability mask is
not cleared in ib_umad_sm_open() if nonseekable_open() fails.
Since container_of() never returns NULL, remove the code that tests
whether container_of() returns NULL. Note: moving the kref_get() call
from the start of ib_umad_*open() to the end is safe since it is the
responsibility of the caller of these functions to ensure that the
cdev pointer remains valid until at least when these functions return.
Signed-off-by: Bart Van Assche bvanass...@acm.org
Cc: Alex Chiang achi...@canonical.com
Cc: Yann Droneaud ydrone...@opteya.com
Cc: sta...@vger.kernel.org
[ydrone...@opteya.com: rework a bit to reduce the amount of code changed]
Signed-off-by: Yann Droneaud ydrone...@opteya.com
---
Hi Bart,
Please find a slightly modified version of your patch to simplify
a bit the error paths (no backward goto's) and to reduces the amount
of lines touched.
I wasn't able to explain it clearly enough in the previous patch review,
so I've made the changes directly in the file and propose the modified
patch for you to review.
As it's only suggestion, feel free to integrate the changes in your
patch or discard them.
Regards.
drivers/infiniband/core/user_mad.c | 50 +-
1 file changed, 27 insertions(+), 22 deletions(-)
diff --git a/drivers/infiniband/core/user_mad.c
b/drivers/infiniband/core/user_mad.c
index f0d588f8859e..055893b870ee 100644
--- a/drivers/infiniband/core/user_mad.c
+++ b/drivers/infiniband/core/user_mad.c
@@ -780,27 +780,19 @@ static int ib_umad_open(struct inode *inode, struct file
*filp)
{
struct ib_umad_port *port;
struct ib_umad_file *file;
- int ret;
+ int ret = -ENXIO;
port = container_of(inode-i_cdev, struct ib_umad_port, cdev);
- if (port)
- kref_get(port-umad_dev-ref);
- else
- return -ENXIO;
mutex_lock(port-file_mutex);
- if (!port-ib_dev) {
- ret = -ENXIO;
+ if (!port-ib_dev)
goto out;
- }
+ ret = -ENOMEM;
file = kzalloc(sizeof *file, GFP_KERNEL);
- if (!file) {
- kref_put(port-umad_dev-ref, ib_umad_release_dev);
- ret = -ENOMEM;
+ if (!file)
goto out;
- }
mutex_init(file-mutex);
spin_lock_init(file-send_lock);
@@ -814,9 +806,16 @@ static int ib_umad_open(struct inode *inode, struct file
*filp)
list_add_tail(file-port_list, port-file_list);
ret = nonseekable_open(inode, filp);
+ if (ret) {
+ list_del(file-port_list);
+ kfree(file);
+ goto out;
+ }
+
+ kref_get(port-umad_dev-ref);
out:
mutex_unlock(port-file_mutex);
return ret;
}
@@ -880,10 +880,6 @@ static int ib_umad_sm_open(struct inode *inode, struct
file *filp)
int ret;
port = container_of(inode-i_cdev, struct ib_umad_port, sm_cdev);
- if (port)
- kref_get(port-umad_dev-ref);
- else
- return -ENXIO;
if (filp-f_flags O_NONBLOCK) {
if (down_trylock(port-sm_sem)) {
@@ -898,17 +894,27 @@ static int ib_umad_sm_open(struct inode *inode, struct
file *filp)
}
ret = ib_modify_port(port-ib_dev, port-port_num, 0, props);
- if (ret) {
- up(port-sm_sem);
- goto fail;
- }
+ if (ret)
+ goto err_up_sem;
filp-private_data = port;
- return nonseekable_open(inode, filp);
+ ret = nonseekable_open(inode, filp);
+ if (ret)
+ goto err_clr_sm_cap;
+
+ kref_get(port-umad_dev-ref);
+
+ return 0;
+
+err_clr_sm_cap:
+ swap(props.set_port_cap_mask, props.clr_port_cap_mask);
+ ib_modify_port(port-ib_dev, port-port_num, 0, props);
+
+err_up_sem:
+ up(port-sm_sem);
fail:
- kref_put(port-umad_dev-ref, ib_umad_release_dev);
return ret;
}
--
1.9.0
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3.1] IB/umad: Fix error handling
On 05/20/14 10:33, Yann Droneaud wrote: Please find a slightly modified version of your patch to simplify a bit the error paths (no backward goto's) and to reduces the amount of lines touched. I wasn't able to explain it clearly enough in the previous patch review, so I've made the changes directly in the file and propose the modified patch for you to review. As it's only suggestion, feel free to integrate the changes in your patch or discard them. Hello Yann, It seems like our opinions about backward goto's are different :-) I thought these are common at the end of error handling code in the Linux kernel. Anyway, what matters to me is that a fix gets upstream, not which fix. But please do not expect me to spend more time testing a patch that has been reworked only because of the coding style aspects mentioned in your e-mail. Bart. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3.1] IB/umad: Fix error handling
Hi, Le mardi 20 mai 2014 à 13:25 +0200, Bart Van Assche a écrit : On 05/20/14 10:33, Yann Droneaud wrote: Please find a slightly modified version of your patch to simplify a bit the error paths (no backward goto's) and to reduces the amount of lines touched. I wasn't able to explain it clearly enough in the previous patch review, so I've made the changes directly in the file and propose the modified patch for you to review. As it's only suggestion, feel free to integrate the changes in your patch or discard them. Hello Yann, It seems like our opinions about backward goto's are different :-) I thought these are common at the end of error handling code in the Linux kernel. Anyway, what matters to me is that a fix gets upstream, not which fix. But please do not expect me to spend more time testing a patch that has been reworked only because of the coding style aspects mentioned in your e-mail. Having a shorter diff might help to get the patch applied upstream. YMMV. Regards. -- Yann Droneaud OPTEYA -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v4 0/9] SRP initiator patches for kernel 3.16
Changes compared to v3: - Fixed test robot complaints (undefined reference to `__udivdi3' and warning: format '%d' expects a matching 'int' argument). - Restored support for HCA's that neither support FMR nor FR. - Simplified memory registration pool reallocation code in srp_create_target_ib(). If FMR or FR is supported, an memory registration pool is allocated. If this fails, an error code is returned. If neither FMR nor FR is supported no attempt is made to allocate a memory registration pool. - Minimized the changes in srp_add_one(). The decision to limit sg_tablesize is now based on whether or not the HCA supports memory registration instead of whether allocation of a memory registration pool succeeded. This change allows to keep the resource allocation order unmodified. - Fixed a (theoretical ?) memory registration failure that was introduced in Linux kernel v2.6.39 (state-npages being compared against SRP_FMR_SIZE instead of max_pages_per_fmr). Changes compared to v2: - Reconnect to the SRP target if a local invalidation work request fails. - Swapped the state-next_fmr / next_fr assignments to improve code readability. - Clarified a comment in patch 1/9. - Fixed error handling in srp_create_target() (was broken in v2). - Added a missing PFX in two shost_printk() statements in patch 9/9. Changes compared to v1: - Modified the FMR code such that one FMR pool is allocated per connection instead of one pool per HCA. - Dropped the patch Make srp_alloc_req_data() reallocate request data. - Moved introduction of the register_always kernel module parameter into a separate patch. - Removed the loop from around ib_create_fmr_pool() and srp_create_fr_pool(). max_pages_per_mr is now computed from max_mr_size and max_fast_reg_page_list_len. - Reduced fast registration pool size from 1024 to scsi_host-can_queue. - Added a patch that should fix a crash that had been reported by Sagi but that I have not yet been able to reproduce myself. This patch series consists of the following nine patches: 0001-IB-srp-Fix-a-sporadic-crash-triggered-by-cable-pulli.patch 0002-IB-srp-Fix-kernel-doc-warnings.patch 0003-IB-srp-Introduce-an-additional-local-variable.patch 0004-IB-srp-Introduce-srp_map_fmr.patch 0005-IB-srp-Introduce-srp_finish_mapping.patch 0006-IB-srp-Introduce-the-register_always-kernel-module-p.patch 0007-IB-srp-One-FMR-pool-per-SRP-connection.patch 0008-IB-srp-Rename-FMR-related-variables.patch 0009-IB-srp-Add-fast-registration-support.patch -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v4 1/9] IB/srp: Fix a sporadic crash triggered by cable pulling
Avoid that the loops that iterate over the request ring can encounter a pointer to a SCSI command in req-scmnd that is no longer associated with that request. If the function srp_unmap_data() is invoked twice for a SCSI command that is not in flight then that would cause ib_fmr_pool_unmap() to be invoked with an invalid pointer as argument, resulting in a kernel oops. Reported-by: Sagi Grimberg sa...@mellanox.com Reference: http://thread.gmane.org/gmane.linux.drivers.rdma/19068/focus=19069 Signed-off-by: Bart Van Assche bvanass...@acm.org Reviewed-by: Sagi Grimberg sa...@mellanox.com Cc: Roland Dreier rol...@purestorage.com Cc: David Dillow d...@thedillows.org Cc: Vu Pham v...@mellanox.com Cc: Sebastian Parschauer sebastian.rie...@profitbricks.com Cc: stable sta...@vger.kernel.org --- drivers/infiniband/ulp/srp/ib_srp.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c index 66a908b..5b2bed8 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.c +++ b/drivers/infiniband/ulp/srp/ib_srp.c @@ -1594,6 +1594,12 @@ err_unmap: err_iu: srp_put_tx_iu(target, iu, SRP_IU_CMD); + /* +* Avoid that the loops that iterate over the request ring can +* encounter a dangling SCSI command pointer. +*/ + req-scmnd = NULL; + spin_lock_irqsave(target-lock, flags); list_add(req-list, target-free_reqs); -- 1.8.4.5 -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v4 3/9] IB/srp: Introduce an additional local variable
This patch does not change any functionality.
Signed-off-by: Bart Van Assche bvanass...@acm.org
Reviewed-by: Sagi Grimberg sa...@mellanox.com
Cc: Roland Dreier rol...@purestorage.com
Cc: David Dillow d...@thedillows.org
Cc: Vu Pham v...@mellanox.com
Cc: Sebastian Parschauer sebastian.rie...@profitbricks.com
---
drivers/infiniband/ulp/srp/ib_srp.c | 9 -
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c
b/drivers/infiniband/ulp/srp/ib_srp.c
index 4f8be37..281f785 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -290,6 +290,7 @@ static int srp_new_cm_id(struct srp_target_port *target)
static int srp_create_target_ib(struct srp_target_port *target)
{
+ struct srp_device *dev = target-srp_host-srp_dev;
struct ib_qp_init_attr *init_attr;
struct ib_cq *recv_cq, *send_cq;
struct ib_qp *qp;
@@ -299,16 +300,14 @@ static int srp_create_target_ib(struct srp_target_port
*target)
if (!init_attr)
return -ENOMEM;
- recv_cq = ib_create_cq(target-srp_host-srp_dev-dev,
- srp_recv_completion, NULL, target,
+ recv_cq = ib_create_cq(dev-dev, srp_recv_completion, NULL, target,
target-queue_size, target-comp_vector);
if (IS_ERR(recv_cq)) {
ret = PTR_ERR(recv_cq);
goto err;
}
- send_cq = ib_create_cq(target-srp_host-srp_dev-dev,
- srp_send_completion, NULL, target,
+ send_cq = ib_create_cq(dev-dev, srp_send_completion, NULL, target,
target-queue_size, target-comp_vector);
if (IS_ERR(send_cq)) {
ret = PTR_ERR(send_cq);
@@ -327,7 +326,7 @@ static int srp_create_target_ib(struct srp_target_port
*target)
init_attr-send_cq = send_cq;
init_attr-recv_cq = recv_cq;
- qp = ib_create_qp(target-srp_host-srp_dev-pd, init_attr);
+ qp = ib_create_qp(dev-pd, init_attr);
if (IS_ERR(qp)) {
ret = PTR_ERR(qp);
goto err_send_cq;
--
1.8.4.5
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v4 4/9] IB/srp: Introduce srp_map_fmr()
This patch does not change any functionality.
Signed-off-by: Bart Van Assche bvanass...@acm.org
Reviewed-by: Sagi Grimberg sa...@mellanox.com
Cc: Roland Dreier rol...@purestorage.com
Cc: David Dillow d...@thedillows.org
Cc: Vu Pham v...@mellanox.com
Cc: Sebastian Parschauer sebastian.rie...@profitbricks.com
---
drivers/infiniband/ulp/srp/ib_srp.c | 77 ++---
1 file changed, 45 insertions(+), 32 deletions(-)
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c
b/drivers/infiniband/ulp/srp/ib_srp.c
index 281f785..be84c94 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -1047,12 +1047,54 @@ static int srp_map_sg_entry(struct srp_map_state *state,
return ret;
}
+static void srp_map_fmr(struct srp_map_state *state,
+ struct srp_target_port *target, struct srp_request *req,
+ struct scatterlist *scat, int count)
+{
+ struct srp_device *dev = target-srp_host-srp_dev;
+ struct ib_device *ibdev = dev-dev;
+ struct scatterlist *sg;
+ int i, use_fmr;
+
+ state-desc = req-indirect_desc;
+ state-pages= req-map_page;
+ state-next_fmr = req-fmr_list;
+
+ use_fmr = dev-fmr_pool ? SRP_MAP_ALLOW_FMR : SRP_MAP_NO_FMR;
+
+ for_each_sg(scat, sg, count, i) {
+ if (srp_map_sg_entry(state, target, sg, i, use_fmr)) {
+ /* FMR mapping failed, so backtrack to the first
+* unmapped entry and continue on without using FMR.
+*/
+ dma_addr_t dma_addr;
+ unsigned int dma_len;
+
+backtrack:
+ sg = state-unmapped_sg;
+ i = state-unmapped_index;
+
+ dma_addr = ib_sg_dma_address(ibdev, sg);
+ dma_len = ib_sg_dma_len(ibdev, sg);
+ dma_len -= (state-unmapped_addr - dma_addr);
+ dma_addr = state-unmapped_addr;
+ use_fmr = SRP_MAP_NO_FMR;
+ srp_map_desc(state, dma_addr, dma_len, target-rkey);
+ }
+ }
+
+ if (use_fmr == SRP_MAP_ALLOW_FMR srp_map_finish_fmr(state, target))
+ goto backtrack;
+
+ req-nfmr = state-nfmr;
+}
+
static int srp_map_data(struct scsi_cmnd *scmnd, struct srp_target_port
*target,
struct srp_request *req)
{
- struct scatterlist *scat, *sg;
+ struct scatterlist *scat;
struct srp_cmd *cmd = req-cmd-buf;
- int i, len, nents, count, use_fmr;
+ int len, nents, count;
struct srp_device *dev;
struct ib_device *ibdev;
struct srp_map_state state;
@@ -,35 +1153,7 @@ static int srp_map_data(struct scsi_cmnd *scmnd, struct
srp_target_port *target,
target-indirect_size, DMA_TO_DEVICE);
memset(state, 0, sizeof(state));
- state.desc = req-indirect_desc;
- state.pages = req-map_page;
- state.next_fmr = req-fmr_list;
-
- use_fmr = dev-fmr_pool ? SRP_MAP_ALLOW_FMR : SRP_MAP_NO_FMR;
-
- for_each_sg(scat, sg, count, i) {
- if (srp_map_sg_entry(state, target, sg, i, use_fmr)) {
- /* FMR mapping failed, so backtrack to the first
-* unmapped entry and continue on without using FMR.
-*/
- dma_addr_t dma_addr;
- unsigned int dma_len;
-
-backtrack:
- sg = state.unmapped_sg;
- i = state.unmapped_index;
-
- dma_addr = ib_sg_dma_address(ibdev, sg);
- dma_len = ib_sg_dma_len(ibdev, sg);
- dma_len -= (state.unmapped_addr - dma_addr);
- dma_addr = state.unmapped_addr;
- use_fmr = SRP_MAP_NO_FMR;
- srp_map_desc(state, dma_addr, dma_len, target-rkey);
- }
- }
-
- if (use_fmr == SRP_MAP_ALLOW_FMR srp_map_finish_fmr(state, target))
- goto backtrack;
+ srp_map_fmr(state, target, req, scat, count);
/* We've mapped the request, now pull as much of the indirect
* descriptor table as we can into the command buffer. If this
@@ -1147,7 +1161,6 @@ backtrack:
* guaranteed to fit into the command, as the SCSI layer won't
* give us more S/G entries than we allow.
*/
- req-nfmr = state.nfmr;
if (state.ndesc == 1) {
/* FMR mapping was able to collapse this to one entry,
* so use a direct descriptor.
--
1.8.4.5
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v4 5/9] IB/srp: Introduce srp_finish_mapping()
This patch does not change any functionality.
Signed-off-by: Bart Van Assche bvanass...@acm.org
Reviewed-by: Sagi Grimberg sa...@mellanox.com
Cc: Roland Dreier rol...@purestorage.com
Cc: David Dillow d...@thedillows.org
Cc: Vu Pham v...@mellanox.com
Cc: Sebastian Parschauer sebastian.rie...@profitbricks.com
---
drivers/infiniband/ulp/srp/ib_srp.c | 42 -
1 file changed, 27 insertions(+), 15 deletions(-)
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c
b/drivers/infiniband/ulp/srp/ib_srp.c
index be84c94..c2e0ad3 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -935,16 +935,6 @@ static int srp_map_finish_fmr(struct srp_map_state *state,
struct ib_pool_fmr *fmr;
u64 io_addr = 0;
- if (!state-npages)
- return 0;
-
- if (state-npages == 1) {
- srp_map_desc(state, state-base_dma_addr, state-fmr_len,
-target-rkey);
- state-npages = state-fmr_len = 0;
- return 0;
- }
-
fmr = ib_fmr_pool_map_phys(dev-fmr_pool, state-pages,
state-npages, io_addr);
if (IS_ERR(fmr))
@@ -954,10 +944,32 @@ static int srp_map_finish_fmr(struct srp_map_state *state,
state-nfmr++;
srp_map_desc(state, 0, state-fmr_len, fmr-fmr-rkey);
- state-npages = state-fmr_len = 0;
+
return 0;
}
+static int srp_finish_mapping(struct srp_map_state *state,
+ struct srp_target_port *target)
+{
+ int ret = 0;
+
+ if (state-npages == 0)
+ return 0;
+
+ if (state-npages == 1)
+ srp_map_desc(state, state-base_dma_addr, state-fmr_len,
+target-rkey);
+ else
+ ret = srp_map_finish_fmr(state, target);
+
+ if (ret == 0) {
+ state-npages = 0;
+ state-fmr_len = 0;
+ }
+
+ return ret;
+}
+
static void srp_map_update_start(struct srp_map_state *state,
struct scatterlist *sg, int sg_index,
dma_addr_t dma_addr)
@@ -998,7 +1010,7 @@ static int srp_map_sg_entry(struct srp_map_state *state,
* avoided using FMR on such page fragments.
*/
if (dma_addr ~dev-fmr_page_mask || dma_len dev-fmr_max_size) {
- ret = srp_map_finish_fmr(state, target);
+ ret = srp_finish_mapping(state, target);
if (ret)
return ret;
@@ -1017,7 +1029,7 @@ static int srp_map_sg_entry(struct srp_map_state *state,
while (dma_len) {
if (state-npages == SRP_FMR_SIZE) {
- ret = srp_map_finish_fmr(state, target);
+ ret = srp_finish_mapping(state, target);
if (ret)
return ret;
@@ -1040,7 +1052,7 @@ static int srp_map_sg_entry(struct srp_map_state *state,
*/
ret = 0;
if (len != dev-fmr_page_size) {
- ret = srp_map_finish_fmr(state, target);
+ ret = srp_finish_mapping(state, target);
if (!ret)
srp_map_update_start(state, NULL, 0, 0);
}
@@ -1083,7 +1095,7 @@ backtrack:
}
}
- if (use_fmr == SRP_MAP_ALLOW_FMR srp_map_finish_fmr(state, target))
+ if (use_fmr == SRP_MAP_ALLOW_FMR srp_finish_mapping(state, target))
goto backtrack;
req-nfmr = state-nfmr;
--
1.8.4.5
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v4 8/9] IB/srp: Rename FMR-related variables
The next patch will cause the renamed variables to be shared between
the code for FMR and for FR memory registration. Make the names of
these variables independent of the memory registration mode. This
patch does not change any functionality. The start of this patch was
the changes applied via the following shell command:
sed -i.orig 's/SRP_FMR_SIZE/SRP_MAX_PAGES_PER_MR/g; \
s/fmr_page_mask/mr_page_mask/g;s/fmr_page_size/mr_page_size/g; \
s/fmr_page_shift/mr_page_shift/g;s/fmr_max_size/mr_max_size/g; \
s/max_pages_per_fmr/max_pages_per_mr/g;s/nfmr/nmdesc/g; \
s/fmr_len/dma_len/g' drivers/infiniband/ulp/srp/ib_srp.[ch]
Signed-off-by: Bart Van Assche bvanass...@acm.org
Cc: Roland Dreier rol...@purestorage.com
Cc: David Dillow d...@thedillows.org
Cc: Sagi Grimberg sa...@mellanox.com
Cc: Vu Pham v...@mellanox.com
Cc: Sebastian Parschauer sebastian.rie...@profitbricks.com
---
drivers/infiniband/ulp/srp/ib_srp.c | 60 ++---
drivers/infiniband/ulp/srp/ib_srp.h | 16 +-
2 files changed, 38 insertions(+), 38 deletions(-)
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c
b/drivers/infiniband/ulp/srp/ib_srp.c
index 80dfe17..c9b3b9e 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -302,8 +302,8 @@ static struct ib_fmr_pool *srp_alloc_fmr_pool(struct
srp_target_port *target)
fmr_param.pool_size = target-scsi_host-can_queue;
fmr_param.dirty_watermark = fmr_param.pool_size / 4;
fmr_param.cache = 1;
- fmr_param.max_pages_per_fmr = dev-max_pages_per_fmr;
- fmr_param.page_shift= ilog2(dev-fmr_page_size);
+ fmr_param.max_pages_per_fmr = dev-max_pages_per_mr;
+ fmr_param.page_shift= ilog2(dev-mr_page_size);
fmr_param.access= (IB_ACCESS_LOCAL_WRITE |
IB_ACCESS_REMOTE_WRITE |
IB_ACCESS_REMOTE_READ);
@@ -657,7 +657,7 @@ static int srp_alloc_req_data(struct srp_target_port
*target)
req = target-req_ring[i];
req-fmr_list = kmalloc(target-cmd_sg_cnt * sizeof(void *),
GFP_KERNEL);
- req-map_page = kmalloc(srp_dev-max_pages_per_fmr *
+ req-map_page = kmalloc(srp_dev-max_pages_per_mr *
sizeof(void *), GFP_KERNEL);
req-indirect_desc = kmalloc(target-indirect_size, GFP_KERNEL);
if (!req-fmr_list || !req-map_page || !req-indirect_desc)
@@ -810,7 +810,7 @@ static void srp_unmap_data(struct scsi_cmnd *scmnd,
return;
pfmr = req-fmr_list;
- while (req-nfmr--)
+ while (req-nmdesc--)
ib_fmr_pool_unmap(*pfmr++);
ib_dma_unmap_sg(ibdev, scsi_sglist(scmnd), scsi_sg_count(scmnd),
@@ -979,9 +979,9 @@ static int srp_map_finish_fmr(struct srp_map_state *state,
return PTR_ERR(fmr);
*state-next_fmr++ = fmr;
- state-nfmr++;
+ state-nmdesc++;
- srp_map_desc(state, 0, state-fmr_len, fmr-fmr-rkey);
+ srp_map_desc(state, 0, state-dma_len, fmr-fmr-rkey);
return 0;
}
@@ -995,14 +995,14 @@ static int srp_finish_mapping(struct srp_map_state *state,
return 0;
if (state-npages == 1 !register_always)
- srp_map_desc(state, state-base_dma_addr, state-fmr_len,
+ srp_map_desc(state, state-base_dma_addr, state-dma_len,
target-rkey);
else
ret = srp_map_finish_fmr(state, target);
if (ret == 0) {
state-npages = 0;
- state-fmr_len = 0;
+ state-dma_len = 0;
}
return ret;
@@ -1047,7 +1047,7 @@ static int srp_map_sg_entry(struct srp_map_state *state,
* that were never quite defined, but went away when the initiator
* avoided using FMR on such page fragments.
*/
- if (dma_addr ~dev-fmr_page_mask || dma_len dev-fmr_max_size) {
+ if (dma_addr ~dev-mr_page_mask || dma_len dev-mr_max_size) {
ret = srp_finish_mapping(state, target);
if (ret)
return ret;
@@ -1066,7 +1066,7 @@ static int srp_map_sg_entry(struct srp_map_state *state,
srp_map_update_start(state, sg, sg_index, dma_addr);
while (dma_len) {
- if (state-npages == dev-max_pages_per_fmr) {
+ if (state-npages == dev-max_pages_per_mr) {
ret = srp_finish_mapping(state, target);
if (ret)
return ret;
@@ -1074,12 +1074,12 @@ static int srp_map_sg_entry(struct srp_map_state *state,
srp_map_update_start(state, sg, sg_index, dma_addr);
}
- len = min_t(unsigned int, dma_len, dev-fmr_page_size);
+
[PATCH v4 7/9] IB/srp: One FMR pool per SRP connection
Allocate one FMR pool per SRP connection instead of one SRP pool
per HCA. This improves scalability of the SRP initiator.
Only request the SCSI mid-layer to retry a SCSI command after a
temporary mapping failure (-ENOMEM) but not after a permanent
mapping failure. This avoids that SCSI commands are retried
indefinitely if a permanent memory mapping failure occurs.
Tell the SCSI mid-layer to reduce queue depth temporarily in the
unlikely case where an application is queuing many requests with
more than max_pages_per_fmr sg-list elements.
For FMR pool allocation, base the max_pages_per_fmr parameter on
the HCA memory registration limit. Only try to allocate an FMR
pool if FMR is supported.
Signed-off-by: Bart Van Assche bvanass...@acm.org
Cc: Sagi Grimberg sa...@mellanox.com
Cc: Roland Dreier rol...@purestorage.com
Cc: David Dillow d...@thedillows.org
Cc: Vu Pham v...@mellanox.com
Cc: Sebastian Parschauer sebastian.rie...@profitbricks.com
---
drivers/infiniband/ulp/srp/ib_srp.c | 129 ++--
drivers/infiniband/ulp/srp/ib_srp.h | 7 +-
2 files changed, 84 insertions(+), 52 deletions(-)
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c
b/drivers/infiniband/ulp/srp/ib_srp.c
index 77ba965..80dfe17 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -293,12 +293,31 @@ static int srp_new_cm_id(struct srp_target_port *target)
return 0;
}
+static struct ib_fmr_pool *srp_alloc_fmr_pool(struct srp_target_port *target)
+{
+ struct srp_device *dev = target-srp_host-srp_dev;
+ struct ib_fmr_pool_param fmr_param;
+
+ memset(fmr_param, 0, sizeof(fmr_param));
+ fmr_param.pool_size = target-scsi_host-can_queue;
+ fmr_param.dirty_watermark = fmr_param.pool_size / 4;
+ fmr_param.cache = 1;
+ fmr_param.max_pages_per_fmr = dev-max_pages_per_fmr;
+ fmr_param.page_shift= ilog2(dev-fmr_page_size);
+ fmr_param.access= (IB_ACCESS_LOCAL_WRITE |
+ IB_ACCESS_REMOTE_WRITE |
+ IB_ACCESS_REMOTE_READ);
+
+ return ib_create_fmr_pool(dev-pd, fmr_param);
+}
+
static int srp_create_target_ib(struct srp_target_port *target)
{
struct srp_device *dev = target-srp_host-srp_dev;
struct ib_qp_init_attr *init_attr;
struct ib_cq *recv_cq, *send_cq;
struct ib_qp *qp;
+ struct ib_fmr_pool *fmr_pool = NULL;
int ret;
init_attr = kzalloc(sizeof *init_attr, GFP_KERNEL);
@@ -341,6 +360,19 @@ static int srp_create_target_ib(struct srp_target_port
*target)
if (ret)
goto err_qp;
+ if (dev-has_fmr) {
+ fmr_pool = srp_alloc_fmr_pool(target);
+ if (IS_ERR(fmr_pool)) {
+ ret = PTR_ERR(fmr_pool);
+ shost_printk(KERN_WARNING, target-scsi_host, PFX
+FMR pool allocation failed (%d)\n, ret);
+ goto err_qp;
+ }
+ if (target-fmr_pool)
+ ib_destroy_fmr_pool(target-fmr_pool);
+ target-fmr_pool = fmr_pool;
+ }
+
if (target-qp)
ib_destroy_qp(target-qp);
if (target-recv_cq)
@@ -377,6 +409,8 @@ static void srp_free_target_ib(struct srp_target_port
*target)
{
int i;
+ if (target-fmr_pool)
+ ib_destroy_fmr_pool(target-fmr_pool);
ib_destroy_qp(target-qp);
ib_destroy_cq(target-send_cq);
ib_destroy_cq(target-recv_cq);
@@ -623,8 +657,8 @@ static int srp_alloc_req_data(struct srp_target_port
*target)
req = target-req_ring[i];
req-fmr_list = kmalloc(target-cmd_sg_cnt * sizeof(void *),
GFP_KERNEL);
- req-map_page = kmalloc(SRP_FMR_SIZE * sizeof(void *),
- GFP_KERNEL);
+ req-map_page = kmalloc(srp_dev-max_pages_per_fmr *
+ sizeof(void *), GFP_KERNEL);
req-indirect_desc = kmalloc(target-indirect_size, GFP_KERNEL);
if (!req-fmr_list || !req-map_page || !req-indirect_desc)
goto out;
@@ -936,11 +970,10 @@ static void srp_map_desc(struct srp_map_state *state,
dma_addr_t dma_addr,
static int srp_map_finish_fmr(struct srp_map_state *state,
struct srp_target_port *target)
{
- struct srp_device *dev = target-srp_host-srp_dev;
struct ib_pool_fmr *fmr;
u64 io_addr = 0;
- fmr = ib_fmr_pool_map_phys(dev-fmr_pool, state-pages,
+ fmr = ib_fmr_pool_map_phys(target-fmr_pool, state-pages,
state-npages, io_addr);
if (IS_ERR(fmr))
return PTR_ERR(fmr);
@@ -1033,7 +1066,7 @@ static int srp_map_sg_entry(struct
RE: [PATCH 0/3] RDMA/core: iWARP Port Mapper Overview
-Original Message- From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Roland Dreier Sent: Monday, May 19, 2014 8:10 PM To: Tatyana Nikolova Cc: Lacombe, John S; Steve Wise; Sean Hefty; linux-rdma@vger.kernel.org Subject: Re: [PATCH 0/3] RDMA/core: iWARP Port Mapper Overview On Wed, Mar 26, 2014 at 3:07 PM, Tatyana Nikolova tatyana.e.nikol...@intel.com wrote: 1) The IWPM functionality, common for both iWarp drivers (nes and cxgb4) is refactored from the drivers source files and is moved to new shared files in infiniband/core which are compiled as part of the iw_cm module. So I'm looking to include this for 3.16. One question: I assume unpatched iwarp drivers (eg cxgb3) continue to work as before? In other words they don't get the new port mapper support but at least they don't regress? Hey Roland, cxgb3-cxgb3 would behave as before; no regression. cxgb3-cxgb4/nes would interoperate only if the port mapping service is disabled on the cxgb4/nes nodes. Steve. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/3] RDMA/core: iWARP Port Mapper Overview
On 20/05/2014 16:46, Steve Wise wrote: But here is it attached thanks -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 0/9] SRP initiator patches for kernel 3.16
On 20/05/2014 16:03, Bart Van Assche wrote: Changes compared to v3: V3 is already applied -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 0/9] SRP initiator patches for kernel 3.16
On Tue, May 20, 2014 at 7:06 AM, Or Gerlitz ogerl...@mellanox.com wrote: V3 is already applied It's OK, I can replace the series. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
CAN YOU HANDLE THE INVESTMENT
Please this is a personal message for you. I need some directives from you to invest some capital in your country. And see how we can partnership into the business. Just reply me to explain better. Mr.Zuberi Muti -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 0/3] RDMA/core: iWARP Port Mapper Overview
Hello Rolland,
Thanks for reviewing the port mapper patches.
Steve's reply is correct.
Unpatched nes will not regress and will continue working as before.
Tatyana
-Original Message-
From: Steve Wise [mailto:sw...@opengridcomputing.com]
Sent: Tuesday, May 20, 2014 8:45 AM
To: 'Roland Dreier'; Nikolova, Tatyana E
Cc: Lacombe, John S; Hefty, Sean; linux-rdma@vger.kernel.org; 'Doug Ledford'
Subject: RE: [PATCH 0/3] RDMA/core: iWARP Port Mapper Overview
-Original Message-
From: linux-rdma-ow...@vger.kernel.org
[mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Roland Dreier
Sent: Monday, May 19, 2014 8:10 PM
To: Tatyana Nikolova
Cc: Lacombe, John S; Steve Wise; Sean Hefty;
linux-rdma@vger.kernel.org
Subject: Re: [PATCH 0/3] RDMA/core: iWARP Port Mapper Overview
On Wed, Mar 26, 2014 at 3:07 PM, Tatyana Nikolova
tatyana.e.nikol...@intel.com wrote:
1) The IWPM functionality, common for both iWarp drivers (nes and cxgb4)
is refactored from the drivers source files and is moved to new shared
files in infiniband/core which are compiled as part of the iw_cm module.
So I'm looking to include this for 3.16. One question: I assume
unpatched iwarp drivers (eg cxgb3) continue to work as before? In
other words they don't get the new port mapper support but at least
they don't regress?
Hey Roland,
cxgb3-cxgb3 would behave as before; no regression. cxgb3-cxgb4/nes would
interoperate only if the port mapping service is disabled on the cxgb4/nes
nodes.
Steve.
N�r��yb�X��ǧv�^�){.n�+{��ٚ�{ay� ʇڙ�,j��f���h���z� �w���
���j:+v���w�j�mzZ+�ݢj��!�i
help with IB_WC_MW_BIND_ERR
Hi- What does it mean when a LOCAL_INV work request fails with a IB_WC_MW_BIND_ERR completion? -- Chuck Lever chuck[dot]lever[at]oracle[dot]com -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: help with IB_WC_MW_BIND_ERR
On Tue, May 20, 2014 at 11:55 AM, Chuck Lever chuck.le...@oracle.com wrote: Hi- What does it mean when a LOCAL_INV work request fails with a IB_WC_MW_BIND_ERR completion? Mapping an IB error code has been a great pain (at least for me) unless you have access to the HCA firmware. In this case, I think it implies memory protection error (registration issues) say in cxgb4 driver, it is associated with invalidate shared MR or invalidate bound memory window (with a QP): case T4_ERR_INVALIDATE_SHARED_MR: case T4_ERR_INVALIDATE_MR_WITH_MW_BOUND: wc-status = IB_WC_MW_BIND_ERR; break; drivers/infiniband/hw/cxgb4/cq.c line 654 of 898 --72%-- col 11-25 You'll probably need to mention the HCA name so the firmware people, if they are reading this, could pinpoint the exact cause. -- Wendy -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: help with IB_WC_MW_BIND_ERR
On May 20, 2014, at 3:39 PM, Wendy Cheng s.wendy.ch...@gmail.com wrote: On Tue, May 20, 2014 at 11:55 AM, Chuck Lever chuck.le...@oracle.com wrote: Hi- What does it mean when a LOCAL_INV work request fails with a IB_WC_MW_BIND_ERR completion? Mapping an IB error code has been a great pain (at least for me) unless you have access to the HCA firmware. In this case, I think it implies memory protection error (registration issues) say in cxgb4 driver, it is associated with invalidate shared MR or invalidate bound memory window (with a QP): case T4_ERR_INVALIDATE_SHARED_MR: case T4_ERR_INVALIDATE_MR_WITH_MW_BOUND: wc-status = IB_WC_MW_BIND_ERR; break; drivers/infiniband/hw/cxgb4/cq.c line 654 of 898 --72%-- col 11-25 You'll probably need to mention the HCA name so the firmware people, if they are reading this, could pinpoint the exact cause. Thanks. ConnectX-2, mlx4 provider. The IB architecture spec lists five conditions that could result in an IB_WC_MW_BIND_ERR completion of LOCAL_INV: 1. Memory access was attempted on an L_Key or R_Key that is in the Invalid State; 2. Memory Region could not be Invalidated, because it is a Shared Memory Region; 3. Memory Region can not be invalidated because it has bound Memory Window; or 4. Memory Region could not be Invalidated, because it was created through a Register Memory Region or Reregister Memory Region. 5. Memory Window could not be Invalidated, because it was a Type 1 Memory Window. This is with FRMR and the MR is not shared (I think?). So I expect I’m dealing with condition 1. But I can’t seem to make any more headway on confirming that, or how it got that way. -- Chuck Lever chuck[dot]lever[at]oracle[dot]com -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
