Re: [patch] IB/mlx5: fix error code in get_port_caps()
On Mon, 12 Jan 2015, Dan Carpenter wrote:
We should return -ENOMEM on allocation failure instead of success.
Probably not a big deal, but the commit message is not clear at all. From
the fix, it doesn't look like the code was returning -ENOMEM on success.
julia
Fixes: e126ba97dba9 ('mlx5: Add driver for Mellanox Connect-IB adapters')
Signed-off-by: Dan Carpenter dan.carpen...@oracle.com
diff --git a/drivers/infiniband/hw/mlx5/main.c
b/drivers/infiniband/hw/mlx5/main.c
index 8a87404..cc4ac1e 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -997,7 +997,7 @@ static int get_port_caps(struct mlx5_ib_dev *dev)
struct ib_device_attr *dprops = NULL;
struct ib_port_attr *pprops = NULL;
struct mlx5_general_caps *gen;
- int err = 0;
+ int err = -ENOMEM;
int port;
gen = dev-mdev-caps.gen;
--
To unsubscribe from this list: send the line unsubscribe kernel-janitors in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[patch v2] IB/mlx5: fix error code in get_port_caps()
The current code returns success when kmalloc() fails. It should return
an error code, -ENOMEM.
Fixes: e126ba97dba9 ('mlx5: Add driver for Mellanox Connect-IB adapters')
Signed-off-by: Dan Carpenter dan.carpen...@oracle.com
---
v2: re-write changelog
diff --git a/drivers/infiniband/hw/mlx5/main.c
b/drivers/infiniband/hw/mlx5/main.c
index 8a87404..cc4ac1e 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -997,7 +997,7 @@ static int get_port_caps(struct mlx5_ib_dev *dev)
struct ib_device_attr *dprops = NULL;
struct ib_port_attr *pprops = NULL;
struct mlx5_general_caps *gen;
- int err = 0;
+ int err = -ENOMEM;
int port;
gen = dev-mdev-caps.gen;
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[patch] IB/mlx5: fix error code in get_port_caps()
We should return -ENOMEM on allocation failure instead of success.
Fixes: e126ba97dba9 ('mlx5: Add driver for Mellanox Connect-IB adapters')
Signed-off-by: Dan Carpenter dan.carpen...@oracle.com
diff --git a/drivers/infiniband/hw/mlx5/main.c
b/drivers/infiniband/hw/mlx5/main.c
index 8a87404..cc4ac1e 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -997,7 +997,7 @@ static int get_port_caps(struct mlx5_ib_dev *dev)
struct ib_device_attr *dprops = NULL;
struct ib_port_attr *pprops = NULL;
struct mlx5_general_caps *gen;
- int err = 0;
+ int err = -ENOMEM;
int port;
gen = dev-mdev-caps.gen;
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [patch] IB/mlx5: fix error code in get_port_caps()
Acked-by: Eli Cohen e...@mellanox.com
Thanks!
On Mon, Jan 12, 2015 at 11:14:29AM +0300, Dan Carpenter wrote:
We should return -ENOMEM on allocation failure instead of success.
Fixes: e126ba97dba9 ('mlx5: Add driver for Mellanox Connect-IB adapters')
Signed-off-by: Dan Carpenter dan.carpen...@oracle.com
diff --git a/drivers/infiniband/hw/mlx5/main.c
b/drivers/infiniband/hw/mlx5/main.c
index 8a87404..cc4ac1e 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -997,7 +997,7 @@ static int get_port_caps(struct mlx5_ib_dev *dev)
struct ib_device_attr *dprops = NULL;
struct ib_port_attr *pprops = NULL;
struct mlx5_general_caps *gen;
- int err = 0;
+ int err = -ENOMEM;
int port;
gen = dev-mdev-caps.gen;
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH v1 06/10] svcrdma: Plant reader function in struct svcxprt_rdma
-Original Message-
From: Chuck Lever [mailto:chuck.le...@oracle.com]
Sent: Sunday, January 11, 2015 6:41 PM
To: Sagi Grimberg; Steve Wise
Cc: linux-rdma@vger.kernel.org; Linux NFS Mailing List
Subject: Re: [PATCH v1 06/10] svcrdma: Plant reader function in struct
svcxprt_rdma
On Jan 11, 2015, at 12:45 PM, Sagi Grimberg sa...@dev.mellanox.co.il wrote:
On 1/9/2015 9:22 PM, Chuck Lever wrote:
The RDMA reader function doesn't change once an svcxprt is
instantiated. Instead of checking sc_devcap during every incoming
RPC, set the reader function once when the connection is accepted.
General question(s),
Any specific reason why to use FRMR in the server side? And why only
for reads and not writes? Sorry if these are dumb questions...
Steve Wise presented patches a few months back to add FRMR, he
would have to answer this. Steve has a selection of iWARP adapters
and maybe could provide some idea of performance impact. I have
only CX-[23] here.
The rdma rpc server has always tried to use FRMR for rdma reads as far as I
recall. The patch I submitted refactored the design in
order to make it more efficient and to fix some bugs. Unlike IB, the iWARP
protocol only allows 1 target/sink SGE in an rdma read
request message, so an FRMR is used to create that single target/sink SGE
allowing 1 read to be submitted instead of many. I
believe that the FRMR allows for more efficient IO since w/o it you end up with
large SGLs of 4K each and lots of read requests.
However, I have no data to back that up. I would think that the write side
(NFS READ) could also benefit from FRMRs too. It also
could use refactoring, because I believe it still creates an intermediate data
structure to hold the write chunks vs just
translating them directly into the RDMA SGLs needed for the IO. See
send_write_chunks() and send_write() and how they create a
svc_rdma_req_map vector first and then translate that into the SGL needed for
the rdma writes.
My next step is to do some performance measurement to see if FRMR
is worth the trouble, at least with the cards on hand.
I notice that the lcl case does not seem to work with my CX-3 Pro.
Probably a bug I will have to address first.
Sagi.
Signed-off-by: Chuck Lever chuck.le...@oracle.com
---
include/linux/sunrpc/svc_rdma.h | 10
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 71
+++---
net/sunrpc/xprtrdma/svc_rdma_transport.c |2 +
3 files changed, 39 insertions(+), 44 deletions(-)
diff --git a/include/linux/sunrpc/svc_rdma.h
b/include/linux/sunrpc/svc_rdma.h
index 2280325..f161e30 100644
--- a/include/linux/sunrpc/svc_rdma.h
+++ b/include/linux/sunrpc/svc_rdma.h
@@ -150,6 +150,10 @@ struct svcxprt_rdma {
struct ib_cq *sc_rq_cq;
struct ib_cq *sc_sq_cq;
struct ib_mr *sc_phys_mr; /* MR for server memory */
+ int (*sc_reader)(struct svcxprt_rdma *,
+struct svc_rqst *,
+struct svc_rdma_op_ctxt *,
+int *, u32 *, u32, u32, u64, bool);
u32 sc_dev_caps; /* distilled device caps */
u32 sc_dma_lkey; /* local dma key */
unsigned int sc_frmr_pg_list_len;
@@ -195,6 +199,12 @@ extern int svc_rdma_xdr_get_reply_hdr_len(struct
rpcrdma_msg *);
/* svc_rdma_recvfrom.c */
extern int svc_rdma_recvfrom(struct svc_rqst *);
+extern int rdma_read_chunk_lcl(struct svcxprt_rdma *, struct svc_rqst *,
+ struct svc_rdma_op_ctxt *, int *, u32 *,
+ u32, u32, u64, bool);
+extern int rdma_read_chunk_frmr(struct svcxprt_rdma *, struct svc_rqst *,
+ struct svc_rdma_op_ctxt *, int *, u32 *,
+ u32, u32, u64, bool);
/* svc_rdma_sendto.c */
extern int svc_rdma_sendto(struct svc_rqst *);
diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
index 577f865..c3aebc1 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
@@ -117,26 +117,16 @@ static int rdma_read_max_sge(struct svcxprt_rdma
*xprt, int sge_count)
return min_t(int, sge_count, xprt-sc_max_sge);
}
-typedef int (*rdma_reader_fn)(struct svcxprt_rdma *xprt,
-struct svc_rqst *rqstp,
-struct svc_rdma_op_ctxt *head,
-int *page_no,
-u32 *page_offset,
-u32 rs_handle,
-u32 rs_length,
-u64 rs_offset,
-int last);
-
/* Issue an RDMA_READ using the local lkey to map the data sink */
-static int rdma_read_chunk_lcl(struct svcxprt_rdma *xprt,
[PATCH infiniband-diags] ibdiag_common.c: Add more supported device IDs
Add support for ConnectX-3 and ConnectX-4.
Signed-off-by: Hal Rosenstock h...@mellanox.com
---
diff --git a/src/ibdiag_common.c b/src/ibdiag_common.c
index 8c749c7..384d342 100644
--- a/src/ibdiag_common.c
+++ b/src/ibdiag_common.c
@@ -499,9 +499,9 @@ conv_cnt_human_readable(uint64_t val64, float *val, int
data)
int is_mlnx_ext_port_info_supported(uint32_t devid)
{
if (ibd_ibnetdisc_flags IBND_CONFIG_MLX_EPI) {
- if (devid == 0xc738)
+ if (devid == 0xc738 || devid == 0xcb84)
return 1;
- if (devid = 0x1003 devid = 0x1011)
+ if (devid = 0x1003 devid = 0x1016)
return 1;
}
return 0;
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH TRIVIAL libibmad] fields.c: Fix commentary typo
Signed-off-by: Hal Rosenstock h...@mellanox.com
---
diff --git a/src/fields.c b/src/fields.c
index 33a6364..9965811 100644
--- a/src/fields.c
+++ b/src/fields.c
@@ -835,7 +835,7 @@ static const ib_field_t ib_mad_f[] = {
{BITSOFFS(16, 16), ThresholdEventCounter, mad_dump_uint},
{BITSOFFS(32, 16), ThresholdCongestionEventMap, mad_dump_hex},
/* XXX: Q3/2010 errata lists offset 48, but that means field is not
-* world aligned. Assume will be aligned to offset 64 later.
+* word aligned. Assume will be aligned to offset 64 later.
*/
{BITSOFFS(64, 32), CurrentTimeStamp, mad_dump_uint},
{0, 0}, /* IB_CC_CONGESTION_LOG_CA_LAST_F */
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH opensm] osm_subnet.c: Add more supported device IDs
Add support for ConnectX-4. Signed-off-by: Hal Rosenstock h...@mellanox.com --- opensm/osm_subnet.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/opensm/osm_subnet.c b/opensm/osm_subnet.c index ba760ff..3463eca 100644 --- a/opensm/osm_subnet.c +++ b/opensm/osm_subnet.c @@ -1476,7 +1476,7 @@ int is_mlnx_ext_port_info_supported(ib_net16_t devid) devid_ho = cl_ntoh16(devid); if ((devid_ho = 0xc738 devid_ho = 0xc73b) || devid_ho == 0xcb20) return 1; - if (devid_ho = 0x1003 devid_ho = 0x1013) + if (devid_ho = 0x1003 devid_ho = 0x1016) return 1; return 0; } -- 1.7.8.2 -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH net-next 0/2] All Chelsio drivers : Cleanup CPL messages macros
From: Anish Bhatt an...@chelsio.com Date: Thu, 8 Jan 2015 21:38:14 -0800 This patch series cleans up all register defines/MACROS defined in t4_msg.h and affected files as part of the continuing cleanup effort The patches series is created against 'net-next' tree and includes patches to the cxgb4, cxgb4vf, iw_cxgb4, cxgb4i and csiostor drivers. We have included all the maintainers of respective drivers. Kindly review the change and let us know in case of any review comments. Looks fine, applied to net-next, thanks. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH infiniband-diags] ibdiag_common.c: Add more supported device IDs
On 1/12/2015 9:08 PM, Hal Rosenstock wrote: Add support for ConnectX-3 and ConnectX-4 So... ConnectX-3 isn't supported today?! on what infrastructure/tools exactly? -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] IB: cxgb3: iwch_cm: Remove some unused functions
Acked-by: Steve Wise sw...@opengridcomputing.com -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 05/20] IB/mad: Change cast in rcv_has_same_class
From: Ira Weiny ira.we...@intel.com
Save dereference and clarifies that rcv_has_same_class can process both IB and
OPA MADs.
Signed-off-by: Ira Weiny ira.we...@intel.com
---
drivers/infiniband/core/mad.c |2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
index 66b3940..819b794 100644
--- a/drivers/infiniband/core/mad.c
+++ b/drivers/infiniband/core/mad.c
@@ -1750,7 +1750,7 @@ static int is_rmpp_data_mad(struct ib_mad_agent_private
*mad_agent_priv,
static inline int rcv_has_same_class(struct ib_mad_send_wr_private *wr,
struct ib_mad_recv_wc *rwc)
{
- return ((struct ib_mad *)(wr-send_buf.mad))-mad_hdr.mgmt_class ==
+ return ((struct ib_mad_hdr *)(wr-send_buf.mad))-mgmt_class ==
rwc-recv_buf.mad-mad_hdr.mgmt_class;
}
--
1.7.1
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 07/20] IB/mad: Convert ib_mad_private allocations from kmem_cache to kmalloc
From: Ira Weiny ira.we...@intel.com
Use the new mad_size specified by devices for the allocations and DMA maps.
kmalloc is more flexible to support devices with different sized MADs and
research and testing showed that the current use of kmem_cache does not provide
performance benefits over kmalloc.
Signed-off-by: Ira Weiny ira.we...@intel.com
---
drivers/infiniband/core/mad.c | 73 +
1 files changed, 30 insertions(+), 43 deletions(-)
diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
index 031ae6f..b1ec225 100644
--- a/drivers/infiniband/core/mad.c
+++ b/drivers/infiniband/core/mad.c
@@ -59,8 +59,6 @@ MODULE_PARM_DESC(send_queue_size, Size of send queue in
number of work requests
module_param_named(recv_queue_size, mad_recvq_size, int, 0444);
MODULE_PARM_DESC(recv_queue_size, Size of receive queue in number of work
requests);
-static struct kmem_cache *ib_mad_cache;
-
static struct list_head ib_mad_port_list;
static u32 ib_mad_client_id = 0;
@@ -717,6 +715,13 @@ static void build_smp_wc(struct ib_qp *qp,
wc-port_num = port_num;
}
+static struct ib_mad_private *alloc_mad_priv(struct ib_device *dev)
+{
+ return (kmalloc(sizeof(struct ib_mad_private_header) +
+ sizeof(struct ib_grh) +
+ dev-attributes.mad_size, GFP_ATOMIC));
+}
+
/*
* Return 0 if SMP is to be sent
* Return 1 if SMP was consumed locally (whether or not solicited)
@@ -771,7 +776,8 @@ static int handle_outgoing_dr_smp(struct
ib_mad_agent_private *mad_agent_priv,
}
local-mad_priv = NULL;
local-recv_mad_agent = NULL;
- mad_priv = kmem_cache_alloc(ib_mad_cache, GFP_ATOMIC);
+
+ mad_priv = alloc_mad_priv(mad_agent_priv-agent.device);
if (!mad_priv) {
ret = -ENOMEM;
dev_err(device-dev, No memory for local response MAD\n);
@@ -801,10 +807,10 @@ static int handle_outgoing_dr_smp(struct
ib_mad_agent_private *mad_agent_priv,
*/
atomic_inc(mad_agent_priv-refcount);
} else
- kmem_cache_free(ib_mad_cache, mad_priv);
+ kfree(mad_priv);
break;
case IB_MAD_RESULT_SUCCESS | IB_MAD_RESULT_CONSUMED:
- kmem_cache_free(ib_mad_cache, mad_priv);
+ kfree(mad_priv);
break;
case IB_MAD_RESULT_SUCCESS:
/* Treat like an incoming receive MAD */
@@ -820,14 +826,14 @@ static int handle_outgoing_dr_smp(struct
ib_mad_agent_private *mad_agent_priv,
* No receiving agent so drop packet and
* generate send completion.
*/
- kmem_cache_free(ib_mad_cache, mad_priv);
+ kfree(mad_priv);
break;
}
local-mad_priv = mad_priv;
local-recv_mad_agent = recv_mad_agent;
break;
default:
- kmem_cache_free(ib_mad_cache, mad_priv);
+ kfree(mad_priv);
kfree(local);
ret = -EINVAL;
goto out;
@@ -1237,7 +1243,7 @@ void ib_free_recv_mad(struct ib_mad_recv_wc *mad_recv_wc)
recv_wc);
priv = container_of(mad_priv_hdr, struct ib_mad_private,
header);
- kmem_cache_free(ib_mad_cache, priv);
+ kfree(priv);
}
}
EXPORT_SYMBOL(ib_free_recv_mad);
@@ -1924,6 +1930,11 @@ static void ib_mad_complete_recv(struct
ib_mad_agent_private *mad_agent_priv,
}
}
+static size_t mad_recv_buf_size(struct ib_device *dev)
+{
+ return(sizeof(struct ib_grh) + dev-attributes.mad_size);
+}
+
static bool generate_unmatched_resp(struct ib_mad_private *recv,
struct ib_mad_private *response)
{
@@ -1964,8 +1975,7 @@ static void ib_mad_recv_done_handler(struct
ib_mad_port_private *port_priv,
recv = container_of(mad_priv_hdr, struct ib_mad_private, header);
ib_dma_unmap_single(port_priv-device,
recv-header.mapping,
- sizeof(struct ib_mad_private) -
- sizeof(struct ib_mad_private_header),
+ mad_recv_buf_size(port_priv-device),
DMA_FROM_DEVICE);
/* Setup MAD receive work completion from normal work completion */
@@ -1982,7 +1992,7 @@ static void ib_mad_recv_done_handler(struct
ib_mad_port_private *port_priv,
if (!validate_mad(recv-mad.mad.mad_hdr, qp_info-qp-qp_num))
goto out;
- response = kmem_cache_alloc(ib_mad_cache, GFP_KERNEL);
+ response = alloc_mad_priv(port_priv-device);
if (!response) {
dev_err(port_priv-device-dev,
[PATCH 16/20] IB/mad: Add Intel Omni-Path Architecture defines
From: Ira Weiny ira.we...@intel.com
OPA_MIN_CLASS_VERSION -- OPA Class versions are 0x80
OPA_SMP_CLASS_VERSION -- Defined at 0x80
OPA_MGMT_BASE_VERSION -- Defined at 0x80
Increase max management version to accommodate OPA
Signed-off-by: Ira Weiny ira.we...@intel.com
---
drivers/infiniband/core/mad_priv.h |4 +++-
include/rdma/ib_mad.h |5 -
2 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/core/mad_priv.h
b/drivers/infiniband/core/mad_priv.h
index d71ddcc..a7041b0 100644
--- a/drivers/infiniband/core/mad_priv.h
+++ b/drivers/infiniband/core/mad_priv.h
@@ -56,11 +56,13 @@
/* Registration table sizes */
#define MAX_MGMT_CLASS 80
-#define MAX_MGMT_VERSION 8
+#define MAX_MGMT_VERSION 0x83
#define MAX_MGMT_OUI 8
#define MAX_MGMT_VENDOR_RANGE2 (IB_MGMT_CLASS_VENDOR_RANGE2_END - \
IB_MGMT_CLASS_VENDOR_RANGE2_START + 1)
+#define OPA_MIN_CLASS_VERSION 0x80
+
struct ib_mad_list_head {
struct list_head list;
struct ib_mad_queue *mad_queue;
diff --git a/include/rdma/ib_mad.h b/include/rdma/ib_mad.h
index 80e7cf4..8938f1e 100644
--- a/include/rdma/ib_mad.h
+++ b/include/rdma/ib_mad.h
@@ -42,8 +42,11 @@
#include rdma/ib_verbs.h
#include uapi/rdma/ib_user_mad.h
-/* Management base version */
+/* Management base versions */
#define IB_MGMT_BASE_VERSION 1
+#define OPA_MGMT_BASE_VERSION 0x80
+
+#define OPA_SMP_CLASS_VERSION 0x80
/* Management classes */
#define IB_MGMT_CLASS_SUBN_LID_ROUTED 0x01
--
1.7.1
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 09/20] IB/mad: Add helper function for smi_handle_dr_smp_recv
From: Ira Weiny ira.we...@intel.com
This helper function will be used for processing both IB and OPA SMP recvs.
Signed-off-by: Ira Weiny ira.we...@intel.com
---
drivers/infiniband/core/smi.c | 80 -
1 files changed, 47 insertions(+), 33 deletions(-)
diff --git a/drivers/infiniband/core/smi.c b/drivers/infiniband/core/smi.c
index 3bac6e6..24670de 100644
--- a/drivers/infiniband/core/smi.c
+++ b/drivers/infiniband/core/smi.c
@@ -137,91 +137,105 @@ enum smi_action smi_handle_dr_smp_send(struct ib_smp
*smp,
smp-dr_slid == IB_LID_PERMISSIVE);
}
-/*
- * Adjust information for a received SMP
- * Return 0 if the SMP should be dropped
- */
-enum smi_action smi_handle_dr_smp_recv(struct ib_smp *smp, u8 node_type,
- int port_num, int phys_port_cnt)
+static inline
+enum smi_action __smi_handle_dr_smp_recv(u8 node_type, int port_num,
+int phys_port_cnt,
+u8 *hop_ptr, u8 hop_cnt,
+u8 *initial_path,
+u8 *return_path,
+u8 direction,
+int dr_dlid_is_permissive,
+int dr_slid_is_permissive)
{
- u8 hop_ptr, hop_cnt;
-
- hop_ptr = smp-hop_ptr;
- hop_cnt = smp-hop_cnt;
-
/* See section 14.2.2.2, Vol 1 IB spec */
/* C14-6 -- valid hop_cnt values are from 0 to 63 */
if (hop_cnt = IB_SMP_MAX_PATH_HOPS)
return IB_SMI_DISCARD;
- if (!ib_get_smp_direction(smp)) {
+ if (!direction) {
/* C14-9:1 -- sender should have incremented hop_ptr */
- if (hop_cnt hop_ptr == 0)
+ if (hop_cnt *hop_ptr == 0)
return IB_SMI_DISCARD;
/* C14-9:2 -- intermediate hop */
- if (hop_ptr hop_ptr hop_cnt) {
+ if (*hop_ptr *hop_ptr hop_cnt) {
if (node_type != RDMA_NODE_IB_SWITCH)
return IB_SMI_DISCARD;
- smp-return_path[hop_ptr] = port_num;
- /* smp-hop_ptr updated when sending */
- return (smp-initial_path[hop_ptr+1] = phys_port_cnt ?
+ return_path[*hop_ptr] = port_num;
+ /* hop_ptr updated when sending */
+ return (initial_path[*hop_ptr+1] = phys_port_cnt ?
IB_SMI_HANDLE : IB_SMI_DISCARD);
}
/* C14-9:3 -- We're at the end of the DR segment of path */
- if (hop_ptr == hop_cnt) {
+ if (*hop_ptr == hop_cnt) {
if (hop_cnt)
- smp-return_path[hop_ptr] = port_num;
- /* smp-hop_ptr updated when sending */
+ return_path[*hop_ptr] = port_num;
+ /* hop_ptr updated when sending */
return (node_type == RDMA_NODE_IB_SWITCH ||
- smp-dr_dlid == IB_LID_PERMISSIVE ?
+ dr_dlid_is_permissive ?
IB_SMI_HANDLE : IB_SMI_DISCARD);
}
/* C14-9:4 -- hop_ptr = hop_cnt + 1 - give to SMA/SM */
/* C14-9:5 -- fail unreasonable hop pointer */
- return (hop_ptr == hop_cnt + 1 ? IB_SMI_HANDLE :
IB_SMI_DISCARD);
+ return (*hop_ptr == hop_cnt + 1 ? IB_SMI_HANDLE :
IB_SMI_DISCARD);
} else {
/* C14-13:1 */
- if (hop_cnt hop_ptr == hop_cnt + 1) {
- smp-hop_ptr--;
- return (smp-return_path[smp-hop_ptr] ==
+ if (hop_cnt *hop_ptr == hop_cnt + 1) {
+ (*hop_ptr)--;
+ return (return_path[*hop_ptr] ==
port_num ? IB_SMI_HANDLE : IB_SMI_DISCARD);
}
/* C14-13:2 */
- if (2 = hop_ptr hop_ptr = hop_cnt) {
+ if (2 = *hop_ptr *hop_ptr = hop_cnt) {
if (node_type != RDMA_NODE_IB_SWITCH)
return IB_SMI_DISCARD;
- /* smp-hop_ptr updated when sending */
- return (smp-return_path[hop_ptr-1] = phys_port_cnt ?
+ /* hop_ptr updated when sending */
+ return (return_path[*hop_ptr-1] = phys_port_cnt ?
IB_SMI_HANDLE : IB_SMI_DISCARD);
}
/* C14-13:3 -- We're at the end of the DR segment of path */
- if (hop_ptr == 1) {
- if (smp-dr_slid == IB_LID_PERMISSIVE) {
[PATCH 15/20] IB/mad: Create jumbo_mad data structures
From: Ira Weiny ira.we...@intel.com
Define jumbo_mad and jumbo_rmpp_mad
Create an RMPP Base header to share between ib_rmpp_mad and jumbo_rmpp_mad
Update code to use the new structures.
Signed-off-by: Ira Weiny ira.we...@intel.com
---
Changes from RFC:
add JUMBO_MGMT_MAD_SIZE
remove jumbo_mad_private structure
drivers/infiniband/core/mad.c | 18 +++---
drivers/infiniband/core/mad_priv.h |2 +
drivers/infiniband/core/mad_rmpp.c | 120 ++--
drivers/infiniband/core/user_mad.c | 16 +++---
include/rdma/ib_mad.h | 26 +++-
5 files changed, 103 insertions(+), 79 deletions(-)
diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
index 6a6efd6..dc78040 100644
--- a/drivers/infiniband/core/mad.c
+++ b/drivers/infiniband/core/mad.c
@@ -883,7 +883,7 @@ static int alloc_send_rmpp_list(struct
ib_mad_send_wr_private *send_wr,
gfp_t gfp_mask)
{
struct ib_mad_send_buf *send_buf = send_wr-send_buf;
- struct ib_rmpp_mad *rmpp_mad = send_buf-mad;
+ struct ib_rmpp_base *rmpp_base = send_buf-mad;
struct ib_rmpp_segment *seg = NULL;
int left, seg_size, pad;
@@ -909,10 +909,10 @@ static int alloc_send_rmpp_list(struct
ib_mad_send_wr_private *send_wr,
if (pad)
memset(seg-data + seg_size - pad, 0, pad);
- rmpp_mad-rmpp_hdr.rmpp_version = send_wr-mad_agent_priv-
+ rmpp_base-rmpp_hdr.rmpp_version = send_wr-mad_agent_priv-
agent.rmpp_version;
- rmpp_mad-rmpp_hdr.rmpp_type = IB_MGMT_RMPP_TYPE_DATA;
- ib_set_rmpp_flags(rmpp_mad-rmpp_hdr, IB_MGMT_RMPP_FLAG_ACTIVE);
+ rmpp_base-rmpp_hdr.rmpp_type = IB_MGMT_RMPP_TYPE_DATA;
+ ib_set_rmpp_flags(rmpp_base-rmpp_hdr, IB_MGMT_RMPP_FLAG_ACTIVE);
send_wr-cur_seg = container_of(send_wr-rmpp_list.next,
struct ib_rmpp_segment, list);
@@ -1748,14 +1748,14 @@ out:
static int is_rmpp_data_mad(struct ib_mad_agent_private *mad_agent_priv,
struct ib_mad_hdr *mad_hdr)
{
- struct ib_rmpp_mad *rmpp_mad;
+ struct ib_rmpp_base *rmpp_base;
- rmpp_mad = (struct ib_rmpp_mad *)mad_hdr;
+ rmpp_base = (struct ib_rmpp_base *)mad_hdr;
return !mad_agent_priv-agent.rmpp_version ||
!ib_mad_kernel_rmpp_agent(mad_agent_priv-agent) ||
- !(ib_get_rmpp_flags(rmpp_mad-rmpp_hdr)
+ !(ib_get_rmpp_flags(rmpp_base-rmpp_hdr)
IB_MGMT_RMPP_FLAG_ACTIVE) ||
- (rmpp_mad-rmpp_hdr.rmpp_type == IB_MGMT_RMPP_TYPE_DATA);
+ (rmpp_base-rmpp_hdr.rmpp_type == IB_MGMT_RMPP_TYPE_DATA);
}
static inline int rcv_has_same_class(struct ib_mad_send_wr_private *wr,
@@ -1897,7 +1897,7 @@ static void ib_mad_complete_recv(struct
ib_mad_agent_private *mad_agent_priv,
spin_unlock_irqrestore(mad_agent_priv-lock, flags);
if (!ib_mad_kernel_rmpp_agent(mad_agent_priv-agent)
ib_is_mad_class_rmpp(mad_recv_wc-recv_buf.mad-mad_hdr.mgmt_class)
- (ib_get_rmpp_flags(((struct ib_rmpp_mad
*)mad_recv_wc-recv_buf.mad)-rmpp_hdr)
+ (ib_get_rmpp_flags(((struct ib_rmpp_base
*)mad_recv_wc-recv_buf.mad)-rmpp_hdr)
IB_MGMT_RMPP_FLAG_ACTIVE)) {
/* user rmpp is in effect
* and this is an active RMPP MAD
diff --git a/drivers/infiniband/core/mad_priv.h
b/drivers/infiniband/core/mad_priv.h
index d1a0b0e..d71ddcc 100644
--- a/drivers/infiniband/core/mad_priv.h
+++ b/drivers/infiniband/core/mad_priv.h
@@ -80,6 +80,8 @@ struct ib_mad_private {
struct ib_mad mad;
struct ib_rmpp_mad rmpp_mad;
struct ib_smp smp;
+ struct jumbo_mad jumbo_mad;
+ struct jumbo_rmpp_mad jumbo_rmpp_mad;
} mad;
} __attribute__ ((packed));
diff --git a/drivers/infiniband/core/mad_rmpp.c
b/drivers/infiniband/core/mad_rmpp.c
index 2379e2d..7184530 100644
--- a/drivers/infiniband/core/mad_rmpp.c
+++ b/drivers/infiniband/core/mad_rmpp.c
@@ -111,10 +111,10 @@ void ib_cancel_rmpp_recvs(struct ib_mad_agent_private
*agent)
}
static void format_ack(struct ib_mad_send_buf *msg,
- struct ib_rmpp_mad *data,
+ struct ib_rmpp_base *data,
struct mad_rmpp_recv *rmpp_recv)
{
- struct ib_rmpp_mad *ack = msg-mad;
+ struct ib_rmpp_base *ack = msg-mad;
unsigned long flags;
memcpy(ack, data-mad_hdr, msg-hdr_len);
@@ -144,7 +144,7 @@ static void ack_recv(struct mad_rmpp_recv *rmpp_recv,
if (IS_ERR(msg))
return;
- format_ack(msg, (struct ib_rmpp_mad *) recv_wc-recv_buf.mad,
rmpp_recv);
+
[PATCH 11/20] IB/mad: Add helper function for SMI processing
From: Ira Weiny ira.we...@intel.com
This helper function will be used for processing both IB and OPA SMPs.
Signed-off-by: Ira Weiny ira.we...@intel.com
---
drivers/infiniband/core/mad.c | 85 +++-
1 files changed, 49 insertions(+), 36 deletions(-)
diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
index b1ec225..4e70e15 100644
--- a/drivers/infiniband/core/mad.c
+++ b/drivers/infiniband/core/mad.c
@@ -1930,6 +1930,52 @@ static void ib_mad_complete_recv(struct
ib_mad_agent_private *mad_agent_priv,
}
}
+enum smi_action handle_ib_smi(struct ib_mad_port_private *port_priv,
+ struct ib_mad_qp_info *qp_info,
+ struct ib_wc *wc,
+ int port_num,
+ struct ib_mad_private *recv,
+ struct ib_mad_private *response)
+{
+ enum smi_forward_action retsmi;
+
+ if (smi_handle_dr_smp_recv(recv-mad.smp,
+ port_priv-device-node_type,
+ port_num,
+ port_priv-device-phys_port_cnt) ==
+ IB_SMI_DISCARD)
+ return IB_SMI_DISCARD;
+
+ retsmi = smi_check_forward_dr_smp(recv-mad.smp);
+ if (retsmi == IB_SMI_LOCAL)
+ return IB_SMI_HANDLE;
+
+ if (retsmi == IB_SMI_SEND) { /* don't forward */
+ if (smi_handle_dr_smp_send(recv-mad.smp,
+ port_priv-device-node_type,
+ port_num) == IB_SMI_DISCARD)
+ return IB_SMI_DISCARD;
+
+ if (smi_check_local_smp(recv-mad.smp, port_priv-device) ==
IB_SMI_DISCARD)
+ return IB_SMI_DISCARD;
+ } else if (port_priv-device-node_type == RDMA_NODE_IB_SWITCH) {
+ /* forward case for switches */
+ memcpy(response, recv, sizeof(*response));
+ response-header.recv_wc.wc = response-header.wc;
+ response-header.recv_wc.recv_buf.mad = response-mad.mad;
+ response-header.recv_wc.recv_buf.grh = response-grh;
+
+ agent_send_response(response-mad.mad,
+ response-grh, wc,
+ port_priv-device,
+ smi_get_fwd_port(recv-mad.smp),
+ qp_info-qp-qp_num);
+
+ return IB_SMI_DISCARD;
+ }
+ return IB_SMI_HANDLE;
+}
+
static size_t mad_recv_buf_size(struct ib_device *dev)
{
return(sizeof(struct ib_grh) + dev-attributes.mad_size);
@@ -2006,45 +2052,12 @@ static void ib_mad_recv_done_handler(struct
ib_mad_port_private *port_priv,
if (recv-mad.mad.mad_hdr.mgmt_class ==
IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE) {
- enum smi_forward_action retsmi;
-
- if (smi_handle_dr_smp_recv(recv-mad.smp,
- port_priv-device-node_type,
- port_num,
- port_priv-device-phys_port_cnt) ==
- IB_SMI_DISCARD)
+ if (handle_ib_smi(port_priv, qp_info, wc, port_num, recv,
+ response)
+ == IB_SMI_DISCARD)
goto out;
-
- retsmi = smi_check_forward_dr_smp(recv-mad.smp);
- if (retsmi == IB_SMI_LOCAL)
- goto local;
-
- if (retsmi == IB_SMI_SEND) { /* don't forward */
- if (smi_handle_dr_smp_send(recv-mad.smp,
- port_priv-device-node_type,
- port_num) == IB_SMI_DISCARD)
- goto out;
-
- if (smi_check_local_smp(recv-mad.smp,
port_priv-device) == IB_SMI_DISCARD)
- goto out;
- } else if (port_priv-device-node_type == RDMA_NODE_IB_SWITCH)
{
- /* forward case for switches */
- memcpy(response, recv, sizeof(*response));
- response-header.recv_wc.wc = response-header.wc;
- response-header.recv_wc.recv_buf.mad =
response-mad.mad;
- response-header.recv_wc.recv_buf.grh = response-grh;
-
- agent_send_response(response-mad.mad,
- response-grh, wc,
- port_priv-device,
- smi_get_fwd_port(recv-mad.smp),
- qp_info-qp-qp_num);
-
- goto out;
- }
}
-local:
/* Give driver
[PATCH 03/20] IB/mad: Change validate_mad signature to take ib_mad_hdr rather than ib_mad
From: Ira Weiny ira.we...@intel.com
validate_mad only needs access to the MAD header
Signed-off-by: Ira Weiny ira.we...@intel.com
---
drivers/infiniband/core/mad.c | 12 ++--
1 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
index 4673262..99b 100644
--- a/drivers/infiniband/core/mad.c
+++ b/drivers/infiniband/core/mad.c
@@ -1708,20 +1708,20 @@ out:
return mad_agent;
}
-static int validate_mad(struct ib_mad *mad, u32 qp_num)
+static int validate_mad(struct ib_mad_hdr *mad_hdr, u32 qp_num)
{
int valid = 0;
/* Make sure MAD base version is understood */
- if (mad-mad_hdr.base_version != IB_MGMT_BASE_VERSION) {
+ if (mad_hdr-base_version != IB_MGMT_BASE_VERSION) {
pr_err(MAD received with unsupported base version %d\n,
- mad-mad_hdr.base_version);
+ mad_hdr-base_version);
goto out;
}
/* Filter SMI packets sent to other than QP0 */
- if ((mad-mad_hdr.mgmt_class == IB_MGMT_CLASS_SUBN_LID_ROUTED) ||
- (mad-mad_hdr.mgmt_class == IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE)) {
+ if ((mad_hdr-mgmt_class == IB_MGMT_CLASS_SUBN_LID_ROUTED) ||
+ (mad_hdr-mgmt_class == IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE)) {
if (qp_num == 0)
valid = 1;
} else {
@@ -1979,7 +1979,7 @@ static void ib_mad_recv_done_handler(struct
ib_mad_port_private *port_priv,
snoop_recv(qp_info, recv-header.recv_wc, IB_MAD_SNOOP_RECVS);
/* Validate MAD */
- if (!validate_mad(recv-mad.mad, qp_info-qp-qp_num))
+ if (!validate_mad(recv-mad.mad.mad_hdr, qp_info-qp-qp_num))
goto out;
response = kmem_cache_alloc(ib_mad_cache, GFP_KERNEL);
--
1.7.1
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 14/20] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
From: Ira Weiny ira.we...@intel.com
Add a device capability flag to flag OPA MAD support on devices.
Signed-off-by: Ira Weiny ira.we...@intel.com
---
Changes from RFC:
Changed flag to indicate OPA MAD base version support
Added device_cap_flags2 as device_cap_flags bits are limited
include/rdma/ib_verbs.h |5 +
1 files changed, 5 insertions(+), 0 deletions(-)
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 5024f1f..83b39d5 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -128,6 +128,10 @@ enum ib_device_cap_flags {
IB_DEVICE_ON_DEMAND_PAGING = (131),
};
+enum ib_device_cap_flags2 {
+ IB_DEVICE_OPA_MAD_SUPPORT = 1
+};
+
enum ib_signature_prot_cap {
IB_PROT_T10DIF_TYPE_1 = 1,
IB_PROT_T10DIF_TYPE_2 = 1 1,
@@ -210,6 +214,7 @@ struct ib_device_attr {
int sig_prot_cap;
int sig_guard_cap;
struct ib_odp_caps odp_caps;
+ u64 device_cap_flags2;
u32 mad_size;
};
--
1.7.1
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 08/20] IB/mad: Add helper function for smi_handle_dr_smp_send
From: Ira Weiny ira.we...@intel.com
This helper function will be used for processing both IB and OPA SMP sends.
Signed-off-by: Ira Weiny ira.we...@intel.com
---
drivers/infiniband/core/smi.c | 81 -
1 files changed, 47 insertions(+), 34 deletions(-)
diff --git a/drivers/infiniband/core/smi.c b/drivers/infiniband/core/smi.c
index 5855e44..3bac6e6 100644
--- a/drivers/infiniband/core/smi.c
+++ b/drivers/infiniband/core/smi.c
@@ -39,84 +39,81 @@
#include rdma/ib_smi.h
#include smi.h
-/*
- * Fixup a directed route SMP for sending
- * Return 0 if the SMP should be discarded
- */
-enum smi_action smi_handle_dr_smp_send(struct ib_smp *smp,
- u8 node_type, int port_num)
+static inline
+enum smi_action __smi_handle_dr_smp_send(u8 node_type, int port_num,
+u8 *hop_ptr, u8 hop_cnt,
+u8 *initial_path,
+u8 *return_path,
+u8 direction,
+int dr_dlid_is_permissive,
+int dr_slid_is_permissive)
{
- u8 hop_ptr, hop_cnt;
-
- hop_ptr = smp-hop_ptr;
- hop_cnt = smp-hop_cnt;
-
/* See section 14.2.2.2, Vol 1 IB spec */
/* C14-6 -- valid hop_cnt values are from 0 to 63 */
if (hop_cnt = IB_SMP_MAX_PATH_HOPS)
return IB_SMI_DISCARD;
- if (!ib_get_smp_direction(smp)) {
+ if (!direction) {
/* C14-9:1 */
- if (hop_cnt hop_ptr == 0) {
- smp-hop_ptr++;
- return (smp-initial_path[smp-hop_ptr] ==
+ if (hop_cnt *hop_ptr == 0) {
+ (*hop_ptr)++;
+ return (initial_path[*hop_ptr] ==
port_num ? IB_SMI_HANDLE : IB_SMI_DISCARD);
}
/* C14-9:2 */
- if (hop_ptr hop_ptr hop_cnt) {
+ if (*hop_ptr *hop_ptr hop_cnt) {
if (node_type != RDMA_NODE_IB_SWITCH)
return IB_SMI_DISCARD;
- /* smp-return_path set when received */
- smp-hop_ptr++;
- return (smp-initial_path[smp-hop_ptr] ==
+ /* return_path set when received */
+ (*hop_ptr)++;
+ return (initial_path[*hop_ptr] ==
port_num ? IB_SMI_HANDLE : IB_SMI_DISCARD);
}
/* C14-9:3 -- We're at the end of the DR segment of path */
- if (hop_ptr == hop_cnt) {
- /* smp-return_path set when received */
- smp-hop_ptr++;
+ if (*hop_ptr == hop_cnt) {
+ /* return_path set when received */
+ (*hop_ptr)++;
return (node_type == RDMA_NODE_IB_SWITCH ||
- smp-dr_dlid == IB_LID_PERMISSIVE ?
+ dr_dlid_is_permissive ?
IB_SMI_HANDLE : IB_SMI_DISCARD);
}
/* C14-9:4 -- hop_ptr = hop_cnt + 1 - give to SMA/SM */
/* C14-9:5 -- Fail unreasonable hop pointer */
- return (hop_ptr == hop_cnt + 1 ? IB_SMI_HANDLE :
IB_SMI_DISCARD);
+ return (*hop_ptr == hop_cnt + 1 ? IB_SMI_HANDLE :
IB_SMI_DISCARD);
} else {
/* C14-13:1 */
- if (hop_cnt hop_ptr == hop_cnt + 1) {
- smp-hop_ptr--;
- return (smp-return_path[smp-hop_ptr] ==
+ if (hop_cnt *hop_ptr == hop_cnt + 1) {
+ (*hop_ptr)--;
+ return (return_path[*hop_ptr] ==
port_num ? IB_SMI_HANDLE : IB_SMI_DISCARD);
}
/* C14-13:2 */
- if (2 = hop_ptr hop_ptr = hop_cnt) {
+ if (2 = *hop_ptr *hop_ptr = hop_cnt) {
if (node_type != RDMA_NODE_IB_SWITCH)
return IB_SMI_DISCARD;
- smp-hop_ptr--;
- return (smp-return_path[smp-hop_ptr] ==
+ (*hop_ptr)--;
+ return (return_path[*hop_ptr] ==
port_num ? IB_SMI_HANDLE : IB_SMI_DISCARD);
}
/* C14-13:3 -- at the end of the DR segment of path */
- if (hop_ptr == 1) {
- smp-hop_ptr--;
+ if (*hop_ptr == 1) {
+ (*hop_ptr)--;
/* C14-13:3 -- SMPs destined for SM shouldn't be here */
return (node_type == RDMA_NODE_IB_SWITCH ||
-
[PATCH 10/20] IB/mad: Add helper function for smi_check_forward_dr_smp
From: Ira Weiny ira.we...@intel.com
This helper function will be used for processing both IB and OPA SMPs.
Signed-off-by: Ira Weiny ira.we...@intel.com
---
drivers/infiniband/core/smi.c | 26 +-
1 files changed, 17 insertions(+), 9 deletions(-)
diff --git a/drivers/infiniband/core/smi.c b/drivers/infiniband/core/smi.c
index 24670de..8a5fb1d 100644
--- a/drivers/infiniband/core/smi.c
+++ b/drivers/infiniband/core/smi.c
@@ -236,21 +236,20 @@ enum smi_action smi_handle_dr_smp_recv(struct ib_smp
*smp, u8 node_type,
smp-dr_slid == IB_LID_PERMISSIVE);
}
-enum smi_forward_action smi_check_forward_dr_smp(struct ib_smp *smp)
+static inline
+enum smi_forward_action __smi_check_forward_dr_smp(u8 hop_ptr, u8 hop_cnt,
+ u8 direction,
+ int dr_dlid_is_permissive,
+ int dr_slid_is_permissive)
{
- u8 hop_ptr, hop_cnt;
-
- hop_ptr = smp-hop_ptr;
- hop_cnt = smp-hop_cnt;
-
- if (!ib_get_smp_direction(smp)) {
+ if (!direction) {
/* C14-9:2 -- intermediate hop */
if (hop_ptr hop_ptr hop_cnt)
return IB_SMI_FORWARD;
/* C14-9:3 -- at the end of the DR segment of path */
if (hop_ptr == hop_cnt)
- return (smp-dr_dlid == IB_LID_PERMISSIVE ?
+ return (dr_dlid_is_permissive ?
IB_SMI_SEND : IB_SMI_LOCAL);
/* C14-9:4 -- hop_ptr = hop_cnt + 1 - give to SMA/SM */
@@ -263,10 +262,19 @@ enum smi_forward_action smi_check_forward_dr_smp(struct
ib_smp *smp)
/* C14-13:3 -- at the end of the DR segment of path */
if (hop_ptr == 1)
- return (smp-dr_slid != IB_LID_PERMISSIVE ?
+ return (dr_slid_is_permissive ?
IB_SMI_SEND : IB_SMI_LOCAL);
}
return IB_SMI_LOCAL;
+
+}
+
+enum smi_forward_action smi_check_forward_dr_smp(struct ib_smp *smp)
+{
+ return __smi_check_forward_dr_smp(smp-hop_ptr, smp-hop_cnt,
+ ib_get_smp_direction(smp),
+ smp-dr_dlid == IB_LID_PERMISSIVE,
+ smp-dr_slid != IB_LID_PERMISSIVE);
}
/*
--
1.7.1
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 20/20] IB/mad: Implement Intel Omni-Path Architecture MAD processing
From: Ira Weiny ira.we...@intel.com
OPA SMP packets must carry a valid pkey
process wc.pkey_index returned by agents for response.
Handle variable length OPA MADs based on the Base Version
Support is provided by:
* Adjusting the 'fake' WC for locally routed SMP's to represent the
proper incoming byte_len
* out_mad_size is used from the local HCA agents
1) when sending agent responses on the wire
2) when passing responses through the local_completions function
NOTE: wc.byte_len includes the GRH length and therefore is different from the
in_mad_size specified to the local HCA agents. out_mad_size should _not_
include the GRH length as it is added by the verbs layer and is not part
of MAD processing.
Signed-off-by: Ira Weiny ira.we...@intel.com
---
Changes from RFC:
No longer use kmem_cache for allocations; use kmalloc instead
Remove supports_jumbo_mads flag
Use mad_size as specified by the devices
Patch clean up and code simplification
remove jumbo_mad_private structure
drivers/infiniband/core/agent.c| 26 -
drivers/infiniband/core/agent.h|2 +-
drivers/infiniband/core/mad.c | 230 +---
drivers/infiniband/core/mad_priv.h |1 +
drivers/infiniband/core/mad_rmpp.c | 30 +++--
drivers/infiniband/core/user_mad.c | 39 ---
6 files changed, 253 insertions(+), 75 deletions(-)
diff --git a/drivers/infiniband/core/agent.c b/drivers/infiniband/core/agent.c
index b6bd305..8dcebcc 100644
--- a/drivers/infiniband/core/agent.c
+++ b/drivers/infiniband/core/agent.c
@@ -80,13 +80,16 @@ ib_get_agent_port(struct ib_device *device, int port_num)
void agent_send_response(struct ib_mad *mad, struct ib_grh *grh,
struct ib_wc *wc, struct ib_device *device,
-int port_num, int qpn)
+int port_num, int qpn, u32 resp_mad_len)
{
struct ib_agent_port_private *port_priv;
struct ib_mad_agent *agent;
struct ib_mad_send_buf *send_buf;
struct ib_ah *ah;
+ size_t data_len;
+ size_t hdr_len;
struct ib_mad_send_wr_private *mad_send_wr;
+ u8 base_version;
if (device-node_type == RDMA_NODE_IB_SWITCH)
port_priv = ib_get_agent_port(device, 0);
@@ -106,16 +109,29 @@ void agent_send_response(struct ib_mad *mad, struct
ib_grh *grh,
return;
}
+ /* base version determines MAD size */
+ base_version = mad-mad_hdr.base_version;
+ if (base_version == OPA_MGMT_BASE_VERSION) {
+ data_len = resp_mad_len - JUMBO_MGMT_MAD_HDR;
+ hdr_len = JUMBO_MGMT_MAD_HDR;
+ } else {
+ data_len = IB_MGMT_MAD_DATA;
+ hdr_len = IB_MGMT_MAD_HDR;
+ }
+
send_buf = ib_create_send_mad(agent, wc-src_qp, wc-pkey_index, 0,
- IB_MGMT_MAD_HDR, IB_MGMT_MAD_DATA,
- GFP_KERNEL,
- IB_MGMT_BASE_VERSION);
+ hdr_len, data_len, GFP_KERNEL,
+ base_version);
if (IS_ERR(send_buf)) {
dev_err(device-dev, ib_create_send_mad error\n);
goto err1;
}
- memcpy(send_buf-mad, mad, sizeof *mad);
+ if (base_version == OPA_MGMT_BASE_VERSION)
+ memcpy(send_buf-mad, mad, JUMBO_MGMT_MAD_HDR + data_len);
+ else
+ memcpy(send_buf-mad, mad, sizeof(*mad));
+
send_buf-ah = ah;
if (device-node_type == RDMA_NODE_IB_SWITCH) {
diff --git a/drivers/infiniband/core/agent.h b/drivers/infiniband/core/agent.h
index 6669287..cb4081d 100644
--- a/drivers/infiniband/core/agent.h
+++ b/drivers/infiniband/core/agent.h
@@ -46,6 +46,6 @@ extern int ib_agent_port_close(struct ib_device *device, int
port_num);
extern void agent_send_response(struct ib_mad *mad, struct ib_grh *grh,
struct ib_wc *wc, struct ib_device *device,
- int port_num, int qpn);
+ int port_num, int qpn, u32 resp_mad_len);
#endif /* __AGENT_H_ */
diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
index 0179d37..1f61b3d 100644
--- a/drivers/infiniband/core/mad.c
+++ b/drivers/infiniband/core/mad.c
@@ -3,6 +3,7 @@
* Copyright (c) 2005 Intel Corporation. All rights reserved.
* Copyright (c) 2005 Mellanox Technologies Ltd. All rights reserved.
* Copyright (c) 2009 HNR Consulting. All rights reserved.
+ * Copyright (c) 2014 Intel Corporation. All rights reserved.
*
* This software is available to you under a choice of one of two
* licenses. You may choose to be licensed under the terms of the GNU
@@ -44,6 +45,7 @@
#include mad_priv.h
#include mad_rmpp.h
#include smi.h
[PATCH 13/20] IB/mad: Add base version parameter to ib_create_send_mad
From: Ira Weiny ira.we...@intel.com
In preparation to support the new OPA MAD Base version, add a base version
parameter ib_create_send_mad and set it to IB_MGMT_BASE_VERSION for current
users.
Definition of the new base version and it's processing will occur in later
patches.
Signed-off-by: Ira Weiny ira.we...@intel.com
---
drivers/infiniband/core/agent.c |3 ++-
drivers/infiniband/core/cm.c|6 --
drivers/infiniband/core/mad.c |3 ++-
drivers/infiniband/core/mad_rmpp.c |6 --
drivers/infiniband/core/sa_query.c |3 ++-
drivers/infiniband/core/user_mad.c |3 ++-
drivers/infiniband/hw/mlx4/mad.c|3 ++-
drivers/infiniband/hw/mthca/mthca_mad.c |3 ++-
drivers/infiniband/hw/qib/qib_iba7322.c |3 ++-
drivers/infiniband/hw/qib/qib_mad.c |3 ++-
drivers/infiniband/ulp/srpt/ib_srpt.c |3 ++-
include/rdma/ib_mad.h |4 +++-
12 files changed, 29 insertions(+), 14 deletions(-)
diff --git a/drivers/infiniband/core/agent.c b/drivers/infiniband/core/agent.c
index f6d2961..b6bd305 100644
--- a/drivers/infiniband/core/agent.c
+++ b/drivers/infiniband/core/agent.c
@@ -108,7 +108,8 @@ void agent_send_response(struct ib_mad *mad, struct ib_grh
*grh,
send_buf = ib_create_send_mad(agent, wc-src_qp, wc-pkey_index, 0,
IB_MGMT_MAD_HDR, IB_MGMT_MAD_DATA,
- GFP_KERNEL);
+ GFP_KERNEL,
+ IB_MGMT_BASE_VERSION);
if (IS_ERR(send_buf)) {
dev_err(device-dev, ib_create_send_mad error\n);
goto err1;
diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c
index e28a494..5767781 100644
--- a/drivers/infiniband/core/cm.c
+++ b/drivers/infiniband/core/cm.c
@@ -267,7 +267,8 @@ static int cm_alloc_msg(struct cm_id_private *cm_id_priv,
m = ib_create_send_mad(mad_agent, cm_id_priv-id.remote_cm_qpn,
cm_id_priv-av.pkey_index,
0, IB_MGMT_MAD_HDR, IB_MGMT_MAD_DATA,
- GFP_ATOMIC);
+ GFP_ATOMIC,
+ IB_MGMT_BASE_VERSION);
if (IS_ERR(m)) {
ib_destroy_ah(ah);
return PTR_ERR(m);
@@ -297,7 +298,8 @@ static int cm_alloc_response_msg(struct cm_port *port,
m = ib_create_send_mad(port-mad_agent, 1, mad_recv_wc-wc-pkey_index,
0, IB_MGMT_MAD_HDR, IB_MGMT_MAD_DATA,
- GFP_ATOMIC);
+ GFP_ATOMIC,
+ IB_MGMT_BASE_VERSION);
if (IS_ERR(m)) {
ib_destroy_ah(ah);
return PTR_ERR(m);
diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
index ec2b664..6a6efd6 100644
--- a/drivers/infiniband/core/mad.c
+++ b/drivers/infiniband/core/mad.c
@@ -930,7 +930,8 @@ struct ib_mad_send_buf * ib_create_send_mad(struct
ib_mad_agent *mad_agent,
u32 remote_qpn, u16 pkey_index,
int rmpp_active,
int hdr_len, int data_len,
- gfp_t gfp_mask)
+ gfp_t gfp_mask,
+ u8 base_version)
{
struct ib_mad_agent_private *mad_agent_priv;
struct ib_mad_send_wr_private *mad_send_wr;
diff --git a/drivers/infiniband/core/mad_rmpp.c
b/drivers/infiniband/core/mad_rmpp.c
index f37878c..2379e2d 100644
--- a/drivers/infiniband/core/mad_rmpp.c
+++ b/drivers/infiniband/core/mad_rmpp.c
@@ -139,7 +139,8 @@ static void ack_recv(struct mad_rmpp_recv *rmpp_recv,
hdr_len =
ib_get_mad_data_offset(recv_wc-recv_buf.mad-mad_hdr.mgmt_class);
msg = ib_create_send_mad(rmpp_recv-agent-agent, recv_wc-wc-src_qp,
recv_wc-wc-pkey_index, 1, hdr_len,
-0, GFP_KERNEL);
+0, GFP_KERNEL,
+IB_MGMT_BASE_VERSION);
if (IS_ERR(msg))
return;
@@ -165,7 +166,8 @@ static struct ib_mad_send_buf *alloc_response_msg(struct
ib_mad_agent *agent,
hdr_len =
ib_get_mad_data_offset(recv_wc-recv_buf.mad-mad_hdr.mgmt_class);
msg = ib_create_send_mad(agent, recv_wc-wc-src_qp,
recv_wc-wc-pkey_index, 1,
-hdr_len, 0, GFP_KERNEL);
+hdr_len, 0, GFP_KERNEL,
+IB_MGMT_BASE_VERSION);
if (IS_ERR(msg))
ib_destroy_ah(ah);
else {
diff --git a/drivers/infiniband/core/sa_query.c
b/drivers/infiniband/core/sa_query.c
index c38f030..32c3fe6
[PATCH 12/20] IB/mad: Add MAD size parameters to process_mad
From: Ira Weiny ira.we...@intel.com
In support of variable length MADs add in and out MAD size parameters to the
process_mad call.
The out MAD size parameter is passed by reference such that it can be updated
by the agent to indicate the proper response length to be sent by the MAD
stack.
The in and out MAD parameters are made generic by specifying them as
ib_mad_hdr.
All MAD sizes are set to the device MAD size specified by the devices.
Drivers are modified to verify the MAD sizes passed to them.
Signed-off-by: Ira Weiny ira.we...@intel.com
---
Changes from RFC:
Fix MLX5 compile
drivers/infiniband/core/mad.c| 29 +-
drivers/infiniband/core/sysfs.c |5 +++-
drivers/infiniband/hw/amso1100/c2_provider.c |5 +++-
drivers/infiniband/hw/cxgb3/iwch_provider.c |5 +++-
drivers/infiniband/hw/cxgb4/provider.c |7 -
drivers/infiniband/hw/ehca/ehca_sqp.c|8 ++-
drivers/infiniband/hw/ipath/ipath_mad.c |8 ++-
drivers/infiniband/hw/ipath/ipath_verbs.h|3 +-
drivers/infiniband/hw/mlx4/mad.c |9 +++-
drivers/infiniband/hw/mlx4/mlx4_ib.h |3 +-
drivers/infiniband/hw/mlx5/mad.c |8 ++-
drivers/infiniband/hw/mlx5/mlx5_ib.h |3 +-
drivers/infiniband/hw/mthca/mthca_dev.h |4 +-
drivers/infiniband/hw/mthca/mthca_mad.c |9 ++-
drivers/infiniband/hw/nes/nes_verbs.c|3 +-
drivers/infiniband/hw/ocrdma/ocrdma_ah.c |3 +-
drivers/infiniband/hw/qib/qib_mad.c |8 ++-
drivers/infiniband/hw/qib/qib_verbs.h|3 +-
include/rdma/ib_verbs.h |8 --
19 files changed, 98 insertions(+), 33 deletions(-)
diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
index 4e70e15..ec2b664 100644
--- a/drivers/infiniband/core/mad.c
+++ b/drivers/infiniband/core/mad.c
@@ -715,11 +715,12 @@ static void build_smp_wc(struct ib_qp *qp,
wc-port_num = port_num;
}
-static struct ib_mad_private *alloc_mad_priv(struct ib_device *dev)
+static struct ib_mad_private *alloc_mad_priv(struct ib_device *dev,
+size_t *mad_size)
{
+ *mad_size = dev-attributes.mad_size;
return (kmalloc(sizeof(struct ib_mad_private_header) +
- sizeof(struct ib_grh) +
- dev-attributes.mad_size, GFP_ATOMIC));
+ sizeof(struct ib_grh) + *mad_size, GFP_ATOMIC));
}
/*
@@ -741,6 +742,8 @@ static int handle_outgoing_dr_smp(struct
ib_mad_agent_private *mad_agent_priv,
u8 port_num;
struct ib_wc mad_wc;
struct ib_send_wr *send_wr = mad_send_wr-send_wr;
+ size_t in_mad_size = mad_agent_priv-agent.device-attributes.mad_size;
+ size_t out_mad_size;
if (device-node_type == RDMA_NODE_IB_SWITCH
smp-mgmt_class == IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE)
@@ -777,7 +780,7 @@ static int handle_outgoing_dr_smp(struct
ib_mad_agent_private *mad_agent_priv,
local-mad_priv = NULL;
local-recv_mad_agent = NULL;
- mad_priv = alloc_mad_priv(mad_agent_priv-agent.device);
+ mad_priv = alloc_mad_priv(mad_agent_priv-agent.device, out_mad_size);
if (!mad_priv) {
ret = -ENOMEM;
dev_err(device-dev, No memory for local response MAD\n);
@@ -792,8 +795,9 @@ static int handle_outgoing_dr_smp(struct
ib_mad_agent_private *mad_agent_priv,
/* No GRH for DR SMP */
ret = device-process_mad(device, 0, port_num, mad_wc, NULL,
- (struct ib_mad *)smp,
- (struct ib_mad *)mad_priv-mad);
+ (struct ib_mad_hdr *)smp, in_mad_size,
+ (struct ib_mad_hdr *)mad_priv-mad,
+ out_mad_size);
switch (ret)
{
case IB_MAD_RESULT_SUCCESS | IB_MAD_RESULT_REPLY:
@@ -2011,6 +2015,7 @@ static void ib_mad_recv_done_handler(struct
ib_mad_port_private *port_priv,
struct ib_mad_agent_private *mad_agent;
int port_num;
int ret = IB_MAD_RESULT_SUCCESS;
+ size_t resp_mad_size;
mad_list = (struct ib_mad_list_head *)(unsigned long)wc-wr_id;
qp_info = mad_list-mad_queue-qp_info;
@@ -2038,7 +2043,7 @@ static void ib_mad_recv_done_handler(struct
ib_mad_port_private *port_priv,
if (!validate_mad(recv-mad.mad.mad_hdr, qp_info-qp-qp_num))
goto out;
- response = alloc_mad_priv(port_priv-device);
+ response = alloc_mad_priv(port_priv-device, resp_mad_size);
if (!response) {
dev_err(port_priv-device-dev,
ib_mad_recv_done_handler no memory for response
buffer\n);
@@ -2063,8 +2068,10 @@ static void ib_mad_recv_done_handler(struct
ib_mad_port_private *port_priv,
[PATCH 19/20] IB/mad: Implement Intel Omni-Path Architecture SMP processing
From: Ira Weiny ira.we...@intel.com
Define the new OPA SMP format, create support functions for this format using
the previously defined helper functions as appropriate.
Signed-off-by: Ira Weiny ira.we...@intel.com
---
drivers/infiniband/core/mad_priv.h |2 +
drivers/infiniband/core/opa_smi.h | 78 ++
drivers/infiniband/core/smi.c | 54 ++
drivers/infiniband/core/smi.h |6 ++
include/rdma/opa_smi.h | 106
5 files changed, 246 insertions(+), 0 deletions(-)
create mode 100644 drivers/infiniband/core/opa_smi.h
create mode 100644 include/rdma/opa_smi.h
diff --git a/drivers/infiniband/core/mad_priv.h
b/drivers/infiniband/core/mad_priv.h
index a7041b0..bef25d2 100644
--- a/drivers/infiniband/core/mad_priv.h
+++ b/drivers/infiniband/core/mad_priv.h
@@ -41,6 +41,7 @@
#include linux/workqueue.h
#include rdma/ib_mad.h
#include rdma/ib_smi.h
+#include rdma/opa_smi.h
#define IB_MAD_QPS_CORE2 /* Always QP0 and QP1 as a minimum */
@@ -84,6 +85,7 @@ struct ib_mad_private {
struct ib_smp smp;
struct jumbo_mad jumbo_mad;
struct jumbo_rmpp_mad jumbo_rmpp_mad;
+ struct opa_smp opa_smp;
} mad;
} __attribute__ ((packed));
diff --git a/drivers/infiniband/core/opa_smi.h
b/drivers/infiniband/core/opa_smi.h
new file mode 100644
index 000..d180179
--- /dev/null
+++ b/drivers/infiniband/core/opa_smi.h
@@ -0,0 +1,78 @@
+/*
+ * Copyright (c) 2014 Intel Corporation. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ *copyright notice, this list of conditions and the following
+ *disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ *copyright notice, this list of conditions and the following
+ *disclaimer in the documentation and/or other materials
+ *provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED AS IS, WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ *
+ */
+
+#ifndef __OPA_SMI_H_
+#define __OPA_SMI_H_
+
+#include rdma/ib_smi.h
+#include rdma/opa_smi.h
+
+#include smi.h
+
+enum smi_action opa_smi_handle_dr_smp_recv(struct opa_smp *smp, u8 node_type,
+ int port_num, int phys_port_cnt);
+int opa_smi_get_fwd_port(struct opa_smp *smp);
+extern enum smi_forward_action opa_smi_check_forward_dr_smp(struct opa_smp
*smp);
+extern enum smi_action opa_smi_handle_dr_smp_send(struct opa_smp *smp,
+ u8 node_type, int port_num);
+
+/*
+ * Return IB_SMI_HANDLE if the SMP should be handled by the local SMA/SM
+ * via process_mad
+ */
+static inline enum smi_action opa_smi_check_local_smp(struct opa_smp *smp,
+ struct ib_device *device)
+{
+ /* C14-9:3 -- We're at the end of the DR segment of path */
+ /* C14-9:4 -- Hop Pointer = Hop Count + 1 - give to SMA/SM */
+ return (device-process_mad
+ !opa_get_smp_direction(smp)
+ (smp-hop_ptr == smp-hop_cnt + 1)) ?
+ IB_SMI_HANDLE : IB_SMI_DISCARD;
+}
+
+/*
+ * Return IB_SMI_HANDLE if the SMP should be handled by the local SMA/SM
+ * via process_mad
+ */
+static inline enum smi_action opa_smi_check_local_returning_smp(struct opa_smp
*smp,
+ struct ib_device *device)
+{
+ /* C14-13:3 -- We're at the end of the DR segment of path */
+ /* C14-13:4 -- Hop Pointer == 0 - give to SM */
+ return (device-process_mad
+ opa_get_smp_direction(smp)
+ !smp-hop_ptr) ? IB_SMI_HANDLE : IB_SMI_DISCARD;
+}
+
+#endif /* __OPA_SMI_H_ */
diff --git a/drivers/infiniband/core/smi.c b/drivers/infiniband/core/smi.c
index 8a5fb1d..a38ccb4 100644
--- a/drivers/infiniband/core/smi.c
+++ b/drivers/infiniband/core/smi.c
@@ -5,6 +5,7 @@
* Copyright (c) 2004, 2005 Topspin Corporation. All rights reserved.
* Copyright (c) 2004-2007
[PATCH 18/20] IB/mad: Implement support for Intel Omni-Path Architecture base version MADs in ib_create_send_mad
From: Ira Weiny ira.we...@intel.com
If the MAD is an OPA base version; verify the device supports Jumbo MAD sizes
Set MAD size and sg lengths as appropriate
Split RMPP MADs as appropriate
Signed-off-by: Ira Weiny ira.we...@intel.com
---
Changes from RFC:
remove use of supports_jumbo_mads
drivers/infiniband/core/mad.c | 38 --
1 files changed, 28 insertions(+), 10 deletions(-)
diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
index 2686393..0179d37 100644
--- a/drivers/infiniband/core/mad.c
+++ b/drivers/infiniband/core/mad.c
@@ -865,11 +865,11 @@ out:
return ret;
}
-static int get_pad_size(int hdr_len, int data_len)
+static int get_pad_size(int hdr_len, int data_len, size_t mad_size)
{
int seg_size, pad;
- seg_size = sizeof(struct ib_mad) - hdr_len;
+ seg_size = mad_size - hdr_len;
if (data_len seg_size) {
pad = seg_size - data_len % seg_size;
return pad == seg_size ? 0 : pad;
@@ -888,14 +888,14 @@ static void free_send_rmpp_list(struct
ib_mad_send_wr_private *mad_send_wr)
}
static int alloc_send_rmpp_list(struct ib_mad_send_wr_private *send_wr,
- gfp_t gfp_mask)
+ size_t mad_size, gfp_t gfp_mask)
{
struct ib_mad_send_buf *send_buf = send_wr-send_buf;
struct ib_rmpp_base *rmpp_base = send_buf-mad;
struct ib_rmpp_segment *seg = NULL;
int left, seg_size, pad;
- send_buf-seg_size = sizeof (struct ib_mad) - send_buf-hdr_len;
+ send_buf-seg_size = mad_size - send_buf-hdr_len;
seg_size = send_buf-seg_size;
pad = send_wr-pad;
@@ -945,20 +945,29 @@ struct ib_mad_send_buf * ib_create_send_mad(struct
ib_mad_agent *mad_agent,
struct ib_mad_send_wr_private *mad_send_wr;
int pad, message_size, ret, size;
void *buf;
+ size_t mad_size;
mad_agent_priv = container_of(mad_agent, struct ib_mad_agent_private,
agent);
- pad = get_pad_size(hdr_len, data_len);
+
+ if (base_version == OPA_MGMT_BASE_VERSION) {
+ if (mad_agent_priv-agent.device-attributes.mad_size
JUMBO_MGMT_MAD_SIZE)
+ return ERR_PTR(-EINVAL);
+ mad_size = sizeof(struct jumbo_mad);
+ } else
+ mad_size = sizeof(struct ib_mad);
+
+ pad = get_pad_size(hdr_len, data_len, mad_size);
message_size = hdr_len + data_len + pad;
if (ib_mad_kernel_rmpp_agent(mad_agent)) {
- if (!rmpp_active message_size sizeof(struct ib_mad))
+ if (!rmpp_active message_size mad_size)
return ERR_PTR(-EINVAL);
} else
- if (rmpp_active || message_size sizeof(struct ib_mad))
+ if (rmpp_active || message_size mad_size)
return ERR_PTR(-EINVAL);
- size = rmpp_active ? hdr_len : sizeof(struct ib_mad);
+ size = rmpp_active ? hdr_len : mad_size;
buf = kzalloc(sizeof *mad_send_wr + size, gfp_mask);
if (!buf)
return ERR_PTR(-ENOMEM);
@@ -973,7 +982,16 @@ struct ib_mad_send_buf * ib_create_send_mad(struct
ib_mad_agent *mad_agent,
mad_send_wr-mad_agent_priv = mad_agent_priv;
mad_send_wr-sg_list[0].length = hdr_len;
mad_send_wr-sg_list[0].lkey = mad_agent-mr-lkey;
- mad_send_wr-sg_list[1].length = sizeof(struct ib_mad) - hdr_len;
+
+ /* OPA MADs don't have to be the full 2048 bytes */
+ if (base_version == OPA_MGMT_BASE_VERSION
+ (mad_agent_priv-agent.device-attributes.device_cap_flags2
+ IB_DEVICE_OPA_MAD_SUPPORT)
+ data_len mad_size - hdr_len)
+ mad_send_wr-sg_list[1].length = data_len;
+ else
+ mad_send_wr-sg_list[1].length = mad_size - hdr_len;
+
mad_send_wr-sg_list[1].lkey = mad_agent-mr-lkey;
mad_send_wr-send_wr.wr_id = (unsigned long) mad_send_wr;
@@ -986,7 +1004,7 @@ struct ib_mad_send_buf * ib_create_send_mad(struct
ib_mad_agent *mad_agent,
mad_send_wr-send_wr.wr.ud.pkey_index = pkey_index;
if (rmpp_active) {
- ret = alloc_send_rmpp_list(mad_send_wr, gfp_mask);
+ ret = alloc_send_rmpp_list(mad_send_wr, mad_size, gfp_mask);
if (ret) {
kfree(buf);
return ERR_PTR(ret);
--
1.7.1
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 17/20] IB/mad: Add registration check for Intel Omni-Path Architecture MADs
From: Ira Weiny ira.we...@intel.com
If the registration specifies an OPA MAD class version and the device does not
support OPA MADs, fail the MAD registration.
Signed-off-by: Ira Weiny ira.we...@intel.com
---
Changes from RFC:
remove use of supports_jumbo_mads
convert to use IB_DEVICE_OPA_MAD_SUPPORT in device_cap_flags2
drivers/infiniband/core/mad.c | 24
1 files changed, 16 insertions(+), 8 deletions(-)
diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
index dc78040..2686393 100644
--- a/drivers/infiniband/core/mad.c
+++ b/drivers/infiniband/core/mad.c
@@ -227,6 +227,14 @@ struct ib_mad_agent *ib_register_mad_agent(struct
ib_device *device,
goto error1;
}
+ /* Validate device and port */
+ port_priv = ib_get_mad_port(device, port_num);
+ if (!port_priv) {
+ dev_notice(device-dev, ib_register_mad_agent: Invalid
port\n);
+ ret = ERR_PTR(-ENODEV);
+ goto error1;
+ }
+
/* Validate MAD registration request if supplied */
if (mad_reg_req) {
if (mad_reg_req-mgmt_class_version = MAX_MGMT_VERSION) {
@@ -235,6 +243,14 @@ struct ib_mad_agent *ib_register_mad_agent(struct
ib_device *device,
mad_reg_req-mgmt_class_version);
goto error1;
}
+ if (mad_reg_req-mgmt_class_version = OPA_MIN_CLASS_VERSION
+ !(port_priv-device-attributes.device_cap_flags2
IB_DEVICE_OPA_MAD_SUPPORT)) {
+ dev_notice(device-dev,
+ ib_register_mad_agent: OPA class Version
0x%x not supported on this device: cap flags 2 0x%llx\n,
+ mad_reg_req-mgmt_class_version,
+
port_priv-device-attributes.device_cap_flags2);
+ goto error1;
+ }
if (!recv_handler) {
dev_notice(device-dev,
ib_register_mad_agent: no recv_handler\n);
@@ -312,14 +328,6 @@ struct ib_mad_agent *ib_register_mad_agent(struct
ib_device *device,
goto error1;
}
- /* Validate device and port */
- port_priv = ib_get_mad_port(device, port_num);
- if (!port_priv) {
- dev_notice(device-dev, ib_register_mad_agent: Invalid
port\n);
- ret = ERR_PTR(-ENODEV);
- goto error1;
- }
-
/* Verify the QP requested is supported. For example, Ethernet devices
* will not have QP0 */
if (!port_priv-qp_info[qpn].qp) {
--
1.7.1
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] rping: create persistent server threads in DETACHED state
Since the persistent server threads aren't joined, they must be created in
the DETACHED state or resources will not be cleaned up when they exit.
This results in pthread_create() failures after thousands of rping
instances are run against a persistent server.
Also check the return from all calls to pthread_create() so we don't
ignore a thread creation failure.
Signed-off-by: Steve Wise sw...@opengridcomputing.com
---
examples/rping.c | 47 ++-
1 files changed, 42 insertions(+), 5 deletions(-)
diff --git a/examples/rping.c b/examples/rping.c
index 58b642e..9486314 100644
--- a/examples/rping.c
+++ b/examples/rping.c
@@ -793,7 +793,11 @@ static void *rping_persistent_server_thread(void *arg)
goto err2;
}
- pthread_create(cb-cqthread, NULL, cq_thread, cb);
+ ret = pthread_create(cb-cqthread, NULL, cq_thread, cb);
+ if (ret) {
+ perror(pthread_create);
+ goto err2;
+ }
ret = rping_accept(cb);
if (ret) {
@@ -825,11 +829,27 @@ static int rping_run_persistent_server(struct rping_cb
*listening_cb)
{
int ret;
struct rping_cb *cb;
+ pthread_attr_t attr;
ret = rping_bind_server(listening_cb);
if (ret)
return ret;
+ /*
+* Set persistent server threads to DEATCHED state so
+* they release all their resources when they exit.
+*/
+ ret = pthread_attr_init(attr);
+ if (ret) {
+ perror(pthread_attr_init);
+ return ret;
+ }
+ ret = pthread_attr_setdetachstate(attr, PTHREAD_CREATE_DETACHED);
+ if (ret) {
+ perror(pthread_attr_setdetachstate);
+ return ret;
+ }
+
while (1) {
sem_wait(listening_cb-sem);
if (listening_cb-state != CONNECT_REQUEST) {
@@ -841,7 +861,12 @@ static int rping_run_persistent_server(struct rping_cb
*listening_cb)
cb = clone_cb(listening_cb);
if (!cb)
return -1;
- pthread_create(cb-persistent_server_thread, NULL,
rping_persistent_server_thread, cb);
+
+ ret = pthread_create(cb-persistent_server_thread, attr,
rping_persistent_server_thread, cb);
+ if (ret) {
+ perror(pthread_create);
+ return ret;
+ }
}
return 0;
}
@@ -880,7 +905,11 @@ static int rping_run_server(struct rping_cb *cb)
goto err2;
}
- pthread_create(cb-cqthread, NULL, cq_thread, cb);
+ ret = pthread_create(cb-cqthread, NULL, cq_thread, cb);
+ if (ret) {
+ perror(pthread_create);
+ goto err2;
+ }
ret = rping_accept(cb);
if (ret) {
@@ -1055,7 +1084,11 @@ static int rping_run_client(struct rping_cb *cb)
goto err2;
}
- pthread_create(cb-cqthread, NULL, cq_thread, cb);
+ ret = pthread_create(cb-cqthread, NULL, cq_thread, cb);
+ if (ret) {
+ perror(pthread_create);
+ goto err2;
+ }
ret = rping_connect_client(cb);
if (ret) {
@@ -1222,7 +1255,11 @@ int main(int argc, char *argv[])
}
DEBUG_LOG(created cm_id %p\n, cb-cm_id);
- pthread_create(cb-cmthread, NULL, cm_thread, cb);
+ ret = pthread_create(cb-cmthread, NULL, cm_thread, cb);
+ if (ret) {
+ perror(pthread_create);
+ goto out2;
+ }
if (cb-server) {
if (persistent_server)
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 04/20] IB/mad: Change ib_response_mad signature to take ib_mad_hdr rather than ib_mad
From: Ira Weiny ira.we...@intel.com
ib_response_mad only needs access to the MAD header
Signed-off-by: Ira Weiny ira.we...@intel.com
---
drivers/infiniband/core/mad.c | 20 ++--
drivers/infiniband/core/user_mad.c |6 +++---
include/rdma/ib_mad.h |2 +-
3 files changed, 14 insertions(+), 14 deletions(-)
diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
index 99b..66b3940 100644
--- a/drivers/infiniband/core/mad.c
+++ b/drivers/infiniband/core/mad.c
@@ -179,12 +179,12 @@ static int is_vendor_method_in_use(
return 0;
}
-int ib_response_mad(struct ib_mad *mad)
+int ib_response_mad(struct ib_mad_hdr *hdr)
{
- return ((mad-mad_hdr.method IB_MGMT_METHOD_RESP) ||
- (mad-mad_hdr.method == IB_MGMT_METHOD_TRAP_REPRESS) ||
- ((mad-mad_hdr.mgmt_class == IB_MGMT_CLASS_BM)
-(mad-mad_hdr.attr_mod IB_BM_ATTR_MOD_RESP)));
+ return ((hdr-method IB_MGMT_METHOD_RESP) ||
+ (hdr-method == IB_MGMT_METHOD_TRAP_REPRESS) ||
+ ((hdr-mgmt_class == IB_MGMT_CLASS_BM)
+(hdr-attr_mod IB_BM_ATTR_MOD_RESP)));
}
EXPORT_SYMBOL(ib_response_mad);
@@ -791,7 +791,7 @@ static int handle_outgoing_dr_smp(struct
ib_mad_agent_private *mad_agent_priv,
switch (ret)
{
case IB_MAD_RESULT_SUCCESS | IB_MAD_RESULT_REPLY:
- if (ib_response_mad(mad_priv-mad.mad)
+ if (ib_response_mad(mad_priv-mad.mad.mad_hdr)
mad_agent_priv-agent.recv_handler) {
local-mad_priv = mad_priv;
local-recv_mad_agent = mad_agent_priv;
@@ -1628,7 +1628,7 @@ find_mad_agent(struct ib_mad_port_private *port_priv,
unsigned long flags;
spin_lock_irqsave(port_priv-reg_lock, flags);
- if (ib_response_mad(mad)) {
+ if (ib_response_mad(mad-mad_hdr)) {
u32 hi_tid;
struct ib_mad_agent_private *entry;
@@ -1765,8 +1765,8 @@ static inline int rcv_has_same_gid(struct
ib_mad_agent_private *mad_agent_priv,
u8 port_num = mad_agent_priv-agent.port_num;
u8 lmc;
- send_resp = ib_response_mad((struct ib_mad *)wr-send_buf.mad);
- rcv_resp = ib_response_mad(rwc-recv_buf.mad);
+ send_resp = ib_response_mad((struct ib_mad_hdr *)wr-send_buf.mad);
+ rcv_resp = ib_response_mad(rwc-recv_buf.mad-mad_hdr);
if (send_resp == rcv_resp)
/* both requests, or both responses. GIDs different */
@@ -1879,7 +1879,7 @@ static void ib_mad_complete_recv(struct
ib_mad_agent_private *mad_agent_priv,
}
/* Complete corresponding request */
- if (ib_response_mad(mad_recv_wc-recv_buf.mad)) {
+ if (ib_response_mad(mad_recv_wc-recv_buf.mad-mad_hdr)) {
spin_lock_irqsave(mad_agent_priv-lock, flags);
mad_send_wr = ib_find_send_mad(mad_agent_priv, mad_recv_wc);
if (!mad_send_wr) {
diff --git a/drivers/infiniband/core/user_mad.c
b/drivers/infiniband/core/user_mad.c
index 928cdd2..66b5217 100644
--- a/drivers/infiniband/core/user_mad.c
+++ b/drivers/infiniband/core/user_mad.c
@@ -426,11 +426,11 @@ static int is_duplicate(struct ib_umad_file *file,
* the same TID, reject the second as a duplicate. This is more
* restrictive than required by the spec.
*/
- if (!ib_response_mad((struct ib_mad *) hdr)) {
- if (!ib_response_mad((struct ib_mad *) sent_hdr))
+ if (!ib_response_mad(hdr)) {
+ if (!ib_response_mad(sent_hdr))
return 1;
continue;
- } else if (!ib_response_mad((struct ib_mad *) sent_hdr))
+ } else if (!ib_response_mad(sent_hdr))
continue;
if (same_destination(packet-mad.hdr, sent_packet-mad.hdr))
diff --git a/include/rdma/ib_mad.h b/include/rdma/ib_mad.h
index 9bb99e9..9c89939 100644
--- a/include/rdma/ib_mad.h
+++ b/include/rdma/ib_mad.h
@@ -263,7 +263,7 @@ struct ib_mad_send_buf {
* ib_response_mad - Returns if the specified MAD has been generated in
* response to a sent request or trap.
*/
-int ib_response_mad(struct ib_mad *mad);
+int ib_response_mad(struct ib_mad_hdr *hdr);
/**
* ib_get_rmpp_resptime - Returns the RMPP response time.
--
1.7.1
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 00/20] IB/mad: Add support for Intel Omni-Path Architecture (OPA) MAD processing.
From: Ira Weiny ira.we...@intel.com The following patch series modifies the kernel MAD processing (ib_mad/ib_umad) and related interfaces to send and receive Intel Omni-Path Architecture MADs on devices which support them. In addition to supporting some IBTA management classes, OPA devices use MADs with lengths up to 2K. These jumbo MADs increase the performance of management traffic. To distinguish IBTA MADs from OPA MADs a new Base Version is introduced. The new format shares the same common header with IBTA MADs which allows us to share most of the MAD processing code when dealing with the new Base Version. The patch series is broken into 3 main areas. 1) Add the ability for devices to indicate MAD size. modify the MAD code to use this MAD size 2) Enhance the interface to the device agents to support larger and variable length MADs. 3) Add support for creating and processing a new OPA MAD Base Version Notes regarding changes from the RFC patches: I did a series of tests using kmalloc vs kmem_cache. This includes running SA, PM, and SM Traffic with both OpenSM and OPA MADs. I have been unable to determine any performance difference in using kmalloc vs kmem_cache. As Roland suggested the use of kmalloc makes device specific MAD sizes much easier to deal with so this series removes the kmem_cache and uses the common kmalloc calls for both IB and OPA devices. Changes based on feedback from the RFC patches: Rebased to latest Rolands for-next (3.18) Convert from kmem_cache to kmalloc for receive buffers remove unneeded jumbo_mad_private structure Add device attribute caching and avoid caching of device flags in the MAD code Add mad_size device attribute Use mad_size in all recv MAD buffer allocations Remove IB_DEVICE_JUMBO_MAD_SUPPORT and internal MAD supports_jumbo_mads flag in favor of an IB_DEVICE_OPA_MAD_SUPPORT flag Added a new device_cap_flags2 field (device_cap_flags was full in 3.18) Limit the use of struct ib_mad to places where the data is actually an IB MAD, common processing functions can use ib_mad_hdr. rename commit messages to IB/* rather than ib/* reorder the patch series to put general refactoring and pre-patches first. Fix MLX5 compile Ira Weiny (20): IB/mad: Rename is_data_mad to is_rmpp_data_mad IB/core: Cache device attributes for use by upper level drivers IB/mad: Change validate_mad signature to take ib_mad_hdr rather than ib_mad IB/mad: Change ib_response_mad signature to take ib_mad_hdr rather than ib_mad IB/mad: Change cast in rcv_has_same_class IB/core: Add mad_size to ib_device_attr IB/mad: Convert ib_mad_private allocations from kmem_cache to kmalloc IB/mad: Add helper function for smi_handle_dr_smp_send IB/mad: Add helper function for smi_handle_dr_smp_recv IB/mad: Add helper function for smi_check_forward_dr_smp IB/mad: Add helper function for SMI processing IB/mad: Add MAD size parameters to process_mad IB/mad: Add base version parameter to ib_create_send_mad IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag IB/mad: Create jumbo_mad data structures IB/mad: Add Intel Omni-Path Architecture defines IB/mad: Add registration check for Intel Omni-Path Architecture MADs IB/mad: Implement support for Intel Omni-Path Architecture base version MADs in ib_create_send_mad IB/mad: Implement Intel Omni-Path Architecture SMP processing IB/mad: Implement Intel Omni-Path Architecture MAD processing drivers/infiniband/core/agent.c | 25 +- drivers/infiniband/core/agent.h |2 +- drivers/infiniband/core/cm.c |6 +- drivers/infiniband/core/device.c |2 + drivers/infiniband/core/mad.c| 521 +- drivers/infiniband/core/mad_priv.h |9 +- drivers/infiniband/core/mad_rmpp.c | 142 drivers/infiniband/core/opa_smi.h| 78 drivers/infiniband/core/sa_query.c |3 +- drivers/infiniband/core/smi.c| 231 drivers/infiniband/core/smi.h|6 + drivers/infiniband/core/sysfs.c |5 +- drivers/infiniband/core/user_mad.c | 42 ++- drivers/infiniband/hw/amso1100/c2_provider.c |5 +- drivers/infiniband/hw/amso1100/c2_rnic.c |1 + drivers/infiniband/hw/cxgb3/iwch_provider.c |6 +- drivers/infiniband/hw/cxgb4/provider.c |8 +- drivers/infiniband/hw/ehca/ehca_hca.c|2 + drivers/infiniband/hw/ehca/ehca_sqp.c|8 +- drivers/infiniband/hw/ipath/ipath_mad.c |8 +- drivers/infiniband/hw/ipath/ipath_verbs.c|1 + drivers/infiniband/hw/ipath/ipath_verbs.h|3 +- drivers/infiniband/hw/mlx4/mad.c | 12 +-
[PATCH 02/20] IB/core: Cache device attributes for use by upper level drivers
From: Ira Weiny ira.we...@intel.com
Signed-off-by: Ira Weiny ira.we...@intel.com
---
drivers/infiniband/core/device.c |2 ++
include/rdma/ib_verbs.h |1 +
2 files changed, 3 insertions(+), 0 deletions(-)
diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c
index 18c1ece..3a6afde 100644
--- a/drivers/infiniband/core/device.c
+++ b/drivers/infiniband/core/device.c
@@ -294,6 +294,8 @@ int ib_register_device(struct ib_device *device,
spin_lock_init(device-event_handler_lock);
spin_lock_init(device-client_data_lock);
+ device-query_device(device, device-attributes);
+
ret = read_port_table_lengths(device);
if (ret) {
printk(KERN_WARNING Couldn't create table lengths cache for
device %s\n,
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 0d74f1d..86fc90f 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1675,6 +1675,7 @@ struct ib_device {
u32 local_dma_lkey;
u8 node_type;
u8 phys_port_cnt;
+ struct ib_device_attrattributes;
};
struct ib_client {
--
1.7.1
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
