Re: [PATCH 10/13] IB/srp: use the new CQ API
On Fri, Dec 11, 2015 at 12:59:01PM -0500, Doug Ledford wrote: > On 12/11/2015 09:22 AM, Christoph Hellwig wrote: > > Hi Bart, > > > > thanks for all the reviews. I've updated the git branch with your > > suggestions and reviewed-by tags. I'm going to wait a little bit > > longer for other reviews to come in before reposting the series. > > Indeed, thanks for all the catches Bart. This patchset, with Bart's > fixups, looks good to me. Allright. How do you want to proceed? The current rdma-cq branch has all kinds of dependencies, but I've also prepared a new rdma-cq.2 branch that could go straight on top of your current queue: http://git.infradead.org/users/hch/rdma.git/shortlog/refs/heads/rdma-cq.2 If you're ready to start the 4.5 tree I can send those out as a patch series. > > > -- > Doug Ledford > GPG KeyID: 0E572FDD > > ---end quoted text--- -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH infiniband-diags] ibportstate.c: Fix unsigned comparison warnings
On Tue, Nov 10, 2015 at 01:23:01PM +0200, Hal Rosenstock wrote:
>
> src/ibportstate.c:450: warning: comparison of unsigned expression < 0 is
> always false
> src/ibportstate.c:454: warning: comparison of unsigned expression < 0 is
> always false
> src/ibportstate.c:458: warning: comparison of unsigned expression < 0 is
> always false
> src/ibportstate.c:462: warning: comparison of unsigned expression < 0 is
> always false
> src/ibportstate.c:483: warning: comparison of unsigned expression < 0 is
> always false
> src/ibportstate.c:500: warning: comparison of unsigned expression < 0 is
> always false
> src/ibportstate.c:504: warning: comparison of unsigned expression < 0 is
> always false
>
> Signed-off-by: Hal Rosenstock
Thanks applied,
Ira
> ---
> diff --git a/src/ibportstate.c b/src/ibportstate.c
> index 47e9133..cb47aa9 100644
> --- a/src/ibportstate.c
> +++ b/src/ibportstate.c
> @@ -447,40 +447,40 @@ int main(int argc, char **argv)
> val = strtoull(argv[i], 0, 0);
> switch (j) {
> case SPEED:
> - if (val < 0 || val > 15)
> + if (val > 15)
> IBEXIT("invalid speed value %ld", val);
> break;
> case ESPEED:
> - if (val < 0 || val > 31)
> + if (val > 31)
> IBEXIT("invalid extended speed value
> %ld", val);
> break;
> case FDR10SPEED:
> - if (val < 0 || val > 1)
> + if (val > 1)
> IBEXIT("invalid fdr10 speed value %ld",
> val);
> break;
> case WIDTH:
> - if (val < 0 || (val > 15 && val != 255))
> + if ((val > 15 && val != 255))
> IBEXIT("invalid width value %ld", val);
> break;
> case VLS:
> - if (val <= 0 || val > 5)
> + if (val == 0 || val > 5)
> IBEXIT("invalid vls value %ld", val);
> break;
> case MTU:
> - if (val <= 0 || val > 5)
> + if (val == 0 || val > 5)
> IBEXIT("invalid mtu value %ld", val);
> break;
> case LID:
> - if (val <= 0 || val >= 0xC000)
> + if (val == 0 || val >= 0xC000)
> IBEXIT("invalid lid value 0x%lx", val);
> break;
> case SMLID:
> - if (val <= 0 || val >= 0xC000)
> + if (val == 0 || val >= 0xC000)
> IBEXIT("invalid smlid value 0x%lx",
> val);
> break;
> case LMC:
> - if (val < 0 || val > 7)
> + if (val > 7)
> IBEXIT("invalid lmc value %ld", val);
> break;
> case MKEY:
> @@ -497,11 +497,11 @@ int main(int argc, char **argv)
> /* All 64-bit values are legal */
> break;
> case MKEYLEASE:
> - if (val < 0 || val > 0x)
> + if (val > 0x)
> IBEXIT("invalid mkey lease time %ld",
> val);
> break;
> case MKEYPROT:
> - if (val < 0 || val > 3)
> + if (val > 3)
> IBEXIT("invalid mkey protection bit
> setting %ld", val);
> }
> *port_args[j].val = val;
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH infiniband-diags] ibsendtrap.c: Eliminate unused parameter from send_trap
On Tue, Nov 10, 2015 at 01:23:39PM +0200, Hal Rosenstock wrote:
>
> src/ibsendtrap.c: In function ?send_trap?:
> src/ibsendtrap.c:124: warning: unused parameter ?name?
>
> Signed-off-by: Hal Rosenstock
Thanks applied,
Ira
> ---
> diff --git a/src/ibsendtrap.c b/src/ibsendtrap.c
> index 46fd6e7..659f2d2 100644
> --- a/src/ibsendtrap.c
> +++ b/src/ibsendtrap.c
> @@ -121,8 +121,7 @@ static void build_trap129(ib_mad_notice_attr_t * n,
> ib_portid_t * port)
> n->data_details.ntc_129_131.port_num = (uint8_t) error_port;
> }
>
> -static int send_trap(const char *name,
> - void (*build) (ib_mad_notice_attr_t *, ib_portid_t *))
> +static int send_trap(void (*build) (ib_mad_notice_attr_t *, ib_portid_t *))
> {
> ib_portid_t sm_port;
> ib_portid_t selfportid;
> @@ -169,7 +168,7 @@ int process_send_trap(char *trap_name)
>
> for (i = 0; traps[i].trap_name; i++)
> if (strcmp(traps[i].trap_name, trap_name) == 0)
> - return send_trap(trap_name, traps[i].build_func);
> + return send_trap(traps[i].build_func);
> ibdiag_show_usage();
> return 1;
> }
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH infiniband-diags] Remove unused pisize parameter from dump_portinfo in ibdiag_common
On Tue, Nov 10, 2015 at 01:22:51PM +0200, Hal Rosenstock wrote:
>
> src/ibdiag_common.c: In function ?dump_portinfo?:
> src/ibdiag_common.c:856: warning: unused parameter ?pisize?
>
> Signed-off-by: Hal Rosenstock
Thanks applied,
Ira
> ---
> diff --git a/include/ibdiag_common.h b/include/ibdiag_common.h
> index af5b3c4..21b0522 100644
> --- a/include/ibdiag_common.h
> +++ b/include/ibdiag_common.h
> @@ -150,7 +150,7 @@ int vsnprint_field(char *buf, size_t n, enum MAD_FIELDS
> f, int spacing,
> const char *format, va_list va_args);
> int snprint_field(char *buf, size_t n, enum MAD_FIELDS f, int spacing,
> const char *format, ...);
> -void dump_portinfo(void *pi, int pisize, int tabs);
> +void dump_portinfo(void *pi, int tabs);
>
> /**
> * Some common command line parsing
> diff --git a/src/ibdiag_common.c b/src/ibdiag_common.c
> index 5ec0167..3ebdbb9 100644
> --- a/src/ibdiag_common.c
> +++ b/src/ibdiag_common.c
> @@ -853,7 +853,7 @@ int snprint_field(char *buf, size_t n, enum MAD_FIELDS f,
> int spacing,
> return ret;
> }
>
> -void dump_portinfo(void *pi, int pisize, int tabs)
> +void dump_portinfo(void *pi, int tabs)
> {
> int field, i;
> char val[64];
> diff --git a/src/saquery.c b/src/saquery.c
> index cc8d8dc..bd70b7b 100644
> --- a/src/saquery.c
> +++ b/src/saquery.c
> @@ -304,7 +304,7 @@ static void dump_one_portinfo_record(void *data, struct
> query_params *p)
> "\t\tOptions.0x%x\n"
> "\tPortInfo dump:\n",
> cl_ntoh16(pir->lid), pir->port_num, pir->options);
> - dump_portinfo(pi, sizeof(*pi), 2);
> + dump_portinfo(pi, 2);
> }
>
> static void dump_one_mcmember_record(void *data, struct query_params *p)
> diff --git a/src/smpquery.c b/src/smpquery.c
> index 187ef61..2bd7132 100644
> --- a/src/smpquery.c
> +++ b/src/smpquery.c
> @@ -161,7 +161,7 @@ static char *port_info(ib_portid_t * dest, char **argv,
> int argc)
> return "port info query failed";
>
> printf("# Port info: %s port %d\n", portid2str(dest), orig_portnum);
> - dump_portinfo(data, sizeof data, 0);
> + dump_portinfo(data, 0);
> return 0;
> }
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 1/6] svcrdma: Do not send XDR roundup bytes for a write chunk
Two small comments.
On 12/7/2015 12:42 PM, Chuck Lever wrote:
Minor optimization: when dealing with write chunk XDR roundup, do
not post a Write WR for the zero bytes in the pad. Simply update
the write segment in the RPC-over-RDMA header to reflect the extra
pad bytes.
The Reply chunk is also a write chunk, but the server does not use
send_write_chunks() to send the Reply chunk. That's OK in this case:
the server Upper Layer typically marshals the Reply chunk contents
in a single contiguous buffer, without a separate tail for the XDR
pad.
The comments and the variable naming refer to "chunks" but what is
really meant is "segments." The existing code sends only one
xdr_write_chunk per RPC reply.
The fix assumes this as well. When the XDR pad in the first write
chunk is reached, the assumption is the Write list is complete and
send_write_chunks() returns.
That will remain a valid assumption until the server Upper Layer can
support multiple bulk payload results per RPC.
Signed-off-by: Chuck Lever
---
net/sunrpc/xprtrdma/svc_rdma_sendto.c |7 +++
1 file changed, 7 insertions(+)
diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c
b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
index 969a1ab..bad5eaa 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
@@ -342,6 +342,13 @@ static int send_write_chunks(struct svcxprt_rdma *xprt,
arg_ch->rs_handle,
arg_ch->rs_offset,
write_len);
+
+ /* Do not send XDR pad bytes */
It might be clearer to say "marshal" instead of "send".
+ if (chunk_no && write_len < 4) {
Why is it necessary to check for chunk_no == 0? It is not
possible for leading data to ever be padding, nor is a leading
data element ever less than 4 bytes long. Right?
Tom.
+ chunk_no++;
+ break;
+ }
+
chunk_off = 0;
while (write_len) {
ret = send_write(xprt, rqstp,
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 5/6] svcrdma: Add infrastructure to receive backwards direction RPC/RDMA replies
Three comments.
On 12/7/2015 12:43 PM, Chuck Lever wrote:
To support the NFSv4.1 backchannel on RDMA connections, add a
capability for receiving an RPC/RDMA reply on a connection
established by a client.
(snip)
+/* By convention, backchannel calls arrive via rdma_msg type
"By convention" is ok, but it's important to note that this is
actually not "by protocol". Therefore, the following check may
reject valid messages. Even though it is unlikely an implementation
will insert chunks, it's not illegal, and ignoring them will
be less harmful. So I'm going to remake my earlier observation
that three checks below should be removed:
+ * messages, and never populate the chunk lists. This makes
+ * the RPC/RDMA header small and fixed in size, so it is
+ * straightforward to check the RPC header's direction field.
+ */
+static bool
+svc_rdma_is_backchannel_reply(struct svc_xprt *xprt, struct rpcrdma_msg *rmsgp)
+{
+ __be32 *p = (__be32 *)rmsgp;
+
+ if (!xprt->xpt_bc_xprt)
+ return false;
+
+ if (rmsgp->rm_type != rdma_msg)
+ return false;
These three:
+ if (rmsgp->rm_body.rm_chunks[0] != xdr_zero)
+ return false;
+ if (rmsgp->rm_body.rm_chunks[1] != xdr_zero)
+ return false;
+ if (rmsgp->rm_body.rm_chunks[2] != xdr_zero)
+ return false;
+
(snip)
diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h
index a1fd74a..3895574 100644
--- a/net/sunrpc/xprtrdma/xprt_rdma.h
+++ b/net/sunrpc/xprtrdma/xprt_rdma.h
@@ -309,6 +309,8 @@ struct rpcrdma_buffer {
u32 rb_bc_srv_max_requests;
spinlock_t rb_reqslock;/* protect rb_allreqs */
struct list_headrb_allreqs;
+
+ u32 rb_bc_max_requests;
Why does this need to be u32? Shouldn't it be an int, and also the
rb_bc_srv_max_requests just above? The forward channel max_requests
are int, btw.
Tom.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/3] IB/core: Align coding style of ib_device_cap_flags structure
From: Leon Romanovsky
Modify enum ib_device_cap_flags such that other patches which add new
enum values pass strict checkpatch.pl checks.
Reviewed-by: Sagi Grimberg
Signed-off-by: Leon Romanovsky
---
include/rdma/ib_verbs.h | 60 -
1 file changed, 30 insertions(+), 30 deletions(-)
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 9a68a19..bcf40ad 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -105,24 +105,24 @@ enum rdma_link_layer {
};
enum ib_device_cap_flags {
- IB_DEVICE_RESIZE_MAX_WR = 1,
- IB_DEVICE_BAD_PKEY_CNTR = (1<<1),
- IB_DEVICE_BAD_QKEY_CNTR = (1<<2),
- IB_DEVICE_RAW_MULTI = (1<<3),
- IB_DEVICE_AUTO_PATH_MIG = (1<<4),
- IB_DEVICE_CHANGE_PHY_PORT = (1<<5),
- IB_DEVICE_UD_AV_PORT_ENFORCE= (1<<6),
- IB_DEVICE_CURR_QP_STATE_MOD = (1<<7),
- IB_DEVICE_SHUTDOWN_PORT = (1<<8),
- IB_DEVICE_INIT_TYPE = (1<<9),
- IB_DEVICE_PORT_ACTIVE_EVENT = (1<<10),
- IB_DEVICE_SYS_IMAGE_GUID= (1<<11),
- IB_DEVICE_RC_RNR_NAK_GEN= (1<<12),
- IB_DEVICE_SRQ_RESIZE= (1<<13),
- IB_DEVICE_N_NOTIFY_CQ = (1<<14),
- IB_DEVICE_LOCAL_DMA_LKEY= (1<<15),
- IB_DEVICE_RESERVED = (1<<16), /* old SEND_W_INV */
- IB_DEVICE_MEM_WINDOW= (1<<17),
+ IB_DEVICE_RESIZE_MAX_WR = (1 << 0),
+ IB_DEVICE_BAD_PKEY_CNTR = (1 << 1),
+ IB_DEVICE_BAD_QKEY_CNTR = (1 << 2),
+ IB_DEVICE_RAW_MULTI = (1 << 3),
+ IB_DEVICE_AUTO_PATH_MIG = (1 << 4),
+ IB_DEVICE_CHANGE_PHY_PORT = (1 << 5),
+ IB_DEVICE_UD_AV_PORT_ENFORCE= (1 << 6),
+ IB_DEVICE_CURR_QP_STATE_MOD = (1 << 7),
+ IB_DEVICE_SHUTDOWN_PORT = (1 << 8),
+ IB_DEVICE_INIT_TYPE = (1 << 9),
+ IB_DEVICE_PORT_ACTIVE_EVENT = (1 << 10),
+ IB_DEVICE_SYS_IMAGE_GUID= (1 << 11),
+ IB_DEVICE_RC_RNR_NAK_GEN= (1 << 12),
+ IB_DEVICE_SRQ_RESIZE= (1 << 13),
+ IB_DEVICE_N_NOTIFY_CQ = (1 << 14),
+ IB_DEVICE_LOCAL_DMA_LKEY= (1 << 15),
+ IB_DEVICE_RESERVED = (1 << 16), /* old SEND_W_INV */
+ IB_DEVICE_MEM_WINDOW= (1 << 17),
/*
* Devices should set IB_DEVICE_UD_IP_SUM if they support
* insertion of UDP and TCP checksum on outgoing UD IPoIB
@@ -130,18 +130,18 @@ enum ib_device_cap_flags {
* incoming messages. Setting this flag implies that the
* IPoIB driver may set NETIF_F_IP_CSUM for datagram mode.
*/
- IB_DEVICE_UD_IP_CSUM= (1<<18),
- IB_DEVICE_UD_TSO= (1<<19),
- IB_DEVICE_XRC = (1<<20),
- IB_DEVICE_MEM_MGT_EXTENSIONS= (1<<21),
- IB_DEVICE_BLOCK_MULTICAST_LOOPBACK = (1<<22),
- IB_DEVICE_MEM_WINDOW_TYPE_2A= (1<<23),
- IB_DEVICE_MEM_WINDOW_TYPE_2B= (1<<24),
- IB_DEVICE_RC_IP_CSUM= (1<<25),
- IB_DEVICE_RAW_IP_CSUM = (1<<26),
- IB_DEVICE_MANAGED_FLOW_STEERING = (1<<29),
- IB_DEVICE_SIGNATURE_HANDOVER= (1<<30),
- IB_DEVICE_ON_DEMAND_PAGING = (1<<31),
+ IB_DEVICE_UD_IP_CSUM= (1 << 18),
+ IB_DEVICE_UD_TSO= (1 << 19),
+ IB_DEVICE_XRC = (1 << 20),
+ IB_DEVICE_MEM_MGT_EXTENSIONS= (1 << 21),
+ IB_DEVICE_BLOCK_MULTICAST_LOOPBACK = (1 << 22),
+ IB_DEVICE_MEM_WINDOW_TYPE_2A= (1 << 23),
+ IB_DEVICE_MEM_WINDOW_TYPE_2B= (1 << 24),
+ IB_DEVICE_RC_IP_CSUM= (1 << 25),
+ IB_DEVICE_RAW_IP_CSUM = (1 << 26),
+ IB_DEVICE_MANAGED_FLOW_STEERING = (1 << 29),
+ IB_DEVICE_SIGNATURE_HANDOVER= (1 << 30),
+ IB_DEVICE_ON_DEMAND_PAGING = (1 << 31),
};
enum ib_signature_prot_cap {
--
1.7.12.4
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/3] Add cross-channel support
From: Leon Romanovsky This patchset adds cross-channel support. The cross-channel feature allows to execute WQEs that involve synchronization of I/O operations’ on different QPs. This capability enables to program complex flows with a single function call, hereby significantly reducing overhead associated with I/O processing. Leon Romanovsky (3): IB/core: Align coding style of ib_device_cap_flags structure IB/core: Add cross-channel support IB/mlx5: Add driver cross-channel support drivers/infiniband/core/uverbs_cmd.c | 5 ++- drivers/infiniband/hw/mlx5/cq.c | 7 +++- drivers/infiniband/hw/mlx5/main.c| 3 ++ drivers/infiniband/hw/mlx5/mlx5_ib.h | 12 ++ drivers/infiniband/hw/mlx5/qp.c | 29 +++ include/linux/mlx5/qp.h | 3 ++ include/rdma/ib_verbs.h | 71 +--- 7 files changed, 98 insertions(+), 32 deletions(-) -- 1.7.12.4 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/3] IB/mlx5: Add driver cross-channel support
From: Leon Romanovsky
Add support of cross-channel functionality to mlx5
driver. This includes ability to ignore overrun for CQ
which intended for cross-channel, export device capability and
configure the QP to be sync master/slave queues.
The cross-channel enabled QP supports combination of
three possible properties:
* WQE processing on the receive queue of this QP
* WQE processing on the send queue of this QP
* WQE are supported on the send queue
Reviewed-by: Sagi Grimberg
Signed-off-by: Leon Romanovsky
---
drivers/infiniband/hw/mlx5/cq.c | 7 ++-
drivers/infiniband/hw/mlx5/main.c| 3 +++
drivers/infiniband/hw/mlx5/mlx5_ib.h | 12
drivers/infiniband/hw/mlx5/qp.c | 29 +
include/linux/mlx5/qp.h | 3 +++
5 files changed, 53 insertions(+), 1 deletion(-)
diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c
index 3dfd287..c363b71 100644
--- a/drivers/infiniband/hw/mlx5/cq.c
+++ b/drivers/infiniband/hw/mlx5/cq.c
@@ -760,7 +760,7 @@ struct ib_cq *mlx5_ib_create_cq(struct ib_device *ibdev,
int eqn;
int err;
- if (attr->flags)
+ if (check_cq_create_flags(attr->flags))
return ERR_PTR(-EINVAL);
if (entries < 0)
@@ -779,6 +779,7 @@ struct ib_cq *mlx5_ib_create_cq(struct ib_device *ibdev,
spin_lock_init(&cq->lock);
cq->resize_buf = NULL;
cq->resize_umem = NULL;
+ cq->create_flags = attr->flags;
if (context) {
err = create_cq_user(dev, udata, context, cq, entries,
@@ -796,6 +797,10 @@ struct ib_cq *mlx5_ib_create_cq(struct ib_device *ibdev,
cq->cqe_size = cqe_size;
cqb->ctx.cqe_sz_flags = cqe_sz_to_mlx_sz(cqe_size) << 5;
+
+ if (cq->create_flags & IB_CQ_FLAGS_IGNORE_OVERRUN)
+ cqb->ctx.cqe_sz_flags |= (1 << 1);
+
cqb->ctx.log_sz_usr_page = cpu_to_be32((ilog2(entries) << 24) | index);
err = mlx5_vector2eqn(dev->mdev, vector, &eqn, &irqn);
if (err)
diff --git a/drivers/infiniband/hw/mlx5/main.c
b/drivers/infiniband/hw/mlx5/main.c
index bdd60a6..be0d994 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -300,6 +300,9 @@ static int mlx5_ib_query_device(struct ib_device *ibdev,
props->odp_caps = dev->odp_caps;
#endif
+ if (MLX5_CAP_GEN(mdev, cd))
+ props->device_cap_flags |= IB_DEVICE_CROSS_CHANNEL;
+
return 0;
}
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h
b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 6333472..7f9c6bf 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -242,6 +242,9 @@ struct mlx5_ib_cq_buf {
enum mlx5_ib_qp_flags {
MLX5_IB_QP_BLOCK_MULTICAST_LOOPBACK = 1 << 0,
MLX5_IB_QP_SIGNATURE_HANDLING = 1 << 1,
+ MLX5_IB_QP_CROSS_CHANNEL= 1 << 2,
+ MLX5_IB_QP_MANAGED_SEND = 1 << 3,
+ MLX5_IB_QP_MANAGED_RECV = 1 << 4,
};
struct mlx5_umr_wr {
@@ -284,6 +287,7 @@ struct mlx5_ib_cq {
struct mlx5_ib_cq_buf *resize_buf;
struct ib_umem *resize_umem;
int cqe_size;
+ u32 create_flags;
};
struct mlx5_ib_srq {
@@ -662,4 +666,12 @@ static inline int is_qp1(enum ib_qp_type qp_type)
#define MLX5_MAX_UMR_SHIFT 16
#define MLX5_MAX_UMR_PAGES (1 << MLX5_MAX_UMR_SHIFT)
+static inline u32 check_cq_create_flags(u32 flags)
+{
+ /*
+* It returns non-zero value for unsupported CQ
+* create flags, otherwise it returns zero.
+*/
+ return (flags & ~IB_CQ_FLAGS_IGNORE_OVERRUN);
+}
#endif /* MLX5_IB_H */
diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index 307bdbc..54ceba8 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -880,6 +880,21 @@ static int create_qp_common(struct mlx5_ib_dev *dev,
struct ib_pd *pd,
}
}
+ if (init_attr->create_flags &
+ (IB_QP_CREATE_CROSS_CHANNEL |
+IB_QP_CREATE_MANAGED_SEND |
+IB_QP_CREATE_MANAGED_RECV)) {
+ if (!MLX5_CAP_GEN(mdev, cd)) {
+ mlx5_ib_dbg(dev, "cross-channel isn't supported\n");
+ return -EINVAL;
+ }
+ if (init_attr->create_flags & IB_QP_CREATE_CROSS_CHANNEL)
+ qp->flags |= MLX5_IB_QP_CROSS_CHANNEL;
+ if (init_attr->create_flags & IB_QP_CREATE_MANAGED_SEND)
+ qp->flags |= MLX5_IB_QP_MANAGED_SEND;
+ if (init_attr->create_flags & IB_QP_CREATE_MANAGED_RECV)
+ qp->flags |= MLX5_IB_QP_MANAGED_RECV;
+ }
if (init_attr->sq_sig_type == IB_SIGNAL_ALL_WR)
qp->sq_signal_bits = MLX5_WQE_CTRL_CQ_UPDATE;
@@ -954,6
[PATCH 2/3] IB/core: Add cross-channel support
From: Leon Romanovsky
The cross-channel feature allows to execute WQEs that involve
synchronization of I/O operations’ on different QPs.
This capability enables to program complex flows with a single
function call, hereby significantly reducing overhead associated
with I/O processing.
Cross-channel operations support is indicated by HCA capability
information.
The queue pairs can be configured to work as a “sync master queue”
or “sync slave queues”.
The added flags are:
1. Device capability flag IB_DEVICE_CROSS_CHANNEL for the
devices that can perform cross-channel operations.
2. CQ property flag IB_CQ_FLAGS_IGNORE_OVERRUN to disable CQ overrun
check. This check is useless in cross-channel scenario.
3. QP property flags to indicate if queues are slave or master:
* IB_QP_CREATE_MANAGED_SEND indicates that posted send work requests
will not be executed immediately and requires enabling.
* IB_QP_CREATE_MANAGED_RECV indicates that posted receive work
requests will not be executed immediately and requires enabling.
* IB_QP_CREATE_CROSS_CHANNEL declares the QP to work in cross-channel
mode. If IB_QP_CREATE_MANAGED_SEND and IB_QP_CREATE_MANAGED_RECV are
not provided, this QP will be sync master queue, else it will be sync
slave.
Reviewed-by: Sagi Grimberg
Signed-off-by: Leon Romanovsky
---
drivers/infiniband/core/uverbs_cmd.c | 5 -
include/rdma/ib_verbs.h | 11 +++
2 files changed, 15 insertions(+), 1 deletion(-)
diff --git a/drivers/infiniband/core/uverbs_cmd.c
b/drivers/infiniband/core/uverbs_cmd.c
index 94816ae..0e9711f 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -1843,7 +1843,10 @@ static int create_qp(struct ib_uverbs_file *file,
sizeof(cmd->create_flags))
attr.create_flags = cmd->create_flags;
- if (attr.create_flags & ~IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK) {
+ if (attr.create_flags & ~(IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK |
+ IB_QP_CREATE_CROSS_CHANNEL |
+ IB_QP_CREATE_MANAGED_SEND |
+ IB_QP_CREATE_MANAGED_RECV)) {
ret = -EINVAL;
goto err_put;
}
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index bcf40ad..370fbdf 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -139,6 +139,13 @@ enum ib_device_cap_flags {
IB_DEVICE_MEM_WINDOW_TYPE_2B= (1 << 24),
IB_DEVICE_RC_IP_CSUM= (1 << 25),
IB_DEVICE_RAW_IP_CSUM = (1 << 26),
+ /*
+* Devices should set IB_DEVICE_CROSS_CHANNEL if they
+* support execution of WQEs that involve synchronization
+* of I/O operations with single completion queue managed
+* by hardware.
+*/
+ IB_DEVICE_CROSS_CHANNEL = (1 << 27),
IB_DEVICE_MANAGED_FLOW_STEERING = (1 << 29),
IB_DEVICE_SIGNATURE_HANDOVER= (1 << 30),
IB_DEVICE_ON_DEMAND_PAGING = (1 << 31),
@@ -184,6 +191,7 @@ struct ib_odp_caps {
enum ib_cq_creation_flags {
IB_CQ_FLAGS_TIMESTAMP_COMPLETION = 1 << 0,
+ IB_CQ_FLAGS_IGNORE_OVERRUN = 1 << 1,
};
struct ib_cq_init_attr {
@@ -866,6 +874,9 @@ enum ib_qp_type {
enum ib_qp_create_flags {
IB_QP_CREATE_IPOIB_UD_LSO = 1 << 0,
IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK = 1 << 1,
+ IB_QP_CREATE_CROSS_CHANNEL = 1 << 2,
+ IB_QP_CREATE_MANAGED_SEND = 1 << 3,
+ IB_QP_CREATE_MANAGED_RECV = 1 << 4,
IB_QP_CREATE_NETIF_QP = 1 << 5,
IB_QP_CREATE_SIGNATURE_EN = 1 << 6,
IB_QP_CREATE_USE_GFP_NOIO = 1 << 7,
--
1.7.12.4
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
