Re: linux-next: build failure after merge of the infiniband tree
On Mon, Dec 15, 2014 at 5:56 PM, Roland Dreier wrote: > I'll add a partial revert of that patch to my tree to get back the > now-used enum values. I rebased my tree on top of the merge-window merge of davem's tree, and added the missing flag on top of the "remove this flag" commit. - R. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux-next: build failure after merge of the infiniband tree
On Mon, Dec 15, 2014 at 5:47 PM, Stephen Rothwell wrote: > Hi all, > > After merging the infiniband tree, today's linux-next build (x86_64 > allmodconfig) failed like this: > > drivers/infiniband/hw/mlx5/main.c: In function 'mlx5_ib_query_device': > drivers/infiniband/hw/mlx5/main.c:248:34: error: > 'MLX5_DEV_CAP_FLAG_ON_DMND_PG' undeclared (first use in this function) > if (dev->mdev->caps.gen.flags & MLX5_DEV_CAP_FLAG_ON_DMND_PG) > ^ > drivers/net/ethernet/mellanox/mlx5/core/fw.c: In function > 'mlx5_query_odp_caps': > drivers/net/ethernet/mellanox/mlx5/core/fw.c:79:30: error: > 'MLX5_DEV_CAP_FLAG_ON_DMND_PG' undeclared (first use in this function) > if (!(dev->caps.gen.flags & MLX5_DEV_CAP_FLAG_ON_DMND_PG)) > ^ > drivers/net/ethernet/mellanox/mlx5/core/eq.c: In function 'mlx5_start_eqs': > drivers/net/ethernet/mellanox/mlx5/core/eq.c:459:28: error: > 'MLX5_DEV_CAP_FLAG_ON_DMND_PG' undeclared (first use in this function) > if (dev->caps.gen.flags & MLX5_DEV_CAP_FLAG_ON_DMND_PG) > ^ > > Really? Code added half way though the merge window not even build > tested? It's not quite as bad as it seems. The infiniband tree itself builds, the problem is the merged tree. The Mellanox guys merged the "cleanup" commit 0c7aac854f52 Author: Eli Cohen Date: Tue Dec 2 02:26:14 2014 net/mlx5_core: Remove unused dev cap enum fields These enumerations are not used so remove them. Signed-off-by: Eli Cohen Signed-off-by: David S. Miller through davem's tree, and then went ahead and used at least MLX5_DEV_CAP_FLAG_ON_DMND_PG (which that patch removes) in patches they merged through my tree. I'll add a partial revert of that patch to my tree to get back the now-used enum values. - R. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
linux-next: build failure after merge of the infiniband tree
Hi all, After merging the infiniband tree, today's linux-next build (x86_64 allmodconfig) failed like this: drivers/infiniband/hw/mlx5/main.c: In function 'mlx5_ib_query_device': drivers/infiniband/hw/mlx5/main.c:248:34: error: 'MLX5_DEV_CAP_FLAG_ON_DMND_PG' undeclared (first use in this function) if (dev->mdev->caps.gen.flags & MLX5_DEV_CAP_FLAG_ON_DMND_PG) ^ drivers/net/ethernet/mellanox/mlx5/core/fw.c: In function 'mlx5_query_odp_caps': drivers/net/ethernet/mellanox/mlx5/core/fw.c:79:30: error: 'MLX5_DEV_CAP_FLAG_ON_DMND_PG' undeclared (first use in this function) if (!(dev->caps.gen.flags & MLX5_DEV_CAP_FLAG_ON_DMND_PG)) ^ drivers/net/ethernet/mellanox/mlx5/core/eq.c: In function 'mlx5_start_eqs': drivers/net/ethernet/mellanox/mlx5/core/eq.c:459:28: error: 'MLX5_DEV_CAP_FLAG_ON_DMND_PG' undeclared (first use in this function) if (dev->caps.gen.flags & MLX5_DEV_CAP_FLAG_ON_DMND_PG) ^ Really? Code added half way though the merge window not even build tested? I have used the infiniband tree from next-20141215 for today. -- Cheers, Stephen Rothwells...@canb.auug.org.au pgpGXnyTds1Qh.pgp Description: OpenPGP digital signature
linux-next: manual merge of the infiniband tree with Linus' tree
Hi all,
Today's linux-next merge of the infiniband tree got a conflict in
drivers/infiniband/hw/mlx5/mr.c between commit 479163f46082 ("mlx5:
don't duplicate kvfree()") from Linus' tree and commit 89c925949c1f
("IB/mlx5: Implement on demand paging by adding support for MMU
notifiers") from the infiniband tree.
I fixed it up (see below) and can carry the fix as necessary (no action
is required).
--
Cheers,
Stephen Rothwells...@canb.auug.org.au
diff --cc drivers/infiniband/hw/mlx5/mr.c
index 5a80dd993761,a2dd7bfc129b..
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@@ -856,7 -1009,8 +1012,8 @@@ static struct mlx5_ib_mr *reg_create(st
goto err_2;
}
mr->umem = umem;
+ mr->live = 1;
- mlx5_vfree(in);
+ kvfree(in);
mlx5_ib_dbg(dev, "mkey = 0x%x\n", mr->mmr.key);
pgpIk2yVaEiSS.pgp
Description: OpenPGP digital signature
RE: [PATCH 1/1] ibacm: Search the pthread and dynamic linker/loader library when linking.
Thanks - applied -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/4] common: dapl_ep_free must serialize CM object destroy
From: Arlin Davis
CM object could be destroyed from time_wait state from
provider in separate thread. Destruction must be serialized
with EP lock.
Signed-off-by: Arlin Davis
---
dapl/common/dapl_ep_free.c | 21 +++--
1 files changed, 11 insertions(+), 10 deletions(-)
diff --git a/dapl/common/dapl_ep_free.c b/dapl/common/dapl_ep_free.c
index 62d7eff..652b61d 100644
--- a/dapl/common/dapl_ep_free.c
+++ b/dapl/common/dapl_ep_free.c
@@ -66,7 +66,7 @@ DAT_RETURN DAT_API dapl_ep_free(IN DAT_EP_HANDLE ep_handle)
DAPL_EP *ep_ptr;
DAPL_IA *ia_ptr;
DAT_EP_PARAM *param;
- dp_ib_cm_handle_t cm_ptr, next_cm_ptr;
+ dp_ib_cm_handle_t cm_ptr;
ib_qp_state_t save_qp_state;
DAT_RETURN dat_status = DAT_SUCCESS;
@@ -110,25 +110,26 @@ DAT_RETURN DAT_API dapl_ep_free(IN DAT_EP_HANDLE
ep_handle)
*/
(void)dapl_ep_disconnect(ep_ptr, DAT_CLOSE_ABRUPT_FLAG);
+ /*
+* Do verification of parameters and the state change atomically.
+*/
+ dapl_os_lock(&ep_ptr->header.lock);
+
/* Free all CM objects */
cm_ptr = (dapl_llist_is_empty(&ep_ptr->cm_list_head)
? NULL : dapl_llist_peek_head(&ep_ptr->cm_list_head));
+
while (cm_ptr != NULL) {
dapl_log(DAPL_DBG_TYPE_EP,
"dapl_ep_free: Free CM: EP=%p CM=%p\n",
ep_ptr, cm_ptr);
-
- next_cm_ptr = dapl_llist_next_entry(&ep_ptr->cm_list_head,
- &cm_ptr->list_entry);
+ dapl_os_unlock(&ep_ptr->header.lock);
dapls_cm_free(cm_ptr); /* blocking call */
- cm_ptr = next_cm_ptr;
+ dapl_os_lock(&ep_ptr->header.lock);
+ cm_ptr = (dapl_llist_is_empty(&ep_ptr->cm_list_head)
+ ? NULL : dapl_llist_peek_head(&ep_ptr->cm_list_head));
}
- /*
-* Do verification of parameters and the state change atomically.
-*/
- dapl_os_lock(&ep_ptr->header.lock);
-
#ifdef DAPL_DBG
/* check if event pending and warn, don't assert, state is valid */
if (ep_ptr->param.ep_state == DAT_EP_STATE_DISCONNECT_PENDING) {
--
1.7.3
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/4] DAPL: bug fixes
From: Arlin Davis Arlin Davis (4): ucm: RTU not retransmitted in TIMEWAIT state dtestx: allow scale up to 1000 EP's common: dapl_ep_free must serialize CM object destroy ucm: add time wait override capability for CM services dapl/common/dapl_ep_free.c | 21 --- dapl/openib_common/dapl_ib_common.h |1 + dapl/openib_ucm/cm.c| 49 +-- dapl/openib_ucm/dapl_ib_util.h |1 + dapl/openib_ucm/device.c|9 +++--- test/dtest/dtestx.c | 21 +- 6 files changed, 55 insertions(+), 47 deletions(-) -- 1.7.3 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 4/4] ucm: add time wait override capability for CM services
From: Arlin Davis
New environment variable DAPL_UCM_WAIT_TIME (ms) to
override the default wait_time for CM services.
Default setting is 60 seconds.
Signed-off-by: Arlin Davis
---
dapl/openib_common/dapl_ib_common.h |1 +
dapl/openib_ucm/cm.c| 52 +-
dapl/openib_ucm/dapl_ib_util.h |1 +
dapl/openib_ucm/device.c|9 +++---
4 files changed, 27 insertions(+), 36 deletions(-)
diff --git a/dapl/openib_common/dapl_ib_common.h
b/dapl/openib_common/dapl_ib_common.h
index c1b9267..d5b26ec 100644
--- a/dapl/openib_common/dapl_ib_common.h
+++ b/dapl/openib_common/dapl_ib_common.h
@@ -225,6 +225,7 @@ typedef uint16_tib_hca_port_t;
#define DCM_RETRY_CNT 10
#define DCM_REP_TIME800/* reply timeout in m_secs */
#define DCM_RTU_TIME800/* rtu timeout in m_secs */
+#define DCM_WAIT_TIME 6 /* wait timeout in m_secs */
#define DCM_QP_SIZE 500 /* uCM tx, rx qp size */
#define DCM_CQ_SIZE 500 /* uCM cq size */
#define DCM_TX_BURST 50 /* uCM signal, every TX burst msgs posted */
diff --git a/dapl/openib_ucm/cm.c b/dapl/openib_ucm/cm.c
index 141086d..04d5eac 100644
--- a/dapl/openib_ucm/cm.c
+++ b/dapl/openib_ucm/cm.c
@@ -231,38 +231,26 @@ static void ucm_check_timers(dp_ib_cm_handle_t cm, int
*timer)
*timer = cm->hca->ib_trans.cm_timer;
if ((time - cm->timer)/1000 >
(cm->hca->ib_trans.rtu_time << cm->retries)) {
- dapl_log(DAPL_DBG_TYPE_CM,
-" CM_TIMEWAIT %d %p [lid, port, cqp, iqp]:"
-" %x %x %x %x -> %x %x %x %x r_pid %x"
-" Time(ms) %d > %d\n",
-cm->retries+1, cm,
-ntohs(cm->msg.saddr.ib.lid),
ntohs(cm->msg.sport),
-ntohl(cm->msg.sqpn),
ntohl(cm->msg.saddr.ib.qpn),
-ntohs(cm->msg.daddr.ib.lid),
ntohs(cm->msg.dport),
-ntohl(cm->msg.dqpn),
ntohl(cm->msg.daddr.ib.qpn),
-ntohl(cm->msg.d_id),
-(time - cm->timer)/1000,
-cm->hca->ib_trans.rtu_time << cm->retries);
cm->retries++;
- }
- if (cm->retries > 2) {
- dapl_log(DAPL_DBG_TYPE_CM_WARN,
-" CM_TIMEWAIT EXPIRED %d %p [lid, port, cqp,
iqp]:"
-" %x %x %x %x -> %x %x %x %x r_pid %x"
-" Time(ms) %d > %d\n",
-cm->retries+1, cm,
-ntohs(cm->msg.saddr.ib.lid),
ntohs(cm->msg.sport),
-ntohl(cm->msg.sqpn),
ntohl(cm->msg.saddr.ib.qpn),
-ntohs(cm->msg.daddr.ib.lid),
ntohs(cm->msg.dport),
-ntohl(cm->msg.dqpn),
ntohl(cm->msg.daddr.ib.qpn),
-ntohl(cm->msg.d_id),
-(time - cm->timer)/1000,
-cm->hca->ib_trans.rtu_time << cm->retries);
- cm->ah = NULL; /* consumer will free AH */
- cm->state = DCM_FREE;
- dapl_os_unlock(&cm->lock);
- dapl_ep_unlink_cm(cm->ep, cm); /* last CM ref */
- return;
+ if ((time - cm->timer)/1000 >
cm->hca->ib_trans.wait_time) {
+ dapl_log(DAPL_DBG_TYPE_CM_WARN,
+" CM_TIMEWAIT EXPIRED %d %p [lid,
port, cqp, iqp]:"
+" %x %x %x %x -> %x %x %x %x r_pid %x"
+" Time(ms) %d > %d\n",
+cm->retries+1, cm,
+ntohs(cm->msg.saddr.ib.lid),
ntohs(cm->msg.sport),
+ntohl(cm->msg.sqpn),
ntohl(cm->msg.saddr.ib.qpn),
+ntohs(cm->msg.daddr.ib.lid),
ntohs(cm->msg.dport),
+ntohl(cm->msg.dqpn),
ntohl(cm->msg.daddr.ib.qpn),
+ntohl(cm->msg.d_id),
+(time - cm->timer)/1000,
+cm->hca->ib_trans.wait_time);
+ cm->ah = NULL; /* consumer will free AH */
+ cm->state = DCM_FREE;
+ dapl_os_unlock(&cm->lock);
+ dapl_ep_unlink_cm(cm->ep, cm); /* last CM ref
*/
+ return;
+ }
}
break;
@@ -737,7 +725,7 @@ void dapls_cm_r
[PATCH 1/4] ucm: RTU not retransmitted in TIMEWAIT state
From: Arlin Davis
Signed-off-by: Arlin Davis
---
dapl/openib_ucm/cm.c | 21 -
1 files changed, 16 insertions(+), 5 deletions(-)
diff --git a/dapl/openib_ucm/cm.c b/dapl/openib_ucm/cm.c
index e462632..141086d 100644
--- a/dapl/openib_ucm/cm.c
+++ b/dapl/openib_ucm/cm.c
@@ -231,7 +231,7 @@ static void ucm_check_timers(dp_ib_cm_handle_t cm, int
*timer)
*timer = cm->hca->ib_trans.cm_timer;
if ((time - cm->timer)/1000 >
(cm->hca->ib_trans.rtu_time << cm->retries)) {
- dapl_log(DAPL_DBG_TYPE_CM_WARN,
+ dapl_log(DAPL_DBG_TYPE_CM,
" CM_TIMEWAIT %d %p [lid, port, cqp, iqp]:"
" %x %x %x %x -> %x %x %x %x r_pid %x"
" Time(ms) %d > %d\n",
@@ -247,8 +247,17 @@ static void ucm_check_timers(dp_ib_cm_handle_t cm, int
*timer)
}
if (cm->retries > 2) {
dapl_log(DAPL_DBG_TYPE_CM_WARN,
- " CM_TIMEWAIT expired (%d ms) for CM %p\n",
-cm, (time - cm->timer)/1000);
+" CM_TIMEWAIT EXPIRED %d %p [lid, port, cqp,
iqp]:"
+" %x %x %x %x -> %x %x %x %x r_pid %x"
+" Time(ms) %d > %d\n",
+cm->retries+1, cm,
+ntohs(cm->msg.saddr.ib.lid),
ntohs(cm->msg.sport),
+ntohl(cm->msg.sqpn),
ntohl(cm->msg.saddr.ib.qpn),
+ntohs(cm->msg.daddr.ib.lid),
ntohs(cm->msg.dport),
+ntohl(cm->msg.dqpn),
ntohl(cm->msg.daddr.ib.qpn),
+ntohl(cm->msg.d_id),
+(time - cm->timer)/1000,
+cm->hca->ib_trans.rtu_time << cm->retries);
cm->ah = NULL; /* consumer will free AH */
cm->state = DCM_FREE;
dapl_os_unlock(&cm->lock);
@@ -382,8 +391,10 @@ static void ucm_process_recv(ib_hca_transport_t *tp,
ucm_connect_rtu(cm, msg);
break;
case DCM_CONNECTED: /* active and passive */
+ case DCM_TIMEWAIT: /* passive */
/* DREQ, change state and process */
- cm->retries = 2;
+ if (cm->state == DCM_CONNECTED)
+ cm->retries = 2;
if (ntohs(msg->op) == DCM_DREQ) {
cm->state = DCM_DISC_RECV;
dapl_os_unlock(&cm->lock);
@@ -395,7 +406,7 @@ static void ucm_process_recv(ib_hca_transport_t *tp,
dapl_log(DAPL_DBG_TYPE_CM_WARN,
" RESEND RTU: op %s st %s [lid, port, cqp,
iqp]:"
" %x %x %x %x -> %x %x %x %x r_pid %x\n",
- dapl_cm_op_str(ntohs(cm->msg.op)),
+ dapl_cm_op_str(ntohs(msg->op)),
dapl_cm_state_str(cm->state),
ntohs(cm->msg.saddr.ib.lid),
ntohs(cm->msg.sport),
ntohl(cm->msg.sqpn),
ntohl(cm->msg.saddr.ib.qpn),
--
1.7.3
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/4] dtestx: allow scale up to 1000 EP's
From: Arlin Davis
Signed-off-by: Arlin Davis
---
test/dtest/dtestx.c | 21 +
1 files changed, 13 insertions(+), 8 deletions(-)
diff --git a/test/dtest/dtestx.c b/test/dtest/dtestx.c
index 8b3294a..931c860 100755
--- a/test/dtest/dtestx.c
+++ b/test/dtest/dtestx.c
@@ -143,7 +143,7 @@ int disconnect_ep(void);
#define RCV_RDMA_BUF_INDEX 1
#define SEND_BUF_INDEX 2
#define RECV_BUF_INDEX 3
-#define MAX_EP_COUNT 8
+#define MAX_EP_COUNT 1000
#define MAX_AH_COUNT (MAX_EP_COUNT * 2)
DAT_VADDR *atomic_buf;
@@ -520,8 +520,8 @@ send_msg(void *data,
if (ud_test) {
/*
* single QP - ep[0] and ah[0] for client and server
-* multi QP - ep[i]->ah[i] for client, to 0
-* ep[0]->ah[i] for server, to all
+* multi QP - ep[i]->ah[i] for client, i to i
+* ep[0]->ah[i] for server, 0 to all
*/
if (multi_eps) {
ah_idx = i;
@@ -704,7 +704,10 @@ void process_conn(int idx)
}
if (ia_attr.extension_version >= 209) {
- /* Destroy CM resources for AH resolution */
+ LOGPRINTF("%s EP_L[%d]=%p -> cm %p CM_FREE\n",
+ ext_event->type == DAT_IB_UD_PASSIVE_REMOTE_AH ?
+ "PASSIVE":"ACTIVE ", ep_l, ep[ep_l],
+ ext_event->context.as_ptr);
dat_ib_ud_cm_free(ep[ep_l], ext_event->context.as_ptr);
}
@@ -723,7 +726,7 @@ int connect_ep(char *hostname, struct sockaddr *ser_sa)
DAT_DTO_COOKIE cookie;
DAT_CONN_QUAL conn_qual;
DAT_BOOLEAN in, out;
- int i, ii, pdata, ctx;
+ int i, ii, pdata, ctx, qdepth = REG_MEM_COUNT;
DAT_DTO_COMPLETION_EVENT_DATA *dto_event =
&event.event_data.dto_completion_event_data;
@@ -762,8 +765,10 @@ int connect_ep(char *hostname, struct sockaddr *ser_sa)
}
ep_attr.qos = 0;
ep_attr.recv_completion_flags = 0;
- ep_attr.max_recv_dtos = eps * 10;
- ep_attr.max_request_dtos = eps * 10;
+ if (ud_test && !multi_eps)
+ qdepth = eps * REG_MEM_COUNT;
+ ep_attr.max_recv_dtos = qdepth;
+ ep_attr.max_request_dtos = qdepth;
ep_attr.max_recv_iov = 1;
ep_attr.max_request_iov = 1;
ep_attr.request_completion_flags = DAT_COMPLETION_DEFAULT_FLAG;
@@ -1548,7 +1553,7 @@ int main(int argc, char **argv)
switch (rc) {
case 'u':
ud_test = 1;
- eps = MAX_EP_COUNT / 2;
+ eps = 4;
break;
case 'm':
multi_eps = 1;
--
1.7.3
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/1] ibacm: Search the pthread and dynamic linker/loader library when linking.
From: John Fleck When building with RHEL 7, there is an undefined reference to the pthread and the dynamic linker/loader libraries. Signed-off-by: John Fleck --- Makefile.am |5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Makefile.am b/Makefile.am index c71ed7c..73b1a31 100644 --- a/Makefile.am +++ b/Makefile.am @@ -1,6 +1,7 @@ AM_CPPFLAGS = -I$(srcdir)/include -I$(srcdir)/linux -I$(srcdir)/src AM_CFLAGS = -g -Wall -D_GNU_SOURCE +AM_LDFLAGS = -lpthread -ldl if HAVE_LD_VERSION_SCRIPT libibacmp_version_script = -Wl,--version-script=$(srcdir)/prov/acmp/src/libibacmp.map @@ -13,14 +14,14 @@ sbin_PROGRAMS = svc/ibacm svc_ibacm_SOURCES = src/acm.c src/acm_util.c util_ib_acme_SOURCES = src/acme.c src/libacm.c linux/libacm_linux.c src/parse.c svc_ibacm_CFLAGS = $(AM_CFLAGS) -svc_ibacm_LDFLAGS = -rdynamic +svc_ibacm_LDFLAGS = -rdynamic $(AM_LDFLAGS) util_ib_acme_CFLAGS = $(AM_CFLAGS) -DACME_PRINTS pkglib_LTLIBRARIES = lib/libibacmp.la lib_libibacmp_la_CFLAGS = $(AM_CFLAGS) lib_libibacmp_la_SOURCES = prov/acmp/src/acmp.c lib_libibacmp_la_LDFLAGS = -version-info 1 -export-dynamic \ - $(libibacmp_version_script) + $(libibacmp_version_script) $(AM_LDFLAGS) lib_libibacmp_la_DEPENDENCIES = $(srcdir)/prov/acmp/src/libibacmp.map ibacmincludedir = $(includedir)/infiniband -- 1.7.1 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/3] dtestx: update IB extension example test with new v2.0.9 features
From: Arlin Davis
Add support for new IB extensions for CM and AH resource cleanup.
Check for v2.0.9 and call dat_ib_ud_cm_free after connection
establishment and dat_ib_ud_ah_free after all data has been
transfered on UD endpoints.
Also add socket based address exchange to eliminate the need
to include lid and qpn parameters on the client side.
Change the multiple EP mode to send from EP 0 to EP[0-3] on
server side and EP[0-3] to EP[0-3] on client side.
Signed-off-by: Arlin Davis
---
test/dtest/dtestx.c | 581 +++
1 files changed, 451 insertions(+), 130 deletions(-)
diff --git a/test/dtest/dtestx.c b/test/dtest/dtestx.c
index 32ffe33..8b3294a 100755
--- a/test/dtest/dtestx.c
+++ b/test/dtest/dtestx.c
@@ -60,7 +60,7 @@
#include
#include
-#define DAPL_PROVIDER "ofa-v2-ib0"
+#define DAPL_PROVIDER "ofa-v2-mlx4_0-1u"
#define F64x "%"PRIx64""
#define F64u "%"PRIu64""
@@ -144,6 +144,7 @@ int disconnect_ep(void);
#define SEND_BUF_INDEX 2
#define RECV_BUF_INDEX 3
#define MAX_EP_COUNT 8
+#define MAX_AH_COUNT (MAX_EP_COUNT * 2)
DAT_VADDR *atomic_buf;
DAT_LMR_HANDLE lmr_atomic;
@@ -174,16 +175,22 @@ int buf_size = BUF_SIZE;
int msg_size = sizeof(DAT_RMR_TRIPLET);
char provider[64] = DAPL_PROVIDER;
char hostname[256] = { 0 };
-DAT_IB_ADDR_HANDLE remote_ah[MAX_EP_COUNT];
+static DAT_IB_ADDR_HANDLE remote_ah[MAX_EP_COUNT][MAX_AH_COUNT];
int eps = 1;
int verbose = 0;
int counters = 0;
int counters_ok = 0;
static int ucm = 0;
static DAT_SOCK_ADDR6 remote;
+static DAT_IA_ATTR ia_attr;
+static DAT_PROVIDER_ATTR prov_attrs;
#define LOGPRINTF if (verbose) printf
+#define CONN_PORT 15828
+#define CONN_MSG_SIZE 128
+#define CONN_MSG_FMT "%04hx:%08x:%08x:%08x:%s"
+
void print_usage(void)
{
printf("\n dtestx usage \n\n");
@@ -196,7 +203,7 @@ void print_usage(void)
printf("h: hostname/address of Server, client and UDP server\n");
printf("c: Client\n");
printf("s: Server, default\n");
- printf("P: provider name (default = ofa-v2-ib0)\n");
+ printf("P: provider name (default = ofa-v2-mlx4_0-1u)\n");
printf("\n");
}
@@ -232,6 +239,265 @@ static void print_ia_address(struct sockaddr *sa)
}
}
+int conn_client_connect(const char *servername, int port)
+{
+
+ struct addrinfo *res, *t;
+ struct addrinfo hints = {
+ .ai_family = AF_UNSPEC,
+ .ai_socktype = SOCK_STREAM
+ };
+ char *service;
+ int n;
+ int sockfd = -1;
+
+ if (asprintf(&service, "%d", port) < 0)
+ return -1;
+
+ n = getaddrinfo(servername, service, &hints, &res);
+
+ if (n < 0) {
+ fprintf(stderr, "%s for %s:%d\n",
+ gai_strerror(n), servername, port);
+ return n;
+ }
+
+ for (t = res; t; t = t->ai_next) {
+ sockfd = socket(t->ai_family, t->ai_socktype, t->ai_protocol);
+ if (sockfd >= 0) {
+ if (!connect(sockfd, t->ai_addr, t->ai_addrlen))
+ break;
+ close(sockfd);
+ sockfd = -1;
+ }
+ }
+
+ freeaddrinfo(res);
+
+ if (sockfd < 0) {
+ fprintf(stderr, "Couldn't connect to %s:%d\n",
+ servername, port);
+ return sockfd;
+ }
+ return sockfd;
+}
+
+int conn_server_connect(int port)
+{
+ struct addrinfo *res, *t;
+ struct addrinfo hints = {
+ .ai_flags= AI_PASSIVE,
+ .ai_family = AF_UNSPEC,
+ .ai_socktype = SOCK_STREAM
+ };
+ char *service;
+ int sockfd = -1, connfd;
+ int n;
+
+ if (asprintf(&service, "%d", port) < 0)
+ return -1;
+
+ n = getaddrinfo(NULL, service, &hints, &res);
+
+ if (n < 0) {
+ fprintf(stderr, "%s for port %d\n", gai_strerror(n), port);
+ return n;
+ }
+
+ for (t = res; t; t = t->ai_next) {
+ sockfd = socket(t->ai_family, t->ai_socktype, t->ai_protocol);
+ if (sockfd >= 0) {
+ n = 1;
+
+ setsockopt(sockfd, SOL_SOCKET, SO_REUSEADDR, &n,
+ sizeof n);
+
+ if (!bind(sockfd, t->ai_addr, t->ai_addrlen))
+ break;
+
+ close(sockfd);
+ sockfd = -1;
+ }
+ }
+
+ freeaddrinfo(res);
+
+ if (sockfd < 0) {
+ fprintf(stderr, "Couldn't listen to port %d\n", port);
+ return sockfd;
+ }
+
+ listen(sockfd, 1);
+ connfd = accept(sockfd, NULL, 0);
+ if (connfd < 0) {
+ perror("server accept");
+ fprintf(stderr, "accept() failed\n");
+ close(sockfd
[PATCH 2/3] dtest: add dtestsrq for SRQ example and provider testing
From: Amir Hanania
Signed-off-by: Arlin Davis
---
test/dtest/dtestsrq.c | 1605 +
1 files changed, 1605 insertions(+), 0 deletions(-)
create mode 100644 test/dtest/dtestsrq.c
diff --git a/test/dtest/dtestsrq.c b/test/dtest/dtestsrq.c
new file mode 100644
index 000..2bc2596
--- /dev/null
+++ b/test/dtest/dtestsrq.c
@@ -0,0 +1,1605 @@
+/*
+ * Copyright (c) 2009 Intel Corporation. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ *copyright notice, this list of conditions and the following
+ *disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ *copyright notice, this list of conditions and the following
+ *disclaimer in the documentation and/or other materials
+ *provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ *
+ * $Id: $
+ */
+#include
+#include
+#include
+
+#ifdef DAPL_PROVIDER
+#undef DAPL_PROVIDER
+#endif
+
+#if defined(_WIN32) || defined(_WIN64)
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include "..\..\..\..\etc\user\getopt.c"
+
+#define getpid() ((int)GetCurrentProcessId())
+#define F64x "%I64x"
+#define F64d "%I64d"
+
+#ifdef DBG
+#define DAPL_PROVIDER "ibnic0v2d"
+#else
+#define DAPL_PROVIDER "ibnic0v2"
+#endif
+
+#define ntohll _byteswap_uint64
+#define htonll _byteswap_uint64
+
+#else // _WIN32 || _WIN64
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#define DAPL_PROVIDER "ofa-v2-ib0"
+
+#define F64x "%"PRIx64""
+#define F64d "%"PRId64""
+
+
+#if __BYTE_ORDER == __BIG_ENDIAN
+#define htonll(x) (x)
+#define ntohll(x) (x)
+#elif __BYTE_ORDER == __LITTLE_ENDIAN
+#define htonll(x) bswap_64(x)
+#define ntohll(x) bswap_64(x)
+#endif
+
+#endif // _WIN32 || _WIN64
+
+/* Header files needed for DAT/uDAPL */
+#include "dat2/udat.h"
+#include "dat2/dat_ib_extensions.h"
+
+/* definitions */
+#define SERVER_CONN_QUAL 45248
+#define CONN_TIMEOUT (1000*1000*100)
+#define DTO_TIMEOUT(1000*1000*5)
+#define CR_TIMEOUT DAT_TIMEOUT_INFINITE
+#define MAX_CONN 100
+#define MAX_BURST 100
+#define MSG_IOV_COUNT 1
+
+/* Global DAT vars */
+static DAT_IA_HANDLE h_ia = DAT_HANDLE_NULL;
+static DAT_PZ_HANDLE h_pz = DAT_HANDLE_NULL;
+static DAT_SRQ_HANDLE h_srq = DAT_HANDLE_NULL;
+static DAT_CR_HANDLE h_cr = DAT_HANDLE_NULL;
+static DAT_PSP_HANDLE h_psp = DAT_HANDLE_NULL;
+static DAT_IB_ADDR_HANDLE *remote_ah = DAT_HANDLE_NULL;
+static DAT_EP_HANDLE *h_ep;
+
+static DAT_EVD_HANDLE h_async_evd = DAT_HANDLE_NULL;
+static DAT_EVD_HANDLE h_dto_req_evd = DAT_HANDLE_NULL;
+static DAT_EVD_HANDLE h_dto_rcv_evd = DAT_HANDLE_NULL;
+static DAT_EVD_HANDLE h_cr_evd = DAT_HANDLE_NULL;
+static DAT_EVD_HANDLE h_conn_evd = DAT_HANDLE_NULL;
+
+static DAT_EP_ATTR ep_attr;
+char hostname[256] = { 0 };
+char provider[64] = DAPL_PROVIDER;
+char addr_str[INET_ADDRSTRLEN];
+
+int status;
+
+/* defaults */
+static int server = 1;
+static int verbose = 0;
+static int connections = 1;
+static int bursts_number = 3;
+static int burst_size = 1;
+static int server_port_id = SERVER_CONN_QUAL;
+static int client_port_id = SERVER_CONN_QUAL + 1;
+static int ucm = 0;
+static int ud_test = 0;
+static int srq_test = 1;
+
+/* RDMA buffers */
+static DAT_LMR_HANDLE h_lmr_send = DAT_HANDLE_NULL;
+static DAT_LMR_HANDLE h_lmr_recv = DAT_HANDLE_NULL;
+static DAT_LMR_CONTEXT lmr_context_send;
+static DAT_LMR_CONTEXT lmr_context_recv;
+static DAT_RMR_CONTEXT rmr_context_send;
+static DAT_RMR_CONTEXT rmr_context_recv;
+static DAT_VLEN registered_size_send;
+static DAT_VLEN registered_size_recv;
+static DAT_VADDR registered_addr_send;
+static DAT_VADDR registered_addr_recv;
+
+static int tx_buf_len = 0, rx_buf_len;
+static int connected = 0;
+char *rbuf = NULL;
+char *sbuf = NULL;
+static DAT_SOCK_ADDR6 remote;
+
+/* forward prototypes */
+cons
[PATCH 1/3] common: add srq support for openib verbs providers
From: Amir Hanania Add necessary components and hooks to support ib_verbs shared receive queues for both RC and UD QP's. External interfaces were already provided per DAT 2.0 specification but internal support was missing. A new dtestsrq will be provided with package for testing and example code. Acked-by: Arlin Davis --- Makefile.am |4 + dapl/common/dapl_adapter_util.h | 13 +++ dapl/common/dapl_cr_callback.c |2 +- dapl/common/dapl_ep_create.c|2 +- dapl/common/dapl_ep_create_with_srq.c | 53 ++ dapl/common/dapl_ep_free.c |5 + dapl/common/dapl_ep_post_recv.c |5 + dapl/common/dapl_ep_util.c | 29 -- dapl/common/dapl_ep_util.h |3 +- dapl/common/dapl_evd_qp_async_error_callb.c | 15 ++- dapl/common/dapl_evd_util.c | 49 - dapl/common/dapl_srq_create.c | 23 ++--- dapl/common/dapl_srq_free.c | 36 +++ dapl/common/dapl_srq_post_recv.c|6 +- dapl/common/dapl_srq_query.c|6 +- dapl/common/dapl_srq_resize.c |9 +- dapl/common/dapl_srq_set_lw.c |4 +- dapl/common/dapl_srq_util.c |2 - dapl/include/dapl.h |2 + dapl/openib_common/cq.c |3 +- dapl/openib_common/dapl_ib_common.h |1 + dapl/openib_common/dapl_ib_dto.h| 48 + dapl/openib_common/qp.c |5 + dapl/openib_common/srq.c| 147 +++ dapl/openib_common/util.c |9 ++- dat/include/dat2/dat.h |3 +- dat/include/dat2/dat_error.h|1 + test/dtest/Makefile.am |6 +- 28 files changed, 388 insertions(+), 103 deletions(-) create mode 100644 dapl/openib_common/srq.c diff --git a/Makefile.am b/Makefile.am index f9c4093..7ad5af1 100755 --- a/Makefile.am +++ b/Makefile.am @@ -273,6 +273,7 @@ dapl_udapl_libdaplofa_la_SOURCES = dapl/udapl/dapl_init.c \ dapl/openib_common/cq.c \ dapl/openib_common/qp.c \ dapl/openib_common/util.c \ +dapl/openib_common/srq.c\ dapl/openib_cma/cm.c\ dapl/openib_cma/device.c $(XPROGRAMS) @@ -389,6 +390,7 @@ dapl_udapl_libdaploscm_la_SOURCES = dapl/udapl/dapl_init.c \ dapl/openib_common/cq.c \ dapl/openib_common/qp.c \ dapl/openib_common/util.c \ +dapl/openib_common/srq.c\ dapl/openib_scm/cm.c\ dapl/openib_scm/device.c $(XPROGRAMS) @@ -505,6 +507,7 @@ dapl_udapl_libdaploucm_la_SOURCES = dapl/udapl/dapl_init.c \ dapl/openib_common/cq.c \ dapl/openib_common/qp.c \ dapl/openib_common/util.c \ +dapl/openib_common/srq.c\ dapl/openib_ucm/cm.c\ dapl/openib_ucm/device.c $(XPROGRAMS) @@ -622,6 +625,7 @@ dapl_udapl_libdaplomcm_la_SOURCES = dapl/udapl/dapl_init.c \ dapl/openib_common/cq.c \ dapl/openib_common/qp.c \ dapl/openib_common/util.c \ +dapl/openib_common/srq.c\ dapl/openib_mcm/cm.c\ dapl/openib_mcm/mix.c \ dapl/openib_mcm/proxy.c \ diff --git a/dapl/common/dapl_adapter_util.h b/dapl/common/dapl_adapter_util.h index 9b12b2d..ba4e73b 100755 --- a/dapl/common/dapl_adapter_util.h +++ b/dapl/common/dapl_adapter_util.h @@ -263,6 +263,19 @@ dapls_evd_dto_wait ( IN DAPL_EVD *evd_ptr, IN uint32_t timeout); +DAT_RETURN +dapls_ib_srq_alloc ( + IN DAPL_SRQ *srq_ptr); + +DAT_RETURN +dapls_ib_srq_free ( + IN DAPL_SRQ *srq_ptr); + +DAT_RETURN +dapls_ib_srq_resize( + IN DAPL_SRQ *srq_ptr, + IN uint32_t new_max_wr); + #ifdef DAT_EXTENSIONS void dapls_cqe_to_event_extension( diff --git a/dapl/common/dapl_cr_callback.c b/dapl/common/dapl_cr_callback.c index 8bfbb3e..1b7e6be 100644 --- a/dapl/common/dapl_cr_callback.c +++ b/dapl/common/dapl_cr_callback.c @@ -366,7 +366,7 @@ dapli_connection_request(IN dp_ib_cm_handle_t ib_cm_handle, * requestor that we cant help them. */ ia_ptr = sp_ptr->header.owner_ia; - ep_ptr = dapl_ep_alloc(ia_ptr, NULL); + ep_ptr = dapl_ep_alloc(ia_ptr, NULL
[PATCH 0/3] DAPL: SRQ support for ofa providers
From: Arlin Davis Amir Hanania (2): common: add srq support for openib verbs providers dtest: add dtestsrq for SRQ example and provider testing Arlin Davis (1): dtestx: update IB extension example test with new v2.0.9 features Makefile.am |4 + dapl/common/dapl_adapter_util.h | 13 + dapl/common/dapl_cr_callback.c |2 +- dapl/common/dapl_ep_create.c|2 +- dapl/common/dapl_ep_create_with_srq.c | 53 +- dapl/common/dapl_ep_free.c |5 + dapl/common/dapl_ep_post_recv.c |5 + dapl/common/dapl_ep_util.c | 29 +- dapl/common/dapl_ep_util.h |3 +- dapl/common/dapl_evd_qp_async_error_callb.c | 15 +- dapl/common/dapl_evd_util.c | 49 +- dapl/common/dapl_srq_create.c | 23 +- dapl/common/dapl_srq_free.c | 36 +- dapl/common/dapl_srq_post_recv.c|6 +- dapl/common/dapl_srq_query.c|6 +- dapl/common/dapl_srq_resize.c |9 +- dapl/common/dapl_srq_set_lw.c |4 +- dapl/common/dapl_srq_util.c |2 - dapl/include/dapl.h |2 + dapl/openib_common/cq.c |3 +- dapl/openib_common/dapl_ib_common.h |1 + dapl/openib_common/dapl_ib_dto.h| 48 + dapl/openib_common/qp.c |5 + dapl/openib_common/srq.c| 147 +++ dapl/openib_common/util.c |9 +- dat/include/dat2/dat.h |3 +- dat/include/dat2/dat_error.h|1 + test/dtest/Makefile.am |6 +- test/dtest/dtestsrq.c | 1605 +++ test/dtest/dtestx.c | 581 --- 30 files changed, 2444 insertions(+), 233 deletions(-) create mode 100644 dapl/openib_common/srq.c create mode 100644 test/dtest/dtestsrq.c -- 1.7.3 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/3] openib: add new TIMEWAIT state for CM
From: Arlin Davis Signed-off-by: Arlin Davis --- dapl/openib_common/dapl_ib_common.h |6 -- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/dapl/openib_common/dapl_ib_common.h b/dapl/openib_common/dapl_ib_common.h index 3b565b4..42391b2 100644 --- a/dapl/openib_common/dapl_ib_common.h +++ b/dapl/openib_common/dapl_ib_common.h @@ -340,6 +340,7 @@ typedef enum dapl_cm_state DCM_RTU_PENDING, DCM_DISC_RECV, DCM_FREE, + DCM_TIMEWAIT } DAPL_CM_STATE; @@ -433,9 +434,10 @@ STATIC _INLINE_ char * dapl_cm_state_str(IN int st) "CM_DESTROY", "CM_RTU_PENDING", "CM_DISC_RECV", - "CM_FREE" + "CM_FREE", + "CM_TIMEWAIT" }; -return ((st < 0 || st > 16) ? "Invalid CM state?" : state[st]); +return ((st < 0 || st > 17) ? "Invalid CM state?" : state[st]); } STATIC _INLINE_ char * dapl_cm_op_str(IN int op) -- 1.7.3 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/3] extension: add IB UD extensions to reduce provider CM and AH memory footprint
From: Arlin Davis
dat_ib_ud_cm_free, dat_ib_ud_ah_free added to allow consumers
the option to free provider CM and AH objects, related to AH resolution,
immediately after consuming CONN events instead of waiting for
EP destroy. With existing UD service providers the CM and AH objects
are linked to EP and not destroyed until consumer calls dat_ep_free.
dat_ib_ud_cm_free() frees CM object after AH and private data are copied
and stored by consumer. Provider will destroy internal object
and memory associated with CM and AH resolution.
MAY be called after CM establishment and before EP destroyed
dat_ib_ud_ah_free() destroys UD Address Handle (AH).
MUST be called after all UD sends are complete and
before UD EP is destroyed.
Signed-off-by: Arlin Davis
---
dapl/openib_common/ib_extensions.c | 40
dat/include/dat2/dat_ib_extensions.h | 47 +++---
2 files changed, 83 insertions(+), 4 deletions(-)
diff --git a/dapl/openib_common/ib_extensions.c
b/dapl/openib_common/ib_extensions.c
index 339dae7..fc03d9c 100644
--- a/dapl/openib_common/ib_extensions.c
+++ b/dapl/openib_common/ib_extensions.c
@@ -188,6 +188,46 @@ dapl_extensions(IN DAT_HANDLE dat_handle,
OP_SEND_UD, comp_flags, ah);
break;
+ case DAT_IB_UD_CM_FREE_OP:
+ {
+ ib_cm_srvc_handle_t cm;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_RTN,
+" UD cm_free extension call\n");
+
+ ep = dat_handle;
+ cm = (ib_cm_srvc_handle_t) va_arg(args, DAT_UINT64);
+
+ if (DAPL_BAD_HANDLE(ep, DAPL_MAGIC_EP))
+ status = DAT_ERROR(DAT_INVALID_HANDLE,
+ DAT_INVALID_HANDLE_EP);
+ else {
+ cm->ah = NULL; /* consumer will free AH */
+ status = dapls_ud_cm_free(ep, cm);
+ }
+ break;
+ }
+ case DAT_IB_UD_AH_FREE_OP:
+ {
+ DAT_IB_ADDR_HANDLE *dat_ah;
+ int ret;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_RTN,
+" UD ah_free extension call\n");
+
+ ep = dat_handle;
+ dat_ah = va_arg(args, DAT_IB_ADDR_HANDLE *);
+
+ if (DAPL_BAD_HANDLE(ep, DAPL_MAGIC_EP)) {
+ status = DAT_ERROR(DAT_INVALID_HANDLE,
+ DAT_INVALID_HANDLE_EP);
+ } else {
+ errno = 0;
+ ret = ibv_destroy_ah(dat_ah->ah);
+ status = dapl_convert_errno(errno, "destroy_ah");
+ }
+ break;
+ }
#ifdef DAPL_COUNTERS
case DAT_QUERY_COUNTERS_OP:
{
diff --git a/dat/include/dat2/dat_ib_extensions.h
b/dat/include/dat2/dat_ib_extensions.h
index 31e7f48..94ded25 100755
--- a/dat/include/dat2/dat_ib_extensions.h
+++ b/dat/include/dat2/dat_ib_extensions.h
@@ -75,9 +75,9 @@
* 2.0.6 - Add MPI over IB collective extensions
* 2.0.7 - Add new IA counters for dapl CM, device LINK, device DIAG
* 2.0.8 - Add DAT_IB_OPEN_QUERY_OP, DAT_IB_CLOSE_QUERY_OP, fast provider
query interface
- *
+ * 2.0.9 - Add DAT_IB_UD_CM_FREE and DAT_IB_UD_AH_FREE
*/
-#define DAT_IB_EXTENSION_VERSION 208 /* 2.0.8 */
+#define DAT_IB_EXTENSION_VERSION 209 /* 2.0.9 */
#define DAT_IB_ATTR_COUNTERS "DAT_COUNTERS"
#define DAT_IB_ATTR_FETCH_AND_ADD "DAT_IB_FETCH_AND_ADD"
#define DAT_IB_ATTR_CMP_AND_SWAP "DAT_IB_CMP_AND_SWAP"
@@ -155,6 +155,8 @@ typedef enum dat_ib_op
DAT_IB_COLLECTIVE_BARRIER_OP,
DAT_IB_START_COUNTERS_OP,
DAT_IB_STOP_COUNTERS_OP,
+ DAT_IB_UD_CM_FREE_OP,
+ DAT_IB_UD_AH_FREE_OP,
/* OPEN and CLOSE extensions require DAT support, set proper range */
DAT_IB_OPEN_QUERY_OP = DAT_OPEN_EXTENSION_BASE,
DAT_IB_CLOSE_QUERY_OP = DAT_CLOSE_EXTENSION_BASE,
@@ -317,8 +319,12 @@ typedef struct dat_ib_collective_event_data
* NOTE: DAT_IB_EXTENSION_EVENT_DATA cannot exceed 64 bytes as defined by
* "DAT_UINT64 extension_data[8]" in DAT_EVENT (dat.h)
*
- * Provide UD address handles via extended connect establishment.
- * ia_addr provided with extended conn events for reference.
+ * DAT_IB_UD_CONNECTION_EVENT_ESTABLISHED
+ *UD address and CM handles via extended UD connect establishment.
+ * ia_addr provided with extended conn events for reference.
+ * CM handle provided via context, for dat_ib_ud_cm_release().
+ * AH provided via remote_ah, for dat_ib_ud_ah_destroy().
+ * NOTE: if CM released, AH must be destroy before dat_ep_free.
*/
typedef struct dat_ib_extension_event_data
{
@@ -331,6 +337,7 @@ typedef struct dat_ib_extension_event_data
DAT_IB_ADDR_HANDLE remote_ah;
DAT_IB_COLLECTIVE_EVENT_DATAcoll;
[PATCH 0/3] Add IB UD calls to free resources after EP/AH resolution
From: Arlin Davis Arlin Davis (3): extension: add IB UD extensions to reduce CM/AH memory footprint openib: add new TIMEWAIT state for CM openib: add IB UD cm_free/ah_free extension support in UCM provider dapl/openib_cma/cm.c |5 ++ dapl/openib_cma/dapl_ib_util.h |1 + dapl/openib_common/dapl_ib_common.h |6 ++- dapl/openib_common/ib_extensions.c | 40 + dapl/openib_mcm/cm.c |5 ++ dapl/openib_mcm/dapl_ib_util.h |1 + dapl/openib_scm/cm.c |5 ++ dapl/openib_scm/dapl_ib_util.h |1 + dapl/openib_ucm/cm.c | 77 +- dapl/openib_ucm/dapl_ib_util.h |1 + dat/include/dat2/dat_ib_extensions.h | 47 +++-- 11 files changed, 181 insertions(+), 8 deletions(-) -- 1.7.3 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/3] openib: add IB UD cm_free/ah_free extension support in UCM provider
From: Arlin Davis
Make changes to UCM provider for new CM and AH destroy extensions.
Allow consumer to schedule CM object destroy after CM connection
event has been processed. Active side will put CM object in
TIMEWAIT in case RTU is dropped, passive side can schedule
CM object destroy immediatly when called. In the case where
consumer requests CM object destroy, the provider will remove
all internal references to AH since consumer will call AH
destroy directly when finished with UD sends.
All other providers, MCM, CMA, SCM will return UNSUPPORTED
if new extensions are called.
See dtestx source for code examples of new extensions.
Signed-off-by: Arlin Davis
---
dapl/openib_cma/cm.c |5 +++
dapl/openib_cma/dapl_ib_util.h |1 +
dapl/openib_mcm/cm.c |5 +++
dapl/openib_mcm/dapl_ib_util.h |1 +
dapl/openib_scm/cm.c |5 +++
dapl/openib_scm/dapl_ib_util.h |1 +
dapl/openib_ucm/cm.c | 77 ++-
dapl/openib_ucm/dapl_ib_util.h |1 +
8 files changed, 94 insertions(+), 2 deletions(-)
diff --git a/dapl/openib_cma/cm.c b/dapl/openib_cma/cm.c
index 34ae3e3..18d9d09 100644
--- a/dapl/openib_cma/cm.c
+++ b/dapl/openib_cma/cm.c
@@ -262,6 +262,11 @@ void dapls_cm_free(dp_ib_cm_handle_t conn)
dapl_ep_unlink_cm(conn->ep, conn);
}
+DAT_RETURN dapls_ud_cm_free(DAPL_EP *ep_ptr, dp_ib_cm_handle_t cm_ptr)
+{
+ return DAT_NOT_IMPLEMENTED;
+}
+
static struct dapl_cm_id *dapli_req_recv(struct dapl_cm_id *conn,
struct rdma_cm_event *event)
{
diff --git a/dapl/openib_cma/dapl_ib_util.h b/dapl/openib_cma/dapl_ib_util.h
index de95485..1f3b63c 100755
--- a/dapl/openib_cma/dapl_ib_util.h
+++ b/dapl/openib_cma/dapl_ib_util.h
@@ -156,6 +156,7 @@ dp_ib_cm_handle_t dapls_ib_cm_create(DAPL_EP *ep);
void dapls_cm_acquire(dp_ib_cm_handle_t cm);
void dapls_cm_release(dp_ib_cm_handle_t cm);
void dapls_cm_free(dp_ib_cm_handle_t cm_ptr);
+DAT_RETURN dapls_ud_cm_free(DAPL_EP *ep_ptr, dp_ib_cm_handle_t cm_ptr);
#ifdef DAPL_COUNTERS
STATIC _INLINE_ void dapls_print_cm_list(IN DAPL_IA * ia_ptr)
diff --git a/dapl/openib_mcm/cm.c b/dapl/openib_mcm/cm.c
index 4c6778a..71f7355 100644
--- a/dapl/openib_mcm/cm.c
+++ b/dapl/openib_mcm/cm.c
@@ -817,6 +817,11 @@ void dapls_cm_free(dp_ib_cm_handle_t cm)
dapl_ep_unlink_cm(cm->ep, cm);
}
+DAT_RETURN dapls_ud_cm_free(DAPL_EP *ep_ptr, dp_ib_cm_handle_t cm_ptr)
+{
+ return DAT_NOT_IMPLEMENTED;
+}
+
/* ACTIVE/PASSIVE: queue up connection object on CM list */
void dapli_queue_conn(dp_ib_cm_handle_t cm)
{
diff --git a/dapl/openib_mcm/dapl_ib_util.h b/dapl/openib_mcm/dapl_ib_util.h
index 9d9d73c..ab4d658 100644
--- a/dapl/openib_mcm/dapl_ib_util.h
+++ b/dapl/openib_mcm/dapl_ib_util.h
@@ -167,6 +167,7 @@ void dapli_cq_event_cb(struct _ib_hca_transport *tp);
void dapls_cm_acquire(dp_ib_cm_handle_t cm_ptr);
void dapls_cm_release(dp_ib_cm_handle_t cm_ptr);
void dapls_cm_free(dp_ib_cm_handle_t cm_ptr);
+DAT_RETURN dapls_ud_cm_free(DAPL_EP *ep_ptr, dp_ib_cm_handle_t cm_ptr);
dp_ib_cm_handle_t dapls_cm_create(DAPL_HCA *hca, DAPL_EP *ep);
DAT_RETURN dapls_modify_qp_rtu(struct ibv_qp *qp, uint32_t qpn, uint16_t lid,
ib_gid_handle_t gid);
diff --git a/dapl/openib_scm/cm.c b/dapl/openib_scm/cm.c
index cf6993f..680d16b 100644
--- a/dapl/openib_scm/cm.c
+++ b/dapl/openib_scm/cm.c
@@ -470,6 +470,11 @@ void dapls_cm_free(dp_ib_cm_handle_t cm_ptr)
dapl_ep_unlink_cm(cm_ptr->ep, cm_ptr);
}
+DAT_RETURN dapls_ud_cm_free(DAPL_EP *ep_ptr, dp_ib_cm_handle_t cm_ptr)
+{
+ return DAT_NOT_IMPLEMENTED;
+}
+
/*
* ACTIVE/PASSIVE: called from CR thread or consumer via ep_disconnect
* or from ep_free.
diff --git a/dapl/openib_scm/dapl_ib_util.h b/dapl/openib_scm/dapl_ib_util.h
index d382226..0f48c44 100644
--- a/dapl/openib_scm/dapl_ib_util.h
+++ b/dapl/openib_scm/dapl_ib_util.h
@@ -141,6 +141,7 @@ void dapli_cq_event_cb(struct _ib_hca_transport *tp);
void dapls_cm_acquire(dp_ib_cm_handle_t cm_ptr);
void dapls_cm_release(dp_ib_cm_handle_t cm_ptr);
void dapls_cm_free(dp_ib_cm_handle_t cm_ptr);
+DAT_RETURN dapls_ud_cm_free(DAPL_EP *ep_ptr, dp_ib_cm_handle_t cm_ptr);
#ifdef DAPL_COUNTERS
void dapls_print_cm_list(IN DAPL_IA *ia_ptr);
diff --git a/dapl/openib_ucm/cm.c b/dapl/openib_ucm/cm.c
index a1e909a..e462632 100644
--- a/dapl/openib_ucm/cm.c
+++ b/dapl/openib_ucm/cm.c
@@ -226,6 +226,37 @@ static void ucm_check_timers(dp_ib_cm_handle_t cm, int
*timer)
return;
}
break;
+ case DCM_TIMEWAIT:
+ /* TIMEWAIT after CM info saved, consumer called
dat_ib_ud_cm_free() */
+ *timer = cm->hca->ib_trans.cm_timer;
+ if ((time - cm->timer)/1000 >
+(cm->hca->ib_trans.rtu_time << cm->retries)) {
+ dapl_log(DAPL_DBG_TYPE_CM_WARN,
+" CM_TIMEWAIT %d
Re: [PATCH ] mlx4_core: Enhance device capabilities flags debug printouts
On Mon, Dec 15, 2014 at 2:25 PM, Yuval Shaia wrote: > On Mon, Dec 15, 2014 at 11:35:52AM +0200, Or Gerlitz wrote: >> On Mon, Dec 15, 2014 at 11:21 AM, Yuval Shaia wrote: >> > On Wed, Dec 10, 2014 at 12:10:07PM +0200, Matan Barak wrote: >> >> >> >> >> >> On 12/8/2014 1:48 AM, Yuval Shaia wrote: >> >> >1. Add indication whether feature is supported or not. >> >> >2. Add descriptions of all features. >> >> >Without this fix there is no way to tell if feature is not supported or >> >> >that description is not exist. >> >> > >> >> >Signed-off-by: Yuval Shaia >> >> >--- >> >> > drivers/net/ethernet/mellanox/mlx4/fw.c | 37 >> >> > +++--- >> >> > 1 files changed, 33 insertions(+), 4 deletions(-) >> >> > >> >> >diff --git a/drivers/net/ethernet/mellanox/mlx4/fw.c >> >> >b/drivers/net/ethernet/mellanox/mlx4/fw.c >> >> >index 2e88a23..79ab326 100644 >> >> >--- a/drivers/net/ethernet/mellanox/mlx4/fw.c >> >> >+++ b/drivers/net/ethernet/mellanox/mlx4/fw.c >> >> >@@ -91,7 +91,10 @@ static void dump_dev_cap_flags(struct mlx4_dev *dev, >> >> >u64 flags) >> >> > [ 8] =P_Key violation counter", >> >> > [ 9] =Q_Key violation counter", >> >> > [10] =VMM", >> >> >+ [11] =Fibre Channel Protocol On Ethernet Ports support", >> >> > [12] =Dual Port Different Protocol (DPDP) support", >> >> >+ [13] =Raw Ethertype support", >> >> >+ [14] =Raw IPv6 support", >> >> > [15] =Big LSO headers", >> >> > [16] =MW support", >> >> > [17] =APM support", >> >> >@@ -99,16 +102,40 @@ static void dump_dev_cap_flags(struct mlx4_dev >> >> >*dev, u64 flags) >> >> > [19] =Raw multicast support", >> >> > [20] =Address vector port checking support", >> >> > [21] =UD multicast support", >> >> >+ [22] =UD IPv4 Multicast support", >> >> > [24] =Demand paging support", >> >> > [25] =Router support", >> >> >+ [26] =L2 Ethernet Multicast support", >> >> >+ [28] =Software parsing support for UD transport", >> >> >+ [29] =TCP checksum off-load support", >> >> > [30] =IBoE support", >> >> >+ [31] =FCoE T11 frame format support", >> >> > [32] =Unicast loopback support", >> >> >+ [33] =Multicast loopback support", >> >> > [34] =FCS header control", >> >> >+ [35] =Address Path ud_force_mgid support", >> >> >+ [36] =Header-Data Split support", >> >> >+ [37] =Wake On LAN support on port 1", >> >> >+ [38] =Wake On LAN support on port 2", >> >> >+ [39] =Fatal Warning Event upon a thermal warning >> >> >condition", >> >> > [38] =Wake On LAN support", >> >> > [40] =UDP RSS support", >> >> > [41] =Unicast VEP steering support", >> >> > [42] =Multicast VEP steering support", >> >> >+ [43] =VLAN Steering mechanism support", >> >> >+ [44] =Steering according to EtherType support", >> >> >+ [45] =WQE format version 1 support", >> >> >+ [46] =Keep Alive Validiation support", >> >> >+ [47] =PTP1588 support", >> >> > [48] =Counters support", >> >> >+ [49] =Advanced Counters support", >> >> >+ [50] =Force Ethernet user priority from QPC support", >> >> >+ [51] =RX Port Num check disabled", >> >> >+ [52] =RSS on fragmented IP datagram support", >> >> >> >> Hi, >> >> >> >> Except for Or's comments, [55] exists twice. Furthermore, please >> >> move [56]-[57] to be after [55] =Port link type sensing support". >> > Thanks, >> > Will send a revise patch soon. >> > By accepting this patch it is agreed that function will dump device >> > capabilities flag and not the combination of dev & driver capabilities. >> > (as the name suggest) >> >> NO, as Matan wrote you "except for Or's comments" - we do want to dump > Sure, was not ignoring this one, it was just a warn :) >> what is supported by both the device (firmware) and the driver, and >> not more. A subset of your patch can be used, let me know if you want >> me to prepare it. > Yes please do as i have no idea from where this info can be retrieved. Oh, that's easy, just follow on the MLX4_DEV_CAP_FLAG_YYY enum values and look for holes in the string array. Or. > How about the idea of maintaining drv_caps_flags field equivalent to > dev_caps_flags? >> >> Or. >> >> >> >> >> >+ [55] =Link Sensing support", >> >> >+ [56] =Reliable Multicast support", >> >> >+ [57] =Fast Drop support", >> >> >+ [58] =Protected FMR support", >> >> > [53] =Port ETS Scheduler support", >> >> > [55] =Por
Re: [PATCH ] mlx4_core: Enhance device capabilities flags debug printouts
On Mon, Dec 15, 2014 at 11:35:52AM +0200, Or Gerlitz wrote: > On Mon, Dec 15, 2014 at 11:21 AM, Yuval Shaia wrote: > > On Wed, Dec 10, 2014 at 12:10:07PM +0200, Matan Barak wrote: > >> > >> > >> On 12/8/2014 1:48 AM, Yuval Shaia wrote: > >> >1. Add indication whether feature is supported or not. > >> >2. Add descriptions of all features. > >> >Without this fix there is no way to tell if feature is not supported or > >> >that description is not exist. > >> > > >> >Signed-off-by: Yuval Shaia > >> >--- > >> > drivers/net/ethernet/mellanox/mlx4/fw.c | 37 > >> > +++--- > >> > 1 files changed, 33 insertions(+), 4 deletions(-) > >> > > >> >diff --git a/drivers/net/ethernet/mellanox/mlx4/fw.c > >> >b/drivers/net/ethernet/mellanox/mlx4/fw.c > >> >index 2e88a23..79ab326 100644 > >> >--- a/drivers/net/ethernet/mellanox/mlx4/fw.c > >> >+++ b/drivers/net/ethernet/mellanox/mlx4/fw.c > >> >@@ -91,7 +91,10 @@ static void dump_dev_cap_flags(struct mlx4_dev *dev, > >> >u64 flags) > >> > [ 8] =P_Key violation counter", > >> > [ 9] =Q_Key violation counter", > >> > [10] =VMM", > >> >+ [11] =Fibre Channel Protocol On Ethernet Ports support", > >> > [12] =Dual Port Different Protocol (DPDP) support", > >> >+ [13] =Raw Ethertype support", > >> >+ [14] =Raw IPv6 support", > >> > [15] =Big LSO headers", > >> > [16] =MW support", > >> > [17] =APM support", > >> >@@ -99,16 +102,40 @@ static void dump_dev_cap_flags(struct mlx4_dev *dev, > >> >u64 flags) > >> > [19] =Raw multicast support", > >> > [20] =Address vector port checking support", > >> > [21] =UD multicast support", > >> >+ [22] =UD IPv4 Multicast support", > >> > [24] =Demand paging support", > >> > [25] =Router support", > >> >+ [26] =L2 Ethernet Multicast support", > >> >+ [28] =Software parsing support for UD transport", > >> >+ [29] =TCP checksum off-load support", > >> > [30] =IBoE support", > >> >+ [31] =FCoE T11 frame format support", > >> > [32] =Unicast loopback support", > >> >+ [33] =Multicast loopback support", > >> > [34] =FCS header control", > >> >+ [35] =Address Path ud_force_mgid support", > >> >+ [36] =Header-Data Split support", > >> >+ [37] =Wake On LAN support on port 1", > >> >+ [38] =Wake On LAN support on port 2", > >> >+ [39] =Fatal Warning Event upon a thermal warning > >> >condition", > >> > [38] =Wake On LAN support", > >> > [40] =UDP RSS support", > >> > [41] =Unicast VEP steering support", > >> > [42] =Multicast VEP steering support", > >> >+ [43] =VLAN Steering mechanism support", > >> >+ [44] =Steering according to EtherType support", > >> >+ [45] =WQE format version 1 support", > >> >+ [46] =Keep Alive Validiation support", > >> >+ [47] =PTP1588 support", > >> > [48] =Counters support", > >> >+ [49] =Advanced Counters support", > >> >+ [50] =Force Ethernet user priority from QPC support", > >> >+ [51] =RX Port Num check disabled", > >> >+ [52] =RSS on fragmented IP datagram support", > >> > >> Hi, > >> > >> Except for Or's comments, [55] exists twice. Furthermore, please > >> move [56]-[57] to be after [55] =Port link type sensing support". > > Thanks, > > Will send a revise patch soon. > > By accepting this patch it is agreed that function will dump device > > capabilities flag and not the combination of dev & driver capabilities. > > (as the name suggest) > > NO, as Matan wrote you "except for Or's comments" - we do want to dump Sure, was not ignoring this one, it was just a warn :) > what is supported by both the device (firmware) and the driver, and > not more. A subset of your patch can be used, let me know if you want > me to prepare it. Yes please do as i have no idea from where this info can be retrieved. How about the idea of maintaining drv_caps_flags field equivalent to dev_caps_flags? > > Or. > > >> > >> >+ [55] =Link Sensing support", > >> >+ [56] =Reliable Multicast support", > >> >+ [57] =Fast Drop support", > >> >+ [58] =Protected FMR support", > >> > [53] =Port ETS Scheduler support", > >> > [55] =Port link type sensing support", > >> > [59] =Port management change event support", > >> >@@ -119,8 +146,9 @@ static void dump_dev_cap_flags(struct mlx4_dev *dev, > >> >u64 flags) > >> > > >> > mlx4_dbg(dev, "DEV_CAP flags:\n"); > >> > for (
Query regarding MAD_DEMUX and Secure Host
Hi, I was going through the mlx4 code for Secure Host and MAD_DEMUX feature. I had a few queries..hoping that i can get these clarified. If i understand correctly mlx4 codebase (mlx4_core/mlx4_en/mlx4_ib drivers) take care of both RoCE and Infiniband adapters ( in both dedicated and shared/SRIOV mode) (1) After reading through the RoCE standard, I understood that Subnet Management features are not supported in RoCE(they are part of Infiniband) (2) Also, as i understand MAD_DEMUX mechanism is used to control which management traffic is passed to the host. So by correlating these two( 1 & 2), am I correct in my understanding when i say that MAD_DEMUX feature is not required to be supported/implemented in Mellanox RoCE Drivers? It is required only for Infiniband drivers? I would be really thankful to get my understanding clarified. Thanks so much Best Regards, Bob -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH ] mlx4_core: Enhance device capabilities flags debug printouts
On Mon, Dec 15, 2014 at 11:21 AM, Yuval Shaia wrote: > On Wed, Dec 10, 2014 at 12:10:07PM +0200, Matan Barak wrote: >> >> >> On 12/8/2014 1:48 AM, Yuval Shaia wrote: >> >1. Add indication whether feature is supported or not. >> >2. Add descriptions of all features. >> >Without this fix there is no way to tell if feature is not supported or >> >that description is not exist. >> > >> >Signed-off-by: Yuval Shaia >> >--- >> > drivers/net/ethernet/mellanox/mlx4/fw.c | 37 >> > +++--- >> > 1 files changed, 33 insertions(+), 4 deletions(-) >> > >> >diff --git a/drivers/net/ethernet/mellanox/mlx4/fw.c >> >b/drivers/net/ethernet/mellanox/mlx4/fw.c >> >index 2e88a23..79ab326 100644 >> >--- a/drivers/net/ethernet/mellanox/mlx4/fw.c >> >+++ b/drivers/net/ethernet/mellanox/mlx4/fw.c >> >@@ -91,7 +91,10 @@ static void dump_dev_cap_flags(struct mlx4_dev *dev, u64 >> >flags) >> > [ 8] =P_Key violation counter", >> > [ 9] =Q_Key violation counter", >> > [10] =VMM", >> >+ [11] =Fibre Channel Protocol On Ethernet Ports support", >> > [12] =Dual Port Different Protocol (DPDP) support", >> >+ [13] =Raw Ethertype support", >> >+ [14] =Raw IPv6 support", >> > [15] =Big LSO headers", >> > [16] =MW support", >> > [17] =APM support", >> >@@ -99,16 +102,40 @@ static void dump_dev_cap_flags(struct mlx4_dev *dev, >> >u64 flags) >> > [19] =Raw multicast support", >> > [20] =Address vector port checking support", >> > [21] =UD multicast support", >> >+ [22] =UD IPv4 Multicast support", >> > [24] =Demand paging support", >> > [25] =Router support", >> >+ [26] =L2 Ethernet Multicast support", >> >+ [28] =Software parsing support for UD transport", >> >+ [29] =TCP checksum off-load support", >> > [30] =IBoE support", >> >+ [31] =FCoE T11 frame format support", >> > [32] =Unicast loopback support", >> >+ [33] =Multicast loopback support", >> > [34] =FCS header control", >> >+ [35] =Address Path ud_force_mgid support", >> >+ [36] =Header-Data Split support", >> >+ [37] =Wake On LAN support on port 1", >> >+ [38] =Wake On LAN support on port 2", >> >+ [39] =Fatal Warning Event upon a thermal warning condition", >> > [38] =Wake On LAN support", >> > [40] =UDP RSS support", >> > [41] =Unicast VEP steering support", >> > [42] =Multicast VEP steering support", >> >+ [43] =VLAN Steering mechanism support", >> >+ [44] =Steering according to EtherType support", >> >+ [45] =WQE format version 1 support", >> >+ [46] =Keep Alive Validiation support", >> >+ [47] =PTP1588 support", >> > [48] =Counters support", >> >+ [49] =Advanced Counters support", >> >+ [50] =Force Ethernet user priority from QPC support", >> >+ [51] =RX Port Num check disabled", >> >+ [52] =RSS on fragmented IP datagram support", >> >> Hi, >> >> Except for Or's comments, [55] exists twice. Furthermore, please >> move [56]-[57] to be after [55] =Port link type sensing support". > Thanks, > Will send a revise patch soon. > By accepting this patch it is agreed that function will dump device > capabilities flag and not the combination of dev & driver capabilities. > (as the name suggest) NO, as Matan wrote you "except for Or's comments" - we do want to dump what is supported by both the device (firmware) and the driver, and not more. A subset of your patch can be used, let me know if you want me to prepare it. Or. >> >> >+ [55] =Link Sensing support", >> >+ [56] =Reliable Multicast support", >> >+ [57] =Fast Drop support", >> >+ [58] =Protected FMR support", >> > [53] =Port ETS Scheduler support", >> > [55] =Port link type sensing support", >> > [59] =Port management change event support", >> >@@ -119,8 +146,9 @@ static void dump_dev_cap_flags(struct mlx4_dev *dev, >> >u64 flags) >> > >> > mlx4_dbg(dev, "DEV_CAP flags:\n"); >> > for (i =; i < ARRAY_SIZE(fname); ++i) >> >- if (fname[i] && (flags & (1LL << i))) >> >- mlx4_dbg(dev, "%s\n", fname[i]); >> >+ if (fname[i]) >> >+ mlx4_dbg(dev, "(%c) %s\n", (flags & (1LL << i)) >> >? >> >+'*' : ' ', fname[i]); >> > } >> >> I think that the strings should be displayed only if both driver and >> FW supports them. You could add a QUERY_DEV_CAP_SUPPORT
[PATCH] mlx4_core: Enhance device capabilities flags debug printouts
1. Add indication whether feature is supported or not. 2. Add descriptions of all features. Without this fix there is no way to tell if feature is not supported or that description is not exist. Signed-off-by: Yuval Shaia --- drivers/net/ethernet/mellanox/mlx4/fw.c | 36 +++--- 1 files changed, 32 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx4/fw.c b/drivers/net/ethernet/mellanox/mlx4/fw.c index 2e88a23..4fd00e4 100644 --- a/drivers/net/ethernet/mellanox/mlx4/fw.c +++ b/drivers/net/ethernet/mellanox/mlx4/fw.c @@ -91,7 +91,10 @@ static void dump_dev_cap_flags(struct mlx4_dev *dev, u64 flags) [ 8] = "P_Key violation counter", [ 9] = "Q_Key violation counter", [10] = "VMM", + [11] = "Fibre Channel Protocol On Ethernet Ports support", [12] = "Dual Port Different Protocol (DPDP) support", + [13] = "Raw Ethertype support", + [14] = "Raw IPv6 support", [15] = "Big LSO headers", [16] = "MW support", [17] = "APM support", @@ -99,18 +102,41 @@ static void dump_dev_cap_flags(struct mlx4_dev *dev, u64 flags) [19] = "Raw multicast support", [20] = "Address vector port checking support", [21] = "UD multicast support", + [22] = "UD IPv4 Multicast support", [24] = "Demand paging support", [25] = "Router support", + [26] = "L2 Ethernet Multicast support", + [28] = "Software parsing support for UD transport", + [29] = "TCP checksum off-load support", [30] = "IBoE support", + [31] = "FCoE T11 frame format support", [32] = "Unicast loopback support", + [33] = "Multicast loopback support", [34] = "FCS header control", + [35] = "Address Path ud_force_mgid support", + [36] = "Header-Data Split support", + [37] = "Wake On LAN support on port 1", + [38] = "Wake On LAN support on port 2", + [39] = "Fatal Warning Event upon a thermal warning condition", [38] = "Wake On LAN support", [40] = "UDP RSS support", [41] = "Unicast VEP steering support", [42] = "Multicast VEP steering support", + [43] = "VLAN Steering mechanism support", + [44] = "Steering according to EtherType support", + [45] = "WQE format version 1 support", + [46] = "Keep Alive Validiation support", + [47] = "PTP1588 support", [48] = "Counters support", + [49] = "Advanced Counters support", + [50] = "Force Ethernet user priority from QPC support", + [51] = "RX Port Num check disabled", + [52] = "RSS on fragmented IP datagram support", [53] = "Port ETS Scheduler support", [55] = "Port link type sensing support", + [56] = "Reliable Multicast support", + [57] = "Fast Drop support", + [58] = "Protected FMR support", [59] = "Port management change event support", [61] = "64 byte EQE support", [62] = "64 byte CQE support", @@ -119,8 +145,9 @@ static void dump_dev_cap_flags(struct mlx4_dev *dev, u64 flags) mlx4_dbg(dev, "DEV_CAP flags:\n"); for (i = 0; i < ARRAY_SIZE(fname); ++i) - if (fname[i] && (flags & (1LL << i))) - mlx4_dbg(dev, "%s\n", fname[i]); + if (fname[i]) + mlx4_dbg(dev, "(%c) %s\n", (flags & (1LL << i)) ? +'*' : ' ', fname[i]); } static void dump_dev_cap_flags2(struct mlx4_dev *dev, u64 flags) @@ -144,8 +171,9 @@ static void dump_dev_cap_flags2(struct mlx4_dev *dev, u64 flags) int i; for (i = 0; i < ARRAY_SIZE(fname); ++i) - if (fname[i] && (flags & (1LL << i))) - mlx4_dbg(dev, "%s\n", fname[i]); + if (fname[i]) + mlx4_dbg(dev, "(%c) %s\n", (flags & (1LL << i)) ? +'*' : ' ', fname[i]); } int mlx4_MOD_STAT_CFG(struct mlx4_dev *dev, struct mlx4_mod_stat_cfg *cfg) -- 1.7.1 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH ] mlx4_core: Enhance device capabilities flags debug printouts
On Wed, Dec 10, 2014 at 12:10:07PM +0200, Matan Barak wrote: > > > On 12/8/2014 1:48 AM, Yuval Shaia wrote: > >1. Add indication whether feature is supported or not. > >2. Add descriptions of all features. > >Without this fix there is no way to tell if feature is not supported or that > >description is not exist. > > > >Signed-off-by: Yuval Shaia > >--- > > drivers/net/ethernet/mellanox/mlx4/fw.c | 37 > > +++--- > > 1 files changed, 33 insertions(+), 4 deletions(-) > > > >diff --git a/drivers/net/ethernet/mellanox/mlx4/fw.c > >b/drivers/net/ethernet/mellanox/mlx4/fw.c > >index 2e88a23..79ab326 100644 > >--- a/drivers/net/ethernet/mellanox/mlx4/fw.c > >+++ b/drivers/net/ethernet/mellanox/mlx4/fw.c > >@@ -91,7 +91,10 @@ static void dump_dev_cap_flags(struct mlx4_dev *dev, u64 > >flags) > > [ 8] =P_Key violation counter", > > [ 9] =Q_Key violation counter", > > [10] =VMM", > >+ [11] =Fibre Channel Protocol On Ethernet Ports support", > > [12] =Dual Port Different Protocol (DPDP) support", > >+ [13] =Raw Ethertype support", > >+ [14] =Raw IPv6 support", > > [15] =Big LSO headers", > > [16] =MW support", > > [17] =APM support", > >@@ -99,16 +102,40 @@ static void dump_dev_cap_flags(struct mlx4_dev *dev, > >u64 flags) > > [19] =Raw multicast support", > > [20] =Address vector port checking support", > > [21] =UD multicast support", > >+ [22] =UD IPv4 Multicast support", > > [24] =Demand paging support", > > [25] =Router support", > >+ [26] =L2 Ethernet Multicast support", > >+ [28] =Software parsing support for UD transport", > >+ [29] =TCP checksum off-load support", > > [30] =IBoE support", > >+ [31] =FCoE T11 frame format support", > > [32] =Unicast loopback support", > >+ [33] =Multicast loopback support", > > [34] =FCS header control", > >+ [35] =Address Path ud_force_mgid support", > >+ [36] =Header-Data Split support", > >+ [37] =Wake On LAN support on port 1", > >+ [38] =Wake On LAN support on port 2", > >+ [39] =Fatal Warning Event upon a thermal warning condition", > > [38] =Wake On LAN support", > > [40] =UDP RSS support", > > [41] =Unicast VEP steering support", > > [42] =Multicast VEP steering support", > >+ [43] =VLAN Steering mechanism support", > >+ [44] =Steering according to EtherType support", > >+ [45] =WQE format version 1 support", > >+ [46] =Keep Alive Validiation support", > >+ [47] =PTP1588 support", > > [48] =Counters support", > >+ [49] =Advanced Counters support", > >+ [50] =Force Ethernet user priority from QPC support", > >+ [51] =RX Port Num check disabled", > >+ [52] =RSS on fragmented IP datagram support", > > Hi, > > Except for Or's comments, [55] exists twice. Furthermore, please > move [56]-[57] to be after [55] =Port link type sensing support". Thanks, Will send a revise patch soon. By accepting this patch it is agreed that function will dump device capabilities flag and not the combination of dev & driver capabilities. (as the name suggest) > > >+ [55] =Link Sensing support", > >+ [56] =Reliable Multicast support", > >+ [57] =Fast Drop support", > >+ [58] =Protected FMR support", > > [53] =Port ETS Scheduler support", > > [55] =Port link type sensing support", > > [59] =Port management change event support", > >@@ -119,8 +146,9 @@ static void dump_dev_cap_flags(struct mlx4_dev *dev, u64 > >flags) > > > > mlx4_dbg(dev, "DEV_CAP flags:\n"); > > for (i =; i < ARRAY_SIZE(fname); ++i) > >- if (fname[i] && (flags & (1LL << i))) > >- mlx4_dbg(dev, "%s\n", fname[i]); > >+ if (fname[i]) > >+ mlx4_dbg(dev, "(%c) %s\n", (flags & (1LL << i)) ? > >+'*' : ' ', fname[i]); > > } > > I think that the strings should be displayed only if both driver and > FW supports them. You could add a QUERY_DEV_CAP_SUPPORTED_FLAGS > which ors all the supported flags in device.h and check if (flags & > QUERY_DEV_CAP_SUPPORTED_FLAGS) & (1LL << i)). > > > > > static void dump_dev_cap_flags2(struct mlx4_dev *dev, u64 flags) > >@@ -144,8 +172,9 @@ static void dump_dev_cap_flags2(struct mlx4_dev *dev, > >u64 flags) > > int i; > > > > for (i =; i < ARRAY_SIZE(fname); ++i) > >- if (
