Re: Socket Direct Protocol: help (2)
I hope to have a fix next week for the first one. Thanks, Amir On 04/14/2010 09:48 PM, Tung, Chien Tin wrote: Tung, Chien Tin wrote: One more thing - Please open a bug regarding the num_sge limitation at: https://bugs.openfabrics.org/ Done, Bug 2027. Chien And 2028 opened to request fastreg support. I am open to test fixes for these two bugs. Chien -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Socket Direct Protocol: help (2)
It should be a simple fix and I plan to do soon - just add yourself as CC in bugzilla - that way I won't forget to notify you. - amir On 04/15/2010 10:07 AM, Andrea Gozzelino wrote: On Apr 15, 2010 08:24 AM, Amir Vadai am...@mellanox.co.il wrote: I hope to have a fix next week for the first one. Thanks, Amir On 04/14/2010 09:48 PM, Tung, Chien Tin wrote: Tung, Chien Tin wrote: One more thing - Please open a bug regarding the num_sge limitation at: https://bugs.openfabrics.org/ Done, Bug 2027. Chien And 2028 opened to request fastreg support. I am open to test fixes for these two bugs. Chien Hi Amir, Hi Chien, I understand that the bug 2027 could be solved next week, so I will test SDP protocol performance on NE020 cards. Is it correct? If yes, could you point out the code modifies? Keep in touch and take care. Regards, Andrea Andrea Gozzelino INFN - Laboratori Nazionali di Legnaro(LNL) Viale dell'Universita' 2 I-35020 - Legnaro (PD)- ITALIA Tel: +39 049 8068346 Fax: +39 049 641925 Mail: andrea.gozzel...@lnl.infn.it -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/4] opensm: added function that dumps PathRecords
Hi Jim, On 15/Apr/10 01:16, Jim Schutt wrote: Hi Yevgeny, On Thu, 2010-04-08 at 07:29 -0600, Yevgeny Kliteynik wrote: Dumping SL, MTU and Rate for all the non-switch-2-non-switch paths in the subnet. PRs that are dumped: for every non-switch source port for every non-switch target LID in the subnet dump PR between source port and target LID This way number of sources is equal to number of physical non-switch ports in the subnet, and only number of targets depends on LMC that is used. Signed-off-by: Yevgeny Kliteynikklit...@dev.mellanox.co.il --- [snip] + p_node = p_src_port-p_node; + if (p_node-node_info.node_type == IB_NODE_TYPE_SWITCH) + return; - return; + continue; Otherwise we stop dumping at the first switch we encounter? Correct, thanks :) -- Yevgeny -- Jim -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: two questions about RDMA-WRITE
static void jm_cq_comp_handler(struct ib_cq *cq, void *context) {
struct jm_rdma_conn *conn = context;
struct ib_wc wc;
struct jm_send_ctx *send;
/* No idea why it should be called twice. */
printk(cq comp for id %p\n, conn-jc_id);
ib_req_notify_cq(cq, IB_CQ_NEXT_COMP);
while (ib_poll_cq(cq, 1, wc) == 1) {
if (wc.opcode != IB_WC_RDMA_WRITE) {
printk(completed unknown opcode %d\n, wc.opcode);
/* continue; */
}
send = (struct jm_send_ctx *)wc.wr_id;
printk(got send=%p\n, send);
printk(completed RDMA_WRITE of IO(%Lu, %u)\n,
send-s_offset, send-s_size);
send-s_done = wc.status == IB_WC_SUCCESS ? 1 : -EIO;
wake_up_all(send-s_wait);
}
ib_req_notify_cq(cq, IB_CQ_NEXT_COMP);
unrelated to your problem, but this second call to ib_req_notify_cq isn't
necessary.
static int jm_rdma_cm_event_handler(struct rdma_cm_id *id, struct rdma_cm_event
*event) {
..
case RDMA_CM_EVENT_DISCONNECTED:
connstate = -ECONNABORTED;
goto connected;
..
connected:
printk(%pI4:%u (event 0x%x)\n,
conn-jc_remoteaddr.sin_addr.s_addr,
ntohs(conn-jc_remoteaddr.sin_port),
event-event 11);
conn-jc_connstate = connstate;
wake_up_all(conn-jc_connect_wait);
break;
How quickly do you respond to the disconnect event? The remote side will wait
until it receives a response or times out, which may be several seconds or
minutes.
- Sean
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] rdma/cm: Randomize local port allocation.
From: Tetsuo Handa penguin-ker...@i-love.sakura.ne.jp
Randomize local port allocation in a way sctp_get_port_local() does.
Update rover at the end of loop since we're likely to pick a valid port
on the first try.
Signed-off-by: Tetsuo Handa penguin-ker...@i-love.sakura.ne.jp
Reviewed-by: Sean Hefty sean.he...@intel.com
---
I like this version, thanks! I'm not sure which tree to merge it through.
Are you needing this for 2.6.34, or is 2.6.35 okay?
drivers/infiniband/core/cma.c | 70 +++--
-
1 file changed, 25 insertions(+), 45 deletions(-)
--- linux-2.6.34-rc4.orig/drivers/infiniband/core/cma.c
+++ linux-2.6.34-rc4/drivers/infiniband/core/cma.c
@@ -79,7 +79,6 @@ static DEFINE_IDR(sdp_ps);
static DEFINE_IDR(tcp_ps);
static DEFINE_IDR(udp_ps);
static DEFINE_IDR(ipoib_ps);
-static int next_port;
struct cma_device {
struct list_headlist;
@@ -1970,47 +1969,33 @@ err1:
static int cma_alloc_any_port(struct idr *ps, struct rdma_id_private *id_priv)
{
- struct rdma_bind_list *bind_list;
- int port, ret, low, high;
-
- bind_list = kzalloc(sizeof *bind_list, GFP_KERNEL);
- if (!bind_list)
- return -ENOMEM;
-
-retry:
- /* FIXME: add proper port randomization per like inet_csk_get_port */
- do {
- ret = idr_get_new_above(ps, bind_list, next_port, port);
- } while ((ret == -EAGAIN) idr_pre_get(ps, GFP_KERNEL));
-
- if (ret)
- goto err1;
+ static unsigned int last_used_port;
+ int low, high, remaining;
+ unsigned int rover;
inet_get_local_port_range(low, high);
- if (port high) {
- if (next_port != low) {
- idr_remove(ps, port);
- next_port = low;
- goto retry;
- }
- ret = -EADDRNOTAVAIL;
- goto err2;
+ remaining = (high - low) + 1;
+ rover = net_random() % remaining + low;
+retry:
+ if (last_used_port != rover
+ !idr_find(ps, (unsigned short) rover)) {
+ int ret = cma_alloc_port(ps, id_priv, rover);
+ /*
+ * Remember previously used port number in order to avoid
+ * re-using same port immediately after it is closed.
+ */
+ if (!ret)
+ last_used_port = rover;
+ if (ret != -EADDRNOTAVAIL)
+ return ret;
}
-
- if (port == high)
- next_port = low;
- else
- next_port = port + 1;
-
- bind_list-ps = ps;
- bind_list-port = (unsigned short) port;
- cma_bind_port(bind_list, id_priv);
- return 0;
-err2:
- idr_remove(ps, port);
-err1:
- kfree(bind_list);
- return ret;
+ if (--remaining) {
+ rover++;
+ if ((rover low) || (rover high))
+ rover = low;
+ goto retry;
+ }
+ return -EADDRNOTAVAIL;
}
static int cma_use_port(struct idr *ps, struct rdma_id_private *id_priv)
@@ -2995,12 +2980,7 @@ static void cma_remove_one(struct ib_dev
static int __init cma_init(void)
{
- int ret, low, high, remaining;
-
- get_random_bytes(next_port, sizeof next_port);
- inet_get_local_port_range(low, high);
- remaining = (high - low) + 1;
- next_port = ((unsigned int) next_port % remaining) + low;
+ int ret;
cma_wq = create_singlethread_workqueue(rdma_cm);
if (!cma_wq)
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[infiniband-diags] [1/2] support libibnetdisc caching overwrite flag
Hi Sasha,
This patch supports a flag in the libibnetdisc caching code to overwrite
a previous cache.
Al
--
Albert Chu
ch...@llnl.gov
Computer Scientist
High Performance Systems Division
Lawrence Livermore National Laboratory
---BeginMessage---
Signed-off-by: Albert Chu ch...@llnl.gov
---
.../libibnetdisc/include/infiniband/ibnetdisc.h|3 +++
.../libibnetdisc/src/ibnetdisc_cache.c | 17 ++---
2 files changed, 17 insertions(+), 3 deletions(-)
diff --git a/infiniband-diags/libibnetdisc/include/infiniband/ibnetdisc.h
b/infiniband-diags/libibnetdisc/include/infiniband/ibnetdisc.h
index 136282c..c83bd0b 100644
--- a/infiniband-diags/libibnetdisc/include/infiniband/ibnetdisc.h
+++ b/infiniband-diags/libibnetdisc/include/infiniband/ibnetdisc.h
@@ -181,6 +181,9 @@ MAD_EXPORT void ibnd_destroy_fabric(ibnd_fabric_t * fabric);
MAD_EXPORT ibnd_fabric_t *ibnd_load_fabric(const char *file,
unsigned int flags);
+#define IBND_CACHE_FABRIC_FLAG_DEFAULT 0x
+#define IBND_CACHE_FABRIC_FLAG_OVERWRITE 0x0001
+
MAD_EXPORT int ibnd_cache_fabric(ibnd_fabric_t * fabric, const char *file,
unsigned int flags);
diff --git a/infiniband-diags/libibnetdisc/src/ibnetdisc_cache.c
b/infiniband-diags/libibnetdisc/src/ibnetdisc_cache.c
index 480a0a2..6cf7d4d 100644
--- a/infiniband-diags/libibnetdisc/src/ibnetdisc_cache.c
+++ b/infiniband-diags/libibnetdisc/src/ibnetdisc_cache.c
@@ -876,9 +876,20 @@ int ibnd_cache_fabric(ibnd_fabric_t * fabric, const char
*file,
return -1;
}
- if (!stat(file, statbuf)) {
- IBND_DEBUG(file '%s' already exists\n, file);
- return -1;
+ if (flags IBND_CACHE_FABRIC_FLAG_OVERWRITE) {
+ if (!stat(file, statbuf)) {
+ if (unlink(file) 0) {
+ IBND_DEBUG(error removing '%s': %s\n,
+ file, strerror(errno));
+ return -1;
+ }
+ }
+ }
+ else {
+ if (!stat(file, statbuf)) {
+ IBND_DEBUG(file '%s' already exists\n, file);
+ return -1;
+ }
}
if ((fd = open(file, O_CREAT | O_EXCL | O_WRONLY, 0644)) 0) {
--
1.5.4.5
---End Message---
[infiniband-diags] [2/2] add ibcacheedit tool
Hey Sasha, This patch adds a new ibcacheedit tool to infiniband-diags. As the name suggests, it offers users options to edit a stored cache. This tool is primarily necessary to allow system administrators to update caches as they make minor modifications to a cluster (e.g. node dies, thus HCA is replaced) rather than regularly regenerating a new cache. Al -- Albert Chu ch...@llnl.gov Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory ---BeginMessage--- Signed-off-by: Albert Chu ch...@llnl.gov --- infiniband-diags/Makefile.am |6 +- infiniband-diags/man/ibcacheedit.8 | 57 ++ infiniband-diags/src/ibcacheedit.c | 361 3 files changed, 422 insertions(+), 2 deletions(-) create mode 100644 infiniband-diags/man/ibcacheedit.8 create mode 100644 infiniband-diags/src/ibcacheedit.c diff --git a/infiniband-diags/Makefile.am b/infiniband-diags/Makefile.am index 1cdb60e..af90b05 100644 --- a/infiniband-diags/Makefile.am +++ b/infiniband-diags/Makefile.am @@ -13,7 +13,7 @@ sbin_PROGRAMS = src/ibaddr src/ibnetdiscover src/ibping src/ibportstate \ src/ibroute src/ibstat src/ibsysstat src/ibtracert \ src/perfquery src/sminfo src/smpdump src/smpquery \ src/saquery src/vendstat src/iblinkinfo \ - src/ibqueryerrors + src/ibqueryerrors src/ibcacheedit if ENABLE_TEST_UTILS sbin_PROGRAMS += src/ibsendtrap src/mcm_rereg_test @@ -62,6 +62,8 @@ src_iblinkinfo_SOURCES = src/iblinkinfo.c src_iblinkinfo_LDFLAGS = -L$(top_builddir)/libibnetdisc -libnetdisc src_ibqueryerrors_SOURCES = src/ibqueryerrors.c src_ibqueryerrors_LDFLAGS = -L$(top_builddir)/libibnetdisc -libnetdisc +src_ibcacheedit_SOURCES = src/ibcacheedit.c +src_ibcacheedit_LDFLAGS = -L$(top_builddir)/libibnetdisc -libnetdisc man_MANS = man/ibaddr.8 man/ibcheckerrors.8 man/ibcheckerrs.8 \ man/ibchecknet.8 man/ibchecknode.8 man/ibcheckport.8 \ @@ -76,7 +78,7 @@ man_MANS = man/ibaddr.8 man/ibcheckerrors.8 man/ibcheckerrs.8 \ man/ibprintswitch.8 man/ibprintca.8 man/ibfindnodesusing.8 \ man/ibdatacounts.8 man/ibdatacounters.8 \ man/ibrouters.8 man/ibprintrt.8 man/ibidsverify.8 \ - man/check_lft_balance.8 + man/check_lft_balance.8 man/ibcacheedit.8 BUILT_SOURCES = ibdiag_version ibdiag_version: diff --git a/infiniband-diags/man/ibcacheedit.8 b/infiniband-diags/man/ibcacheedit.8 new file mode 100644 index 000..b977827 --- /dev/null +++ b/infiniband-diags/man/ibcacheedit.8 @@ -0,0 +1,57 @@ +.TH IBCACHEEDIT 8 Apr 15, 2010 OpenIB OpenIB Diagnostics + +.SH NAME +ibcacheedit \- edit an ibnetdiscover cache + +.SH SYNOPSIS +.B ibcacheedit +[\-\-switchguid BEFOREGUID:AFTERGUID] [\-\-caguid BEFORE:AFTER] +[\-\-sysimgguid BEFOREGUID:AFTERGUID] [\-\-portguid NODEGUID:BEFOREGUID:AFTERGUID] +[\-h(elp)] orig.cache new.cache + +.SH DESCRIPTION +.PP +ibcacheedit allows users to edit an ibnetdiscover cache created through the +\fB\-\-cache\fR option in +.B ibnetdiscover(8). + +.SH OPTIONS + +.PP +.TP +\fB\-\-switchguid\fR BEFOREGUID:AFTERGUID +Specify a switchguid that should be changed. The before and after guid +should be separated by a colon. On switches, port guids are identical +to the switch guid, so port guids will be adjusted as well on switches. +.TP +\fB\-\-caguid\fR BEFOREGUID:AFTERGUID +Specify a caguid that should be changed. The before and after guid +should be separated by a colon. +.TP +\fB\-\-sysimgguid\fR BEFOREGUID:AFTERGUID +Specify a sysimgguid that should be changed. The before and after guid +should be spearated by a colon. +.TP +\fB\-\-portguid\fR NODEGUID:BEFOREGUID:AFTERGUID +Specify a portguid that should be changed. The nodeguid of the port +(e.g. switchguid or caguid) should be specified first, followed by a +colon, the before port guid, another colon, then the after port guid. +On switches, port guids are identical to the switch guid, so the +switch guid will be adjusted as well on switches. + +.SH COMMON OPTIONS + +Most OpenIB diagnostics take the following common flags. The exact list of +supported flags per utility can be found in the usage message and can be shown +using the util_name -h syntax. + +# Debugging flags +.PP +\-h show the usage message +.PP +\-V show the version info. + +.SH AUTHORS +.TP +Albert Chu +.RI ch...@llnl.gov diff --git a/infiniband-diags/src/ibcacheedit.c b/infiniband-diags/src/ibcacheedit.c new file mode 100644 index 000..1f2558b --- /dev/null +++ b/infiniband-diags/src/ibcacheedit.c @@ -0,0 +1,361 @@ +/* + * Copyright (c) 2010 Lawrence Livermore National Lab. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD
Re: [PATCH] rdma/cm: Randomize local port allocation.
Sean Hefty wrote: From: Tetsuo Handa penguin-ker...@i-love.sakura.ne.jp Randomize local port allocation in a way sctp_get_port_local() does. Update rover at the end of loop since we're likely to pick a valid port on the first try. Signed-off-by: Tetsuo Handa penguin-ker...@i-love.sakura.ne.jp Reviewed-by: Sean Hefty sean.he...@intel.com Thanks, everyone! I like this version, thanks! I'm not sure which tree to merge it through. Are you needing this for 2.6.34, or is 2.6.35 okay? As soon as possible, so 2.6.34. :) -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
