RE: [PATCH RFC 2/9] IB/core: Introduce Signature Verbs API

2013-10-17 Thread Hefty, Sean
@@ -885,6 +901,19 @@ struct ib_send_wr { u32 rkey; struct ib_mw_bind_info bind_info; } bind_mw; + struct { + struct ib_sig_attrs*sig_attrs; + struct ib_mr

RE: [PATCH librdmacm] man/rdma_create_id.3: Add RDMA_PS_IB port space description

2013-09-20 Thread Hefty, Sean
applied - thanks! -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

RE: [PATCH librdmacm] rstream.c: Indicate when specified address family is unknown

2013-09-20 Thread Hefty, Sean
applied - thanks! -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

RE: [PATCH librdmacm] man/rstream.1: Update man page to be consistent with rstream -h

2013-09-20 Thread Hefty, Sean
applied - thanks! -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

RE: [PATCH librdmacm] examples: Remove unneeded include of header file when using rocket.h

2013-09-20 Thread Hefty, Sean
Subject: [PATCH librdmacm] examples: Remove unneeded include of header file when using rocket.h as it's included in rsocket.h My preference is to include the file explicitly. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to

RE: [ceph-users] Help needed porting Ceph to RSockets

2013-09-20 Thread Hefty, Sean
I would not call these workarounds a real fix, but they should point out the problems which I am trying to solve. Thanks for the update. I haven't had the time to investigate this, but did want to at least acknowledge that this hasn't gotten lost. - Sean -- To unsubscribe from this list:

RE: rsockets addressing

2013-09-16 Thread Hefty, Sean
which kind of system is used by rsockets to address the remote host? Is using IPoIB ? rsockets is built over the RDMA CM. The rdma cm uses ipoib indirectly when connecting over IB, by mapping the IP addresses to GIDs. rsockets can also support using native IB addresses, though this is a new

RE: [PATCH V2 2/4] IB/usnic: Cisco VIC - driver part 1/3

2013-09-12 Thread Hefty, Sean
We are using the RoCE ethertype in the MAC frames, and that is all of the similarity between usNIC protocol and RoCE. L3 is not the GRH, but rather a usNIC L3 header. Customers can distinguish between RoCE traffic and usNIC traffic by looking at the version of the L3 frame. It is set to 6

RE: [PATCH] IB/cma: use cached gids

2013-09-10 Thread Hefty, Sean
The cma_acquire_dev function was changed by commit 3c86aa70bf67 to use find_gid_port because multiport devices might have either IB or IBoE formatted gids. The old function assumed that all ports on the same device used the same GID format. However, when it was changed to use find_gid_port,

RE: [PATCH 0/4] RDMA/nes and RDMA/cxgb4: iWARP Port Mapper Overview

2013-09-03 Thread Hefty, Sean
I think there are lots of #defines and enums replicated in both nes_netlink.h and c4iw_netlink.h. For example, the IWPM_* defines and enums. It seems like we could put all this in a common header file to be included by all iwarp providers? Say include/rdma/iw_portmap.h or something.

RE: Slow performance with librspreload.so

2013-08-30 Thread Hefty, Sean
with 2 parallel connection i'm able to reach rate speed with iperf, the same speed archived with rstream. Is iperf affected by IPoIB MTU size when used with librspreload.so ? Not directly. The ipoib mtu is usually set based on the mtu of the IB link. The latter does affect rsocket

RE: Slow performance with librspreload.so

2013-08-30 Thread Hefty, Sean
Another strange issue: $ sudo LD_PRELOAD=/usr/local/lib/rsocket/librspreload.so iperf -c 172.17.0.2 Client connecting to 172.17.0.2, TCP port 5001 TCP window size: 128 KByte (default) Increasing the window size may improve the

RE: Slow performance with librspreload.so

2013-08-28 Thread Hefty, Sean
i've connected just one port between two hosts. Ports is detected properly as 20Gb/s (4x DDR) but i'm unable to reach speed over 5Gbit/s: It's possible that this is falling back to using normal TCP sockets. Can you run the rstream test program to verify that you can get faster than 5 Gbps?

RE: [PATCH librdmacm 0/3] no IBV_SEND_INLINE thus memory registration required on QLogic/Intel HCA

2013-08-28 Thread Hefty, Sean
Perhaps it would be best to have ULP's ask for 0 and have providers return their = 0 max, which would implement this as the man page suggests? Based on performance tests, increasing the maximum inline can result in a significant decrease in overall bandwidth. So, I don't believe that we want

RE: [PATCH librdmacm 0/3] no IBV_SEND_INLINE thus memory registration required on QLogic/Intel HCA

2013-08-28 Thread Hefty, Sean
Based on performance tests, increasing the maximum inline can result in a significant decrease in overall bandwidth. So, I don't believe that we want providers to increase max_inline automatically to what could be supported. So why would providers return a max_inline value that

RE: Slow performance with librspreload.so

2013-08-28 Thread Hefty, Sean
2013/8/28 Hefty, Sean sean.he...@intel.com: Can you explain your environment more? The performance seems low. Ubuntu 13.04 Server on both nodes. node1: $ cat /proc/cpuinfo | grep 'model name' model name : Intel(R) Xeon(R) CPU E5-2603 0 @ 1.80GHz $ cat /proc/cpuinfo | grep 'model

RE: Slow performance with librspreload.so

2013-08-28 Thread Hefty, Sean
Ubuntu 13.04 Server on both nodes. node1: $ cat /proc/cpuinfo | grep 'model name' model name : Intel(R) Xeon(R) CPU E5-2603 0 @ 1.80GHz If you can provide your PCIe information and the results from running the perftest tools (rdma_bw), that could help as well. -- To unsubscribe from this

RE: [PATCH librdmacm 0/3] no IBV_SEND_INLINE thus memory registration required on QLogic/Intel HCA

2013-08-27 Thread Hefty, Sean
While doing some tests, I've found that rdma_client failed on my QLogic/Intel QLE7340 / QLE7342 HCA: # rdma_client rdma_client: start rdma_post_send 22 rdma_client: end -1 I had a deeper look on the examples and found that max_inline_data was returned as 0, The man

RE: [PATCH librdmacm] examples: adds cmtime to .gitignore

2013-08-27 Thread Hefty, Sean
thanks - applied -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

RE: RDMA CM EVENT rejected with status = 4 (timeout)

2013-08-23 Thread Hefty, Sean
P1- addr_resolve() select(cm_event_fd) gets RDMA_CM_ADDR_RESOLVED rout_resolve() gets RDMA_CM_ROUTE_RESOLVE connect(): P2: Gets Connect request P1: - Dies When process 1 dies after trying to establish a connection, this generates a connection

RE: [ceph-users] Help needed porting Ceph to RSockets

2013-08-20 Thread Hefty, Sean
I have added the patch and re-tested: I still encounter hangs of my application. I am not quite sure whether the I hit the same error on the shutdown because now I don't hit the error always, but only every now and then. I guess this is at least some progress... :/ WHen adding the patch to

RE: [PATCH] Add new verb: uv_query_port_max_datagram()

2013-08-20 Thread Hefty, Sean
Where is the documentation for this? Multiple people have referred to it, but I don't see any mention of it in libibverbs.git. This is an unmerged, yet to be accepted patch set. Extensions were added as part of adding support for XRC. Yishai Hadas posted v9 of the series on 8/1 - Add

RE: [ceph-users] Help needed porting Ceph to RSockets

2013-08-19 Thread Hefty, Sean
Can you see if the patch below fixes the hang? Signed-off-by: Sean Hefty sean.he...@intel.com --- src/rsocket.c | 11 ++- 1 files changed, 10 insertions(+), 1 deletions(-) diff --git a/src/rsocket.c b/src/rsocket.c index d544dd0..e45b26d 100644 --- a/src/rsocket.c +++ b/src/rsocket.c

RE: [PATCH] Add new verb: uv_query_port_max_datagram()

2013-08-19 Thread Hefty, Sean
Bumped the ABI version to 7 (the new verb will return -ENOSYS if abi_verb is 7). How does this break the ABI? -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at

RE: [PATCH] Add new verb: uv_query_port_max_datagram()

2013-08-19 Thread Hefty, Sean
It doesn't *break* the ABI, but it does add a new downcall into the kernel. That requires bumping the ABI version to 7, no? No - adding a new command is fine. Older kernels will return ENOSYS if that command is not supported. In that case, you can handle things like Jason suggested. - Sean

RE: [PATCH] Add new verb: uv_query_port_max_datagram()

2013-08-19 Thread Hefty, Sean
This breaks the libibverbs ABI. You can't modify ibv_context_ops because it changes struct ibv_context. Any suggestions on how one adds a new driver call without breaking ABI? It could be built on the verbs extension mechanism. Is it necessary to call into a provider library, versus

RE: [PATCH libibverbs] XRC - Sample application issues

2013-08-16 Thread Hefty, Sean
@@ -884,6 +884,13 @@ int main(int argc, char *argv[]) if (ctx.use_event) ibv_ack_cq_events(ctx.recv_cq, num_cq_events); + /* Process should wait before closing its resources to make sure + * latest daemon's response sent via its target QP destined to an XSRQ +

RE: [ceph-users] Help needed porting Ceph to RSockets

2013-08-16 Thread Hefty, Sean
I am looking at a multithreaded application here, and I believe that the race is between thread A calling the rpoll() for POLLIN event and thread B calling the shutdown(SHUT_RDWR) for reading and writing of the (r)socket almost immediately afterwards. I modified a test program, and I can

RE: [ceph-users] Help needed porting Ceph to RSockets

2013-08-14 Thread Hefty, Sean
The first question I would have is: why is the rpoll() split into these two pieces? There must have been some reason to do a busy loop on some local state information rather than just call the real poll() directly. As Scott mentioned in his email, this is done for performance reasons. The

RE: [ceph-users] Help needed porting Ceph to RSockets

2013-08-13 Thread Hefty, Sean
I found a workaround for my (our) problem: in the librdmacm code, rsocket.c, there is a global constant polling_time, which is set to 10 microseconds at the moment. I raise this to 1 - and all of a sudden things work nicely. I am adding the linux-rdma list to CC so Sean might see

RE: [PATCH 3/3] read_config: skip file/directory with unsecure permissions

2013-08-12 Thread Hefty, Sean
This is slightly off topic, but what is the underlying reason for having the libibverbs config directory? Is it simply to support the library name being different than the device name, or is there something more? - Sean -- To unsubscribe from this list: send the line unsubscribe linux-rdma in

RE: using rsockets via librspreload: poll() support?

2013-08-08 Thread Hefty, Sean
I am currently testing rsockets in connection with ceph. I am using LD_PRELOAD and the librspreload.so to force the application (ceph) to use rsockets instead of regular tcp/ip sockets. All this works pretty well - until the point where an established connection is shut down: this seems to

RE: using rsockets via librspreload: poll() support?

2013-08-08 Thread Hefty, Sean
I don't believe that errno is supposed to be cleared by calls. It is only set when an error occurs. This way, errno does not get lost. Is poll() returning -1? I begin to believe that this may be a more general problem: it seems to me that errno is not always initialized to 0 when the

RE: [PATCH librdmacm] cmtime: Add example program that times rdma cm calls

2013-08-08 Thread Hefty, Sean
and this is obviously missing the actual source code :P -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

RE: [PATCH ibacm 4/8] acm.c: Fix endian of transaction ID

2013-08-07 Thread Hefty, Sean
It should be in network rather than host endian. The TID is basically an abstract 64-bit handle. I don't see that it matters what value is used, provided that the sender use it consistently. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to

RE: [PATCH ibacm 2/8] acm.c: In acm_port_up, handle noncontiguous pkeys

2013-08-07 Thread Hefty, Sean
There can be holes in pkey table although this is not usually the case but since IBA spec allows for this, it should be handled. Signed-off-by: Hal Rosenstock h...@mellanox.com --- src/acm.c | 14 +++--- 1 files changed, 7 insertions(+), 7 deletions(-) diff --git a/src/acm.c

RE: [PATCH ibacm] acm.c: Handle client reregister local event

2013-08-05 Thread Hefty, Sean
+ case IBV_EVENT_CLIENT_REREGISTER: + if (dev-port[i].state == IBV_PORT_ACTIVE) { + acm_port_join(dev-port[i]); + acm_log(1, %s %d has reregistered\n, +

RE: [PATCH TRIVIAL ibacm] acm.c: In acm_ep_up, add pkey to error message

2013-08-01 Thread Hefty, Sean
applied -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

RE: [PATCH for-3.11 2/3] rdma/cm: Fix accessing invalid private data for UD

2013-07-31 Thread Hefty, Sean
If a application is using AF_IB with a UD QP, but does not provide any private data, we will end up accessing invalid memory. Check for this case and handle it appropriately. Signed-off-by: Sean Hefty sean.he...@intel.com Roland, hold off on this patch, but please push the other 2 for

RE: [PATCH 4/4] Declare 'server_port' as an unsigned variable

2013-07-30 Thread Hefty, Sean
thanks - applied all 4 -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

RE: Dual star topology

2013-07-27 Thread Hefty, Sean
This is still unclear. From the README ( http://git.openfabrics.org/git?p=~shefty/librdmacm.git;a=blob;f=README;h=e1f222 740144ed8f7d8bc935ea6643355b077bcd;hb=refs/heads/master ) I can read: Using multiple interfaces The librdmacm does support multiple interfaces. so, multiple

RE: Dual star topology

2013-07-25 Thread Hefty, Sean
IPoIB is very slow, I prefere to try with rsockets and the preloader library. Currently, rsockets cannot fail over between local HCA ports. This is an implementation restriction resulting from no automatic path migration support. - Sean -- To unsubscribe from this list: send the line

RE: [PATCH ibacm] Add support for name cache

2013-07-25 Thread Hefty, Sean
thanks - applied -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

RE: [PATCH V3 1/4] RDMA/cma: Add IPv6 support for iWARP.

2013-07-25 Thread Hefty, Sean
From: Steve Wise sw...@opengridcomputing.com This patch modifies the type of local_addr and remote_addr fields in struct iw_cm_id from struct sockaddr_in to struct sockaddr_storage to hold IPv6 and IPv4 addresses uniformly. It changes the references of local_addr and remote_addr in

RE: [PATCH] ib/mad: remove Device Mgmt from RMPP class list

2013-07-24 Thread Hefty, Sean
See Annex 8; DevMgt class version 2 rather than 1 is currently supported. Could older devices still return version 1? If so the kernel should allow DevMgt without RMPP, correct? This check has been this way since 2.6.17. I think it's reasonable to say that there aren't any devices using

RE: [PATCH] ib/mad: remove Device Mgmt from RMPP class list

2013-07-24 Thread Hefty, Sean
See Annex 8; DevMgt class version 2 rather than 1 is currently supported. Could older devices still return version 1? If so the kernel should allow DevMgt without RMPP, correct? This check has been this way since 2.6.17. I think it's reasonable to say that there aren't

RE: infiniband build warning

2013-07-21 Thread Hefty, Sean
I am seeing build warnings in drivers/infiniband/core/cma.c starting with v3.11-rc1. These can be reproduced with gcc 4.6.3. Would you consider applying the following fix ? A patch to fix this was submitted to the linux-rdma list last week. Thanks. - Sean -- To unsubscribe from this

RE: [PATCH ibacm] Update documentation for the new preload option names

2013-07-19 Thread Hefty, Sean
thanks - committed -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

RE: [PATCH V3 for-next 3/4] IB/core: Export ib_create/destroy_flow through uverbs

2013-07-18 Thread Hefty, Sean
+ kern_spec = kern_flow_attr + 1; + ib_spec = flow_attr + 1; + for (i = 0; i flow_attr-num_of_specs; i++) { + err = kern_spec_to_ib_spec(kern_spec, ib_spec); + if (err) + goto err_free; + flow_attr-size += + ((struct

RE: [PATCH REPOST FIXES for-3.11 1/4] IB/core: Create QP1 using the pkey index which contains the default pkey

2013-07-17 Thread Hefty, Sean
diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c index dc3fd1e..9be6754 100644 --- a/drivers/infiniband/core/mad.c +++ b/drivers/infiniband/core/mad.c @@ -2663,6 +2663,7 @@ static int ib_mad_port_start(struct ib_mad_port_private *port_priv) int ret, i;

RE: [PATCH v2] RDMA/cma: silence GCC warning

2013-07-17 Thread Hefty, Sean
Building cma.o triggers this GCC warning: drivers/infiniband/core/cma.c: In function ‘rdma_resolve_addr’: drivers/infiniband/core/cma.c:465:23: warning: ‘port’ may be used uninitialized in this function [-Wmaybe-uninitialized] drivers/infiniband/core/cma.c:426:5: note: ‘port’ was

RE: [PATCH V2] libibverbs: Allow arbitrary int values for MTU

2013-07-17 Thread Hefty, Sean
I hadn't looked at the kernel side yet; I was waiting for the userspace side to sort itself out first. I think it makes sense to start with how user space can get the data. Without eating up reserved fields, we're starting with 8 bit values. Hmm. 16 bits is probably enough for the MTU

RE: [PATCH V3 for-next 3/4] IB/core: Export ib_create/destroy_flow through uverbs

2013-07-17 Thread Hefty, Sean
+ssize_t ib_uverbs_create_flow(struct ib_uverbs_file *file, + const char __user *buf, int in_len, + int out_len) +{ + struct ib_uverbs_create_flow cmd; + struct ib_uverbs_create_flow_resp resp; + struct ib_uobject

RE: [PATCH V2] libibverbs: Allow arbitrary int values for MTU

2013-07-16 Thread Hefty, Sean
Jeff's patch doesn't break old binaries, old binaries, running with normal IB MTUs work fine. The structure layouts all stay the same, etc. FWIW, I did a simple test to confirm this. I installed a stock git HEAD libibverbs into $HOME/libibverbs-HEAD and a libibverbs with the MTU patch

RE: [PATCH V2] libibverbs: Allow arbitrary int values for MTU

2013-07-16 Thread Hefty, Sean
A source change is completely unvaoidable. Supporting the new MTU values requires updated source. I don't really care one way or the other; I'll submit whatever patch people want. :-) But FWIW, I tend to believe the Doug/Jason position: - MTU really needs to be a plain integer

RE: [PATCH librdmacm 3/8] autogen.sh: Use autoreconf in autogen.sh

2013-07-16 Thread Hefty, Sean
Thanks - I pulled in these patches, but see below: diff --git a/autogen.sh b/autogen.sh index f433312..6c9233e 100755 --- a/autogen.sh +++ b/autogen.sh @@ -1,9 +1,4 @@ #! /bin/sh set -x -test -d ./config || mkdir ./config Without the above line, the build fails. I added it back in.

RE: [PATCH] RDMA/cma: silence GCC warning

2013-07-15 Thread Hefty, Sean
1) Perhaps a better way to silence GCC is to drop port entirely, and assign to id_priv-id.port_num directly. Would that be acceptable? Yes, this would work. - Sean

RE: [PATCH 00/15 ibacm] ACM Cache Preloading (and some cosmetic cleanups)

2013-06-28 Thread Hefty, Sean
I've pulled in this series, with the changes mentioned in a separate email, plus updates to the patches below (mostly renames of functions, variables, enums, etc.). The updated patches are available from my ibacm.git tree in the preload branch. Please look over those changes and see if you

RE: [PATCH 00/15 ibacm] ACM Cache Preloading (and some cosmetic cleanups)

2013-06-28 Thread Hefty, Sean
I looked them over and retested this. I have a few minor fixups to follow shortly. I've incorporated your changes into the patch set and updated the preload branch. The one thing I noticed is that the check for using this (address) preload option without the previous (route) preload option

RE: [PATCH v2 15/16 ibacm] Add support for preloading IPv4 and IPv6 ACM caches

2013-06-28 Thread Hefty, Sean
+static void acm_parse_hosts_file(struct acm_ep *ep) +{ + FILE *f; + char s[120]; + char addr[INET6_ADDRSTRLEN], gid[INET6_ADDRSTRLEN]; + uint8_t name[ACM_MAX_ADDRESS]; + struct in6_addr ip_addr, ib_addr; + struct acm_dest *dest, *new_dest; + uint8_t addr_type;

RE: [PATCH V2 for-next 1/4] IB/core: Add receive Flow Steering support

2013-06-27 Thread Hefty, Sean
The TCP/IP filters are broken into separate filters based in L4/L3. It would seem to make sense if the IB filters were similarly divided into L2/L3/L4 filters. IB and IPv6 could probably share the same filter definition. IPv6 filters wasn't defined through this submission, but as I

RE: [PATCH V2 for-next 1/4] IB/core: Add receive Flow Steering support

2013-06-27 Thread Hefty, Sean
Still for the initial set of patches that goes in I tend to just remove the IB filter structure and define the different IB filters along your proposal in a follow-up patches/es, OK? That sounds fine to me. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a

RE: [PATCH 00/15 ibacm] ACM Cache Preloading (and some cosmetic cleanups)

2013-06-27 Thread Hefty, Sean
Patches 1-9 clean up documentation and sample config files. Rather than trying to keep the sample ibacm_addr/opts files in sync, I elected to remove them from the git tree. There's really no use for them, and I don't think they're included with the released package anyway. Doing this drops

RE: [PATCH 03/15 ibacm] acm.c: Cosmetic formatting change to acm log message

2013-06-27 Thread Hefty, Sean
diff --git a/src/acm.c b/src/acm.c index e956b09..a124e93 100644 --- a/src/acm.c +++ b/src/acm.c @@ -1277,7 +1277,7 @@ static void acm_process_acm_recv(struct acm_ep *ep, struct ibv_wc *wc, struct ac rec = (struct acm_resolve_rec *) mad-data; acm_format_name(2, log_data, sizeof

RE: [PATCH 10/15 ibacm] Add the ability to preload the destination GID and LID caches

2013-06-26 Thread Hefty, Sean
To preload these caches, ibacm service is started with -p option which can also take optional path_rec_file which defaults to ACM_CONF_DIR/ibacm_path_records.dump I would rather use the config file versus adding command line parameters. So add a new preload option in the config file

RE: [PATCH V2 for-next 1/4] IB/core: Add receive Flow Steering support

2013-06-26 Thread Hefty, Sean
The input to ib_create_flow is instance of struct ib_flow_attr which contain few mandatory control elements and optional flow specs. struct ib_flow_attr { enum ib_flow_attr_type type; u16 size; u16 priority; u8 num_of_specs; u8 port;

RE: [PATCH 10/15 ibacm] Add the ability to preload the destination GID and LID caches

2013-06-25 Thread Hefty, Sean
Preloading of these caches is supported via a file which is produced by OpenSM by the dump_pr plugin which contains sufficient SA PathRecord information. Details on this file format and configuring OpenSM for this are found in dump_pr_notes.txt in dump_pr. File format is specified in

RE: [PATCH V2] libibverbs: Allow arbitrary int values for MTU.

2013-06-21 Thread Hefty, Sean
raises hand I'm ready for a new project. I already had one in mind, but it required some changes to libibverbs that are pretty deep changes. Instead of doing that deep work in libibverbs, I can start here and move on to my next project when this is up and running. That would be

RE: [PATCH] libibverbs: Allow arbitrary int values for MTU

2013-06-20 Thread Hefty, Sean
I used the name num_to_ibv_mtu because it is in the spirit of the other enum- to-int/int-to-enum function pair naming conventions: int ibv_rate_to_mult(enum ibv_rate rate); enum ibv_rate mult_to_ibv_rate(int mult); int ibv_rate_to_mbps(enum ibv_rate rate); enum ibv_rate

RE: [PATCH V2] libibverbs: Allow arbitrary int values for MTU.

2013-06-20 Thread Hefty, Sean
I would argue that this is because the libraries are so disjoint (that librdmacm needs the deep internal knowledge it needs of libibverbs indicates that maybe these two shouldn't be separate from each other for example, or that maybe libibverbs should provide a unified connection API to the

RE: [PATCH] IB/core: Fix error return code in add_port()

2013-06-20 Thread Hefty, Sean
Subject: [PATCH] IB/core: Fix error return code in add_port() From: Wei Yongjun yongjun_...@trendmicro.com.cn Fix to return -ENOMEM in the alloc_group_attrs() error handling Patch itself looks fine, but please change alloc_group_attrs() - add_port() in the description. - Sean -- To

RE: [PATCH v5 01/28] rdma/cm: define native IB address

2013-06-10 Thread Hefty, Sean
Define AF_IB and sockaddr_ib to allow the rdma_cm to use native IB addressing. Signed-off-by: Sean Hefty sean.he...@intel.com --- include/linux/socket.h |2 + include/rdma/ib.h | 89 2 files changed, 91 insertions(+), 0

RE: [PATCH] libibverbs: A possible solution for allowing arbitrary MTU values.

2013-06-05 Thread Hefty, Sean
The concept of a libibverbs 2.0 has been NAK's by pretty much everyone involved. This is why we are suffering with the complex extension mechanism. Are you saying that libibverbs must always always always be backwards compatible, and there will never be an ABI break at any version in the

RE: [PATCH libmlx4 v6 1/2] libmlx4: Infra-structure changes to support verbs extensions

2013-06-04 Thread Hefty, Sean
+ #ifdef HAVE_IBV_REGISTER_DRIVER static __attribute__((constructor)) void mlx4_register_driver(void) { - ibv_register_driver(mlx4, mlx4_driver_init); + verbs_register_driver(mlx4, mlx4_driver_init); + } #else Shouldn't ibv_register_driver() need to be called in

RE: [PATCH libibverbs v6 3/7] livibverbs: Add support for XRC SRQs

2013-06-04 Thread Hefty, Sean
enum verbs_context_mask { VERBS_CONTEXT_XRCD = 1 0, - VERBS_CONTEXT_RESERVED = 1 1 + VERBS_CONTEXT_SRQ = 1 1, + VERBS_CONTEXT_RESERVED = 1 2 }; Why is _RESERVED being re-numbered here? That worries me.. For that matter, what is it for? I called it

RE: [PATCH libibverbs v6 6/7] libibverbs: Add man page for ibv_open_qp

2013-06-04 Thread Hefty, Sean
+uint32_t qp_num; /* QP number */ +void *qp_context; /* Associated context of the QP */ Would be nice to clarify what this value is too. I took a quick peak and it wasn't clear to me. The XRC example does not use it. Is it a user defined cookie? If so can we

RE: [PATCH libibverbs v5 3/7] livibverbs: Add support for XRC SRQs

2013-06-03 Thread Hefty, Sean
enum verbs_context_mask { VERBS_CONTEXT_XRCD = 1 0, - VERBS_CONTEXT_RESERVED = 1 1 + VERBS_CONTEXT_SRQ = 1 1, + VERBS_CONTEXT_RESERVED = 1 2 }; Would VERBS_CONTEXT_XSRQ or VERBS_CONTEXT_SRQ_EX be more clear? The structure being extended is ibv_srq, so

RE: [PATCH v3] Add trap details to notice log message

2013-05-31 Thread Hefty, Sean
Are you suggesting to do this in addition to ib_types.h or instead of ? umad_sm.h seems to hold only a small subset of what's found in ib_types.h. umad_sm.h was only recently added. I'm suggesting to put the definitions there instead of within the opensm header file, so that the definitions

RE: [PATCH v3] Add trap details to notice log message

2013-05-30 Thread Hefty, Sean
diff --git a/include/iba/ib_types.h b/include/iba/ib_types.h index d5cf287..5c238ae 100644 --- a/include/iba/ib_types.h +++ b/include/iba/ib_types.h @@ -3,6 +3,7 @@ * Copyright (c) 2002-2011 Mellanox Technologies LTD. All rights reserved. * Copyright (c) 1996-2003 Intel Corporation. All

RE: mlx4/xrc problem

2013-05-22 Thread Hefty, Sean
[root@hpc-hn1 libibverbs-1.1.4]# ibv_xsrq_pingpong -d mlx4_0 192.168.174.52 local: LID 0001, QPN RECV 98004b SEND 18004c, PSN 5b6d99, SRQN 0042 remote: LID 0002, QPN RECV d4004a SEND 54004b, PSN d7ba7a, SRQN 0042 The same SRQN on both sides looks suspicious. [root@hpc-cn2

RE: better understanding rdma-cm UNREACHABLE/ETIMEDOUT scheme

2013-05-21 Thread Hefty, Sean
So under some unknown conditions, many of the clients connection attempts fail with RDMA_CM_EVENT_UNREACHABLE event and the status is -ETIMEDOUT. Looking on the rdma-cm kernel code, I see that the only location which generates this event is in cma_ib_handler when getting IB_CM_REQ_ERROR (or

RE: better understanding rdma-cm UNREACHABLE/ETIMEDOUT scheme

2013-05-21 Thread Hefty, Sean
One thing seen in the nodes dmesg is a message from an old patch of yours which exists in ofed1.5.3 but didn't hit (or wasn't accepted?) upstream saying ib_cm: calculated mra timeout 67584 8192, decreasing used timeout_ms does this provides any insight into the problem? I don't

RE: [PATCH libibverbs v5 5/7] libibverbs: Add ibv_open_qp

2013-05-20 Thread Hefty, Sean
So is it intended that cooperating processes sharing an xrc domain will choose one process to create the xrc qp, and the rest will open it? yes - The QPN of the shared QP must be known between the cooperating processes. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in

RE: rockets feedbacks?

2013-05-20 Thread Hefty, Sean
Do we have some public quoted usages/feedback for rsockets? I think you've mentioned something during the panel at the Linux EU summit last week but I am not sure... Most feedback I can think of has come via private emails or personal interactions, especially specific details of various usage

RE: MLX4 Cq Question

2013-05-20 Thread Hefty, Sean
My reading of the code (i.e. hw/mlx4/cq.c) is that the hardware cqe owner_sr_opcode field contains MLX4_CQE_OPCODE_ERROR when there is an error and therefore, the only way to recover what the opcode was is through the wr_id you used when submitting the WR. Is my reading of the code correct?

RE: rockets feedbacks?

2013-05-20 Thread Hefty, Sean
So if you were pushing these private conversations to linux-rdma, more have been known on rsockets for the benefit of all... oh well. I think you mentioned something re Intel HPC group, or I am wrong? rsockets will continue to be supported by myself and Intel going forward. The rsocket work

RE: ibv_send_wr extension

2013-05-10 Thread Hefty, Sean
- Remove the pointer to the ibv_send_wr_ext pointer from the wr union in ibv_send_wr, and put the ibv_send_wr (less the duplicated data structures) after the union, with the usual comp_mask flag to indicate what is supported. This would be our #1 preference, because of the performance

RE: ibv_send_wr extension

2013-05-10 Thread Hefty, Sean
Previously we had agreed to use a new send_flag , IBV_SEND_EXTENDED, which would indicate the presence of the extended data structure. With new functionality, there may also be new extended opcodes, but this is not a requirement. I think that's fine. -- To unsubscribe from this list: send

RE: ibv_send_wr extension

2013-05-10 Thread Hefty, Sean
So are you are OK with this option ? There were 2 options, and this one is our preference. Remove the pointer to the ibv_send_wr_ext pointer from the wr union in ibv_send_wr, and put the ibv_send_wr (less the duplicated data structures) after the union, with the usual comp_mask flag to

RE: race in ULPs when processing RDMA_CM_EVENT_DEVICE_REMOVAL

2013-05-06 Thread Hefty, Sean
What would be the correct way to go for consumers, is that making sure they destroy 1st all their IB objects (PDs, MRs, CQs, QPs, etc) prior to destroying the last rdma_cm id on a device removal event? any other idea? The user should free any objects that are associated with the device

RE: race in ULPs when processing RDMA_CM_EVENT_DEVICE_REMOVAL

2013-05-06 Thread Hefty, Sean
So think on the case, e.g for iser where a ULP maintains a per-device context where they stroe IB objects used for all connections of that device (e.g PD, CQs, DMA-MR, etc) and per connection object where the QP and RDMA-CM ID are storage. The per-device context is created on demand

RE: [PATCH V4 for-next 1/5] IB/core: Add RSS and TSS QP groups - suggesting BOF during OFA conf to further discuss that

2013-04-25 Thread Hefty, Sean
Conceptually, RSS/TSS are a set of send/receive work queues all belonging to the same transport level address. There's no parent-child relationship or needed pairing of send and receive queues together in order to form a group. This view makes sense to me as well. Can someone also

RE: [PATCH V4 for-next 1/5] IB/core: Add RSS and TSS QP groups - suggesting BOF during OFA conf to further discuss that

2013-04-23 Thread Hefty, Sean
On Mon, Apr 22, 2013 at 7:46 PM, Or Gerlitz or.gerl...@gmail.com wrote: Sean, Tzahi -- I understand now that there have been few talkings @ the OFA meeting re this patch set. So what's the way to move forward, any concrete feedback that needs to be addressed here? This series is hanging

RE: [PATCH 2/2] Ad IB_MTU_1500|9000 enums.

2013-04-20 Thread Hefty, Sean
This seems reasonable, but still concerns me a bit. The original version was flat out wrong because you can't re-arrange any exposed enum like this without requiring that all user space apps be recompiled. This is especially true because ibv_mtu_enum_to_int is an inline ib_mtu_enum_to_int()

RE: [PATCH 1/2] Use autoreconf in autogen.sh

2013-04-19 Thread Hefty, Sean
Any thoughts on these two patches? They're pretty trivial, enable use with modern versions of Autotools, and now feature the proper Signed-off-by line. It may help if you identify the library this patch is against. :) -- To unsubscribe from this list: send the line unsubscribe linux-rdma in

RE: [RFC PATCH] libibumad: change sm_key to be be64_t

2013-04-18 Thread Hefty, Sean
Was there a reason to define sm_key as an array of uint8_t's vs a be64_t? I don't think the sm_key is 64-bit aligned, so you would need to pack the structure and deal with misaligned accesses. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to

RE: [PATCH V4 for-next 1/5] IB/core: Add RSS and TSS QP groups

2013-04-11 Thread Hefty, Sean
I have no issue with RSS/TSS. But the 'qp group' interface to using this seems kludgy. lets try to be more specific On a node, this is multiple send/receive queues grouped together to form a larger construct. On the wire, this is a single QP - maybe? I'm still not clear on

RE: [PATCH V3 for-next 0/5] IB/IPoIB: Add multi-queue TSS and RSS support

2013-04-09 Thread Hefty, Sean
any feedback? I have no issue with RSS/TSS. But the 'qp group' interface to using this seems kludgy. On a node, this is multiple send/receive queues grouped together to form a larger construct. On the wire, this is a single QP - maybe? I'm still not clear on that. From what's written,

RE: [PATCH 2/2] Ad IB_MTU_1500|9000 enums.

2013-04-08 Thread Hefty, Sean
Agreed, but changing to an int would seem to have some fairly serious backwards compatibility issues. Why can't IB_MTU_1500 = 1500? -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at

RE: [PATCH 1/4] Add IBV_*_USNIC enums for the Cisco Ethernet Virtual NIC.

2013-04-05 Thread Hefty, Sean
RDMA_NODE_VENDOR would be great, actually. Should I work up a patch for that? I would prefer taking this approach and would be fine accepting such a change. Roland, do you have an opinion on this? - Sean -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a

<    1   2   3   4   5   6   7   8   9   10   >