RE: [RFC v2] [OFED] libibverbs: Support both OFED verbs and ibverbs

2011-07-20 Thread Hefty, Sean
I was able to run mvapich2 osu benchmarks with xrc support using the following options: configure --with-rdma=gen2 --enable-xrc=yes CFALGS='-DOFED_VERBS -D_ENABLE_XRC' export MV2_USE_SHARED_MEM=0 export MV2_USE_XRC=1 Testing revealed a couple of issues in the libraries and kernel patches. All

Re: [RFC v2] [OFED] libibverbs: Support both OFED verbs and ibverbs

2011-07-14 Thread Jack Morgenstein
Hi Sean, I am pleased that you are putting in the effort to enable the existing OFED user base to continue using its code without changes to the XRC calls. Regarding XRC and MPI, see below. On Wednesday 13 July 2011 20:21, Hefty, Sean wrote: I was able to build and run mvapich2 successfully

Re: [RFC v2] [OFED] libibverbs: Support both OFED verbs and ibverbs

2011-07-14 Thread Jeff Squyres
Sean pinged me last night about XRC in Open MPI last night (note that I am no longer on the linux-rdma list). Open MPI uses XRC, but in a non-default manner -- the user has to specifically ask for it at run time. On Jul 14, 2011, at 9:13 AM, Jack Morgenstein wrote: Hi Sean, I am pleased

Re: [RFC v2] [OFED] libibverbs: Support both OFED verbs and ibverbs

2011-07-14 Thread Jack Morgenstein
On Thursday 14 July 2011 16:39, Dhabaleswar Panda wrote: Both MVAPICH1 and MVAPICH2 have XRC support for many years and are being used by many large-scale production clusters. Sorry about that! My bad. -Jack -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body

Re: [RFC v2] [OFED] libibverbs: Support both OFED verbs and ibverbs

2011-07-14 Thread Dhabaleswar Panda
Sean, Both MVAPICH1 and MVAPICH2 have XRC support for many years and are being used by many large-scale production clusters. For MVAPICH1, this support is available in both the standard Gen2 interface and the Hybrid (UD-RC/XRC) interface. From MVAPICH1 user guide (sec 4.4.1), you can find the

RE: [RFC v2] [OFED] libibverbs: Support both OFED verbs and ibverbs

2011-07-14 Thread Hefty, Sean
Both MVAPICH1 and MVAPICH2 have XRC support for many years and are being used by many large-scale production clusters. To clarify my comment: --with-rdma=gen2 CFLAGS='-DOFED_VERBS -D_ENABLE_XRC_ It didn't appear that mvapichs ever used XRC I only meant that mvapich2 didn't invoke any

[RFC v2] [OFED] libibverbs: Support both OFED verbs and ibverbs

2011-07-13 Thread Hefty, Sean
This patch allows libibverbs to support both libibverbs API that shipped with OFED 1.5 and the upstream libibverbs API. This supports existing apps that are compiled against the upstream libibverbs (ibverbs). And in ideal cases, an application coded to the OFED version of libibverbs (ofverbs)