Re: [OMPI devel] OMPI over ofed udapl over iwarp

2007-05-10 Thread Steve Wise
> > There are two new issues so far: > > 1) this has uncovered a connection migration issue in the Chelsio > driver/firmware. We are developing and testing a fix for this now. > Should be ready tomorrow hopefully. > I have a fix for the above issue and I can continue with OMPI testing. To wo

Re: [OMPI devel] OMPI over ofed udapl over iwarp

2007-05-10 Thread Caitlin Bestler
devel-boun...@open-mpi.org wrote: >> There are two new issues so far: >> >> 1) this has uncovered a connection migration issue in the Chelsio >> driver/firmware. We are developing and testing a fix for this now. >> Should be ready tomorrow hopefully. >> > > I have a fix for the above issue and

Re: [OMPI devel] OMPI over ofed udapl over iwarp

2007-05-10 Thread Jeff Squyres
Steve -- Can you file a trac bug about this? On May 10, 2007, at 6:15 PM, Steve Wise wrote: There are two new issues so far: 1) this has uncovered a connection migration issue in the Chelsio driver/firmware. We are developing and testing a fix for this now. Should be ready tomorrow hopef

Re: [OMPI devel] OMPI over ofed udapl over iwarp

2007-05-10 Thread Donald Kerr
Caitlin Bestler wrote: devel-boun...@open-mpi.org wrote: There are two new issues so far: 1) this has uncovered a connection migration issue in the Chelsio driver/firmware. We are developing and testing a fix for this now. Should be ready tomorrow hopefully. I have a fix for the

Re: [OMPI devel] OMPI over ofed udapl over iwarp

2007-05-11 Thread Steve Wise
On Thu, 2007-05-10 at 23:10 -0400, Donald Kerr wrote: > > Caitlin Bestler wrote: > > >devel-boun...@open-mpi.org wrote: > > > > > >>>There are two new issues so far: > >>> > >>>1) this has uncovered a connection migration issue in the Chelsio > >>>driver/firmware. We are developing and testing

Re: [OMPI devel] OMPI over ofed udapl over iwarp

2007-05-11 Thread Caitlin Bestler
Donal Kerr wrote: >>> order of business after connection establishment >>> (mba_btl_udapl_sendrecv(). The RECV buffer post for this exchange, >>> however, should really be done _before_ the >>> dat_ep_connect() on the active side, and _before_ the >>> dat_cr_accept() on the server side. >>> Curre

Re: [OMPI devel] OMPI over ofed udapl over iwarp

2007-05-11 Thread Galen Shipman
More like trying to work around the race condition that exists: The server side sends an rdma message first thus violating the iwarp protocol. For those who want the gory details: when the server sends first -and- that rdma message arrives at the client _before_ the client transitions into r

Re: [OMPI devel] OMPI over ofed udapl over iwarp

2007-05-11 Thread Paul H. Hargrove
Caitlin Bestler wrote: [snip] > The DAPL semantics are very clear that send/recv operations must > be matched one to one, that the receive buffer must be large > enough for the received message and that there must be a receive > buffer for each incoming send/recv message. That means that > the send

Re: [OMPI devel] OMPI over ofed udapl over iwarp

2007-05-11 Thread Galen Shipman
As an aside, my personal feeling is that even when running over IB the preposting of recvs is worth the small overhead of piggybacking a credit system on the messages that already cross the wire. If nothing else, this avoids adding congestion of RNR-NAKS and the resends they trigger. Put

Re: [OMPI devel] OMPI over ofed udapl over iwarp

2007-05-13 Thread Donald Kerr
Caitlin Bestler wrote: Donal Kerr wrote: order of business after connection establishment (mba_btl_udapl_sendrecv(). The RECV buffer post for this exchange, however, should really be done _before_ the dat_ep_connect() on the active side, and _before_ the dat_cr_accept() on the server sid

Re: [OMPI devel] OMPI over ofed udapl over iwarp

2007-05-14 Thread Steve Wise
On Sun, 2007-05-13 at 21:26 -0400, Donald Kerr wrote: > > Caitlin Bestler wrote: > > >Donal Kerr wrote: > > > > > > > order of business after connection establishment > (mba_btl_udapl_sendrecv(). The RECV buffer post for this exchange, > however, should really be done _before_ the