On Tue, Feb 19, 2008 at 02:13:30PM -0500, George Bosilca wrote:
> Few days ago during some testing I realize that the RDMA pipeline was
> disabled for MX and Elan (I didn't check for the others). A quick look
> into the source code, pinpointed the problem into the pml_ob1_rdma.c
> file, and it seems that the problem was introduced by commit 15247. The
> problem comes from the usage of the dummy registration, which is set for
> all non mpool friendly BTL. Later on this is checked against NULL (and of
> course it fails), which basically disable the RDMA pipeline.
Do you mean that mca_pml_ob1_send_request_start_rdma() is used for
rendezvous sends? I will be very surprised if ompi 1.2 works
differently. It assumes that if btl has no mpool then entire message buffer
is registered and no pipeline is needed. Trunk does the same but
differently. OpenIB also choose this route if buffer memory is allocated
by MPI_alloc_mem().
>
> I'll enable the RDMA pipeline back in 2 days if I don't hear anything
> back. Attached is the patch that fix this problem.
>
I am not sure why you need pipeline for BTLs that don't require
registration, but by applying this patch you'll change how ompi behaves
from v1.0. (unless I miss something, then please provide more
explanations).
--
Gleb.
