Re: [OMPI devel] RDMA pipeline

[George Bosilca]

Actually, it restores the original behavior. The RDMA operations were pipelined before the r15247 commit, independent of the fact that they had mpool or not. We were actively using this behavior in the message logging framework to hide the cost of the local storage of the payload, and we were quite surprised when we realized that it disappeared.

  Thanks,
    george.


On Feb 19, 2008, at 3:03 PM, Gleb Natapov wrote:

If a BTL don't want to use pipeline for RDMA operations, it can set the RDMA fragment size to the max value, and this will automatically disable the pipeline. However, if the BTL support pipeline with the trunk version today it is not possible to activate it. Moreover, in the current version the parameters that define the BTL behavior are blatantly ignored, as the PML make high level assumption about what they want to do.

On Tue, Feb 19, 2008 at 02:13:30PM -0500, George Bosilca wrote:

Few days ago during some testing I realize that the RDMA pipeline was

into the source code, pinpointed the problem into the pml_ob1_rdma.c

course it fails), which basically disable the RDMA pipeline.

disabled for MX and Elan (I didn't check for the others). A quick look file, and it seems that the problem was introduced by commit 15247. The problem comes from the usage of the dummy registration, which is set for all non mpool friendly BTL. Later on this is checked against NULL (and of

Do you mean that mca_pml_ob1_send_request_start_rdma() is used for
rendezvous sends? I will be very surprised if ompi 1.2 works

is registered and no pipeline is needed. Trunk does the same but

by MPI_alloc_mem().

differently. It assumes that if btl has no mpool then entire message buffer differently. OpenIB also choose this route if buffer memory is allocated

I'll enable the RDMA pipeline back in 2 days if I don't hear anything
back. Attached is the patch that fix this problem.

I am not sure why you need pipeline for BTLs that don't require

from v1.0. (unless I miss something, then please provide more
explanations).

--
                        Gleb.
_______________________________________________
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel

registration, but by applying this patch you'll change how ompi behaves

smime.p7s
Description: S/MIME cryptographic signature