Thanks for your explanation, George. However:
suppose we have leave_pinned = FALSE. Then we go to
mca_mpool_rdma_find. There we try to find suitable registration in the
cache. Suppose we cannot (cache's empty). Then NULL registration is
returned, BTL is skipped and 0 is returned from mca_pml_ob1_rdma_btls.
This way RDMA never happends.
(To make it clear: this is not some theoretical situation, this is
what I'm actually observing. OpenMPI 1.2.9) Can somebody explain
please?
2009/9/29 George Bosilca <bosi...@eecs.utk.edu>:
> Roman,
>
> Before going into explaining the logic, let me state that the memory is
> registered (if required/supported) by the BTLs. However, this is done only
> at the moment when the memory segment is involved in any kind of
> communication.
>
> We do not want to replicate this at the PML level, in order to make sure
> that the amount of memory registered at any moment is minimal. In other
> words, the PML leave the decision on when to register and when to unregister
> to the BTLs. However, in order to speedup the code a little bit (and to keep
> things tidy), the PML will help the BTLs to work around the memory
> registration issue. And the code you pointed out is exactly the place where
> we do it.
>
> We need to register the memory if leave_pinned is TRUE, as registering will
> leave a trace. If leave_pinned is FALSE then we only check if somehow this
> memory is not already registered (by some BTL). In this case, there is no
> need to create a registration in the PML (if required the BTL will do it
> when needed).
>
> george.
>
> On Sep 28, 2009, at 13:44 , Roman Cheplyaka wrote:
>
>> Hi,
>> I'm trying to dig into OpenMPI sources but have some problems. Can
>> anyone explain the logic of the following code from
>> mca/pml/ob1/pml_ob1_rdma.c please?
>>
>> if(!mca_pml_ob1.leave_pinned) {
>> /* look through existing registrations */
>> btl_mpool->mpool_find(btl_mpool, base, size, ®);
>> } else {
>> /* register the memory */
>> btl_mpool->mpool_register(btl_mpool, base, size, 0, ®);
>> }
>>
>> It seems to me that we should register new memory if leave_pinned is
>> FALSE (i.e. no existing registrations avaliable) and search through
>> existing registrations otherwise, but the logic is inversed here.
>>
>> --
>> Roman I. Cheplyaka
>> _______________________________________________
>> devel mailing list
>> de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>
> _______________________________________________
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>
--
Roman I. Cheplyaka
