Re: [O-MPI devel] Registration Cache changes

[Gleb Natapov]

Hello Galen,

It seams this issue is still present and can be easily triggered.
(see attached program). Do you have plans to fix it?

On Wed, Sep 21, 2005 at 12:06:18PM -0600, Galen M. Shipman wrote:
> Gleb,
> 
> >
> > Gleb Natapov wrote:
> >
> >> Hello Galen,
> >>
> >> Finally I've got some time to look through the new code.
> >> I have couple of notes.  In pml_ob1_rdma.c you try to merge
> >> registrations in the number of places. The code looks like this:
> >>   btl_mpool->mpool_deregister(btl_mpool, reg);
> >>   btl_mpool->mpool_register(btl_mpool,
> >>                     new_base,
> >>             new_len,
> >>             MCA_MPOOL_FLAGS_CACHE,
> >>             &reg);
> >> How do you know reg is not in use? You can't deregister it if  
> >> somebody
> >> is using the registration!
> >>
> >
> > Good catch... this should check the reference count and
> > only deregister when the reference count actually goes to zero...
> > _______________________________________________
> > devel mailing list
> > de...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/devel
> >
> 
> Yes, this was a good catch.. This was causing all sorts of fun for us!
> Thanks,
> 

--
                        Gleb.

#include "mpi.h"
#include <stdio.h>
#include <stdlib.h>

#define MB (1024*1024)

char buf[5*MB];

int main( int argc, char *argv[])
{
    int myid, numprocs;
    int  namelen;
    char processor_name[MPI_MAX_PROCESSOR_NAME], *b;

    MPI_Init(&argc,&argv);
    MPI_Comm_size(MPI_COMM_WORLD,&numprocs);
    MPI_Comm_rank(MPI_COMM_WORLD,&myid);
    MPI_Get_processor_name(processor_name,&namelen);

    fprintf(stderr,"Process %d on %s\n",
	    myid, processor_name);

    if (myid == 0)
    {
	MPI_Request req[4];
	MPI_Status stat[4];
	MPI_Send (buf, 4*MB, MPI_BYTE, 1, 0, MPI_COMM_WORLD);
	MPI_Isend (buf, MB, MPI_BYTE, 1, 0, MPI_COMM_WORLD, &req[0]);
	MPI_Isend (buf + MB, MB, MPI_BYTE, 1, 0, MPI_COMM_WORLD, &req[1]);
	MPI_Isend (buf + 2*MB, MB, MPI_BYTE, 1, 0, MPI_COMM_WORLD, &req[2]);
	MPI_Isend (buf + 3*MB, 2*MB, MPI_BYTE, 1, 0, MPI_COMM_WORLD, &req[3]);
	MPI_Waitall (4, req, stat);
    }
    else
    {
        MPI_Status stat;
	MPI_Recv (buf, 4*MB, MPI_BYTE, 0, 0, MPI_COMM_WORLD, &stat);
	MPI_Recv (buf, MB, MPI_BYTE, 0, 0, MPI_COMM_WORLD, &stat);
	MPI_Recv (buf + MB, MB, MPI_BYTE, 0, 0, MPI_COMM_WORLD, &stat);
	MPI_Recv (buf + 2*MB, MB, MPI_BYTE, 0, 0, MPI_COMM_WORLD, &stat);
	MPI_Recv (buf + 3*MB, 2*MB, MPI_BYTE, 0, 0, MPI_COMM_WORLD, &stat);
    }

    fprintf(stderr,"Process %d on %s end\n",
	    myid, processor_name);
    MPI_Finalize();

    return 0;
}