Jonas, The section 5.1.6 in MPI 4.0 should give you a better idea about the differences between size, extent and true extent. There are also few examples in Section 5.1.14 on how to manipulate the datatype using extent. I think you should find Examples 5.13 to 5.16 of particular interest.
Best, George. On Thu, Dec 16, 2021 at 5:15 PM Jonas Thies <j.th...@tudelft.nl> wrote: > Hi George, > > thanks, I'm starting to understand this now. > > Still not quite intuitive that "Type_create_resized" allows me to reset > the extent but not the size (just from a naming perspective). > > The man page is talking about extent, upper and lower bounds, but the > upper bound cannot be specified: > > NAME > MPI_Type_create_resized - Returns a new data type with new extent > and > upper and lower bounds. > > SYNTAX > C Syntax > #include <mpi.h> > int MPI_Type_create_resized(MPI_Datatype oldtype, MPI_Aint lb, > MPI_Aint extent, MPI_Datatype *newtype) > > > Jonas > On 16-12-2021 22:39, George Bosilca wrote: > > You are confusing the size and extent of the datatype. The size (aka the > physical number of bytes described by the memory layout) would be > m*nloc*sizeof(type), while the extent will be related to where you expect > the second element of the same type to start. If you do resize, you will > incorporate the leading dimension in your pointer computation, and will see > the gaps you were reporting. > > George. > > > > > On Thu, Dec 16, 2021 at 3:03 PM Jonas Thies via users < > users@lists.open-mpi.org> wrote: > >> Dear Gilles, >> >> thanks, the resizing fixes the issue, it seems. It is not really >> intuitive, though, because the actual extent of the data type is >> m*nloc*sizeof(int) and I have to make MPI believe that it is >> nloc*sizeof(int). And indeed, this seems to be not OpenMPI-specific, sorry >> for that. >> >> Best, >> >> Jonas >> >> MPI_Type_vector (Gilles Gouaillardet) >> >> >> ---------------------------------------------------------------------- >> >> Message: 1 >> Date: Thu, 16 Dec 2021 10:29:27 +0100 >> From: Jonas Thies <j.th...@tudelft.nl> <j.th...@tudelft.nl> >> To: users@lists.open-mpi.org >> Subject: [OMPI users] unexpected behavior when combining MPI_Gather >> and MPI_Type_vector >> Message-ID: <64075574-7a58-b194-208f-d455c10c8...@tudelft.nl> >> <64075574-7a58-b194-208f-d455c10c8...@tudelft.nl> >> Content-Type: text/plain; charset="utf-8"; Format="flowed" >> >> Dear OpenMPI community, >> >> Here's a little puzzle for the Christmas holidays (although I would >> really appreciate a quick solution!). >> >> I'm stuck with the following relatively basic problem: given a local >> nloc x m matrix X_p in column-major ordering on each MPI process p, >> perform a single MPI_Gather operation to construct the matrix >> X_0 >> X_1 >> ... >> >> X_nproc >> >> again, in col-major ordering. My approach is to use MPI_Type_vector to >> define an stype and an rtype, where stype has stride nloc, and rtype has >> stride nproc*nloc. The observation is that there is an unexpected >> displacement of (m-1)*n*p in the result array for the part arriving from >> process p. >> >> The MFE code is attached, and I use OpenMPI 4.0.5 with GCC 11.2 >> (although other versions and even distributions seem to display the same >> behavior). Example (nloc=3, nproc=3, m=2, with some additional columns >> printed for the sake of demonstration): >> >> >> > mpicxx -o matrix_gather matrix_gather.cpp >> mpirun -np 3 ./matrix_gather >> >> v_loc on P0: 3x2 >> 0 9 >> 1 10 >> 2 11 >> >> v_loc on P1: 3x2 >> 3 12 >> 4 13 >> 5 14 >> >> v_loc on P2: 3x2 >> 6 15 >> 7 16 >> 8 17 >> >> v_glob on P0: 9x4 >> 0 9 0 0 >> 1 10 0 0 >> 2 11 0 0 >> 0 3 12 0 >> 0 4 13 0 >> 0 5 14 0 >> 0 0 6 15 >> 0 0 7 16 >> 0 0 8 17 >> >> Any ideas? >> >> Thanks, >> >> Jonas >> >> >> >> -- >> *J. Thies* >> Assistant Professor >> >> TU Delft >> Faculty Electrical Engineering, Mathematics and Computer Science >> Institute of Applied Mathematics and High Performance Computing Center >> Mekelweg 4 >> 2628 CD Delft >> >> T +31 15 27 XXXX >> *j.th...@tudelft.nl <j.th...@tudelft.nl>* >> > -- > *J. Thies* > Assistant Professor > > TU Delft > Faculty Electrical Engineering, Mathematics and Computer Science > Institute of Applied Mathematics and High Performance Computing Center > Mekelweg 4 > 2628 CD Delft > > T +31 15 27 XXXX > *j.th...@tudelft.nl <j.th...@tudelft.nl>* >
