Eugene -- somehow this slipped by me in the disaster that is my inbox. Sorry!
Answers below...
On Jul 31, 2012, at 11:17 AM, Eugene Loh wrote:
>> What specifically do you propose? I don't remember offhand if the status
>> conversion macros are the same as the regular int/INTEGER conversion macros
>> -- we want to keep the no-op behavior for the regular int/INTEGER conversion
>> macros and only handle the conversion of MPI_Status separately, I think.
>> Specifically: for MPI_Status, we can probably still have those shortcuts for
>> the int/INTEGERs, but treat the copying to the size_t separately.
> I'm embarrassingly unfamiliar with the code. My impression is that
> internally we deal with C status structures and so our requirements for
> Fortran status are:
> *) enough bytes to hold whatever is in a C status
> *) several words are addressable via the indices MPI_SOURCE, MPI_TAG, and
> MPI_ERROR
Correct.
> So, I think what we do today is sufficient in most respects. Copying between
> Fortran and C integer-by-integer is fine. It might be a little nonsensical
> for an 8-byte size_t component to be handled as two 4-byte words, but if we
> do so only for copying and otherwise only use that component from the C side,
> things should be fine.
>
> The only problem is if we try to use the Fortran array in-place. It's big
> enough, but its alignment might be wrong.
Ok.
So the issue is when (for example) Fortran MPI_Recv says "hey, C ints are the
same as Fortran INEGERs, so I don't need a temporary MPI_Status buffer; I'll
just use the INTEGER array that I was given, and pass it to the back-end C
MPI_Recv() routine." Then C MPI_Recv() tries to write to the size_t variable,
and it might be poorly aligned. Kaboom.
> So, specifically, what I propose is getting rid of the short-cuts that try to
> use Fortran statuses in-place if Fortran INTEGERs are as big as C ints. I
> can make the changes. Sanity checks on all that are welcome.
Hmm. I'm not excited about this -- the whole point is that if we don't need to
do an extra copy, let's not do it.
Is there a better way to fix this?
Off the top of my head -- for example, could we change some of those
compile-time checks to run-time checks, and add in an alignment check? E.g.:
----
#if OMPI_SIZEOF_FORTRAN_INTEGER == SIZEOF_INT
c_status = (MPI_Status *) status;
#else
c_status = &c_status2;
#endif
-----
to
----
/* The constant checks will be resolved at compile time; assume
alignment_is_good() is an inline macro checking for "good" alignment
on platforms where alignment(int) != alignment(size_t) */
if (OMPI_SIZEOF_FORTRAN_INTEGER == SIZEOF_INT && alignment_is_good(status))
{
c_status = (MPI_Status *) status;
} else {
c_status = &c_status2;
}
-----
Would something like that work?
I'm thinking that the benefit here is that we only penalize platforms (with an
extra "if" statement) where alignment(int) != alignment(size_t).
>> Related issue: do we need to (conditionally) add padding for the size_t in
>> the Fortran array?
> I guess so, but once again am unsure of myself. If I look in
> ompi/config/ompi_setup_mpi_fortran.m4, we compute the size of 4 C ints and a
> size_t in units of Fortran INTEGERs. I'm guessing that usually works for us
> today since any padding is at the very end and doesn't need to be copied. If
> someone reorganized MPI_Status, however, there could be internal padding and
> we would start losing parts of the status. So, it might make the code a
> little more robust if the padding were accounted for. I'm not keen on making
> such a change myself.
You're probably right -- it's not needed at this time (if it's 4 int's plus a
size_t).
> Meanwhile, the config code errors out if the size turns out not to be an even
> multiple of Fortran INTEGER size. I don't get this. I would think one could
> just round up to the next multiple. I'm worried my understanding of what's
> going on is faulty.
I probably did that just as a sanity check -- i.e., why wouldn't the Fortran
status size be an even multiple of integers? If it isn't, that would seem to
be a very strange machine, and I'd be worried of something else breaking (e.g.,
copying the status values over int-by-int might actually result in a wonky
error).
It's also possible/likely that that m4 code pre-dates us having a size_t in the
C status.
>> [regarding IBM test suite / MPI_STATUS_IGNORE and MPROBE]
>> If not, we should probably extend them to do so.
> I suppose so. Horse having left the barn, we had better close the doors. I
> can add to the tests, albeit making some judgment calls about how carried
> away to get with this task. Clearly, that test suite is not very aggressive
> about exercising all code paths.
Correct. That suite started off its life as the IBM test suite, and as morphed
over time with LAM and Open MPI tests being added to it. I wrote the MPROBE
tests and obviously didn't check for all the proper corner cases...
--
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/
