Re: [OMPI devel] mpi/java question

2013-02-20 Thread Dmitri Gribenko
On Wed, Feb 20, 2013 at 10:05 PM, Ralph Castain  wrote:
>
> On Feb 20, 2013, at 11:39 AM, Dmitri Gribenko  wrote:
>
>> On Wed, Feb 20, 2013 at 9:34 PM, Jeff Squyres (jsquyres)
>>  wrote:
>>> If someone could write some generic java code to figure out the size of a 
>>> java type (and either printf it out, or write it to a file, or otherwise be 
>>> able to give that value to a shell script), that would be a good start.
>>
>> No need for that -- type sizes in Java are fixed.
>>
>> http://docs.oracle.com/javase/tutorial/java/nutsandbolts/datatypes.html
>
> True - but the ones on the C-side are not, and that's the problem.

My point was that there is no need to write java code to detect type
sizes.  About C types -- don't we already check those anyway?  Sure,
we need to match these with java side, but there's no need to write
new code to check type sizes.

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/


Re: [OMPI devel] mpi/java question

2013-02-20 Thread Dmitri Gribenko
On Wed, Feb 20, 2013 at 9:34 PM, Jeff Squyres (jsquyres)
 wrote:
> If someone could write some generic java code to figure out the size of a 
> java type (and either printf it out, or write it to a file, or otherwise be 
> able to give that value to a shell script), that would be a good start.

No need for that -- type sizes in Java are fixed.

http://docs.oracle.com/javase/tutorial/java/nutsandbolts/datatypes.html

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/


Re: [OMPI devel] [EXTERNAL] Re: Compile-time MPI_Datatype checking

2012-11-19 Thread Dmitri Gribenko
On Mon, Nov 19, 2012 at 6:51 PM, Jeff Squyres  wrote:
> On Nov 19, 2012, at 11:47 AM, Dmitri Gribenko wrote:
>
>>> 3. the warnings are disabled for buffer types of (char*) or (void*)
>>
>> I think that disabling warnings for char* might make sense.  But
>> during the LLVM developer meeting I talked with Hal Finkel (MPICH2
>> developer) about this.  These annotations are already integrated into
>> MPICH2.  Hal said that he found real bugs with these warnings
>> involving char* buffers.
>>
>> Is it really not an option for the codebase in question to build with
>> -D... or -Wno-...?
>
>
> It is pedantic, but I can still side with: it's valid MPI (and those who do 
> it usually do it for a reason), so we shouldn't warn about it.
>
> How about a phased implementation:
>
> 3a. Have an option to *also* warn about (char*) mismatches?
> 3b. Over time (perhaps in coordination with MPICH?), and polling once this 
> initial implementation has been out in the world for a while: if user want, 
> we can change the default to warn about (char*) mismatches by default

We can move warnings for char* to a separate warning group,
-Wtype-safety-char-ptr and make it off by default.

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/


Re: [OMPI devel] [EXTERNAL] Re: Compile-time MPI_Datatype checking

2012-11-19 Thread Dmitri Gribenko
On Mon, Nov 19, 2012 at 6:35 PM, Jeff Squyres  wrote:
> (SC is now over, and, aside from the US holiday later this week, I'm picking 
> up the items that I've let fall on the floor for the past 1-2 months...)
>
> On Oct 31, 2012, at 3:51 PM, Barrett, Brian W wrote:
>
>> That said, if we didn't throw a warning if the pointer is of type void* or
>> char*, I think I'd be mostly ok with the patch being on by default.  I'm
>> not sure if that's possible or not...
>
>
> This might be a good compromise.  How about the following (assuming this all 
> only happens for compilers that support this stuff, which, at the moment, 
> will be clang version >= X.Y.Z, for some values of X, Y, and Z):
>
> 1. the warnings are enabled by default in mpi.h

Awesome!

> 2. the warnings can be disabled by a magic -D (e.g., 
> -DDISABLE_MPI_TYPE_CHECKING or whatever -- the -Wno-type-safety option sounds 
> scary; does it really disable *ALL* type safety?)

-Wno-type-safety just turns off warnings from 'type-safety' group.
Just like -Wno-unitialize turns off warnings about 'uninitialized'.

> 3. the warnings are disabled for buffer types of (char*) or (void*)

I think that disabling warnings for char* might make sense.  But
during the LLVM developer meeting I talked with Hal Finkel (MPICH2
developer) about this.  These annotations are already integrated into
MPICH2.  Hal said that he found real bugs with these warnings
involving char* buffers.

Is it really not an option for the codebase in question to build with
-D... or -Wno-...?

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/


Re: [OMPI devel] [EXTERNAL] Re: Compile-time MPI_Datatype checking

2012-10-31 Thread Dmitri Gribenko
On Wed, Oct 31, 2012 at 9:51 PM, Barrett, Brian W  wrote:
> On 10/31/12 1:39 PM, "Paul Hargrove"  wrote:
>
>>No, I don't have specific usage cases that concern me.
>>
>>
>>As I said a minute or two ago in a reply to Ralph, my concern is that the
>>Sandia codes provide an "existence proof" that "really smart people" can
>>write questionable code at times.  So, I fear that a larger-than-expected
>>fraction of real codes would generate warnings.
>
> Not surprisingly, most of the codes I'm concerned about are really old
> (like pre-MPI old).  The authors were dealing with more than one
> communication library, so they wrote wrappers inside their code for
> communication.  The wrappers were for a bunch of different communication
> interfaces and so fairly agnostic, but looked a lot like MPI (because MPI
> looks a lot like NX, PVM, etc.).  Anyway, they squashed down everything to
> either a void* or char* (remember, this is when void* was not always
> supported), passed that to MPI with a datatype, and off we go.

Thank you for sharing the context.

> That said, if we didn't throw a warning if the pointer is of type void* or
> char*, I think I'd be mostly ok with the patch being on by default.  I'm
> not sure if that's possible or not...

It is certainly possible to do that.  However, we would lose the
ability to diagnose type mismatches for MPI_CHARACTER, but it is a
rarely-used type tag (is it?).

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/


Re: [OMPI devel] Compile-time MPI_Datatype checking

2012-10-31 Thread Dmitri Gribenko
On Wed, Oct 31, 2012 at 9:11 PM, Paul Hargrove  wrote:
> Ralph,
>
> I work at a National Lab, and like many of my peers I develop/prototype
> codes on my desktop and/or laptop.  So, I think the default behavior of
> mpicc on a Clang-based Mac is entirely relevant.
>
> FWIW:
> I agree w/ Jeff that these datatype checking warnings "feel" like a
> candidate for "-Wall" (or "-Wextra"), rather than enabled by default.

Hi Paul,

Is there any particular case you are concerned about?

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/


Re: [OMPI devel] Compile-time MPI_Datatype checking

2012-10-31 Thread Dmitri Gribenko
On Wed, Oct 31, 2012 at 9:04 PM, Ralph Castain  wrote:
> Understood, but also remember that the national labs don't have Mac clusters
> - and so they couldn't care less about Clang.

Clang is also the new system compiler for FreeBSD.  But there are not
many FreeBSD clusters either.

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/


Re: [OMPI devel] Compile-time MPI_Datatype checking

2012-10-31 Thread Dmitri Gribenko
On Wed, Oct 31, 2012 at 5:04 PM, Jeff Squyres  wrote:
> On Oct 31, 2012, at 9:38 AM, Dmitri Gribenko wrote:
>
>>> The rationale here is that correct MPI applications should not need to add 
>>> any extra compiler files to compile without warnings.
>>
>> I would disagree with this.  Compiler warnings are most useful when
>> they are on by default.  Only a few developers will turn on a warning
>> because warnings are hard to discover and enabling a warning requires
>> an explicit action from the developer.
>
> Understood, but:
>
> a) MPI explicitly allows this kind of deliberate mismatch.  It does not make 
> sense to warn for things that are correct in MPI.

I don't think it is MPI.  It is the C standard that allows one to
store any pointer in void* and char*.  But C standard also considers
lots of other weird things to be valid, see below.

> b) Warnings are significantly less useful if the user looks at them and says, 
> "the compiler is wrong; I know that MPI says that this deliberate mismatch in 
> my code is ok."

Well, one can also argue that since the following is valid C, the
warning in question should not be implemented at all:

long *b = malloc(sizeof(int));
MPI_Recv(b, 1, MPI_INT, ...);

But this code is clearly badly written, so we are left with a question
about where to draw the line.

> c) as such, these warnings are really only useful for the application where 
> type/MPI_Datatype matching is expected/desired.

Compilers already warn about valid C code.  Silencing many warnings
relies on conventions that are derived from best practices of being
explicit about something unusual.  For example:

$ cat /tmp/aaa.c
void foo(void *a) {
  for(int i = a; i < 10; i++)
  {
if(i = 5)
  return;
  }
}
$ clang -fsyntax-only -std=c99 /tmp/aaa.c
/tmp/aaa.c:2:11: warning: incompatible pointer to integer conversion
initializing 'int' with an expression of type 'void *'
[-Wint-conversion]
  for(int i = a; i < 10; i++)
  ^   ~
/tmp/aaa.c:4:10: warning: using the result of an assignment as a
condition without parentheses [-Wparentheses]
if(i = 5)
   ~~^~~
/tmp/aaa.c:4:10: note: place parentheses around the assignment to
silence this warning
if(i = 5)
 ^
   ()
/tmp/aaa.c:4:10: note: use '==' to turn this assignment into an
equality comparison
if(i = 5)
 ^
 ==
2 warnings generated.

According to C standard this is valid C code, but clang emits two
warnings on this.

> Can these warnings be enabled as part of the warnings rollup -Wall option?  
> That would be an easy way to find/enable these warnings.

IIRC, -Wall warning set is frozen in clang.  -Wall is misleading in
that it does not turn on all warnings implemented in the compiler.
Clang has -Weverything to really turn on all warnings.  But
-Weverything is very noisy (by design, not to be fixed) unless one
also turns off all warnings that are not interesting for the project
with -Wno-foo.

I don't think it is possible to disable this warning by default
because off-by-default warnings are discouraged in Clang.  There is no
formal policy, but the rule of thumb is: either make the warning good
enough for everyone or don't implement it; if some particular app does
something strange, it can disable this warning.

>> The pattern you described is an important one, but most MPI
>> applications will have matching buffer types/type tags.
>
> I agree that most applications *probably* don't do this.  But significant 
> developer in this community (i.e., Sandia) has at least multiple applications 
> that *do* do it.  I can't ignore that.  :-(

Here are a few approaches to solving this in order of preference:

0. Is this really a concern for Sandia?  (I.e., do they target Clang?)

1. Ask the developer to silence the warning with a cast to 'void *' or
-Wno-type-safety.  Rationale: compilers already do warn about valid
but suspicious code.

2. Turn off checking for char* just like for void*.  Rationale: C
standard allows char* to alias a pointer of any type.  Note that char*
is special in this regard (strict aliasing rules).

3. Turn off annotations by default in mpi.h.

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/


Re: [OMPI devel] Compile-time MPI_Datatype checking

2012-10-31 Thread Dmitri Gribenko
On Wed, Oct 31, 2012 at 2:36 PM, Jeff Squyres  wrote:
> On Oct 31, 2012, at 3:45 AM, Dmitri Gribenko wrote:
>
>>> With this patch, they'd get warnings about these uses, even though they are 
>>> completely valid according to MPI.
>>>
>>> A suggestion was that this functionality could be disabled by default, and 
>>> enabled with a magic macro.  Perhaps something like:
>>
>> This is a vaild concern and a good idea for a FAQ entry.
>
> Can we flip the orientation such that -Wmpi-type-safety needs to be specified 
> to enable this behavior, rather than -Wno-mpi-type-safety is required to 
> disable it?  (or the equivalent -D's)
>
> The rationale here is that correct MPI applications should not need to add 
> any extra compiler files to compile without warnings.

I would disagree with this.  Compiler warnings are most useful when
they are on by default.  Only a few developers will turn on a warning
because warnings are hard to discover and enabling a warning requires
an explicit action from the developer.

The pattern you described is an important one, but most MPI
applications will have matching buffer types/type tags.  Applications
that use code like that can disable the warning or silence it.

If the pattern you describe is extremely frequent, we could disable
checking for 'char *' buffer type, just like for 'void *'.

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/


Re: [OMPI devel] Compile-time MPI_Datatype checking

2012-10-31 Thread Dmitri Gribenko
On Wed, Oct 31, 2012 at 4:25 AM, Jeff Squyres  wrote:
> On Oct 28, 2012, at 10:28 AM, Dmitri Gribenko wrote:
>
>> Thank you for the feedback!  Hopefully the attached patch fixes both of 
>> these.
>>
>> 1. There are two helper structs with complex numbers.  I predicated
>> the struct declarations and use to appear only in C99.
>>
>> 2. These macros were indeed missing.
>
> I did a few tests and this now looks good; no more warnings.
>
> I brought up this functionality on the weekly OMPI dev telecon today and got 
> an important piece of feedback: apparently there are a large class of apps 
> that wrap their messages as transparent blobs, and then use either 
> non-blob-like or derived MPI datatypes.  (I said something similar to this 
> earlier in the thread, but I didn't know that there was a large class of apps 
> that actually did it)
>
> A very simple example is:
>
> char *foo = malloc(...);
> // ...fill foo...
> MPI_Send(foo, x, MPI_INT, ...);
>
> Another not-uncommon example is:
>
> char *foo = malloc(...);
> // Receive some INTEGERs from a Fortran sender
> MPI_Recv(foo, x, MPI_INTEGER, ...);
>
> With this patch, they'd get warnings about these uses, even though they are 
> completely valid according to MPI.
>
> A suggestion was that this functionality could be disabled by default, and 
> enabled with a magic macro.  Perhaps something like:
>
> mpicc -DOMPI_DDT_CHECKING ...
>
> or something like that.
>
> Thoughts?

This is a vaild concern and a good idea for a FAQ entry.

Q: How to silence warnings about buffer type/type tag mismatch?

A: There are multiple ways for an MPI application to silence these
warnings.  To silence a particular warning, change the type of the
buffer to 'void *' or explicitly cast it to 'void *':

 void *foo = malloc(...);
 // ...fill foo...
 MPI_Send(foo, x, MPI_INT, ...);

or:

 char *foo = malloc(...);
 // ...fill foo...
 MPI_Send((void *) foo, x, MPI_INT, ...);

To turn off all warnings of this kind in clang, use the
-Wno-type-safety command line option.

It is also possible to completely remove annotations from mpi.h by
defining a macro OMPI_NO_ATTR_TYPE_TAGS in the source code or on the
compiler's command line.

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/



Re: [OMPI devel] Compile-time MPI_Datatype checking

2012-10-28 Thread Dmitri Gribenko
On Sun, Oct 28, 2012 at 3:51 PM, Jeff Squyres  wrote:
> On Oct 26, 2012, at 12:48 PM, Dmitri Gribenko wrote:
>
>> Thank you for reviewing and helping me to get this up to the project's
>> standards!
>
> Thank *you*.
>
> My prior reviews were from reading the code only.  I just tried a few compile 
> tests and got the following:
>
> 1. With gcc 4.4.6, I got a bunch of these:
>
> -
> ../../../ompi/include/mpi.h:997: warning: ISO C90 does not support complex 
> types
> ../../../ompi/include/mpi.h:998: warning: ISO C90 does not support complex 
> types
> -
>
> 2. With clang 3.1, I got a bunch of these:
>
> -
> ../../../ompi/include/mpi.h:1084:5: warning: 'OMPI_HAVE_FORTRAN_COMPLEX8' is 
> not defined, evaluates to 0 [-Wundef]
> #if OMPI_HAVE_FORTRAN_COMPLEX8
> ^
> ../../../ompi/include/mpi.h:1089:5: warning: 'OMPI_HAVE_FORTRAN_COMPLEX16' is 
> not defined, evaluates to 0 [-Wundef]
> #if OMPI_HAVE_FORTRAN_COMPLEX16
> ^
> ../../../ompi/include/mpi.h:1094:5: warning: 'OMPI_HAVE_FORTRAN_COMPLEX32' is 
> not defined, evaluates to 0 [-Wundef]
> #if OMPI_HAVE_FORTRAN_COMPLEX32
> ^
> -

Thank you for the feedback!  Hopefully the attached patch fixes both of these.

1. There are two helper structs with complex numbers.  I predicated
the struct declarations and use to appear only in C99.

2. These macros were indeed missing.

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/


ompi-v7.patch
Description: Binary data


Re: [OMPI devel] Compile-time MPI_Datatype checking

2012-10-26 Thread Dmitri Gribenko
On Fri, Oct 26, 2012 at 6:37 PM, Jeff Squyres  wrote:
> This all now looks good to me

Thank you for reviewing and helping me to get this up to the project's
standards!

> For me to commit this, can you do two things:
>
> 1. I hate to do this, but this is more than a "trivial" patch that we could 
> accept without attribution.  Can you do one of two things:
>1a. Sign a contributor agreement (almost identical to the Apache 
> contributor agreement): http://www.open-mpi.org/community/contribute/

I sent it synchronously with v1 of the patch.  I understand that it
was a while ago, but it should be sitting somewhere in your mail.
IIRC, I sent a "corporate" one as an employee of High Performance
Computing Center at NTUU "KPI". If you can not find it -- I can send
another one (but it is a bit complicated to get legal documents like
this signed, so please try to find the one I sent).

>1b. Send a v6 of your patch to this public mailing list with a 
> BSD-compatible license at the top, thereby releasing your patch under that 
> license (which is compatible with OMPI's).

Attached.

> 2. Give me a short description that I can use to put into the README / FAQ / 
> etc. about what this functionality does, what prerequisites are needed (e.g., 
> version of clang, etc.).

Here's what I came up with.  Feel free to edit or ask for clarifications.

=== NEWS entry ===
mpi.h header was annotated to enable compile-time type checking of
buffers and type tags.  These annotations are understood by Clang 3.2
and later versions.  This imposes no requirements on the compiler
which is used to compile OpenMPI itself (for example, one can compile
OpenMPI with GCC and use Clang to compile an MPI application).

=== FAQ entry ===
How to diagnose mismatches between buffer type and type tag?

In OpenMPI communication functions' declarations and predefined type
tags are annotated with pointer_with_type_tag and
type_tag_for_datatype attributes.  These attributes allow
compilers to check that buffer type matches the type tag.

These attributes are understood by Clang 3.2 and later versions.
Please note that this imposes no requirements on the compiler which is
used to compile OpenMPI itself (for example, one can compile OpenMPI
with GCC and use Clang to compile an MPI application).

For example, consider the following code:

double *double_buf = /* ... */;
MPI_Send(double_buf, 1, MPI_INT, /* ... */);

Clang produces a diagnostic:

wrong.c:151:12: warning: argument type 'double *' doesn't match
specified 'mpi' type tag that requires 'int *' [-Wtype-safety]
  MPI_Send(double_buf, 1, MPI_INT);
   ^~ ~~~

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/


ompi-v6.patch
Description: Binary data


Re: [OMPI devel] Compile-time MPI_Datatype checking

2012-10-19 Thread Dmitri Gribenko
On Thu, Oct 18, 2012 at 7:32 PM, Jeff Squyres  wrote:
> On Oct 7, 2012, at 2:25 PM, Dmitri Gribenko wrote:
>> I tried to follow your advice about Fortran datatypes and updated the
>> patch accordingly (attached).  This patch is against OpenMPI 1.9.
>> Please review.
>
> Comments in the attached.  Look for "*** JMS".

Thank you for the review!  Please find attached an updated patch.

*** JMS The thought occurs to me that Fortran's built-in types
CHARACTER, COMPLEX, LOGICAL, INTEGER, REAL, DOUBLE COMPLEX, and DOUBLE
PRECISION will *always* exist if we are including Fortran support.
The other (LOGICAL1, LOGICAL4, ...etc.) are optional/compiler
extensions and may or may not exist.  Regardless, the point is that I
wonder if we don't need these extra #defines (E.g.,
OMPI_HAVE_FORTRAN_CHARACTER) -- we could just use #if
(OMPI_BUILD_FORTRAN_MPIFH_BINDINGS ||
OMPI_BUILD_FORTRAN_USEMPIF08_BINDINGS ||
OMPI_BUILD_FORTRAN_USEMPI_BINDINGS) (or have an abbreviation for
that)...

Thank you for the suggestion, I introduced OMPI_BUILD_FORTRAN_BINDINGS
into mpi.h.in for this.

*** JMS Per my above ramble, CHARACTER and INTEGER (and others) are
built-in to Fortran, so we'll always have these if we're building
Fortran at all.  So how about simplifying these cases a little;
perhaps something like:
*** JMS I think these "2foo" types might be able to be simplified per
my above ramble...?

Simplified thanks to OMPI_BUILD_FORTRAN_BINDINGS.

*** JMS Can we do anything with ## to simplify these macros a bit,
perchance?  (I haven't thought this through)

+OMPI_DECLSPEC extern struct ompi_predefined_datatype_t ompi_mpi_logical1
+#if OMPI_HAVE_FORTRAN_LOGICAL1
+  OMPI_ATTR_TYPE_TAG(ompi_fortran_logical1_t)
+#endif
+  ;

I could not think of anything that could help to simplify that part.
All of these attributes are predicated on different conditions.

 typedef struct {
   ompi_fortran_real_t real;
   ompi_fortran_real_t imag;
-} ompi_fortran_complex_t;
+} ompi_fortran_complex_struct_t;
 #endif

*** JMS I was initially curious as to why you renamed these, but now I
see: it's because we added the same names into mpi.h.in.  Do we really
still need them here in ompi_config.h.in?  I.e., aren't the
definitions the same?

They are different because macro names in mpi.h.in are defined to
expand standard complex types' names (like float _Complex); there is
special syntax to access real and imaginary parts of these.  Types in
ompi_config.h.in are custom structs that are layout-compatible with
standard complex types; real and imaginary parts can be accessed with
usual member access operators.

So, while we could change op_base_functions.c that uses these structs
to use standard syntax (and remove struct declarations), I doubt that
it is desirable. As far as I understand, one should be able to build
OpenMPI with a compiler that does not support standard complex types.

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/


ompi-v5.patch
Description: Binary data


Re: [OMPI devel] Compile-time MPI_Datatype checking

2012-10-18 Thread Dmitri Gribenko
On Thu, Oct 18, 2012 at 7:32 PM, Jeff Squyres  wrote:
> On Oct 7, 2012, at 2:25 PM, Dmitri Gribenko wrote:
> Hmm.  I'm not sure how to do that -- I don't know of any C compiler that has 
> built-in #defines for what Fortran types exist.
>
> I'm open to suggestions, though -- can you suggest an easier solution?

Sorry, that was unclear.  I meant just not annotating Fortran
datatypes -- they will not get any checking, so we will not get false
positives, but such a change is much simpler to review for 1.7.

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/


Re: [OMPI devel] Compile-time MPI_Datatype checking

2012-10-18 Thread Dmitri Gribenko
Dear OpenMPI developers,

Could someone please review the attached patch?

Dmitri

On Sun, Oct 7, 2012 at 9:25 PM, Dmitri Gribenko  wrote:
> On Thu, May 31, 2012 at 2:38 PM, Jeff Squyres  wrote:
>> On May 31, 2012, at 7:29 AM, Jeff Squyres wrote:
>>
>>>>> We should have AC macros for all of these already.
>>>>
>>>> OK, I'll try find them to support (1) usecase described below.
>>>
>>> No, I'll find them -- sorry, I meant to look them up before I sent the last 
>>> mail.  Let me look them up and get back to you.  Our configury is quite 
>>> complicated, and I know the right places to look.  :-)
>>
>> Ok, this might get a little complicated.  You'll probably need to use a pair 
>> of them (this is trunk only; it's different in v1.6 because we wholly 
>> revamped the trunk's Fortran support recently):
>>
>> 1. You can see all the OMPI_HAVE_FORTRAN_'s at the top of mpi.h.in.  
>> These indicate whether the Fortran compiler supports these types or not.
>>
>> 2. We currently define *one* Fortran type in mpi.h.in: 
>> ompi_fortran_integer_t.  It looks like we need to add the rest of them: 
>> ompi_fortran__t (these are all in opal/include/opal_config.h, but 
>> mpi.h is a standalone, user-includeable file, which is why it replicates a 
>> subset of all the configure-generated results).  Here's a first stab at what 
>> I think will be needed in mpi.h.in:
>
> Hello Jeff,
>
> I would like to continue this discussion.  Corresponding changes in
> Clang are already in SVN and the feature should be released with the
> upcoming Clang 3.2.
>
> I tried to follow your advice about Fortran datatypes and updated the
> patch accordingly (attached).  This patch is against OpenMPI 1.9.
> Please review.
>
> Is there any chance we can get a less invasive (header-only, without
> autotools magic for Fortran datatypes support) change in the OpenMPI
> 1.7?
>
>> Does clang link together with gfortran?  I.e., does the following work:
>>
>> ./configure CC=clang CXX=clang++ FC=gfortran ...
>
> Seems like it works.
>
> Dmitri
>
> --
> main(i,j){for(i=2;;i++){for(j=2;j (j){printf("%d\n",i);}}} /*Dmitri Gribenko */



-- 
main(i,j){for(i=2;;i++){for(j=2;j*/


ompi-v4.patch
Description: Binary data


Re: [OMPI devel] Compile-time MPI_Datatype checking

2012-10-07 Thread Dmitri Gribenko
On Thu, May 31, 2012 at 2:38 PM, Jeff Squyres  wrote:
> On May 31, 2012, at 7:29 AM, Jeff Squyres wrote:
>
 We should have AC macros for all of these already.
>>>
>>> OK, I'll try find them to support (1) usecase described below.
>>
>> No, I'll find them -- sorry, I meant to look them up before I sent the last 
>> mail.  Let me look them up and get back to you.  Our configury is quite 
>> complicated, and I know the right places to look.  :-)
>
> Ok, this might get a little complicated.  You'll probably need to use a pair 
> of them (this is trunk only; it's different in v1.6 because we wholly 
> revamped the trunk's Fortran support recently):
>
> 1. You can see all the OMPI_HAVE_FORTRAN_'s at the top of mpi.h.in.  
> These indicate whether the Fortran compiler supports these types or not.
>
> 2. We currently define *one* Fortran type in mpi.h.in: 
> ompi_fortran_integer_t.  It looks like we need to add the rest of them: 
> ompi_fortran__t (these are all in opal/include/opal_config.h, but mpi.h 
> is a standalone, user-includeable file, which is why it replicates a subset 
> of all the configure-generated results).  Here's a first stab at what I think 
> will be needed in mpi.h.in:

Hello Jeff,

I would like to continue this discussion.  Corresponding changes in
Clang are already in SVN and the feature should be released with the
upcoming Clang 3.2.

I tried to follow your advice about Fortran datatypes and updated the
patch accordingly (attached).  This patch is against OpenMPI 1.9.
Please review.

Is there any chance we can get a less invasive (header-only, without
autotools magic for Fortran datatypes support) change in the OpenMPI
1.7?

> Does clang link together with gfortran?  I.e., does the following work:
>
> ./configure CC=clang CXX=clang++ FC=gfortran ...

Seems like it works.

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/


ompi-v4.patch
Description: Binary data


Re: [OMPI devel] 1.6.1rc3 - 3 of 5 tests failed on OSX 10.8

2012-08-20 Thread Dmitri Gribenko
On Mon, Aug 20, 2012 at 12:30 PM, Jeff Squyres  wrote:
> I see a clang 3.1 on http://llvm.org/releases/, but I don't see a 4.0.  Is 
> that a released version?

Apple Clang has a different and unrelated version numbering compared
to open source clang.

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/


Re: [OMPI devel] Compile-time MPI_Datatype checking

2012-05-31 Thread Dmitri Gribenko
On Thu, May 31, 2012 at 2:29 PM, Jeff Squyres  wrote:
> Ah, good point.  But in those cases, MPI specifically defines that those 
> arguments are wholly ignored on non-root processes.  So you could still even 
> pass <&foo, MPI_DATATYPE_NULL>, or , and it would be ok.

I see.  If this warning is imposing some coding style guideline, then
we would better not annotate MPI_DATATYPE_NULL or some users will find
these diagnostics to be too noisy.

>>> We should have AC macros for all of these already.
>>
>> OK, I'll try find them to support (1) usecase described below.
>
> No, I'll find them -- sorry, I meant to look them up before I sent the last 
> mail.  Let me look them up and get back to you.  Our configury is quite 
> complicated, and I know the right places to look.  :-)

Thanks in advance.

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/



Re: [OMPI devel] Compile-time MPI_Datatype checking

2012-05-31 Thread Dmitri Gribenko
On Thu, May 31, 2012 at 2:07 PM, Jeff Squyres  wrote:
> On May 30, 2012, at 7:04 PM, Dmitri Gribenko wrote:
>> +must_be_null specifies that the expression should be a null
>> +pointer constant, for example:
>> +
>> +
>> +
>> +/* In mpi.h */
>> +extern struct mpi_datatype mpi_datatype_null
>> +    __attribute__(( type_tag_for_datatype(mpi, void, must_be_null) ));
>> +
>> +#define MPI_DATATYPE_NULL ((MPI_Datatype) &mpi_datatype_null)
>> +
>> +/* In user code */
>> +MPI_Send(buffer, 1, MPI_DATATYPE_NULL /*, ... */); // warning:
>> MPI_DATATYPE_NULL
>> +                                                   // was specified but 
>> buffer
>> +                                                   // is not a null pointer
>
> I'm not sure that this is a warning we want to set.
>
> MPI__NULL constants are defined as "invalid handles" -- in most cases, 
> it's an error to pass them to an MPI function (e.g., the MPI_Send example, 
> above, would generate an MPI exception).  They're usually used for comparison.
>
> I can't think of a case offhand where it's acceptable to pass 
> MPI_DATATYPE_NULL to an MPI function.  I could be missing something, 
> though...  (actually, I guess I can think of some cases -- we have some 
> regression test programs that specifically pass MPI_DATATYPE_NULL, just to 
> ensure that it properly invokes an MPI exception)

I usually pass  as arguments which are
"significant only at root" and that function call is not in the root
process code path.

> But even so, if one passes MPI_DATATYPE_NULL, the buffer and count arguments 
> will be ignored.  So I don't think that we want to require that 
> MPI_DATATYPE_NULL *requires* a NULL pointer.

It just doesn't make sense to pass non-null pointer and
MPI_DATATYPE_NULL (because null pointer will be ignored).  Hence the
warning.

>> *** JMS I'm a bit confused as to why 2int got a tag, but the others
>>    did not.  We do have C equivalents for all types -- do you need a
>>    listing of the configure results where we figured out all the C
>>    equivalents?  (i.e., I can look them up for you -- our configury
>>    is a bit... deep :-) )
>>
>> I did not annotate them because those are Fortran types.  If there are
>> some known C equivalents in OpenMPI, I will happily add them.  But
>> please note that if these equivalents are discovered during configure,
>> we can not hardcode them into mpi.h.in.  Some autoconf macros will be
>> needed probably.
>
> We should have AC macros for all of these already.

OK, I'll try find them to support (1) usecase described below.

> The point of this is because all MPI datatypes are available in all 
> languages, meaning that I could MPI_Send an MPI_INTEGER from fortran and 
> receive it in C code (that MPI_Recv's an MPI_INTEGER).  FWIW, I've seen apps 
> do this in two general cases:
>
> 1. Fortran sends to C, C just holds the blob without looking at/understanding 
> the value, C eventually sends the blob back to Fortran.
>
> 2. Fortran sends to C, and C interprets the value because it knows the 
> interoperable type (e.g., sends MPI_INTEGER, receives into a C int).
>
> If the app doesn't know the exact equivalent C type corresponding to the 
> Fortran type (e.g., case 1), they may need one of the examples I provided 
> above (e.g., cast the buffer to (void*)).

Agreed.

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/



Re: [OMPI devel] Compile-time MPI_Datatype checking

2012-05-30 Thread Dmitri Gribenko
Hi Jeff,

On Thu, May 31, 2012 at 12:57 AM, Jeff Squyres  wrote:
> I've reviewed the patch.  Good stuff!

Thank you very much for the review.  Answers to comments below.
Updated patch attached.

*** JMS What do the 3-argument forms of type_tag_for_datatype() do?
They aren't described in

http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20120521/057992.html

My bad: that page has documentation for the previous version of clang
patch.  Below is a relevant excerpt of current docs.  Basically, the
third argument states additional instructions on how to compare types.
 Please excuse me for a long prose copypaste.

+type_tag_for_datatype(...)
+
+Clang supports annotating type tags of two forms.
+
+
+Type tag that is an expression containing a reference to some declared
+identifier. Use __attribute__((type_tag_for_datatype(kind, type)))
+on a declaration with that identifier:
+
+
+
+extern struct mpi_datatype mpi_datatype_int
+__attribute__(( type_tag_for_datatype(mpi,int) ));
+#define MPI_INT ((MPI_Datatype) &mpi_datatype_int)
+
+
+
+Type tag that is an integral literal.  Introduce a static
+const variable with a corresponding initializer value and attach
+__attribute__((type_tag_for_datatype(kind, type))) on that
+declaration, for example:
+
+
+
+#define MPI_INT ((MPI_Datatype) 42)
+static const MPI_Datatype mpi_datatype_int
+__attribute__(( type_tag_for_datatype(mpi,int) )) = 42
+
+
+
+
+The attribute also accepts an optional third argument that determines how
+the expression is compared to the type tag.  There are two supported flags:
+
+layout_compatible will cause types to be compared according to
+layout-compatibility rules (C++11 [class.mem] p 17, 18).  This is
+implemented to support annotating types like MPI_DOUBLE_INT.
+
+For example:
+
+
+/* In mpi.h */
+struct internal_mpi_double_int { double d; int i; };
+extern struct mpi_datatype mpi_datatype_double_int
+__attribute__(( type_tag_for_datatype(mpi, struct internal_mpi_double_int,
+  layout_compatible) ));
+
+#define MPI_DOUBLE_INT ((MPI_Datatype) &mpi_datatype_double_int)
+
+/* In user code */
+struct my_pair { double a; int b; };
+struct my_pair *buffer;
+MPI_Send(buffer, 1, MPI_DOUBLE_INT /*, ... */); // no warning
+
+struct my_int_pair { int a; int b; }
+struct my_int_pair *buffer2;
+MPI_Send(buffer2, 1, MPI_DOUBLE_INT /*, ... */); // warning: actual
buffer element
+ // type 'struct my_int_pair'
+ // doesn't match
specified MPI_Datatype
+
+
+
+
+must_be_null specifies that the expression should be a null
+pointer constant, for example:
+
+
+
+/* In mpi.h */
+extern struct mpi_datatype mpi_datatype_null
+__attribute__(( type_tag_for_datatype(mpi, void, must_be_null) ));
+
+#define MPI_DATATYPE_NULL ((MPI_Datatype) &mpi_datatype_null)
+
+/* In user code */
+MPI_Send(buffer, 1, MPI_DATATYPE_NULL /*, ... */); // warning:
MPI_DATATYPE_NULL
+   // was specified but buffer
+   // is not a null pointer
+
+
+
+

*** JMS What happens if the argument type is (void*)?  Does that match
the tag type?  E.g.:

char recvbuf;
void *foo = &recvbuf;
MPI_Send(foo, 1, MPI_CHAR, ...)

If buffer has type void* then no warning is emitted.

*** JMS What if I deliberately want to receive into the wrong type
(and for some reason, I know it's ok)?  Is casting ok enough to
defeat the tag testing?  E.g.:

char recvbuf[100];
MPI_Recv((int*) recvbuf, 1, MPI_INT, ...);

Explicit casts are always honored. In this case buffer type is thought
to be int*, hence no warning is emitted.

*** JMS Random suggestion: how about CXX instead of CPP?  CPP could
also mean C preprocessor.

Right, CXX is better.  Done.

*** JMS Why doesn't ompi_mpi_byte have an attribute?  And what does it
mean to not have an attribute?  Will the type checking be silently
ignored at compile-time if there's no matching (MPI,foo)
type_tag_for_datatype?  (same question applies to other instances
with no tags, like ompi_mpi_packed, but I won't bother repeating
it everywhere)

ompi_mpi_byte doesn't have an attr because I annotated only those
types that have C or C++ equivalents stated in MPI 2.2 specification.
If OpenMPI provides additional guarantees, for example that Fortran
types always have some known equivalent C types, we can annotate them,
too.

If a function is passed a type tag (that is, MPI_Datatype) that does
not have an annotation, then that function call is not checked.  That
is, in MPI_Send(buf, 1, MPI_BYTE, ...) buf can be of any pointer type.

*** JMS I'm a bit confused as to why 2int got a tag, but the others
did not.  We do have C equivalents for all types -- do you need a
listing of the configure results where we figured out all the C
equivalents?  (i.e., I can look them 

[OMPI devel] Compile-time MPI_Datatype checking

2012-05-29 Thread Dmitri Gribenko
Hello,

I've implemented a patch for clang that enables compile-time checking
of  arguments to functions.  When applied to MPI,
this enables the compiler to check that buffer type and MPI_Datatype
match.

Latest version of clang patch can be found here. [1]  Please note that
clang patch was not yet accepted to clang trunk.

On the OpenMPI side we need to:
* add attributes to MPI functions to mark them as accepting pointers
with type tags;
* add attributes to ompi_mpi_* declarations to mark them as type tags.

All in all, the changes boil down to:
1. Annotate type tags:
OMPI_DECLSPEC extern struct ompi_predefined_datatype_t
ompi_mpi_float OMPI_ATTR_TYPE_TAG(float);

2. Annotate functions:
 OMPI_DECLSPEC  int MPI_Send(void *buf, int count, MPI_Datatype
datatype, int dest,
int tag, MPI_Comm comm)
OMPI_ATTR_POINTER_WITH_TYPE_TAG(1,3);


where OMPI_ATTR* are macros that are defined to attributes when
compiling under clang with this feature.

Attached is the OpenMPI patch I've arrived to.  Although I tried to be
attentive, changes to mpi.h are very repetitive and error-prone, so
please review them closely.

I've implemented a similar patch for MPICH2. [2]  OpenMPI developers
might want to follow that discussion, too.

Dmitri

[1] 
http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20120521/058137.html
[2] http://lists.mcs.anl.gov/pipermail/mpich2-dev/2012-May/000938.html

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/


ompi-v2.patch
Description: Binary data


Re: [OMPI devel] Building otfcompress with binutils-gold fails

2012-02-18 Thread Dmitri Gribenko
On Sat, Feb 18, 2012 at 11:59 AM, Matthias Jurenz
 wrote:
> Does that mean that the workaround I suggested (extra linkage of -lz) is not
> needed anymore?

It is needed for SVN checkout until it is migrated to the same libtool
as 1.7 tarball.

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/


Re: [OMPI devel] Building otfcompress with binutils-gold fails

2012-02-17 Thread Dmitri Gribenko
Hi Jeff,

On Fri, Feb 17, 2012 at 1:14 PM, Jeff Squyres  wrote:
> Are you compiling a nightly 1.7/trunk tarball, or an SVN checkout where you 
> ran autogen.pl?

I was compiling an SVN checkout.

> The 1.7 nightly tarballs are built with the latest Libtool (2.4.2).  If that 
> isn't doing the Right Things, that would be a bit surprising.

It really does.  1.7 nightly tarball compiles and links without any
problems.  Thank you for pointing that out.

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/



Re: [OMPI devel] Building otfcompress with binutils-gold fails

2012-02-15 Thread Dmitri Gribenko
On Wed, Feb 15, 2012 at 8:08 PM, Jeff Squyres  wrote:
> What is "gold"?  Can you send all the information listed here (e.g., I don't 
> know what version of Open MPI you're reporting about):

gold is the new binutils linker.

OS: Debian sid amd64 with g++-4.6, binutils-gold packages installed
OpenMPI: latest svn trunk

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/


ompi-output.tar.bz2
Description: BZip2 compressed data


[OMPI devel] Building otfcompress with binutils-gold fails

2012-02-15 Thread Dmitri Gribenko
Hi,

I've found that otfcompress doesn't build with gold:

make[3]: Entering directory
`/home/storage_3/grib/ompi-build/ompi/contrib/vt/vt/extlib/otf/tools/otfcompress'
/bin/sh ../../libtool  --tag=CC   --mode=link clang  -O3 -DNDEBUG
-finline-functions -fno-strict-aliasing -pthread
-I/home/storage_3/grib/ompi-trunk/opal/mca/hwloc/hwloc132/hwloc/include
 -L/home/storage_3/grib/ompi-build/ompi/contrib/vt/vt/../../../.libs
-o otfcompress otfcompress.o ../../otflib/libotf.la
libtool: link: clang -O3 -DNDEBUG -finline-functions
-fno-strict-aliasing -pthread
-I/home/storage_3/grib/ompi-trunk/opal/mca/hwloc/hwloc132/hwloc/include
-o .libs/otfcompress otfcompress.o
-L/home/storage_3/grib/ompi-build/ompi/contrib/vt/vt/../../../.libs
../../otflib/.libs/libotf.so -pthread -Wl,-rpath
-Wl,/home/storage_3/grib/ompi-install/lib
otfcompress.o:../../../../../../../../../ompi-trunk/ompi/contrib/vt/vt/extlib/otf/tools/otfcompress/otfcompress.c:function
decompressFile: error: undefined reference to 'inflateInit_'
otfcompress.o:../../../../../../../../../ompi-trunk/ompi/contrib/vt/vt/extlib/otf/tools/otfcompress/otfcompress.c:function
decompressFile: error: undefined reference to 'inflate'
otfcompress.o:../../../../../../../../../ompi-trunk/ompi/contrib/vt/vt/extlib/otf/tools/otfcompress/otfcompress.c:function
decompressFile: error: undefined reference to 'inflateEnd'
otfcompress.o:../../../../../../../../../ompi-trunk/ompi/contrib/vt/vt/extlib/otf/tools/otfcompress/otfcompress.c:function
compressFile: error: undefined reference to 'deflateInit_'
otfcompress.o:../../../../../../../../../ompi-trunk/ompi/contrib/vt/vt/extlib/otf/tools/otfcompress/otfcompress.c:function
compressFile: error: undefined reference to 'deflate'
otfcompress.o:../../../../../../../../../ompi-trunk/ompi/contrib/vt/vt/extlib/otf/tools/otfcompress/otfcompress.c:function
compressFile: error: undefined reference to 'deflateEnd'
clang: error: linker command failed with exit code 1 (use -v to see invocation)

The problem is that --no-add-needed is default gold behavior.  All
libraries have to be specified explicitly to the linker.  In this case
-lz is missing.

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/


Re: [OMPI devel] VT build failure with Clang++

2012-02-15 Thread Dmitri Gribenko
On Wed, Feb 15, 2012 at 10:56 AM, Paul Hargrove  wrote:
> I strongly suspect that this is a Clang++ bug.

I don't know if it is a Clang bug, but here's my understanding of the problem.

TokenFactoryScopeC::create() boils down to this:

template 
uint32_t
TokenFactoryScopeC::create( const void * localDef, uint32_t globalToken )
{
   const T & local_def = *static_cast(localDef);
   typename std::set::const_iterator it = m_globDefs->find(
local_def ); // need operator< for T
   return 0;
}

When clang tries to instantiate
TokenFactoryScopeC::getPackSize(), it tries to
instantiate create() too.  It fails to do so because there is no
operator< defined for DefRec_BaseS.

Anyway, this looks like a bug in VT code, here's why.  DefRec_* have
DefRec_*::SortS defined, but it is not used in
TokenFactoryScopeC::m_globDefs.  Probably it should be, something
like this:

std::set * m_globDefs;

I'm not sure if this is a bug in Clang because I don't know if Clang
should have tried to instantiate create().

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/


Re: [OMPI devel] Compile-time MPI_Datatype checking

2012-02-02 Thread Dmitri Gribenko
On Thu, Feb 2, 2012 at 7:31 AM, Christopher Samuel
 wrote:
> On 29/01/12 10:07, Dmitri Gribenko wrote:
>> My colleague and I want to implement a compile-time check (warning)
>> for clang compiler that specified buffer type matches passed
>> MPI_Datatype.
>
> Interesting, is it possible to do the same for GCC with its plugin
> architecture ?

I haven't worked with GCC plugins, but after looking at the
documentation I think it is possible.

Dmitri Gribenko

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/


Re: [OMPI devel] Compile-time MPI_Datatype checking

2012-02-01 Thread Dmitri Gribenko
On Wed, Feb 1, 2012 at 10:01 PM, Jeff Squyres  wrote:
> On Jan 28, 2012, at 6:07 PM, Dmitri Gribenko wrote:
>> Here's what is required on the OpenMPI part: all MPI functions that
>> accept a buffer and MPI_Datatype should be annotated with
>> mpi_typed_arg(buffer-arg-index, type-arg-index) attribute.  For
>> example:
>>
>> int MPI_Send(void *buf, ...etc...) __attribute__(( mpi_typed_arg(1,3) ));
>
> This isn't terrible; we do similar things throughout the code base.
>
> Will you be able to do this for MPI functions that take 2 choice buffers?  
> E.g., MPI_Gather.

Yes, just put two attributes: __attribute__(( mpi_typed_arg(1,3),
mpi_typed_arg(4,6) ))

>> Predefined datatypes should be annotated with corresponding C types:
>> mpi_datatype(var-decl-with-corresponding-type):
>>
>> extern int ompi_mpi_int_dummy;
>> extern struct ompi_predefined_datatype_t ompi_mpi_int
>>    __attribute__(( mpi_datatype(ompi_mpi_int_dummy) ));
>
> Just to make sure I understand this syntax: is ompi_mpi_int_dummy just to 
> indicate the *type* that is congruent with the ompi_predefined_datatype_t?

Yes, I did it that way because parsing a type-name as attribute
argument is not possible currently in clang, but I wanted to check how
it will work as quickly as possible.  But I'm now hacking the parser
to support the cleaner syntax.

> I.e., I'd do something like this for MPI_DOUBLE:
>
> extern double ompi_mpi_double_dummy;
> extern struct ompi_predefined_datatype_t ompi_mpi_double
>   __attribute__(( mpi_datatype(ompi_mpi_double_dummy) ));
>
> ?

Right, but I'm going for the cleaner syntax anyway.

>> Users can annotate their types too:
>> int my_int_dummy;
>> MPI_Datatype my_int_type __attribute__(( mpi_datatype(my_int_dummy) ));
>
> More importantly, can they do:
>
> struct my_struct_t my_struct_dummy;
> MPI_Datatype my_struct_type __attribute__(( mpi_datatype(my_struct_dummy) ));
>
> ?

Yes, specifying any type is possible.

>> I would like to hear if there is any issue with this idea or
>> implementation design that could prevent the corresponding patch for
>> mpi.h to be accepted.
>
> I'm supportive of this (mpi.h.in, you mean :-) ).
>
> Depending on the patch, however, we might need a signed Open MPI contribution 
> agreement from you/your employer

That's fine for us.

Dmitri Gribenko

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/



[OMPI devel] Compile-time MPI_Datatype checking

2012-01-28 Thread Dmitri Gribenko
Hello,

My colleague and I want to implement a compile-time check (warning)
for clang compiler that specified buffer type matches passed
MPI_Datatype.

For example:

MPI_Send(long_buf, 1, MPI_INT, 1, 1, MPI_COMM_WORLD); //
expected-warning {{actual buffer element type 'long' doesn't match
specified MPI_Datatype}}

We've hacked a prototype patch for clang, but first we wanted to
discuss this idea with OpenMPI developers.

Here's what is required on the OpenMPI part: all MPI functions that
accept a buffer and MPI_Datatype should be annotated with
mpi_typed_arg(buffer-arg-index, type-arg-index) attribute.  For
example:

int MPI_Send(void *buf, ...etc...) __attribute__(( mpi_typed_arg(1,3) ));

Predefined datatypes should be annotated with corresponding C types:
mpi_datatype(var-decl-with-corresponding-type):

extern int ompi_mpi_int_dummy;
extern struct ompi_predefined_datatype_t ompi_mpi_int
__attribute__(( mpi_datatype(ompi_mpi_int_dummy) ));

Users can annotate their types too:
int my_int_dummy;
MPI_Datatype my_int_type __attribute__(( mpi_datatype(my_int_dummy) ));

This design is not final, I'd really like to have mpi_datatype(type)
(e.g., mpi_datatype(long long)) but clang doesn't currently have
support for parsing that.  (But I think that clang developers won't
accept the patch unless we implement that).  So type annotations will
probably look like:

extern struct ompi_predefined_datatype_t ompi_mpi_int
__attribute__(( mpi_datatype(int) ));

The attributes can be hidden with macros from compilers that don't
support them, so compatibility should not be a problem.

See attached test and clang output to see what will it look like.

Any comments are welcome.  Please also tell me if you have any ideas
for additional compile-time checks.

I would like to hear if there is any issue with this idea or
implementation design that could prevent the corresponding patch for
mpi.h to be accepted.

Dmitri Gribenko

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/
../../../llvm/tools/clang/test/Sema/warn-mpi-type-mismatch.c:10:21: error: 
attribute requires exactly 2 arguments
__attribute__(( mpi_typed_arg )); // expected-error {{attribute requires 
exactly 2 arguments}}
^
../../../llvm/tools/clang/test/Sema/warn-mpi-type-mismatch.c:13:21: error: 
'mpi_typed_arg' attribute parameter 1 is out of bounds
__attribute__(( mpi_typed_arg(0,7) )); // expected-error {{attribute 
parameter 1 is out of bounds}}
^ ~
../../../llvm/tools/clang/test/Sema/warn-mpi-type-mismatch.c:16:21: error: 
'mpi_typed_arg' attribute parameter 1 is out of bounds
__attribute__(( mpi_typed_arg(5,7) )); // expected-error {{attribute 
parameter 1 is out of bounds}}
^ ~
../../../llvm/tools/clang/test/Sema/warn-mpi-type-mismatch.c:19:21: error: 
'mpi_typed_arg' attribute parameter 2 is out of bounds
__attribute__(( mpi_typed_arg(3,0) )); // expected-error {{attribute 
parameter 2 is out of bounds}}
^   ~
../../../llvm/tools/clang/test/Sema/warn-mpi-type-mismatch.c:22:21: error: 
'mpi_typed_arg' attribute parameter 2 is out of bounds
__attribute__(( mpi_typed_arg(3,5) )); // expected-error {{attribute 
parameter 2 is out of bounds}}
^   ~
../../../llvm/tools/clang/test/Sema/warn-mpi-type-mismatch.c:27:21: error: 
'mpi_typed_arg' attribute requires parameter 1 to be an integer constant
__attribute__(( mpi_typed_arg((x+1),7) )); // expected-error {{attribute 
requires parameter 1 to be an integer constant}}
^ ~
../../../llvm/tools/clang/test/Sema/warn-mpi-type-mismatch.c:30:21: error: 
'mpi_typed_arg' attribute requires parameter 2 to be an integer constant
__attribute__(( mpi_typed_arg(3,x) )); // expected-error {{attribute 
requires parameter 2 to be an integer constant}}
^   ~
../../../llvm/tools/clang/test/Sema/warn-mpi-type-mismatch.c:32:34: error: 
'mpi_typed_arg' attribute only applies to functions and methods
int MPI_Wrong8() __attribute__(( mpi_typed_arg(1,3) )); // expected-error 
{{attribute only applies to functions and methods}}
 ^
MPITypedArgAttr1 3
../../../llvm/tools/clang/test/Sema/warn-mpi-type-mismatch.c:45:74: error: 
attribute takes one argument
extern struct ompi_predefined_datatype_t ompi_mpi_wrong1 __attribute__(( 
mpi_datatype )); // expected-error {{attribute takes one argument}}
 ^
../../../llvm/tools/clang/test/Sema/warn-mpi-type-mismatch.c:46:74: error: 
attribute takes one argument
extern struct ompi_predefined_datatype_t ompi_mpi_wrong2 __attribute__(( 
mpi_datatype(1,2) )); // expected-error {{attribute takes one argument}}