Re: [OMPI devel] RFC: convert send to ssend
On Aug 24, 2009, at 9:24 PM, Patrick Geoffray wrote: Ralph Castain wrote: Not quite that simple, Patrick. Think of things like MPI_Sendrecv, where the "send" call is below that of the user's code. You have a point, Ralph. Although, that would be 8 more lines to add to the user MPI code to define a MPI_Sendrecv macro :-) Seriously, this particular proposal is not the most flaming example of OpenMPI doing too much or going too far. Agreed I personally thought that the discussion about affinity was much more revealing in itself, like the part about in effect replacing the OS scheduler. Ummm...though this is a separate discussion, please note that we never said we would replace the OS scheduler. All we said was that OMPI didn't currently detect and/or obey any OS scheduler binding directives when binding processes. This has now been corrected in the devel trunk, though it remains to be fully tested. The issue of what to do in the absence of any OS scheduler binding directives is a topic for a separate discussion - which I believe we already have partially had. :-) Patrick ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [OMPI devel] RFC: convert send to ssend
Ralph Castain wrote: Not quite that simple, Patrick. Think of things like MPI_Sendrecv, where the "send" call is below that of the user's code. You have a point, Ralph. Although, that would be 8 more lines to add to the user MPI code to define a MPI_Sendrecv macro :-) Seriously, this particular proposal is not the most flaming example of OpenMPI doing too much or going too far. I personally thought that the discussion about affinity was much more revealing in itself, like the part about in effect replacing the OS scheduler. Patrick
Re: [OMPI devel] RFC: convert send to ssend
- "George Bosilca" wrote: > Do people know that there exist tools for checking MPI code > correctness? Many, many tools and most of them are freely > available. Yes, but have yet to be able to persuade any of our users to use them (and have no control over them). :-( -- Christopher Samuel - (03) 9925 4751 - Systems Manager The Victorian Partnership for Advanced Computing P.O. Box 201, Carlton South, VIC 3053, Australia VPAC is a not-for-profit Registered Research Agency
Re: [OMPI devel] RFC: convert send to ssend
On Aug 24, 2009, at 5:33 PM, Patrick Geoffray wrote: George Bosilca wrote: I know the approach "because we can". We develop an MPI library, and we should keep it that way. Our main focus should not diverge to provide I would join George in the minority on this one. "Because we can" is a slippery slope, there is value in keeping things simple, having less knobs and bells and whistles. On this particular whistle, the user could add one line to his MPI code to define send to ssend and be done with it. If he does not have the code in the first place, there is nothing he can't do about it anyway. So, it's just a matter of convenience for a lazy user. Not quite that simple, Patrick. Think of things like MPI_Sendrecv, where the "send" call is below that of the user's code. Frankly, I'm surprised at the fuss this has kicked up. It is a barely a handful of lines of code, totally protected by a configure switch. If we spent this much effort arguing over every such small thing, nearly every configure option that currently exists would never have made it. Patrick ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [OMPI devel] RFC: convert send to ssend
George Bosilca wrote: I know the approach "because we can". We develop an MPI library, and we should keep it that way. Our main focus should not diverge to provide I would join George in the minority on this one. "Because we can" is a slippery slope, there is value in keeping things simple, having less knobs and bells and whistles. On this particular whistle, the user could add one line to his MPI code to define send to ssend and be done with it. If he does not have the code in the first place, there is nothing he can't do about it anyway. So, it's just a matter of convenience for a lazy user. Patrick
Re: [OMPI devel] RFC: convert send to ssend
On Aug 24, 2009, at 1:35 PM, Ashley Pittman wrote: > The point of b is for sysadmins (or individual developers) who want to > force there to *always* be correct MPI applications. But couldn't the sysadmin equally well write a config file to achieve the same effect should they want to? Yes, probably so. I was only pulling from the precedent of the MPI param checking for the tri-state. Having it enabled (and on) in the standard "debug" build is going to change the behaviour of applications with using a debug library, may well render bugs un-reproducible in debug mode or worse you may end up with end-user applications that only run in debug mode and not with a normal build. Agreed. Just to be totally clear: I, too, would not advocate that this feature be automatically enabled in debug builds. It should default to compiled-out in all cases -- only enabled via a configure switch. -- Jeff Squyres jsquy...@cisco.com
Re: [OMPI devel] RFC: convert send to ssend
We spent more time over emails on this thread than the time required to write the patch. As apparently I'm the only one concerned about what we have in our code base or the only one that do not perceive the usefulness of such a feature, I belong to an ignorable minority. As long as you don't make it compile by default, please ignore my rambling. On Aug 24, 2009, at 15:30 , Jeff Squyres wrote: On Aug 24, 2009, at 2:26 PM, George Bosilca wrote: > My point is that this is a fairly trivial-to-implement feature. It > can even be done in a way that doesn't impact performance at all > (default to compile out). It is more trivial than this: mpirun -np 2 --mca btl_tcp_rndv_eager_limit 0 --mca btl_tcp_eager_limit 72 --bynode ./ NPmpi ? You would have a user do that rather than mpirun --mca mpi_force_ssend 1 -np 2 --bynode ./NPmpi ? I'm an OMPI implementor and I'll never remember that the TCP BTL requires *2* eager limits, much less what their values are. :-) My version is for free, it doesn't require _any_ modification in the source code and no long term maintenance. In fact, what I'm trying to say it's that we already have such a feature, except it doesn't have a cool name (mpi_force_ssend) and it is slightly PML dependent. george.
Re: [OMPI devel] RFC: convert send to ssend
On Aug 24, 2009, at 2:26 PM, George Bosilca wrote: > My point is that this is a fairly trivial-to-implement feature. It > can even be done in a way that doesn't impact performance at all > (default to compile out). It is more trivial than this: mpirun -np 2 --mca btl_tcp_rndv_eager_limit 0 --mca btl_tcp_eager_limit 72 --bynode ./ NPmpi ? You would have a user do that rather than mpirun --mca mpi_force_ssend 1 -np 2 --bynode ./NPmpi ? I'm an OMPI implementor and I'll never remember that the TCP BTL requires *2* eager limits, much less what their values are. :-) -- Jeff Squyres jsquy...@cisco.com
Re: [OMPI devel] RFC: convert send to ssend
On Aug 24, 2009, at 13:25 , Jeff Squyres wrote: On Aug 24, 2009, at 11:35 AM, George Bosilca wrote: As a side note, a very similar effect can be obtained by decreasing the eager size of the BTLs to be equal to the size of the match header, which is about 24 bytes. I disagree with this statement. ;-) We currently don't export the BTL or PML header size, so you can't possibly know what value to set the eager limit to. And even if we did, as the conversation between you, me, and Brian from the last Chicago Forum meeting proved, the exact definition of "eager_limit" is a fairly nebulous thing. It's enough to ask somebody who knows. While I agree that we don't have a simple definition of the eager_limit, for this particular topic it's enough to set it as low as possible. My point is that this is a fairly trivial-to-implement feature. It can even be done in a way that doesn't impact performance at all (default to compile out). It is more trivial than this: mpirun -np 2 --mca btl_tcp_rndv_eager_limit 0 --mca btl_tcp_eager_limit 72 --bynode ./ NPmpi ? We all know that there are many MPI correctness tools that are available, but it can be difficult to get users to actually use them. If they can flip a switch a mpirun time to turn on some semantic checking, that's a Good Thing. I know the approach "because we can". We develop an MPI library, and we should keep it that way. Our main focus should not diverge to provide features for a hand of users, features that will barely be maintained and that might hit us back in the future. There are way to many critical features that we need now, to focus on something as trivial as transforming sends in ssends. Anyway, we are a community based project and the vote of the community will decide the fate of this RFC. george.
Re: [OMPI devel] RFC: convert send to ssend
Hi Ashley,
My understanding is that this behavior would not be enabled by default
in the standard debug build. The "always convert to synchronous sends"
mode would be an additional configure-time option.
Samuel K. Gutierrez
Ashley Pittman wrote:
On Mon, 2009-08-24 at 13:27 -0400, Jeff Squyres wrote:
It's the difference between:
a. if (0) { ... convert ... } Modern compilers will remove this code
as part of dead-code removal.
b. if (1) { ... convert ... } Modern compilers will remove the "if
(1)" and always execute the code.
c. if (some_variable) { ... convert ...} An MCA parameter can load
some_variable with 0 or 1.
The point of b is for sysadmins (or individual developers) who want to
force there to *always* be correct MPI applications.
But couldn't the sysadmin equally well write a config file to achieve
the same effect should they want to?
Having it enabled (and on) in the standard "debug" build is going to
change the behaviour of applications with using a debug library, may
well render bugs un-reproducible in debug mode or worse you may end up
with end-user applications that only run in debug mode and not with a
normal build.
I'm all for having as much error checking enabled in debug builds as
possible but to change the behaviour risks masking problems elsewhere
IMHO.
Ashley,
Re: [OMPI devel] RFC: convert send to ssend
On Mon, 2009-08-24 at 13:27 -0400, Jeff Squyres wrote:
> It's the difference between:
>
> a. if (0) { ... convert ... } Modern compilers will remove this code
> as part of dead-code removal.
> b. if (1) { ... convert ... } Modern compilers will remove the "if
> (1)" and always execute the code.
> c. if (some_variable) { ... convert ...} An MCA parameter can load
> some_variable with 0 or 1.
>
> The point of b is for sysadmins (or individual developers) who want to
> force there to *always* be correct MPI applications.
But couldn't the sysadmin equally well write a config file to achieve
the same effect should they want to?
Having it enabled (and on) in the standard "debug" build is going to
change the behaviour of applications with using a debug library, may
well render bugs un-reproducible in debug mode or worse you may end up
with end-user applications that only run in debug mode and not with a
normal build.
I'm all for having as much error checking enabled in debug builds as
possible but to change the behaviour risks masking problems elsewhere
IMHO.
Ashley,
--
Ashley Pittman, Bath, UK.
Padb - A parallel job inspection tool for cluster computing
http://padb.pittman.org.uk
Re: [OMPI devel] RFC: convert send to ssend
On Aug 24, 2009, at 12:14 PM, Ashley Pittman wrote:
> - compiled out
> - compiled in, always convert standard send to sync send
> - compiled in, use MCA parameter to determine whether to convert
> standard -> sync
>
> And we can leave the default as "compiled out".
>
> Howzat?
I don't understand, what the purpose of the middle state? It seems
like
a bad idea to me.
It's the difference between:
a. if (0) { ... convert ... } Modern compilers will remove this code
as part of dead-code removal.
b. if (1) { ... convert ... } Modern compilers will remove the "if
(1)" and always execute the code.
c. if (some_variable) { ... convert ...} An MCA parameter can load
some_variable with 0 or 1.
The point of b is for sysadmins (or individual developers) who want to
force there to *always* be correct MPI applications.
--
Jeff Squyres
jsquy...@cisco.com
Re: [OMPI devel] RFC: convert send to ssend
On Aug 24, 2009, at 11:35 AM, George Bosilca wrote: As a side note, a very similar effect can be obtained by decreasing the eager size of the BTLs to be equal to the size of the match header, which is about 24 bytes. I disagree with this statement. ;-) We currently don't export the BTL or PML header size, so you can't possibly know what value to set the eager limit to. And even if we did, as the conversation between you, me, and Brian from the last Chicago Forum meeting proved, the exact definition of "eager_limit" is a fairly nebulous thing. My point is that this is a fairly trivial-to-implement feature. It can even be done in a way that doesn't impact performance at all (default to compile out). We all know that there are many MPI correctness tools that are available, but it can be difficult to get users to actually use them. If they can flip a switch a mpirun time to turn on some semantic checking, that's a Good Thing. -- Jeff Squyres jsquy...@cisco.com
Re: [OMPI devel] RFC: convert send to ssend
On Mon, 2009-08-24 at 10:52 -0400, Jeff Squyres wrote: > Adapting that to this RFC, perhaps something like this: > > - compiled out > - compiled in, always convert standard send to sync send > - compiled in, use MCA parameter to determine whether to convert > standard -> sync > > And we can leave the default as "compiled out". > > Howzat? I don't understand, what the purpose of the middle state? It seems like a bad idea to me. Ashley, -- Ashley Pittman, Bath, UK. Padb - A parallel job inspection tool for cluster computing http://padb.pittman.org.uk
Re: [OMPI devel] RFC: convert send to ssend
For the record, I see an big interest in this. Sometimes, you have to answer calls for tender featuring applications that must work with no code change, even if the code is completely not MPI-compliant. That's sad, but true (no pun intended :-)) Sylvain On Mon, 24 Aug 2009, George Bosilca wrote: Do people know that there exist tools for checking MPI code correctness? Many, many tools and most of them are freely available. Personally I don't see any interest of doing this, absolutely no interest. There is basically no added value to our MPI, except for a very limited number of users, and these users if they manage to write a parallel application that need this checking I'm sure they will greatly benefit from a real tool to help them correct their MPI code. As a side note, a very similar effect can be obtained by decreasing the eager size of the BTLs to be equal to the size of the match header, which is about 24 bytes. george. On Aug 24, 2009, at 11:11 , Samuel K. Gutierrez wrote: Hi Jeff, Sounds good to me. Samuel K. Gutierrez Jeff Squyres wrote: The debug builds already have quite a bit of performance overhead. It might be desirable to change this RFC to have a similar tri-state as the MPI parameter checking: - compiled out - compiled in, always check - compiled in, use MCA parameter to determine whether to check Adapting that to this RFC, perhaps something like this: - compiled out - compiled in, always convert standard send to sync send - compiled in, use MCA parameter to determine whether to convert standard -> sync And we can leave the default as "compiled out". Howzat? On Aug 23, 2009, at 9:07 PM, Samuel K. Gutierrez wrote: Hi all, How about exposing this functionality as a run-time parameter that is only available in debug builds? This will make debugging easier and won't impact the performance of optimized builds. Just an idea... Samuel K. Gutierrez - "Jeff Squyres" wrote: Does anyone have any suggestions? Or are we stuck with compile-time checking? I didn't see this until now, but I'd be happy with just a compile time option so we could produce an install just for debugging purposes and have our users explicitly select it with modules. I have to say that this is of interest to us as we're trying to help a researcher at one of our member uni's to track down a bug where a message appears to go missing. cheers! Chris -- Christopher Samuel - (03) 9925 4751 - Systems Manager The Victorian Partnership for Advanced Computing P.O. Box 201, Carlton South, VIC 3053, Australia VPAC is a not-for-profit Registered Research Agency ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [OMPI devel] RFC: convert send to ssend
Do people know that there exist tools for checking MPI code correctness? Many, many tools and most of them are freely available. Personally I don't see any interest of doing this, absolutely no interest. There is basically no added value to our MPI, except for a very limited number of users, and these users if they manage to write a parallel application that need this checking I'm sure they will greatly benefit from a real tool to help them correct their MPI code. As a side note, a very similar effect can be obtained by decreasing the eager size of the BTLs to be equal to the size of the match header, which is about 24 bytes. george. On Aug 24, 2009, at 11:11 , Samuel K. Gutierrez wrote: Hi Jeff, Sounds good to me. Samuel K. Gutierrez Jeff Squyres wrote: The debug builds already have quite a bit of performance overhead. It might be desirable to change this RFC to have a similar tri- state as the MPI parameter checking: - compiled out - compiled in, always check - compiled in, use MCA parameter to determine whether to check Adapting that to this RFC, perhaps something like this: - compiled out - compiled in, always convert standard send to sync send - compiled in, use MCA parameter to determine whether to convert standard -> sync And we can leave the default as "compiled out". Howzat? On Aug 23, 2009, at 9:07 PM, Samuel K. Gutierrez wrote: Hi all, How about exposing this functionality as a run-time parameter that is only available in debug builds? This will make debugging easier and won't impact the performance of optimized builds. Just an idea... Samuel K. Gutierrez > > - "Jeff Squyres" wrote: > >> Does anyone have any suggestions? Or are we stuck >> with compile-time checking? > > I didn't see this until now, but I'd be happy with > just a compile time option so we could produce an > install just for debugging purposes and have our > users explicitly select it with modules. > > I have to say that this is of interest to us as we're > trying to help a researcher at one of our member uni's > to track down a bug where a message appears to go missing. > > cheers! > Chris > -- > Christopher Samuel - (03) 9925 4751 - Systems Manager > The Victorian Partnership for Advanced Computing > P.O. Box 201, Carlton South, VIC 3053, Australia > VPAC is a not-for-profit Registered Research Agency > ___ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel > ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [OMPI devel] RFC: convert send to ssend
Hi Jeff, Sounds good to me. Samuel K. Gutierrez Jeff Squyres wrote: The debug builds already have quite a bit of performance overhead. It might be desirable to change this RFC to have a similar tri-state as the MPI parameter checking: - compiled out - compiled in, always check - compiled in, use MCA parameter to determine whether to check Adapting that to this RFC, perhaps something like this: - compiled out - compiled in, always convert standard send to sync send - compiled in, use MCA parameter to determine whether to convert standard -> sync And we can leave the default as "compiled out". Howzat? On Aug 23, 2009, at 9:07 PM, Samuel K. Gutierrez wrote: Hi all, How about exposing this functionality as a run-time parameter that is only available in debug builds? This will make debugging easier and won't impact the performance of optimized builds. Just an idea... Samuel K. Gutierrez > > - "Jeff Squyres" wrote: > >> Does anyone have any suggestions? Or are we stuck >> with compile-time checking? > > I didn't see this until now, but I'd be happy with > just a compile time option so we could produce an > install just for debugging purposes and have our > users explicitly select it with modules. > > I have to say that this is of interest to us as we're > trying to help a researcher at one of our member uni's > to track down a bug where a message appears to go missing. > > cheers! > Chris > -- > Christopher Samuel - (03) 9925 4751 - Systems Manager > The Victorian Partnership for Advanced Computing > P.O. Box 201, Carlton South, VIC 3053, Australia > VPAC is a not-for-profit Registered Research Agency > ___ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel > ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [OMPI devel] RFC: convert send to ssend
The debug builds already have quite a bit of performance overhead. It might be desirable to change this RFC to have a similar tri-state as the MPI parameter checking: - compiled out - compiled in, always check - compiled in, use MCA parameter to determine whether to check Adapting that to this RFC, perhaps something like this: - compiled out - compiled in, always convert standard send to sync send - compiled in, use MCA parameter to determine whether to convert standard -> sync And we can leave the default as "compiled out". Howzat? On Aug 23, 2009, at 9:07 PM, Samuel K. Gutierrez wrote: Hi all, How about exposing this functionality as a run-time parameter that is only available in debug builds? This will make debugging easier and won't impact the performance of optimized builds. Just an idea... Samuel K. Gutierrez > > - "Jeff Squyres" wrote: > >> Does anyone have any suggestions? Or are we stuck >> with compile-time checking? > > I didn't see this until now, but I'd be happy with > just a compile time option so we could produce an > install just for debugging purposes and have our > users explicitly select it with modules. > > I have to say that this is of interest to us as we're > trying to help a researcher at one of our member uni's > to track down a bug where a message appears to go missing. > > cheers! > Chris > -- > Christopher Samuel - (03) 9925 4751 - Systems Manager > The Victorian Partnership for Advanced Computing > P.O. Box 201, Carlton South, VIC 3053, Australia > VPAC is a not-for-profit Registered Research Agency > ___ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel > ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel -- Jeff Squyres jsquy...@cisco.com
Re: [OMPI devel] RFC: convert send to ssend
Hi all, How about exposing this functionality as a run-time parameter that is only available in debug builds? This will make debugging easier and won't impact the performance of optimized builds. Just an idea... Samuel K. Gutierrez > > - "Jeff Squyres" wrote: > >> Does anyone have any suggestions? Or are we stuck >> with compile-time checking? > > I didn't see this until now, but I'd be happy with > just a compile time option so we could produce an > install just for debugging purposes and have our > users explicitly select it with modules. > > I have to say that this is of interest to us as we're > trying to help a researcher at one of our member uni's > to track down a bug where a message appears to go missing. > > cheers! > Chris > -- > Christopher Samuel - (03) 9925 4751 - Systems Manager > The Victorian Partnership for Advanced Computing > P.O. Box 201, Carlton South, VIC 3053, Australia > VPAC is a not-for-profit Registered Research Agency > ___ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel >
Re: [OMPI devel] RFC: convert send to ssend
- "Jeff Squyres" wrote: > Does anyone have any suggestions? Or are we stuck > with compile-time checking? I didn't see this until now, but I'd be happy with just a compile time option so we could produce an install just for debugging purposes and have our users explicitly select it with modules. I have to say that this is of interest to us as we're trying to help a researcher at one of our member uni's to track down a bug where a message appears to go missing. cheers! Chris -- Christopher Samuel - (03) 9925 4751 - Systems Manager The Victorian Partnership for Advanced Computing P.O. Box 201, Carlton South, VIC 3053, Australia VPAC is a not-for-profit Registered Research Agency
Re: [OMPI devel] RFC: convert send to ssend
Given the limited scope, would it make sense to somehow add this to the trace library (or a separate debug lib) - i.e., can we do it via a lib that inserts itself between the MPI binding and PMPI call? I would hate to duplicate the code in something like sendrecv, but I wonder if we could refactor that to allow for this added capability. Just a thought. It would allow someone to switch back-and-forth without recompiling or switching MPI modules. On Sat, Aug 8, 2009 at 6:03 AM, Jeff Squyres wrote: > WHAT: MCA parameter for converting all standard mode MPI sends to > synchronous mode sends > > WHY: helpful in debugging user apps > > WHERE: here's the output from "svn st" > > M ompi/runtime/params.h > M ompi/runtime/ompi_mpi_params.c > M ompi/mpi/c/send.c > M ompi/mpi/c/send_init.c > M ompi/mpi/c/sendrecv.c > M ompi/mpi/c/isend.c > > WHEN: could be 1.3.4, could be 1.5 -- don't really care which (there's no > rush) > > TIMEOUT: COB Friday 14 Aug 2009 > > More details: > > A feature we've long talked about is having an MCA parameter to switch all > standard mode MPI sends to synchronous mode sends (MPI_SEND, MPI_ISEND, > MPI_SEND_INIT, MPI_SENDRECV). This helps users identify that their > application relies on internal MPI buffering. > > Sam from LANL took a crack at implementing this; attached is the patch. > > The only concern I have about this patch (echoed by Brian to me in IM) is > that it replaces a compile-time constant with a variable lookup in the > critical performance code path -- we have to look up the value of a new > global variable during MPI_SEND to determine if the send is going to be > _SEND_STANDARD or _SEND_SYNCHRONOUS. This could cause a cache miss. > > Brian suggested making this stuff compile-out-able via some > --configure-cmd-line-switch, similar to how the MPI parameter checking stuff > is done (i.e., configure specifies either: always force sync, never force > sync, or force to sync based on an MCA parameter value at runtime). This is > certainly do-able. However, I'm sending this RFC just in case anyone can > think of a better way. Having a compile-time option to effectively remove > this capability works fine, but it does reduce the usability of this > feature: you effectively have to link your app against a different libmpi.so > in order to turn it on. > > Does anyone have any suggestions? Or are we stuck with compile-time > checking? > > Thanks. > > -- > Jeff Squyres > jsquy...@cisco.com > > ___ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel >
[OMPI devel] RFC: convert send to ssend
WHAT: MCA parameter for converting all standard mode MPI sends to synchronous mode sends WHY: helpful in debugging user apps WHERE: here's the output from "svn st" M ompi/runtime/params.h M ompi/runtime/ompi_mpi_params.c M ompi/mpi/c/send.c M ompi/mpi/c/send_init.c M ompi/mpi/c/sendrecv.c M ompi/mpi/c/isend.c WHEN: could be 1.3.4, could be 1.5 -- don't really care which (there's no rush) TIMEOUT: COB Friday 14 Aug 2009 More details: A feature we've long talked about is having an MCA parameter to switch all standard mode MPI sends to synchronous mode sends (MPI_SEND, MPI_ISEND, MPI_SEND_INIT, MPI_SENDRECV). This helps users identify that their application relies on internal MPI buffering. Sam from LANL took a crack at implementing this; attached is the patch. The only concern I have about this patch (echoed by Brian to me in IM) is that it replaces a compile-time constant with a variable lookup in the critical performance code path -- we have to look up the value of a new global variable during MPI_SEND to determine if the send is going to be _SEND_STANDARD or _SEND_SYNCHRONOUS. This could cause a cache miss. Brian suggested making this stuff compile-out-able via some -- configure-cmd-line-switch, similar to how the MPI parameter checking stuff is done (i.e., configure specifies either: always force sync, never force sync, or force to sync based on an MCA parameter value at runtime). This is certainly do-able. However, I'm sending this RFC just in case anyone can think of a better way. Having a compile-time option to effectively remove this capability works fine, but it does reduce the usability of this feature: you effectively have to link your app against a different libmpi.so in order to turn it on. Does anyone have any suggestions? Or are we stuck with compile-time checking? Thanks. -- Jeff Squyres jsquy...@cisco.com send-to-ssend.patch Description: Binary data
