Re: [OMPI devel] calling sendi earlier in the PML

2009-03-04 Thread Eugene Loh
Brian W. Barrett wrote: How about removing the MCA parameter from my earlier proposal and just having r2 filter out the sendi calls if there are multiple BTLs with heterogeneous BTLs (ie, some with sendi and some without) to the same peer. That way, the early sendi will be bypassed in that ca

Re: [OMPI devel] calling sendi earlier in the PML

2009-03-04 Thread Eugene Loh
George Bosilca wrote: On Mar 4, 2009, at 14:44 , Eugene Loh wrote: Let me try another thought here. Why do we have BTL sendi functions at all? I'll make an assertion and would appreciate feedback: a BTL sendi function contributes nothing to optimizing send latency. To optimize send la

Re: [OMPI devel] calling sendi earlier in the PML

2009-03-04 Thread Brian W. Barrett
On Wed, 4 Mar 2009, George Bosilca wrote: I'm churning a lot and not making much progress, but I'll try chewing on that idea (unless someone points out it's utterly ridiculous). I'll look into having PML ignore sendi functions altogether and just make the "send-immediate" path work fast with

Re: [OMPI devel] calling sendi earlier in the PML

2009-03-04 Thread George Bosilca
On Mar 4, 2009, at 14:44 , Eugene Loh wrote: Let me try another thought here. Why do we have BTL sendi functions at all? I'll make an assertion and would appreciate feedback: a BTL sendi function contributes nothing to optimizing send latency. To optimize send latency in the "immediate

Re: [OMPI devel] calling sendi earlier in the PML

2009-03-04 Thread Eugene Loh
Jeff Squyres wrote: On Mar 3, 2009, at 4:04 PM, Eugene Loh wrote: How about an MCA parameter to switch between this mechanism (early sendi) and the original behavior (late sendi)? This is the usual way that we resolve "I want to do X / I want to do Y" disputes. :-) I see the smiley fac

Re: [OMPI devel] calling sendi earlier in the PML

2009-03-04 Thread Jeff Squyres
On Mar 3, 2009, at 4:04 PM, Eugene Loh wrote: How about an MCA parameter to switch between this mechanism (early sendi) and the original behavior (late sendi)? This is the usual way that we resolve "I want to do X / I want to do Y" disputes. :-) I see the smiley face, but am unsure how

Re: [OMPI devel] calling sendi earlier in the PML

2009-03-04 Thread Terry Dontje
I didn't see exchange between you and Jeff at the end of this email. It basically nullifies my half-baked concern. thanks, --td Eugene Loh wrote: Terry Dontje wrote: Eugene Loh wrote: I'm on the verge of giving up moving the sendi call in the PML. I will try one or two last things, incl

Re: [OMPI devel] calling sendi earlier in the PML

2009-03-03 Thread Eugene Loh
Jeff Squyres wrote: How about an MCA parameter to switch between this mechanism (early sendi) and the original behavior (late sendi)? This is the usual way that we resolve "I want to do X / I want to do Y" disputes. :-) I see the smiley face, but am unsure how much of the message to appl

Re: [OMPI devel] calling sendi earlier in the PML

2009-03-03 Thread Eugene Loh
Brian W. Barrett wrote: On Tue, 3 Mar 2009, Eugene Loh wrote: First, this behavior is basically what I was proposing and what George didn't feel comfortable with. It is arguably no compromise at all. (Uggh, why must I be so honest?) For eager messages, it favors BTLs with sendi functions,

Re: [OMPI devel] calling sendi earlier in the PML

2009-03-03 Thread Brian W. Barrett
On Tue, 3 Mar 2009, Jeff Squyres wrote: On Mar 3, 2009, at 3:31 PM, Eugene Loh wrote: First, this behavior is basically what I was proposing and what George didn't feel comfortable with. It is arguably no compromise at all. (Uggh, why must I be so honest?) For eager messages, it favors BTL

Re: [OMPI devel] calling sendi earlier in the PML

2009-03-03 Thread Eugene Loh
Terry Dontje wrote: Eugene Loh wrote: I'm on the verge of giving up moving the sendi call in the PML. I will try one or two last things, including this e-mail asking for feedback. The idea is that when a BTL goes over a very low-latency interconnect (like sm), we really want to shave off

Re: [OMPI devel] calling sendi earlier in the PML

2009-03-03 Thread Jeff Squyres
On Mar 3, 2009, at 3:31 PM, Eugene Loh wrote: First, this behavior is basically what I was proposing and what George didn't feel comfortable with. It is arguably no compromise at all. (Uggh, why must I be so honest?) For eager messages, it favors BTLs with sendi functions, which could le

Re: [OMPI devel] calling sendi earlier in the PML

2009-03-03 Thread Brian W. Barrett
On Tue, 3 Mar 2009, Eugene Loh wrote: First, this behavior is basically what I was proposing and what George didn't feel comfortable with. It is arguably no compromise at all. (Uggh, why must I be so honest?) For eager messages, it favors BTLs with sendi functions, which could lead to those

Re: [OMPI devel] calling sendi earlier in the PML

2009-03-03 Thread Eugene Loh
Jeff Squyres wrote: How about a compromise... Keep a separate list somewhere of the sendi-enabled BTLs (this avoids looping over all the btl's and testing -- you can just loop over the btl's that you *know* have a sendi). Put that at the top of the PML and avoid the costly overhead, yadd

Re: [OMPI devel] calling sendi earlier in the PML

2009-03-03 Thread Jeff Squyres
How about a compromise... Keep a separate list somewhere of the sendi-enabled BTLs (this avoids looping over all the btl's and testing -- you can just loop over the btl's that you *know* have a sendi). Put that at the top of the PML and avoid the costly overhead, yadda yadda yadda. But i

Re: [OMPI devel] calling sendi earlier in the PML

2009-03-03 Thread Terry Dontje
Eugene Loh wrote: I'm on the verge of giving up moving the sendi call in the PML. I will try one or two last things, including this e-mail asking for feedback. The idea is that when a BTL goes over a very low-latency interconnect (like sm), we really want to shave off whatever we can from th

[OMPI devel] calling sendi earlier in the PML

2009-03-02 Thread Eugene Loh
I'm on the verge of giving up moving the sendi call in the PML. I will try one or two last things, including this e-mail asking for feedback. The idea is that when a BTL goes over a very low-latency interconnect (like sm), we really want to shave off whatever we can from the software stack.