Jed,
Ok I buy what you are saying as a possible solution but now I am
wondering why you need the request argument. Instead, as soon as you hit the
first End or PetscCommSplitReductionBegin why don't you start a new batch
(internally) that all new Begin() calls feed into? Then each End call just
checks through the previous (still incompleted) batches to find its begin
partner (or we can require always first in, first out). Thus your first
VecNormEnd(Y, would match the first begin with Y posted and your second would
match the second.
No API changes.
Barry
> On Mar 23, 2017, at 11:07 PM, Jed Brown <[email protected]> wrote:
>
> Barry Smith <[email protected]> writes:
>
>>> On Mar 23, 2017, at 10:54 PM, Jed Brown <[email protected]> wrote:
>>>
>>> Barry Smith <[email protected]> writes:
>>>>> Meh,
>>>>>
>>>>> VecNormBegin(X,&request1x);
>>>>> VecNormBegin(Y,&request1y);
>>>>> VecNormEnd(X,request1x,&norm);
>>>>> VecAXPY(Y,-1,X);
>>>>> VecNormBegin(Y,&request2y);
>>>>> VecNormEnd(Y,request2y,&norm2y);
>>>>> VecNormEnd(Y,request1y,&norm1y);
>>>>
>>>> I don't understand what you are getting at here. You don't seem to be
>>>> understanding my use case where multiple inner products/norms share the
>>>> same MPI communication (which was the original reason for
>>>> VecNormBegin/End) see for example KSPSolve_CR
>>>>
>>>> Are you somehow (incompetently) saying that the first two VecNorms
>>>> somehow share the same parallel communication (even though they
>>>> have different request values) while the third Norm has its own
>>>> MPI communication.
>>>
>>> Yeah, same as now. Every time you call *Begin() using a communicator,
>>> you get a new request for something in that "batch". When the batch is
>>> closed, either by a *End() or PetscCommSplitReductionBegin(), any future
>>> *Begin() calls will go into a new batch.
>>
>> Ok, so to support their use case they would need to explicitly call
>> PetscCommSplitReductionBegin?
>
> Yes, just like all the other pipelined methods do now.
>
>> Start first inner product
>> Do mat mult
>> do PetscCommSplitReductionBegin() to indicate the first batch is done?
>> start second inner product (will be handled in a second batch)
>> do stuff
>> get result of first inner product
>> ....
>>
>> Ok I can live with this model. But my "hoist" model would have, in some
>> cases fewer objects managed by the user but it would require an explicit
>> PetscCommSplitReductionCreate().
>
> Yeah, but that sucks if an outer scope wants to start a reduction
> (perhaps a time integrator's CFL check) without changing the API of its
> solver to plumb the reduction in.
