Re: [DISCUSS] Controlling event dispatch to AsyncEventListener (review by Aug 22)

2019-08-20 Thread Anilkumar Gingade
Thanks for all the great feedback and comments.

*API Name change:*
*Suggestion: *startPaused*,*setManualStart*,
*startWithEventDispatcherPaused*?, createPaused()*

*Start/Stop behavior:*


*- Manual start has caused a lot of trouble over the years.- Explain
starting AEQ in a paused state is different from creating gateway senders
with manual start*

Yes, we can change/adopt the name which is meaningful with the
functionality. The name suggested "pauseEventDispatchToListener()" is to
make its usage/action clear; and address any ambiguity with its usage;
between adding/removing event from AEQ and dispatching events to AEQ
listener.

To emphasis, this is not same as GatewaySender manual start (start/stop);
the manual start/stop is with enqueuing and dequeuing events from
GatewaySender itself, and as Mike pointed out there are issues with this
(during recovery with parallel gateway) and the reason its been deprecated.

The new functionality is similar to the "pause" and "resume" operation on
the GatewaySender. Except that here with the new api, the AEQ is created
with pause state.

The new api doesn't control adding and removing event from AEQ. Its to
control dispatching event to the listener. When created in paused state,
the events are continued be added into the AEQ and removed from it
(expiry). The new API will allow applications to create/manage any required
state/resource for the events before processing those events in the
application code.

*Cache level setting:*
*- Will it be more feasible if we can set the flag at cache level.*

The cache level configuration affects all the AEQs, which may not be the
requirement. Having at AEQ level help the application to use this
capability only at the required AEQ, gives more controlling capability.

-Anil.







On Tue, Aug 20, 2019 at 2:01 PM Nabarun Nag  wrote:

> Hi Anil,
>
> Will it be possible to explain to the community how the starting AEQ in a
> paused state is different from creating gateway senders with manual start
> set to true. It may be of concern as  'manual start'  for gateways is a
> deprecated.
>
> Just thinking out loud, will it be more feasible if we can set the flag at
> cache level. Any framework that is starting up Apache Geode (E.g: Spring) ,
> creates the cache -> cache.pauseProcessing(); -> create regions -> create
> AEQs -> cache.unpauseProcessing()
>
> We can gate the processing of all event listener at dispatchBatch().
>
> The advantage I feel is that
>  - we avoid introducing a new API to the AEQ creation factory.
>  - if we created 100 AEQs in paused state then we avoid having to have 100
> AEQ.unpause calls.
>
>
> Regards
> Naba
>
>
> On Tue, Aug 20, 2019 at 9:07 AM Michael Stolz  wrote:
>
> > Manual start has caused a lot of trouble over the years. We should
> > definitely circle back on those issues before traveling very far down
> this
> > road.
> >
> > --
> > Mike Stolz
> > Principal Engineer, Pivotal Cloud Cache
> > Mobile: +1-631-835-4771
> >
> >
> >
> > On Tue, Aug 20, 2019 at 11:56 AM Juan José Ramos 
> > wrote:
> >
> > > Hello Anil,
> > >
> > > +1 for the proposed solution.
> > > I'd change the method name from *pauseEventDispatchToListener* to
> > something
> > > more meaningful and understandable for our users, maybe *startPaused*?,
> > > *setManualStart* (as we currently have for the
> *GatewaySenderFactory*)?,
> > > *startWithEventDispatcherPaused*?.
> > > Best regards.
> > >
> > >
> > >
> > > On Sat, Aug 17, 2019 at 12:55 AM Anilkumar Gingade <
> aging...@pivotal.io>
> > > wrote:
> > >
> > > > I have updated the wiki based on Dan's comment.
> > > > Changes with api:
> > > >
> > > > *On "AsyncEventQueueFactory" interface - *
> > > >
> > > > *AsyncEventQueueFactory pauseEventDispatchToListener();  *// This
> > causes
> > > > AEQ to be created with paused state.
> > > >
> > > >
> > > >
> > > >
> > > > On Fri, Aug 16, 2019 at 4:36 PM Anilkumar Gingade <
> aging...@pivotal.io
> > >
> > > > wrote:
> > > >
> > > > > Dan,
> > > > >
> > > > > If you look into the API; the AEQ will be created with the pause
> > state.
> > > > > The user (application) has to call resume to dispatch the events.
> > > > >
> > > > > It will be slightly different from GatewaySender behavior; where
> > > > > GatewaySender will be created with run mode and then application
> has
> > to
> > > > > call pause on it. Here in this case AEQ will be created with paused
> > > > state.
> > > > >
> > > > > -Anil.
> > > > >
> > > > >
> > > > > On Fri, Aug 16, 2019 at 4:31 PM Dan Smith 
> wrote:
> > > > >
> > > > >> Hi Anil,
> > > > >>
> > > > >> While I like the idea of matching the API of GatewaySender, I'm
> not
> > > > sure I
> > > > >> see how this solves the problem. Is it required of the user to
> call
> > > > pause
> > > > >> on the AsyncEventQueue as soon as it is created? How would someone
> > do
> > > > that
> > > > >> when creating AEQs with xml or cluster configuration? Maybe it
> would
> > > be
> > > > >> better to not dispatch any events until 

Re: [DISCUSS] Controlling event dispatch to AsyncEventListener (review by Aug 22)

2019-08-20 Thread John Blum
FTR, I am not opposed to *Naba's* idea either.   +1

I kind of like the idea of having a global, cache-wide call that can
coordinate the background initialization/processing of all other Geode
objects that have lifecycle processes starting in the background  (e.g.
Gateways, AEQs, etc).



On Tue, Aug 20, 2019 at 2:38 PM Kirk Lund  wrote:

> If we need to add pause/resume processing to Cache, I suggest adding
> setAutostart(boolean) to CacheFactory and start() to Cache to do something
> like this:
>
> Cache cache = new CacheFactory()
> *.setAutostart(false)*
> .create();
> cache.createRegionFactory(...)...
> cache.createAsyncEventQueueFactory(...)...
> cache.addCacheServer()...
> cache.*start()*;
>
> AEQs, CacheServers, GatewaySenders/Receivers and any other endpoints that
> are created would not be started until invoking cache.start().
>
> Cheers,
> Kirk
>
> On Tue, Aug 20, 2019 at 2:01 PM Nabarun Nag  wrote:
>
> > Hi Anil,
> >
> > Will it be possible to explain to the community how the starting AEQ in a
> > paused state is different from creating gateway senders with manual start
> > set to true. It may be of concern as  'manual start'  for gateways is a
> > deprecated.
> >
> > Just thinking out loud, will it be more feasible if we can set the flag
> at
> > cache level. Any framework that is starting up Apache Geode (E.g:
> Spring) ,
> > creates the cache -> cache.pauseProcessing(); -> create regions -> create
> > AEQs -> cache.unpauseProcessing()
> >
> > We can gate the processing of all event listener at dispatchBatch().
> >
> > The advantage I feel is that
> >  - we avoid introducing a new API to the AEQ creation factory.
> >  - if we created 100 AEQs in paused state then we avoid having to have
> 100
> > AEQ.unpause calls.
> >
> >
> > Regards
> > Naba
> >
> >
> > On Tue, Aug 20, 2019 at 9:07 AM Michael Stolz  wrote:
> >
> > > Manual start has caused a lot of trouble over the years. We should
> > > definitely circle back on those issues before traveling very far down
> > this
> > > road.
> > >
> > > --
> > > Mike Stolz
> > > Principal Engineer, Pivotal Cloud Cache
> > > Mobile: +1-631-835-4771
> > >
> > >
> > >
> > > On Tue, Aug 20, 2019 at 11:56 AM Juan José Ramos 
> > > wrote:
> > >
> > > > Hello Anil,
> > > >
> > > > +1 for the proposed solution.
> > > > I'd change the method name from *pauseEventDispatchToListener* to
> > > something
> > > > more meaningful and understandable for our users, maybe
> *startPaused*?,
> > > > *setManualStart* (as we currently have for the
> > *GatewaySenderFactory*)?,
> > > > *startWithEventDispatcherPaused*?.
> > > > Best regards.
> > > >
> > > >
> > > >
> > > > On Sat, Aug 17, 2019 at 12:55 AM Anilkumar Gingade <
> > aging...@pivotal.io>
> > > > wrote:
> > > >
> > > > > I have updated the wiki based on Dan's comment.
> > > > > Changes with api:
> > > > >
> > > > > *On "AsyncEventQueueFactory" interface - *
> > > > >
> > > > > *AsyncEventQueueFactory pauseEventDispatchToListener();  *// This
> > > causes
> > > > > AEQ to be created with paused state.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Aug 16, 2019 at 4:36 PM Anilkumar Gingade <
> > aging...@pivotal.io
> > > >
> > > > > wrote:
> > > > >
> > > > > > Dan,
> > > > > >
> > > > > > If you look into the API; the AEQ will be created with the pause
> > > state.
> > > > > > The user (application) has to call resume to dispatch the events.
> > > > > >
> > > > > > It will be slightly different from GatewaySender behavior; where
> > > > > > GatewaySender will be created with run mode and then application
> > has
> > > to
> > > > > > call pause on it. Here in this case AEQ will be created with
> paused
> > > > > state.
> > > > > >
> > > > > > -Anil.
> > > > > >
> > > > > >
> > > > > > On Fri, Aug 16, 2019 at 4:31 PM Dan Smith 
> > wrote:
> > > > > >
> > > > > >> Hi Anil,
> > > > > >>
> > > > > >> While I like the idea of matching the API of GatewaySender, I'm
> > not
> > > > > sure I
> > > > > >> see how this solves the problem. Is it required of the user to
> > call
> > > > > pause
> > > > > >> on the AsyncEventQueue as soon as it is created? How would
> someone
> > > do
> > > > > that
> > > > > >> when creating AEQs with xml or cluster configuration? Maybe it
> > would
> > > > be
> > > > > >> better to not dispatch any events until we are done creating all
> > > > > regions?
> > > > > >>
> > > > > >> -Dan
> > > > > >>
> > > > > >> On Fri, Aug 16, 2019 at 2:31 PM Anilkumar Gingade <
> > > > aging...@pivotal.io>
> > > > > >> wrote:
> > > > > >>
> > > > > >> > Proposal to support controlling capability with event dispatch
> > to
> > > > > >> > AsyncEventQueue Listener.
> > > > > >> >
> > > > > >> > Wiki proposal page:
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/GEODE/%5BDraft%5D+Controlling+event+dispatch+to+AsyncEventListener
> > > > > >> >
> > > > > >> > Here is the details from the wiki page:
> > > > > >> > *Problem*
> 

Re: [DISCUSS] Controlling event dispatch to AsyncEventListener (review by Aug 22)

2019-08-20 Thread Kirk Lund
If we need to add pause/resume processing to Cache, I suggest adding
setAutostart(boolean) to CacheFactory and start() to Cache to do something
like this:

Cache cache = new CacheFactory()
*.setAutostart(false)*
.create();
cache.createRegionFactory(...)...
cache.createAsyncEventQueueFactory(...)...
cache.addCacheServer()...
cache.*start()*;

AEQs, CacheServers, GatewaySenders/Receivers and any other endpoints that
are created would not be started until invoking cache.start().

Cheers,
Kirk

On Tue, Aug 20, 2019 at 2:01 PM Nabarun Nag  wrote:

> Hi Anil,
>
> Will it be possible to explain to the community how the starting AEQ in a
> paused state is different from creating gateway senders with manual start
> set to true. It may be of concern as  'manual start'  for gateways is a
> deprecated.
>
> Just thinking out loud, will it be more feasible if we can set the flag at
> cache level. Any framework that is starting up Apache Geode (E.g: Spring) ,
> creates the cache -> cache.pauseProcessing(); -> create regions -> create
> AEQs -> cache.unpauseProcessing()
>
> We can gate the processing of all event listener at dispatchBatch().
>
> The advantage I feel is that
>  - we avoid introducing a new API to the AEQ creation factory.
>  - if we created 100 AEQs in paused state then we avoid having to have 100
> AEQ.unpause calls.
>
>
> Regards
> Naba
>
>
> On Tue, Aug 20, 2019 at 9:07 AM Michael Stolz  wrote:
>
> > Manual start has caused a lot of trouble over the years. We should
> > definitely circle back on those issues before traveling very far down
> this
> > road.
> >
> > --
> > Mike Stolz
> > Principal Engineer, Pivotal Cloud Cache
> > Mobile: +1-631-835-4771
> >
> >
> >
> > On Tue, Aug 20, 2019 at 11:56 AM Juan José Ramos 
> > wrote:
> >
> > > Hello Anil,
> > >
> > > +1 for the proposed solution.
> > > I'd change the method name from *pauseEventDispatchToListener* to
> > something
> > > more meaningful and understandable for our users, maybe *startPaused*?,
> > > *setManualStart* (as we currently have for the
> *GatewaySenderFactory*)?,
> > > *startWithEventDispatcherPaused*?.
> > > Best regards.
> > >
> > >
> > >
> > > On Sat, Aug 17, 2019 at 12:55 AM Anilkumar Gingade <
> aging...@pivotal.io>
> > > wrote:
> > >
> > > > I have updated the wiki based on Dan's comment.
> > > > Changes with api:
> > > >
> > > > *On "AsyncEventQueueFactory" interface - *
> > > >
> > > > *AsyncEventQueueFactory pauseEventDispatchToListener();  *// This
> > causes
> > > > AEQ to be created with paused state.
> > > >
> > > >
> > > >
> > > >
> > > > On Fri, Aug 16, 2019 at 4:36 PM Anilkumar Gingade <
> aging...@pivotal.io
> > >
> > > > wrote:
> > > >
> > > > > Dan,
> > > > >
> > > > > If you look into the API; the AEQ will be created with the pause
> > state.
> > > > > The user (application) has to call resume to dispatch the events.
> > > > >
> > > > > It will be slightly different from GatewaySender behavior; where
> > > > > GatewaySender will be created with run mode and then application
> has
> > to
> > > > > call pause on it. Here in this case AEQ will be created with paused
> > > > state.
> > > > >
> > > > > -Anil.
> > > > >
> > > > >
> > > > > On Fri, Aug 16, 2019 at 4:31 PM Dan Smith 
> wrote:
> > > > >
> > > > >> Hi Anil,
> > > > >>
> > > > >> While I like the idea of matching the API of GatewaySender, I'm
> not
> > > > sure I
> > > > >> see how this solves the problem. Is it required of the user to
> call
> > > > pause
> > > > >> on the AsyncEventQueue as soon as it is created? How would someone
> > do
> > > > that
> > > > >> when creating AEQs with xml or cluster configuration? Maybe it
> would
> > > be
> > > > >> better to not dispatch any events until we are done creating all
> > > > regions?
> > > > >>
> > > > >> -Dan
> > > > >>
> > > > >> On Fri, Aug 16, 2019 at 2:31 PM Anilkumar Gingade <
> > > aging...@pivotal.io>
> > > > >> wrote:
> > > > >>
> > > > >> > Proposal to support controlling capability with event dispatch
> to
> > > > >> > AsyncEventQueue Listener.
> > > > >> >
> > > > >> > Wiki proposal page:
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/GEODE/%5BDraft%5D+Controlling+event+dispatch+to+AsyncEventListener
> > > > >> >
> > > > >> > Here is the details from the wiki page:
> > > > >> > *Problem*
> > > > >> >
> > > > >> > *The Geode system requires AEQs to be configured before regions
> > are
> > > > >> > created. If an AEQ listener is operating on a secondary region,
> > this
> > > > >> could
> > > > >> > cause listener to operate on a region which is not yet created
> or
> > > > fully
> > > > >> > initialized (for region with co-located regions) which could
> > result
> > > in
> > > > >> > missing events or dead-lock scenario between region (co-located
> > > > region)
> > > > >> > creation threads. This scenario is likely to happen during
> > > persistence
> > > > >> > recovery; when AEQs are created in the start, the recovered AEQ
> > > 

Re: [DISCUSS] Controlling event dispatch to AsyncEventListener (review by Aug 22)

2019-08-20 Thread John Blum
After talking with *Alexander* this morning (and taking *Mike's* concerns
into consideration), and to not adversely affect users/customers today, I
think...

1). AEQ's can (should) have a configuration setting to "manually" start the
queue's background processor (i.e. Thread) to begin processing (existing,
persistent) events. This would be consistent with GatewaySender processing.

2) I think that the default behavior should remain as is, the AEQ starts
automatically, as before...
2.1) As such, this will not present any surprises for existing
users/customers that expect the AEQ to start processing automatically as it
currently does.
2.2) If users wants to have the AEQ start manually, then they must
"explicitly" configure it to do so (as with GatewaySenders today), which is
a "conscious" choice since they probably have encountered some
non-apparent, distributed, cyclic dependency (e.g. Region A -> AEQ -> AEQ
Listener -> Region B) which could cause initialization safety problems,
race conditions or other distributed deadlock problems (e.g. deadly
embrace) if not properly coordinated, and perhaps the only way to achieve
that arrangement reliably is by starting an AEQ manually in these
situations.  Either way, it is an explicit/conscious choice by the user.

3) If we are talking about API, then I would envision a very "consistent"
and "familiar" API to the GatewaySenderFactory API already, i.e.
GatewaySenderFactory.setManualStart(start:boolean).

You could possibly provide an overloaded AsyncEventQueueFactory.create(..)
method, such as create(id:String, :AsyncEventListener, manualStart:boolean),
but meh.


Food for thought,
John




On Tue, Aug 20, 2019 at 2:01 PM Nabarun Nag  wrote:

> Hi Anil,
>
> Will it be possible to explain to the community how the starting AEQ in a
> paused state is different from creating gateway senders with manual start
> set to true. It may be of concern as  'manual start'  for gateways is a
> deprecated.
>
> Just thinking out loud, will it be more feasible if we can set the flag at
> cache level. Any framework that is starting up Apache Geode (E.g: Spring) ,
> creates the cache -> cache.pauseProcessing(); -> create regions -> create
> AEQs -> cache.unpauseProcessing()
>
> We can gate the processing of all event listener at dispatchBatch().
>
> The advantage I feel is that
>  - we avoid introducing a new API to the AEQ creation factory.
>  - if we created 100 AEQs in paused state then we avoid having to have 100
> AEQ.unpause calls.
>
>
> Regards
> Naba
>
>
> On Tue, Aug 20, 2019 at 9:07 AM Michael Stolz  wrote:
>
> > Manual start has caused a lot of trouble over the years. We should
> > definitely circle back on those issues before traveling very far down
> this
> > road.
> >
> > --
> > Mike Stolz
> > Principal Engineer, Pivotal Cloud Cache
> > Mobile: +1-631-835-4771
> >
> >
> >
> > On Tue, Aug 20, 2019 at 11:56 AM Juan José Ramos 
> > wrote:
> >
> > > Hello Anil,
> > >
> > > +1 for the proposed solution.
> > > I'd change the method name from *pauseEventDispatchToListener* to
> > something
> > > more meaningful and understandable for our users, maybe *startPaused*?,
> > > *setManualStart* (as we currently have for the
> *GatewaySenderFactory*)?,
> > > *startWithEventDispatcherPaused*?.
> > > Best regards.
> > >
> > >
> > >
> > > On Sat, Aug 17, 2019 at 12:55 AM Anilkumar Gingade <
> aging...@pivotal.io>
> > > wrote:
> > >
> > > > I have updated the wiki based on Dan's comment.
> > > > Changes with api:
> > > >
> > > > *On "AsyncEventQueueFactory" interface - *
> > > >
> > > > *AsyncEventQueueFactory pauseEventDispatchToListener();  *// This
> > causes
> > > > AEQ to be created with paused state.
> > > >
> > > >
> > > >
> > > >
> > > > On Fri, Aug 16, 2019 at 4:36 PM Anilkumar Gingade <
> aging...@pivotal.io
> > >
> > > > wrote:
> > > >
> > > > > Dan,
> > > > >
> > > > > If you look into the API; the AEQ will be created with the pause
> > state.
> > > > > The user (application) has to call resume to dispatch the events.
> > > > >
> > > > > It will be slightly different from GatewaySender behavior; where
> > > > > GatewaySender will be created with run mode and then application
> has
> > to
> > > > > call pause on it. Here in this case AEQ will be created with paused
> > > > state.
> > > > >
> > > > > -Anil.
> > > > >
> > > > >
> > > > > On Fri, Aug 16, 2019 at 4:31 PM Dan Smith 
> wrote:
> > > > >
> > > > >> Hi Anil,
> > > > >>
> > > > >> While I like the idea of matching the API of GatewaySender, I'm
> not
> > > > sure I
> > > > >> see how this solves the problem. Is it required of the user to
> call
> > > > pause
> > > > >> on the AsyncEventQueue as soon as it is created? How would someone
> > do
> > > > that
> > > > >> when creating AEQs with xml or cluster configuration? Maybe it
> would
> > > be
> > > > >> better to not dispatch any events until we are done creating all
> > > > regions?
> > > > >>
> > > > >> -Dan
> > > > >>
> > > > >> On Fri, Aug 16, 2019 at 2:31 PM Anilkumar 

Re: [DISCUSS] Controlling event dispatch to AsyncEventListener (review by Aug 22)

2019-08-20 Thread Nabarun Nag
Hi Anil,

Will it be possible to explain to the community how the starting AEQ in a
paused state is different from creating gateway senders with manual start
set to true. It may be of concern as  'manual start'  for gateways is a
deprecated.

Just thinking out loud, will it be more feasible if we can set the flag at
cache level. Any framework that is starting up Apache Geode (E.g: Spring) ,
creates the cache -> cache.pauseProcessing(); -> create regions -> create
AEQs -> cache.unpauseProcessing()

We can gate the processing of all event listener at dispatchBatch().

The advantage I feel is that
 - we avoid introducing a new API to the AEQ creation factory.
 - if we created 100 AEQs in paused state then we avoid having to have 100
AEQ.unpause calls.


Regards
Naba


On Tue, Aug 20, 2019 at 9:07 AM Michael Stolz  wrote:

> Manual start has caused a lot of trouble over the years. We should
> definitely circle back on those issues before traveling very far down this
> road.
>
> --
> Mike Stolz
> Principal Engineer, Pivotal Cloud Cache
> Mobile: +1-631-835-4771
>
>
>
> On Tue, Aug 20, 2019 at 11:56 AM Juan José Ramos 
> wrote:
>
> > Hello Anil,
> >
> > +1 for the proposed solution.
> > I'd change the method name from *pauseEventDispatchToListener* to
> something
> > more meaningful and understandable for our users, maybe *startPaused*?,
> > *setManualStart* (as we currently have for the *GatewaySenderFactory*)?,
> > *startWithEventDispatcherPaused*?.
> > Best regards.
> >
> >
> >
> > On Sat, Aug 17, 2019 at 12:55 AM Anilkumar Gingade 
> > wrote:
> >
> > > I have updated the wiki based on Dan's comment.
> > > Changes with api:
> > >
> > > *On "AsyncEventQueueFactory" interface - *
> > >
> > > *AsyncEventQueueFactory pauseEventDispatchToListener();  *// This
> causes
> > > AEQ to be created with paused state.
> > >
> > >
> > >
> > >
> > > On Fri, Aug 16, 2019 at 4:36 PM Anilkumar Gingade  >
> > > wrote:
> > >
> > > > Dan,
> > > >
> > > > If you look into the API; the AEQ will be created with the pause
> state.
> > > > The user (application) has to call resume to dispatch the events.
> > > >
> > > > It will be slightly different from GatewaySender behavior; where
> > > > GatewaySender will be created with run mode and then application has
> to
> > > > call pause on it. Here in this case AEQ will be created with paused
> > > state.
> > > >
> > > > -Anil.
> > > >
> > > >
> > > > On Fri, Aug 16, 2019 at 4:31 PM Dan Smith  wrote:
> > > >
> > > >> Hi Anil,
> > > >>
> > > >> While I like the idea of matching the API of GatewaySender, I'm not
> > > sure I
> > > >> see how this solves the problem. Is it required of the user to call
> > > pause
> > > >> on the AsyncEventQueue as soon as it is created? How would someone
> do
> > > that
> > > >> when creating AEQs with xml or cluster configuration? Maybe it would
> > be
> > > >> better to not dispatch any events until we are done creating all
> > > regions?
> > > >>
> > > >> -Dan
> > > >>
> > > >> On Fri, Aug 16, 2019 at 2:31 PM Anilkumar Gingade <
> > aging...@pivotal.io>
> > > >> wrote:
> > > >>
> > > >> > Proposal to support controlling capability with event dispatch to
> > > >> > AsyncEventQueue Listener.
> > > >> >
> > > >> > Wiki proposal page:
> > > >> >
> > > >> >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/GEODE/%5BDraft%5D+Controlling+event+dispatch+to+AsyncEventListener
> > > >> >
> > > >> > Here is the details from the wiki page:
> > > >> > *Problem*
> > > >> >
> > > >> > *The Geode system requires AEQs to be configured before regions
> are
> > > >> > created. If an AEQ listener is operating on a secondary region,
> this
> > > >> could
> > > >> > cause listener to operate on a region which is not yet created or
> > > fully
> > > >> > initialized (for region with co-located regions) which could
> result
> > in
> > > >> > missing events or dead-lock scenario between region (co-located
> > > region)
> > > >> > creation threads. This scenario is likely to happen during
> > persistence
> > > >> > recovery; when AEQs are created in the start, the recovered AEQ
> > events
> > > >> are
> > > >> > dispatched immediately, thus invoking the AEQ listeners.*
> > > >> > Anti-Goals
> > > >> >
> > > >> > None
> > > >> > *Solution*
> > > >> >
> > > >> > *The proposed solution is to provide a way to control dispatching
> > AEQ
> > > >> > events to the AEQ Listeners, this could be done by adding "pause"
> > and
> > > >> > "resume" capability to the AEQ, which will allow application to
> > decide
> > > >> when
> > > >> > to dispatch events to the listeners. *
> > > >> >
> > > >> >
> > > >> > *The proposal is similar to existing "pause" and "resume" behavior
> > on
> > > >> the
> > > >> > GatewaySender, on which the AEQ is based on (AEQ implementation
> is a
> > > >> > wrapper around GatewaySender). *
> > > >> > Changes and Additions to Public Interfaces
> > > >> >
> > > >> > *The proposed APIs are:*
> > > >> >
> > > >> > *On "AsyncEventQueueFactory" interface - 

Re: [DISCUSS] Controlling event dispatch to AsyncEventListener (review by Aug 22)

2019-08-20 Thread Michael Stolz
Manual start has caused a lot of trouble over the years. We should
definitely circle back on those issues before traveling very far down this
road.

--
Mike Stolz
Principal Engineer, Pivotal Cloud Cache
Mobile: +1-631-835-4771



On Tue, Aug 20, 2019 at 11:56 AM Juan José Ramos  wrote:

> Hello Anil,
>
> +1 for the proposed solution.
> I'd change the method name from *pauseEventDispatchToListener* to something
> more meaningful and understandable for our users, maybe *startPaused*?,
> *setManualStart* (as we currently have for the *GatewaySenderFactory*)?,
> *startWithEventDispatcherPaused*?.
> Best regards.
>
>
>
> On Sat, Aug 17, 2019 at 12:55 AM Anilkumar Gingade 
> wrote:
>
> > I have updated the wiki based on Dan's comment.
> > Changes with api:
> >
> > *On "AsyncEventQueueFactory" interface - *
> >
> > *AsyncEventQueueFactory pauseEventDispatchToListener();  *// This causes
> > AEQ to be created with paused state.
> >
> >
> >
> >
> > On Fri, Aug 16, 2019 at 4:36 PM Anilkumar Gingade 
> > wrote:
> >
> > > Dan,
> > >
> > > If you look into the API; the AEQ will be created with the pause state.
> > > The user (application) has to call resume to dispatch the events.
> > >
> > > It will be slightly different from GatewaySender behavior; where
> > > GatewaySender will be created with run mode and then application has to
> > > call pause on it. Here in this case AEQ will be created with paused
> > state.
> > >
> > > -Anil.
> > >
> > >
> > > On Fri, Aug 16, 2019 at 4:31 PM Dan Smith  wrote:
> > >
> > >> Hi Anil,
> > >>
> > >> While I like the idea of matching the API of GatewaySender, I'm not
> > sure I
> > >> see how this solves the problem. Is it required of the user to call
> > pause
> > >> on the AsyncEventQueue as soon as it is created? How would someone do
> > that
> > >> when creating AEQs with xml or cluster configuration? Maybe it would
> be
> > >> better to not dispatch any events until we are done creating all
> > regions?
> > >>
> > >> -Dan
> > >>
> > >> On Fri, Aug 16, 2019 at 2:31 PM Anilkumar Gingade <
> aging...@pivotal.io>
> > >> wrote:
> > >>
> > >> > Proposal to support controlling capability with event dispatch to
> > >> > AsyncEventQueue Listener.
> > >> >
> > >> > Wiki proposal page:
> > >> >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/GEODE/%5BDraft%5D+Controlling+event+dispatch+to+AsyncEventListener
> > >> >
> > >> > Here is the details from the wiki page:
> > >> > *Problem*
> > >> >
> > >> > *The Geode system requires AEQs to be configured before regions are
> > >> > created. If an AEQ listener is operating on a secondary region, this
> > >> could
> > >> > cause listener to operate on a region which is not yet created or
> > fully
> > >> > initialized (for region with co-located regions) which could result
> in
> > >> > missing events or dead-lock scenario between region (co-located
> > region)
> > >> > creation threads. This scenario is likely to happen during
> persistence
> > >> > recovery; when AEQs are created in the start, the recovered AEQ
> events
> > >> are
> > >> > dispatched immediately, thus invoking the AEQ listeners.*
> > >> > Anti-Goals
> > >> >
> > >> > None
> > >> > *Solution*
> > >> >
> > >> > *The proposed solution is to provide a way to control dispatching
> AEQ
> > >> > events to the AEQ Listeners, this could be done by adding "pause"
> and
> > >> > "resume" capability to the AEQ, which will allow application to
> decide
> > >> when
> > >> > to dispatch events to the listeners. *
> > >> >
> > >> >
> > >> > *The proposal is similar to existing "pause" and "resume" behavior
> on
> > >> the
> > >> > GatewaySender, on which the AEQ is based on (AEQ implementation is a
> > >> > wrapper around GatewaySender). *
> > >> > Changes and Additions to Public Interfaces
> > >> >
> > >> > *The proposed APIs are:*
> > >> >
> > >> > *On "AsyncEventQueueFactory" interface - *
> > >> >
> > >> > *AsyncEventQueue pauseEventDispatchToListener();*
> > >> >
> > >> > *On "AsyncEventQueue" interface -*
> > >> >
> > >> > *boolean resumeEventDispatchToListener(); **returns true or false if
> > the
> > >> > event dispatch is resumed successfully.*
> > >> >
> > >> >
> > >> > *The constraints on the pauseEventDispatchToListener() will remain
> > >> similar
> > >> > to as in "GatewaySender.pause()" :*
> > >> >
> > >> > "It should be kept in mind that the events will still be getting
> > queued
> > >> > into the queue. The scope of this operation is the VM on which it is
> > >> > invoked. In case the AEQ is parallel, the AEQ will be paused on
> > >> individual
> > >> > node where this API is called and the AEQ on other VM's can still
> > >> dispatch
> > >> > events. In case the AEQ is not parallel, and the running AEQ on
> which
> > >> this
> > >> > API is invoked is not primary then primary AEQ will still continue
> > >> > dispatching events."
> > >> > Performance Impact
> > >> >
> > >> >
> > >> > *This will have similar performance and resource implication as with
> > 

Re: [DISCUSS] Controlling event dispatch to AsyncEventListener (review by Aug 22)

2019-08-20 Thread Ryan McMahon
+1 to Juan's comment.  I wonder if it would make sense just to have a
variation on the existing create() method.  Either have a createPaused()
method or add an overload for create() that takes a startPaused boolean.
That will really drive home that the AsyncEventQueue will be created in a
paused state.  I personally would prefer the overload with extra parameter,
but I think either approach would be fine.

Thanks,
Ryan

On Tue, Aug 20, 2019 at 8:56 AM Juan José Ramos  wrote:

> Hello Anil,
>
> +1 for the proposed solution.
> I'd change the method name from *pauseEventDispatchToListener* to something
> more meaningful and understandable for our users, maybe *startPaused*?,
> *setManualStart* (as we currently have for the *GatewaySenderFactory*)?,
> *startWithEventDispatcherPaused*?.
> Best regards.
>
>
>
> On Sat, Aug 17, 2019 at 12:55 AM Anilkumar Gingade 
> wrote:
>
> > I have updated the wiki based on Dan's comment.
> > Changes with api:
> >
> > *On "AsyncEventQueueFactory" interface - *
> >
> > *AsyncEventQueueFactory pauseEventDispatchToListener();  *// This causes
> > AEQ to be created with paused state.
> >
> >
> >
> >
> > On Fri, Aug 16, 2019 at 4:36 PM Anilkumar Gingade 
> > wrote:
> >
> > > Dan,
> > >
> > > If you look into the API; the AEQ will be created with the pause state.
> > > The user (application) has to call resume to dispatch the events.
> > >
> > > It will be slightly different from GatewaySender behavior; where
> > > GatewaySender will be created with run mode and then application has to
> > > call pause on it. Here in this case AEQ will be created with paused
> > state.
> > >
> > > -Anil.
> > >
> > >
> > > On Fri, Aug 16, 2019 at 4:31 PM Dan Smith  wrote:
> > >
> > >> Hi Anil,
> > >>
> > >> While I like the idea of matching the API of GatewaySender, I'm not
> > sure I
> > >> see how this solves the problem. Is it required of the user to call
> > pause
> > >> on the AsyncEventQueue as soon as it is created? How would someone do
> > that
> > >> when creating AEQs with xml or cluster configuration? Maybe it would
> be
> > >> better to not dispatch any events until we are done creating all
> > regions?
> > >>
> > >> -Dan
> > >>
> > >> On Fri, Aug 16, 2019 at 2:31 PM Anilkumar Gingade <
> aging...@pivotal.io>
> > >> wrote:
> > >>
> > >> > Proposal to support controlling capability with event dispatch to
> > >> > AsyncEventQueue Listener.
> > >> >
> > >> > Wiki proposal page:
> > >> >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/GEODE/%5BDraft%5D+Controlling+event+dispatch+to+AsyncEventListener
> > >> >
> > >> > Here is the details from the wiki page:
> > >> > *Problem*
> > >> >
> > >> > *The Geode system requires AEQs to be configured before regions are
> > >> > created. If an AEQ listener is operating on a secondary region, this
> > >> could
> > >> > cause listener to operate on a region which is not yet created or
> > fully
> > >> > initialized (for region with co-located regions) which could result
> in
> > >> > missing events or dead-lock scenario between region (co-located
> > region)
> > >> > creation threads. This scenario is likely to happen during
> persistence
> > >> > recovery; when AEQs are created in the start, the recovered AEQ
> events
> > >> are
> > >> > dispatched immediately, thus invoking the AEQ listeners.*
> > >> > Anti-Goals
> > >> >
> > >> > None
> > >> > *Solution*
> > >> >
> > >> > *The proposed solution is to provide a way to control dispatching
> AEQ
> > >> > events to the AEQ Listeners, this could be done by adding "pause"
> and
> > >> > "resume" capability to the AEQ, which will allow application to
> decide
> > >> when
> > >> > to dispatch events to the listeners. *
> > >> >
> > >> >
> > >> > *The proposal is similar to existing "pause" and "resume" behavior
> on
> > >> the
> > >> > GatewaySender, on which the AEQ is based on (AEQ implementation is a
> > >> > wrapper around GatewaySender). *
> > >> > Changes and Additions to Public Interfaces
> > >> >
> > >> > *The proposed APIs are:*
> > >> >
> > >> > *On "AsyncEventQueueFactory" interface - *
> > >> >
> > >> > *AsyncEventQueue pauseEventDispatchToListener();*
> > >> >
> > >> > *On "AsyncEventQueue" interface -*
> > >> >
> > >> > *boolean resumeEventDispatchToListener(); **returns true or false if
> > the
> > >> > event dispatch is resumed successfully.*
> > >> >
> > >> >
> > >> > *The constraints on the pauseEventDispatchToListener() will remain
> > >> similar
> > >> > to as in "GatewaySender.pause()" :*
> > >> >
> > >> > "It should be kept in mind that the events will still be getting
> > queued
> > >> > into the queue. The scope of this operation is the VM on which it is
> > >> > invoked. In case the AEQ is parallel, the AEQ will be paused on
> > >> individual
> > >> > node where this API is called and the AEQ on other VM's can still
> > >> dispatch
> > >> > events. In case the AEQ is not parallel, and the running AEQ on
> which
> > >> this
> > >> > API is invoked is not primary then 

Re: [DISCUSS] Controlling event dispatch to AsyncEventListener (review by Aug 22)

2019-08-20 Thread Juan José Ramos
Hello Anil,

+1 for the proposed solution.
I'd change the method name from *pauseEventDispatchToListener* to something
more meaningful and understandable for our users, maybe *startPaused*?,
*setManualStart* (as we currently have for the *GatewaySenderFactory*)?,
*startWithEventDispatcherPaused*?.
Best regards.



On Sat, Aug 17, 2019 at 12:55 AM Anilkumar Gingade 
wrote:

> I have updated the wiki based on Dan's comment.
> Changes with api:
>
> *On "AsyncEventQueueFactory" interface - *
>
> *AsyncEventQueueFactory pauseEventDispatchToListener();  *// This causes
> AEQ to be created with paused state.
>
>
>
>
> On Fri, Aug 16, 2019 at 4:36 PM Anilkumar Gingade 
> wrote:
>
> > Dan,
> >
> > If you look into the API; the AEQ will be created with the pause state.
> > The user (application) has to call resume to dispatch the events.
> >
> > It will be slightly different from GatewaySender behavior; where
> > GatewaySender will be created with run mode and then application has to
> > call pause on it. Here in this case AEQ will be created with paused
> state.
> >
> > -Anil.
> >
> >
> > On Fri, Aug 16, 2019 at 4:31 PM Dan Smith  wrote:
> >
> >> Hi Anil,
> >>
> >> While I like the idea of matching the API of GatewaySender, I'm not
> sure I
> >> see how this solves the problem. Is it required of the user to call
> pause
> >> on the AsyncEventQueue as soon as it is created? How would someone do
> that
> >> when creating AEQs with xml or cluster configuration? Maybe it would be
> >> better to not dispatch any events until we are done creating all
> regions?
> >>
> >> -Dan
> >>
> >> On Fri, Aug 16, 2019 at 2:31 PM Anilkumar Gingade 
> >> wrote:
> >>
> >> > Proposal to support controlling capability with event dispatch to
> >> > AsyncEventQueue Listener.
> >> >
> >> > Wiki proposal page:
> >> >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/GEODE/%5BDraft%5D+Controlling+event+dispatch+to+AsyncEventListener
> >> >
> >> > Here is the details from the wiki page:
> >> > *Problem*
> >> >
> >> > *The Geode system requires AEQs to be configured before regions are
> >> > created. If an AEQ listener is operating on a secondary region, this
> >> could
> >> > cause listener to operate on a region which is not yet created or
> fully
> >> > initialized (for region with co-located regions) which could result in
> >> > missing events or dead-lock scenario between region (co-located
> region)
> >> > creation threads. This scenario is likely to happen during persistence
> >> > recovery; when AEQs are created in the start, the recovered AEQ events
> >> are
> >> > dispatched immediately, thus invoking the AEQ listeners.*
> >> > Anti-Goals
> >> >
> >> > None
> >> > *Solution*
> >> >
> >> > *The proposed solution is to provide a way to control dispatching AEQ
> >> > events to the AEQ Listeners, this could be done by adding "pause"  and
> >> > "resume" capability to the AEQ, which will allow application to decide
> >> when
> >> > to dispatch events to the listeners. *
> >> >
> >> >
> >> > *The proposal is similar to existing "pause" and "resume" behavior on
> >> the
> >> > GatewaySender, on which the AEQ is based on (AEQ implementation is a
> >> > wrapper around GatewaySender). *
> >> > Changes and Additions to Public Interfaces
> >> >
> >> > *The proposed APIs are:*
> >> >
> >> > *On "AsyncEventQueueFactory" interface - *
> >> >
> >> > *AsyncEventQueue pauseEventDispatchToListener();*
> >> >
> >> > *On "AsyncEventQueue" interface -*
> >> >
> >> > *boolean resumeEventDispatchToListener(); **returns true or false if
> the
> >> > event dispatch is resumed successfully.*
> >> >
> >> >
> >> > *The constraints on the pauseEventDispatchToListener() will remain
> >> similar
> >> > to as in "GatewaySender.pause()" :*
> >> >
> >> > "It should be kept in mind that the events will still be getting
> queued
> >> > into the queue. The scope of this operation is the VM on which it is
> >> > invoked. In case the AEQ is parallel, the AEQ will be paused on
> >> individual
> >> > node where this API is called and the AEQ on other VM's can still
> >> dispatch
> >> > events. In case the AEQ is not parallel, and the running AEQ on which
> >> this
> >> > API is invoked is not primary then primary AEQ will still continue
> >> > dispatching events."
> >> > Performance Impact
> >> >
> >> >
> >> > *This will have similar performance and resource implication as with
> the
> >> > "GatewaySender.pause()" functionality. If the AEQ is not resumed or
> >> kept in
> >> > "pause" state for long, it may start consuming the configured memory
> and
> >> > overflow it into disk and may cause disk full scenario.*
> >> > Backwards Compatibility and Upgrade Path
> >> >
> >> > *Impact with rolling upgrade: *
> >> >
> >> > *As the api is applicable at individual VM level, there is no message
> >> > serialization changes involved. And only applicable to the events
> >> getting
> >> > dispatched to the listeners on that VM. And the AEQ which are
> replicated
> >> > (for 

Re: [DISCUSS] Controlling event dispatch to AsyncEventListener (review by Aug 22)

2019-08-16 Thread Anilkumar Gingade
I have updated the wiki based on Dan's comment.
Changes with api:

*On "AsyncEventQueueFactory" interface - *

*AsyncEventQueueFactory pauseEventDispatchToListener();  *// This causes
AEQ to be created with paused state.




On Fri, Aug 16, 2019 at 4:36 PM Anilkumar Gingade 
wrote:

> Dan,
>
> If you look into the API; the AEQ will be created with the pause state.
> The user (application) has to call resume to dispatch the events.
>
> It will be slightly different from GatewaySender behavior; where
> GatewaySender will be created with run mode and then application has to
> call pause on it. Here in this case AEQ will be created with paused state.
>
> -Anil.
>
>
> On Fri, Aug 16, 2019 at 4:31 PM Dan Smith  wrote:
>
>> Hi Anil,
>>
>> While I like the idea of matching the API of GatewaySender, I'm not sure I
>> see how this solves the problem. Is it required of the user to call pause
>> on the AsyncEventQueue as soon as it is created? How would someone do that
>> when creating AEQs with xml or cluster configuration? Maybe it would be
>> better to not dispatch any events until we are done creating all regions?
>>
>> -Dan
>>
>> On Fri, Aug 16, 2019 at 2:31 PM Anilkumar Gingade 
>> wrote:
>>
>> > Proposal to support controlling capability with event dispatch to
>> > AsyncEventQueue Listener.
>> >
>> > Wiki proposal page:
>> >
>> >
>> https://cwiki.apache.org/confluence/display/GEODE/%5BDraft%5D+Controlling+event+dispatch+to+AsyncEventListener
>> >
>> > Here is the details from the wiki page:
>> > *Problem*
>> >
>> > *The Geode system requires AEQs to be configured before regions are
>> > created. If an AEQ listener is operating on a secondary region, this
>> could
>> > cause listener to operate on a region which is not yet created or fully
>> > initialized (for region with co-located regions) which could result in
>> > missing events or dead-lock scenario between region (co-located region)
>> > creation threads. This scenario is likely to happen during persistence
>> > recovery; when AEQs are created in the start, the recovered AEQ events
>> are
>> > dispatched immediately, thus invoking the AEQ listeners.*
>> > Anti-Goals
>> >
>> > None
>> > *Solution*
>> >
>> > *The proposed solution is to provide a way to control dispatching AEQ
>> > events to the AEQ Listeners, this could be done by adding "pause"  and
>> > "resume" capability to the AEQ, which will allow application to decide
>> when
>> > to dispatch events to the listeners. *
>> >
>> >
>> > *The proposal is similar to existing "pause" and "resume" behavior on
>> the
>> > GatewaySender, on which the AEQ is based on (AEQ implementation is a
>> > wrapper around GatewaySender). *
>> > Changes and Additions to Public Interfaces
>> >
>> > *The proposed APIs are:*
>> >
>> > *On "AsyncEventQueueFactory" interface - *
>> >
>> > *AsyncEventQueue pauseEventDispatchToListener();*
>> >
>> > *On "AsyncEventQueue" interface -*
>> >
>> > *boolean resumeEventDispatchToListener(); **returns true or false if the
>> > event dispatch is resumed successfully.*
>> >
>> >
>> > *The constraints on the pauseEventDispatchToListener() will remain
>> similar
>> > to as in "GatewaySender.pause()" :*
>> >
>> > "It should be kept in mind that the events will still be getting queued
>> > into the queue. The scope of this operation is the VM on which it is
>> > invoked. In case the AEQ is parallel, the AEQ will be paused on
>> individual
>> > node where this API is called and the AEQ on other VM's can still
>> dispatch
>> > events. In case the AEQ is not parallel, and the running AEQ on which
>> this
>> > API is invoked is not primary then primary AEQ will still continue
>> > dispatching events."
>> > Performance Impact
>> >
>> >
>> > *This will have similar performance and resource implication as with the
>> > "GatewaySender.pause()" functionality. If the AEQ is not resumed or
>> kept in
>> > "pause" state for long, it may start consuming the configured memory and
>> > overflow it into disk and may cause disk full scenario.*
>> > Backwards Compatibility and Upgrade Path
>> >
>> > *Impact with rolling upgrade: *
>> >
>> > *As the api is applicable at individual VM level, there is no message
>> > serialization changes involved. And only applicable to the events
>> getting
>> > dispatched to the listeners on that VM. And the AEQ which are replicated
>> > (for redundancy) continues to work as before.*
>> >
>> > *Backward compatibility requirements: *
>> >
>> > *None. The AEQs are configured and managed at the server side. There is
>> no
>> > messaging involved between client/server.*
>> >
>> > *Disk formatting changes:*
>> >
>> > *None.*
>> >
>> > *Deprecation and Application Changes:*
>> >
>> >
>> > *None. If needed, the existing application can be modified to control
>> event
>> > dispatch with AEQ listener.*
>> > Prior Art
>> >
>> > *Without this, the AEQ listeners operating on other regions could
>> > experience missing events or dead lock, if there are co-located
>> regions.*
>> >
>> > 

Re: [DISCUSS] Controlling event dispatch to AsyncEventListener (review by Aug 22)

2019-08-16 Thread Anilkumar Gingade
Dan,

If you look into the API; the AEQ will be created with the pause state. The
user (application) has to call resume to dispatch the events.

It will be slightly different from GatewaySender behavior; where
GatewaySender will be created with run mode and then application has to
call pause on it. Here in this case AEQ will be created with paused state.

-Anil.


On Fri, Aug 16, 2019 at 4:31 PM Dan Smith  wrote:

> Hi Anil,
>
> While I like the idea of matching the API of GatewaySender, I'm not sure I
> see how this solves the problem. Is it required of the user to call pause
> on the AsyncEventQueue as soon as it is created? How would someone do that
> when creating AEQs with xml or cluster configuration? Maybe it would be
> better to not dispatch any events until we are done creating all regions?
>
> -Dan
>
> On Fri, Aug 16, 2019 at 2:31 PM Anilkumar Gingade 
> wrote:
>
> > Proposal to support controlling capability with event dispatch to
> > AsyncEventQueue Listener.
> >
> > Wiki proposal page:
> >
> >
> https://cwiki.apache.org/confluence/display/GEODE/%5BDraft%5D+Controlling+event+dispatch+to+AsyncEventListener
> >
> > Here is the details from the wiki page:
> > *Problem*
> >
> > *The Geode system requires AEQs to be configured before regions are
> > created. If an AEQ listener is operating on a secondary region, this
> could
> > cause listener to operate on a region which is not yet created or fully
> > initialized (for region with co-located regions) which could result in
> > missing events or dead-lock scenario between region (co-located region)
> > creation threads. This scenario is likely to happen during persistence
> > recovery; when AEQs are created in the start, the recovered AEQ events
> are
> > dispatched immediately, thus invoking the AEQ listeners.*
> > Anti-Goals
> >
> > None
> > *Solution*
> >
> > *The proposed solution is to provide a way to control dispatching AEQ
> > events to the AEQ Listeners, this could be done by adding "pause"  and
> > "resume" capability to the AEQ, which will allow application to decide
> when
> > to dispatch events to the listeners. *
> >
> >
> > *The proposal is similar to existing "pause" and "resume" behavior on the
> > GatewaySender, on which the AEQ is based on (AEQ implementation is a
> > wrapper around GatewaySender). *
> > Changes and Additions to Public Interfaces
> >
> > *The proposed APIs are:*
> >
> > *On "AsyncEventQueueFactory" interface - *
> >
> > *AsyncEventQueue pauseEventDispatchToListener();*
> >
> > *On "AsyncEventQueue" interface -*
> >
> > *boolean resumeEventDispatchToListener(); **returns true or false if the
> > event dispatch is resumed successfully.*
> >
> >
> > *The constraints on the pauseEventDispatchToListener() will remain
> similar
> > to as in "GatewaySender.pause()" :*
> >
> > "It should be kept in mind that the events will still be getting queued
> > into the queue. The scope of this operation is the VM on which it is
> > invoked. In case the AEQ is parallel, the AEQ will be paused on
> individual
> > node where this API is called and the AEQ on other VM's can still
> dispatch
> > events. In case the AEQ is not parallel, and the running AEQ on which
> this
> > API is invoked is not primary then primary AEQ will still continue
> > dispatching events."
> > Performance Impact
> >
> >
> > *This will have similar performance and resource implication as with the
> > "GatewaySender.pause()" functionality. If the AEQ is not resumed or kept
> in
> > "pause" state for long, it may start consuming the configured memory and
> > overflow it into disk and may cause disk full scenario.*
> > Backwards Compatibility and Upgrade Path
> >
> > *Impact with rolling upgrade: *
> >
> > *As the api is applicable at individual VM level, there is no message
> > serialization changes involved. And only applicable to the events getting
> > dispatched to the listeners on that VM. And the AEQ which are replicated
> > (for redundancy) continues to work as before.*
> >
> > *Backward compatibility requirements: *
> >
> > *None. The AEQs are configured and managed at the server side. There is
> no
> > messaging involved between client/server.*
> >
> > *Disk formatting changes:*
> >
> > *None.*
> >
> > *Deprecation and Application Changes:*
> >
> >
> > *None. If needed, the existing application can be modified to control
> event
> > dispatch with AEQ listener.*
> > Prior Art
> >
> > *Without this, the AEQ listeners operating on other regions could
> > experience missing events or dead lock, if there are co-located regions.*
> >
> > *This approach is simple and can take advantage of the existing
> > functionality that is already supported in GatewaySender on which AEQ is
> > based on.*
> >
>


Re: [DISCUSS] Controlling event dispatch to AsyncEventListener (review by Aug 22)

2019-08-16 Thread Dan Smith
Hi Anil,

While I like the idea of matching the API of GatewaySender, I'm not sure I
see how this solves the problem. Is it required of the user to call pause
on the AsyncEventQueue as soon as it is created? How would someone do that
when creating AEQs with xml or cluster configuration? Maybe it would be
better to not dispatch any events until we are done creating all regions?

-Dan

On Fri, Aug 16, 2019 at 2:31 PM Anilkumar Gingade 
wrote:

> Proposal to support controlling capability with event dispatch to
> AsyncEventQueue Listener.
>
> Wiki proposal page:
>
> https://cwiki.apache.org/confluence/display/GEODE/%5BDraft%5D+Controlling+event+dispatch+to+AsyncEventListener
>
> Here is the details from the wiki page:
> *Problem*
>
> *The Geode system requires AEQs to be configured before regions are
> created. If an AEQ listener is operating on a secondary region, this could
> cause listener to operate on a region which is not yet created or fully
> initialized (for region with co-located regions) which could result in
> missing events or dead-lock scenario between region (co-located region)
> creation threads. This scenario is likely to happen during persistence
> recovery; when AEQs are created in the start, the recovered AEQ events are
> dispatched immediately, thus invoking the AEQ listeners.*
> Anti-Goals
>
> None
> *Solution*
>
> *The proposed solution is to provide a way to control dispatching AEQ
> events to the AEQ Listeners, this could be done by adding "pause"  and
> "resume" capability to the AEQ, which will allow application to decide when
> to dispatch events to the listeners. *
>
>
> *The proposal is similar to existing "pause" and "resume" behavior on the
> GatewaySender, on which the AEQ is based on (AEQ implementation is a
> wrapper around GatewaySender). *
> Changes and Additions to Public Interfaces
>
> *The proposed APIs are:*
>
> *On "AsyncEventQueueFactory" interface - *
>
> *AsyncEventQueue pauseEventDispatchToListener();*
>
> *On "AsyncEventQueue" interface -*
>
> *boolean resumeEventDispatchToListener(); **returns true or false if the
> event dispatch is resumed successfully.*
>
>
> *The constraints on the pauseEventDispatchToListener() will remain similar
> to as in "GatewaySender.pause()" :*
>
> "It should be kept in mind that the events will still be getting queued
> into the queue. The scope of this operation is the VM on which it is
> invoked. In case the AEQ is parallel, the AEQ will be paused on individual
> node where this API is called and the AEQ on other VM's can still dispatch
> events. In case the AEQ is not parallel, and the running AEQ on which this
> API is invoked is not primary then primary AEQ will still continue
> dispatching events."
> Performance Impact
>
>
> *This will have similar performance and resource implication as with the
> "GatewaySender.pause()" functionality. If the AEQ is not resumed or kept in
> "pause" state for long, it may start consuming the configured memory and
> overflow it into disk and may cause disk full scenario.*
> Backwards Compatibility and Upgrade Path
>
> *Impact with rolling upgrade: *
>
> *As the api is applicable at individual VM level, there is no message
> serialization changes involved. And only applicable to the events getting
> dispatched to the listeners on that VM. And the AEQ which are replicated
> (for redundancy) continues to work as before.*
>
> *Backward compatibility requirements: *
>
> *None. The AEQs are configured and managed at the server side. There is no
> messaging involved between client/server.*
>
> *Disk formatting changes:*
>
> *None.*
>
> *Deprecation and Application Changes:*
>
>
> *None. If needed, the existing application can be modified to control event
> dispatch with AEQ listener.*
> Prior Art
>
> *Without this, the AEQ listeners operating on other regions could
> experience missing events or dead lock, if there are co-located regions.*
>
> *This approach is simple and can take advantage of the existing
> functionality that is already supported in GatewaySender on which AEQ is
> based on.*
>