If we need to add pause/resume processing to Cache, I suggest adding
setAutostart(boolean) to CacheFactory and start() to Cache to do something
like this:
Cache cache = new CacheFactory()
*.setAutostart(false)*
.create();
cache.createRegionFactory(...)...
cache.createAsyncEventQueueFactory(...)...
cache.addCacheServer()...
cache.*start()*;
AEQs, CacheServers, GatewaySenders/Receivers and any other endpoints that
are created would not be started until invoking cache.start().
Cheers,
Kirk
On Tue, Aug 20, 2019 at 2:01 PM Nabarun Nag <n...@apache.org> wrote:
> Hi Anil,
>
> Will it be possible to explain to the community how the starting AEQ in a
> paused state is different from creating gateway senders with manual start
> set to true. It may be of concern as 'manual start' for gateways is a
> deprecated.
>
> Just thinking out loud, will it be more feasible if we can set the flag at
> cache level. Any framework that is starting up Apache Geode (E.g: Spring) ,
> creates the cache -> cache.pauseProcessing(); -> create regions -> create
> AEQs -> cache.unpauseProcessing()
>
> We can gate the processing of all event listener at dispatchBatch().
>
> The advantage I feel is that
> - we avoid introducing a new API to the AEQ creation factory.
> - if we created 100 AEQs in paused state then we avoid having to have 100
> AEQ.unpause calls.
>
>
> Regards
> Naba
>
>
> On Tue, Aug 20, 2019 at 9:07 AM Michael Stolz <mst...@pivotal.io> wrote:
>
> > Manual start has caused a lot of trouble over the years. We should
> > definitely circle back on those issues before traveling very far down
> this
> > road.
> >
> > --
> > Mike Stolz
> > Principal Engineer, Pivotal Cloud Cache
> > Mobile: +1-631-835-4771
> >
> >
> >
> > On Tue, Aug 20, 2019 at 11:56 AM Juan José Ramos <jra...@pivotal.io>
> > wrote:
> >
> > > Hello Anil,
> > >
> > > +1 for the proposed solution.
> > > I'd change the method name from *pauseEventDispatchToListener* to
> > something
> > > more meaningful and understandable for our users, maybe *startPaused*?,
> > > *setManualStart* (as we currently have for the
> *GatewaySenderFactory*)?,
> > > *startWithEventDispatcherPaused*?.
> > > Best regards.
> > >
> > >
> > >
> > > On Sat, Aug 17, 2019 at 12:55 AM Anilkumar Gingade <
> aging...@pivotal.io>
> > > wrote:
> > >
> > > > I have updated the wiki based on Dan's comment.
> > > > Changes with api:
> > > >
> > > > *On "AsyncEventQueueFactory" interface - *
> > > >
> > > > *AsyncEventQueueFactory pauseEventDispatchToListener(); *// This
> > causes
> > > > AEQ to be created with paused state.
> > > >
> > > >
> > > >
> > > >
> > > > On Fri, Aug 16, 2019 at 4:36 PM Anilkumar Gingade <
> aging...@pivotal.io
> > >
> > > > wrote:
> > > >
> > > > > Dan,
> > > > >
> > > > > If you look into the API; the AEQ will be created with the pause
> > state.
> > > > > The user (application) has to call resume to dispatch the events.
> > > > >
> > > > > It will be slightly different from GatewaySender behavior; where
> > > > > GatewaySender will be created with run mode and then application
> has
> > to
> > > > > call pause on it. Here in this case AEQ will be created with paused
> > > > state.
> > > > >
> > > > > -Anil.
> > > > >
> > > > >
> > > > > On Fri, Aug 16, 2019 at 4:31 PM Dan Smith <dsm...@pivotal.io>
> wrote:
> > > > >
> > > > >> Hi Anil,
> > > > >>
> > > > >> While I like the idea of matching the API of GatewaySender, I'm
> not
> > > > sure I
> > > > >> see how this solves the problem. Is it required of the user to
> call
> > > > pause
> > > > >> on the AsyncEventQueue as soon as it is created? How would someone
> > do
> > > > that
> > > > >> when creating AEQs with xml or cluster configuration? Maybe it
> would
> > > be
> > > > >> better to not dispatch any events until we are done creating all
> > > > regions?
> > > > >>
> > > > >> -Dan
> > > > >>
> > > > >> On Fri, Aug 16, 2019 at 2:31 PM Anilkumar Gingade <
> > > aging...@pivotal.io>
> > > > >> wrote:
> > > > >>
> > > > >> > Proposal to support controlling capability with event dispatch
> to
> > > > >> > AsyncEventQueue Listener.
> > > > >> >
> > > > >> > Wiki proposal page:
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/GEODE/%5BDraft%5D+Controlling+event+dispatch+to+AsyncEventListener
> > > > >> >
> > > > >> > Here is the details from the wiki page:
> > > > >> > *Problem*
> > > > >> >
> > > > >> > *The Geode system requires AEQs to be configured before regions
> > are
> > > > >> > created. If an AEQ listener is operating on a secondary region,
> > this
> > > > >> could
> > > > >> > cause listener to operate on a region which is not yet created
> or
> > > > fully
> > > > >> > initialized (for region with co-located regions) which could
> > result
> > > in
> > > > >> > missing events or dead-lock scenario between region (co-located
> > > > region)
> > > > >> > creation threads. This scenario is likely to happen during
> > > persistence
> > > > >> > recovery; when AEQs are created in the start, the recovered AEQ
> > > events
> > > > >> are
> > > > >> > dispatched immediately, thus invoking the AEQ listeners.*
> > > > >> > Anti-Goals
> > > > >> >
> > > > >> > None
> > > > >> > *Solution*
> > > > >> >
> > > > >> > *The proposed solution is to provide a way to control
> dispatching
> > > AEQ
> > > > >> > events to the AEQ Listeners, this could be done by adding
> "pause"
> > > and
> > > > >> > "resume" capability to the AEQ, which will allow application to
> > > decide
> > > > >> when
> > > > >> > to dispatch events to the listeners. *
> > > > >> >
> > > > >> >
> > > > >> > *The proposal is similar to existing "pause" and "resume"
> behavior
> > > on
> > > > >> the
> > > > >> > GatewaySender, on which the AEQ is based on (AEQ implementation
> > is a
> > > > >> > wrapper around GatewaySender). *
> > > > >> > Changes and Additions to Public Interfaces
> > > > >> >
> > > > >> > *The proposed APIs are:*
> > > > >> >
> > > > >> > *On "AsyncEventQueueFactory" interface - *
> > > > >> >
> > > > >> > *AsyncEventQueue pauseEventDispatchToListener();*
> > > > >> >
> > > > >> > *On "AsyncEventQueue" interface -*
> > > > >> >
> > > > >> > *boolean resumeEventDispatchToListener(); **returns true or
> false
> > if
> > > > the
> > > > >> > event dispatch is resumed successfully.*
> > > > >> >
> > > > >> >
> > > > >> > *The constraints on the pauseEventDispatchToListener() will
> remain
> > > > >> similar
> > > > >> > to as in "GatewaySender.pause()" :*
> > > > >> >
> > > > >> > "It should be kept in mind that the events will still be getting
> > > > queued
> > > > >> > into the queue. The scope of this operation is the VM on which
> it
> > is
> > > > >> > invoked. In case the AEQ is parallel, the AEQ will be paused on
> > > > >> individual
> > > > >> > node where this API is called and the AEQ on other VM's can
> still
> > > > >> dispatch
> > > > >> > events. In case the AEQ is not parallel, and the running AEQ on
> > > which
> > > > >> this
> > > > >> > API is invoked is not primary then primary AEQ will still
> continue
> > > > >> > dispatching events."
> > > > >> > Performance Impact
> > > > >> >
> > > > >> >
> > > > >> > *This will have similar performance and resource implication as
> > with
> > > > the
> > > > >> > "GatewaySender.pause()" functionality. If the AEQ is not resumed
> > or
> > > > >> kept in
> > > > >> > "pause" state for long, it may start consuming the configured
> > memory
> > > > and
> > > > >> > overflow it into disk and may cause disk full scenario.*
> > > > >> > Backwards Compatibility and Upgrade Path
> > > > >> >
> > > > >> > *Impact with rolling upgrade: *
> > > > >> >
> > > > >> > *As the api is applicable at individual VM level, there is no
> > > message
> > > > >> > serialization changes involved. And only applicable to the
> events
> > > > >> getting
> > > > >> > dispatched to the listeners on that VM. And the AEQ which are
> > > > replicated
> > > > >> > (for redundancy) continues to work as before.*
> > > > >> >
> > > > >> > *Backward compatibility requirements: *
> > > > >> >
> > > > >> > *None. The AEQs are configured and managed at the server side.
> > There
> > > > is
> > > > >> no
> > > > >> > messaging involved between client/server.*
> > > > >> >
> > > > >> > *Disk formatting changes:*
> > > > >> >
> > > > >> > *None.*
> > > > >> >
> > > > >> > *Deprecation and Application Changes:*
> > > > >> >
> > > > >> >
> > > > >> > *None. If needed, the existing application can be modified to
> > > control
> > > > >> event
> > > > >> > dispatch with AEQ listener.*
> > > > >> > Prior Art
> > > > >> >
> > > > >> > *Without this, the AEQ listeners operating on other regions
> could
> > > > >> > experience missing events or dead lock, if there are co-located
> > > > >> regions.*
> > > > >> >
> > > > >> > *This approach is simple and can take advantage of the existing
> > > > >> > functionality that is already supported in GatewaySender on
> which
> > > AEQ
> > > > is
> > > > >> > based on.*
> > > > >> >
> > > > >>
> > > > >
> > > >
> > >
> > >
> > > --
> > > Juan José Ramos Cassella
> > > Senior Software Engineer
> > > Email: jra...@pivotal.io
> > >
> >
>
