Vladimir, Communication should stop reading from connection is there are too many unprocessed messages. Sender will be blocked on putting message to queue.
--Yakov 2016-03-30 11:11 GMT+03:00 Vladimir Ozerov <voze...@gridgain.com>: > Guys, > > Can you explain how backpressure control is implemented? What if event > arrival speed is greater than filter processing speed? > > On Wed, Mar 30, 2016 at 10:37 AM, Semyon Boikov <sboi...@gridgain.com> > wrote: > > > Andrey, > > > > I agree that current situation with threading in Ignite is very > > inconvenient when user callbacks execute some non-trivial code. But > > changing this to async dispatch is huge refactoring, even changing this > > just for continuous queries callback is not so easy task. > > > > We can start with https://issues.apache.org/jira/browse/IGNITE-2004, and > > if > > more users complains arise we can think about changing others parts of > > system. > > > > For now we need decisions for these points: > > - how to specify that callback should be run asynchronously (Nikolay > > suggested marker interface IgniteAsyncCallback, or @IgniteAsyncCallback) > > - where these callbacks are executed, AFAIK Nikolay added special pool > > which is configured in IgniteConfiguration (something like > > IgniteConfiguration.asyncCallbackThreadPoolSize) > > > > Regards > > > > > > On Tue, Mar 29, 2016 at 10:45 PM, Andrey Kornev < > andrewkor...@hotmail.com> > > wrote: > > > > > Vladimir, Igniters > > > > > > Here are my 2 cents. > > > > > > The current situation with threading when it comes to executing user > > > callbacks -- the CQ filters (either local or remote), the CQ listeners, > > the > > > event listeners, the messaging listeners, the entry processors (did I > > miss > > > anything?) -- is pretty sad. The callbacks may get executed on a system > > > pool's thread, public pool's, utility pool's, discovery worker thread, > > > application thread, to name a few. It causes a lot of grief and > > suffering, > > > hard-to-fix races, dead locks and other bugs. > > > > > > I guess it's always possible to come up with a more or less reasonable > > > explanation to such predicament (which usually boils down to "It is so > > > because this is how it's implemented"), but I, as a user, could not > care > > > less. I want consistency. I want all my callbacks (including Entry > > > Processors!) to be executed on the public pool's threads, to be > precise. > > > This is not the first time I complain about this, and I really think > it's > > > time to fix this mess. > > > > > > For a good example of how to implement ordered async dispatch of > > callbacks > > > on large scale, one only needs to look at Akka (or Reactor > > > https://github.com/reactor/reactor). Coherence also managed to get it > > > right (in my opinion, that is). > > > > > > Regards > > > Andrey > > > > > > > > >
