Hi,
We are using Flink as a timer scheduler and delay in timer execution is
a huge problem for us. What we have experienced is that as the number of
Timers we register increases the timers start getting delayed (for more
than 5 seconds). Can anyone point us in the right direction to figure
out wha
I have a couple of questions related to this:
1. We store state per key (Rocksdb backend). Currently, the state size
is ~1.5Gb. Checkpointing time sometimes reaches ~10-20 seconds. Is it
possible that checkpointing is affecting timer execution?
2. Does checkpointing cause Flink to stop consumption
Checkpoints are largely asynchronous, but the checkpointing of timers has
some synchronous component (which we are currently working on getting rid
of).
So when you have a lot of timers, streams stall for a short time while the
timers are checkpointed. If all goes as planned, Flink 1.6 will not hav
It is true that onTimer and processElement are never called at the same
time.
I'm not entirely sure whether there is any prioritization/fairness
between these methods
(if not if could be that onTimer is starved) , looping in Aljoscha who
hopefully knows more
about this.
On 10.09.2017 09:31,
Hi,
Yes, execution of these methods is protected by a synchronized block. This is
not a fair lock so incoming data might starve timer callbacks. What is the
number of timers we are talking about here?
Best,
Aljoscha
> On 11. Sep 2017, at 19:38, Chesnay Schepler wrote:
>
> It is true that onT
The number of timers is about 400 per second. We have observed that onTimer
calls are delayed only when the number of scheduled timers starts
increasing from a minima. It would be great if you can share pointers to
code I can look at to understand it better. :)
Narendra Joshi
On 14 Sep 2017 16:04,
