[
https://issues.apache.org/jira/browse/BEAM-2402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16034187#comment-16034187
]
Aljoscha Krettek commented on BEAM-2402:
----------------------------------------
[~pei...@gmail.com], what do you mean by that timer scan? Flink keeps the
timer's in a {{PriorityQueue}} by timestamp (on heap, which comes with it's on
set of problems that we're hoping to address soon) and when a watermark comes
(or processing time advances) the queue is worked off. So yes, if all keys set
a timer for timestamp T and that time T comes then the system will fire all
those timers at once (still serially). I think this is a "problem" of all timer
implementations, though.
> Support AfterPane.elementGapAtMost() trigger and its combination with
> elementCountAtLeast()
> -------------------------------------------------------------------------------------------
>
> Key: BEAM-2402
> URL: https://issues.apache.org/jira/browse/BEAM-2402
> Project: Beam
> Issue Type: New Feature
> Components: sdk-java-core
> Reporter: Pei He
> Assignee: Pei He
>
> We need a timestamp-driven trigger to use as a cheaper (or more efficient)
> version of the ProcessingTime trigger.
> The problem of using ProcessingTime trigger is that current runners' supports
> are not very efficient, and couldn't work for pipelines that have lots of
> keys (for example, flink runner will scan timers for all keys when watermark
> advance).
> We have used AfterPane.elementGapAtMost() trigger in our production, and want
> to merge it back. And, we believe it could be the solution for people who
> have the similar issue.
> Implementation for reference:
> https://github.com/apache/beam/compare/master...peihe:custom-after-pane?expand=1
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
