[
https://issues.apache.org/jira/browse/BEAM-8212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212475#comment-17212475
]
Kenneth Knowles commented on BEAM-8212:
---------------------------------------
Makes sense. Note that {{StatefulDoFnRunner}} is not any definition of the Beam
model, but just utility code that a runner may optionally use. So it makes
sense to remove the GC timer setting for situations where it is global window,
with documentation that the runner must still call OnWindowExpiration (if
needed) in a drain scenario.
Users who were working before we had OnWindowExpiration will encounter problems
since we will not special case their timers.
> StatefulParDoFn creates GC timers for every record
> ---------------------------------------------------
>
> Key: BEAM-8212
> URL: https://issues.apache.org/jira/browse/BEAM-8212
> Project: Beam
> Issue Type: Bug
> Components: runner-core
> Reporter: Akshay Iyangar
> Priority: P3
>
> Hi
> So currently the StatefulParDoFn create timers for all the records.
> [https://github.com/apache/beam/blob/master/runners/core-java/src/main/java/org/apache/beam/runners/core/StatefulDoFnRunner.java#L211]
> This becomes a problem if you are using GlobalWindows for streaming where
> these timers get created and never get closed since the window will never
> close.
> This is a problem especially if your memory bound in rocksDB where these
> timers take up potential space and sloe the pipelines considerably.
> Was wondering that if the pipeline runs in global windows we should avoid
> adding timers to it at all?
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
