[ https://issues.apache.org/jira/browse/BEAM-9794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17088812#comment-17088812 ]
Maximilian Michels commented on BEAM-9794: ------------------------------------------ That's also what I found when I started working on this. Instead, I went for a fixed number of buffers which are rotated: https://github.com/apache/beam/pull/11478 > Flink pipeline with RequiresStableInput fails after Short.MAX_VALUE > checkpoints. > -------------------------------------------------------------------------------- > > Key: BEAM-9794 > URL: https://issues.apache.org/jira/browse/BEAM-9794 > Project: Beam > Issue Type: Bug > Components: runner-flink > Affects Versions: 2.14.0, 2.15.0, 2.16.0, 2.17.0, 2.18.0, 2.19.0, 2.20.0 > Reporter: David Morávek > Assignee: Maximilian Michels > Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Full original report: > https://lists.apache.org/thread.html/rb2ebfad16d85bcf668978b3defd442feda0903c20db29c323497a672%40%3Cuser.beam.apache.org%3E > The exception comes from: > https://github.com/apache/flink/blob/release-1.8.0/flink-runtime/src/main/java/org/apache/flink/runtime/state/OperatorBackendSerializationProxy.java#L68 > In the Flink Runner code, each checkpoint results in a new OperatorState (or > KeyedState if the stream is keyed): > https://github.com/apache/beam/blob/v2.14.0/runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/stableinput/BufferingDoFnRunner.java#L91-L103 > https://github.com/apache/beam/blob/v2.14.0/runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/stableinput/BufferingDoFnRunner.java#L136-L143 -- This message was sent by Atlassian Jira (v8.3.4#803005)