[ https://issues.apache.org/jira/browse/BEAM-9945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105937#comment-17105937 ]
Luke Cwik commented on BEAM-9945: --------------------------------- Fix in https://github.com/apache/beam/pull/11689 > Use consistent element count for progress counter. > -------------------------------------------------- > > Key: BEAM-9945 > URL: https://issues.apache.org/jira/browse/BEAM-9945 > Project: Beam > Issue Type: Improvement > Components: beam-model, sdk-go, sdk-java-harness, sdk-py-harness > Reporter: Robert Bradshaw > Assignee: Robert Bradshaw > Priority: Critical > Fix For: 2.21.0 > > Time Spent: 20m > Remaining Estimate: 0h > > This influences how the SDK communicates progress and how splitting is > performed. > We currently inspect the element count of the output (which has inconsistent > definitions across SDKs, see BEAM-9934). Instead, we can move to using an > explicit, separate metric. > This can currently lead to incorrect progress reporting and in some cases > even a crash in the UW depending on the SDK, and makes it more difficult (but > not impossible) to fix element counts in the future. -- This message was sent by Atlassian Jira (v8.3.4#803005)