[ https://issues.apache.org/jira/browse/BEAM-9945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luke Cwik reopened BEAM-9945: ----------------------------- When testing the implementation against runner v2, I found that the Go SDK's read index ends at *# elements* or the *stop index* when splitting while the Python/Java implements always stop at *# elements - 1* or *stop index - 1* I think it makes sense to use Go's definition. > Use consistent element count for progress counter. > -------------------------------------------------- > > Key: BEAM-9945 > URL: https://issues.apache.org/jira/browse/BEAM-9945 > Project: Beam > Issue Type: Improvement > Components: beam-model, sdk-go, sdk-java-harness, sdk-py-harness > Reporter: Robert Bradshaw > Assignee: Robert Bradshaw > Priority: Critical > Fix For: 2.21.0 > > Time Spent: 20m > Remaining Estimate: 0h > > This influences how the SDK communicates progress and how splitting is > performed. > We currently inspect the element count of the output (which has inconsistent > definitions across SDKs, see BEAM-9934). Instead, we can move to using an > explicit, separate metric. > This can currently lead to incorrect progress reporting and in some cases > even a crash in the UW depending on the SDK, and makes it more difficult (but > not impossible) to fix element counts in the future. -- This message was sent by Atlassian Jira (v8.3.4#803005)