[ https://issues.apache.org/jira/browse/BEAM-9945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105926#comment-17105926 ]
Luke Cwik edited comment on BEAM-9945 at 5/13/20, 3:23 AM: ----------------------------------------------------------- When testing the implementation against runner v2, I found that the Go SDK's read index ends at *# elements* or the *stop index* when splitting while the Python/Java implementations always stop at *# elements - 1* or *stop index - 1* I think it makes sense to use Go's definition. was (Author: lcwik): When testing the implementation against runner v2, I found that the Go SDK's read index ends at *# elements* or the *stop index* when splitting while the Python/Java implements always stop at *# elements - 1* or *stop index - 1* I think it makes sense to use Go's definition. > Use consistent element count for progress counter. > -------------------------------------------------- > > Key: BEAM-9945 > URL: https://issues.apache.org/jira/browse/BEAM-9945 > Project: Beam > Issue Type: Improvement > Components: beam-model, sdk-go, sdk-java-harness, sdk-py-harness > Reporter: Robert Bradshaw > Assignee: Robert Bradshaw > Priority: Critical > Fix For: 2.21.0 > > Time Spent: 20m > Remaining Estimate: 0h > > This influences how the SDK communicates progress and how splitting is > performed. > We currently inspect the element count of the output (which has inconsistent > definitions across SDKs, see BEAM-9934). Instead, we can move to using an > explicit, separate metric. > This can currently lead to incorrect progress reporting and in some cases > even a crash in the UW depending on the SDK, and makes it more difficult (but > not impossible) to fix element counts in the future. -- This message was sent by Atlassian Jira (v8.3.4#803005)