Robert Burke created BEAM-11084:
-----------------------------------
Summary: [Go SDK] Validate Windowing
Key: BEAM-11084
URL: https://issues.apache.org/jira/browse/BEAM-11084
Project: Beam
Issue Type: Task
Components: sdk-go
Reporter: Robert Burke
While Windowing basics are implemented in the Go SDK, there is precious little
validation that it's correct. Much work has been done largely for the benefit
of Global Windowing.
DoFn Authors can specify event times for elements by returning a
`beam.EventTime` along with their element, allowing windowing transforms by the
runner to take effect. They can observe windows in their DoFns by having a
`[]beam.Window` parameter, and extract results.
Pipeline authors can insert `beam.WindowInto` transforms to have elements be
batched into the appropriate window slices.
But very little of this has been validated end to end. This umbrella task is
to test and validate specific aspects of the window implementation to ensure
correctness.
The tricky part is that windows are selected by runners, meaning many of these
must be integration tests.
* Windows are correctly observed in DoFns.
* Validate windows are aggregated correctly in GBKs
* Validate that multi-window lifted combiners
** In particular sliding windows where a single element could contribute to
multiple aggregates
* Validate Side Input behavior WRT windows (implicit observation of windows)
** In particular, Side Inputs are valid per window.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
