Correct, however I think our triggering model is close to useless (or at least close to unusable) without such a guarantee, for both accumulating and discarding. What's worse - AFAIK all streaming runners today practically do provide these panes in order; this means that likely many users implicitly rely on this "non guarantee," probably without even knowing they are relying on it.
On Wed, Jun 26, 2019 at 10:02 PM Robert Bradshaw <rober...@google.com> wrote: > There is no promise that panes will arrive in order (especially the > further you get "downstream"). Though they may be approximately so, > it's dangerous to assume that. You can inspect the sequential index in > PaneInfo to determine whether a pane is older than other panes you > have seen. > > On Wed, Jun 26, 2019 at 7:03 PM Rui Wang <ruw...@google.com> wrote: > > > > Hi Community, > > > > I am trying to understand Beam model and having a question related to > accumulating mode and panes: > > > > Accumulating mode means every time when a trigger fires, it emits all > values seen so far in a window(so it's called accumulating), an example > from Beam programming model guide[1] sets a repeating order has a repeating > trigger that fires every time 3 elements arrive on a 10-min fixed windowing > with the following emitted results: > > > > First trigger firing: [5, 8, 3] > > Second trigger firing: [5, 8, 3, 15, 19, 23] > > Third trigger firing: [5, 8, 3, 15, 19, 23, 9, 13, 10] > > > > > > The original dataflow paper[2] also mentions that accumulating mode is > useful to downstream consumers to overwrite old value with new value. > > > > In order to help such "overwriting" use case, seems to me that Beam > model provides: > > 1. triggers fire in order. In the example above, second trigger firing > should after first trigger firing such that downstream transforms should > see [5, 8, 3] before [5, 8, 3, 15, 19, 23]. > > 2. downstream transforms execute panes in order. If this is not true, it > might end with that new values(from later panes) are overwritten by old > values(earlier panes) > > > > Do I have a correct understanding? > > > > > > Thanks, > > Rui > > > > > > > > [1]: > https://beam.apache.org/documentation/programming-guide/#setting-a-trigger > > [2]: https://www.vldb.org/pvldb/vol8/p1792-Akidau.pdf >
