There is no promise that panes will arrive in order (especially the further you get "downstream"). Though they may be approximately so, it's dangerous to assume that. You can inspect the sequential index in PaneInfo to determine whether a pane is older than other panes you have seen.
On Wed, Jun 26, 2019 at 7:03 PM Rui Wang <[email protected]> wrote: > > Hi Community, > > I am trying to understand Beam model and having a question related to > accumulating mode and panes: > > Accumulating mode means every time when a trigger fires, it emits all values > seen so far in a window(so it's called accumulating), an example from Beam > programming model guide[1] sets a repeating order has a repeating trigger > that fires every time 3 elements arrive on a 10-min fixed windowing with the > following emitted results: > > First trigger firing: [5, 8, 3] > Second trigger firing: [5, 8, 3, 15, 19, 23] > Third trigger firing: [5, 8, 3, 15, 19, 23, 9, 13, 10] > > > The original dataflow paper[2] also mentions that accumulating mode is useful > to downstream consumers to overwrite old value with new value. > > In order to help such "overwriting" use case, seems to me that Beam model > provides: > 1. triggers fire in order. In the example above, second trigger firing should > after first trigger firing such that downstream transforms should see [5, 8, > 3] before [5, 8, 3, 15, 19, 23]. > 2. downstream transforms execute panes in order. If this is not true, it > might end with that new values(from later panes) are overwritten by old > values(earlier panes) > > Do I have a correct understanding? > > > Thanks, > Rui > > > > [1]: > https://beam.apache.org/documentation/programming-guide/#setting-a-trigger > [2]: https://www.vldb.org/pvldb/vol8/p1792-Akidau.pdf
