Makes sense. At least for accumulating mode, maintaining pane ordering
cross stages will be very useful but it is indeed difficult to do so.
Now I can see why trigger at sinks might be a better approach.
-Rui
On Thu, Jun 27, 2019 at 9:35 AM Reuven Lax wrote:
>
>
> On Thu, Jun 27, 2019 at 3:
On Thu, Jun 27, 2019 at 3:32 AM Robert Bradshaw wrote:
> On Thu, Jun 27, 2019 at 1:52 AM Rui Wang wrote:
> >>
> >>
> >> AFAIK all streaming runners today practically do provide these panes
> in order;
> >
> > Does it refer to "the stage immediately after GBK itself processes fired
> panes in or
On Thu, Jun 27, 2019 at 1:52 AM Rui Wang wrote:
>>
>>
>> AFAIK all streaming runners today practically do provide these panes in
>> order;
>
> Does it refer to "the stage immediately after GBK itself processes fired
> panes in order" in streaming runners? Could you share more information?
>
>
>
>
>
> AFAIK all streaming runners today practically do provide these panes in
> order;
>
Does it refer to "the stage immediately after GBK itself processes fired
panes in order" in streaming runners? Could you share more information?
> this means that likely many users implicitly rely on this "
Correct, however I think our triggering model is close to useless (or at
least close to unusable) without such a guarantee, for both accumulating
and discarding. What's worse - AFAIK all streaming runners today
practically do provide these panes in order; this means that likely many
users implicitl
Thanks! That thread was really helpful!
-Rui
On Wed, Jun 26, 2019 at 1:18 PM Steve Niemitz wrote:
> There was a thread about this a few months ago as well:
>
> https://lists.apache.org/thread.html/20d11046d26174969ef44a781e409a1cb9f7c736e605fa40fdf98397@%3Cuser.beam.apache.org%3E
>
>
> On Wed,
There was a thread about this a few months ago as well:
https://lists.apache.org/thread.html/20d11046d26174969ef44a781e409a1cb9f7c736e605fa40fdf98397@%3Cuser.beam.apache.org%3E
On Wed, Jun 26, 2019 at 4:02 PM Robert Bradshaw wrote:
> There is no promise that panes will arrive in order (especial
There is no promise that panes will arrive in order (especially the
further you get "downstream"). Though they may be approximately so,
it's dangerous to assume that. You can inspect the sequential index in
PaneInfo to determine whether a pane is older than other panes you
have seen.
On Wed, Jun 2
Hi Community,
I am trying to understand Beam model and having a question related to
accumulating mode and panes:
Accumulating mode means every time when a trigger fires, it emits all
values seen so far in a window(so it's called accumulating), an example
from Beam programming model guide[1] sets
