Hi Amit, I'll comment in more detail later, but meanwhile please take a look at https://github.com/apache/incubator-beam/pull/1565 There is a small amount of relevant changes to spark runner. Take a look at implementation of SplittableParDo (already committed) in particular ProcessFn and it's usage in direct runner - this is exactly what you're looking for, a new DoFn that with per-runner support is able to emit multi-windowed values. On Sun, Dec 11, 2016 at 4:28 AM Amit Sela <amitsel...@gmail.com> wrote:
> Hi all, > > I've been working on migrating the Spark runner to new DoFn and I've > stumbled upon a couple of cases where OldDoFn is used in a way that > accessed windowInternals (outputWindowedValue) such as AssignWindowsDoFn. > > Since changing windows is no longer the responsibility of DoFn I was > wondering who and how is this done. > > Thanks, > Amit >
