So basically using new DoFn with *SplittableParDo*, *Window.Bound*, and *GroupAlsoByWindow* requires a custom implementation by per runner as they are not handled by DoFn anymore, right ?
On Sun, Dec 11, 2016 at 3:42 PM Eugene Kirpichov <kirpic...@google.com.invalid> wrote: > Hi Amit, I'll comment in more detail later, but meanwhile please take a > look at https://github.com/apache/incubator-beam/pull/1565 > There is a small amount of relevant changes to spark runner. > Take a look at implementation of SplittableParDo (already committed) in > particular ProcessFn and it's usage in direct runner - this is exactly what > you're looking for, a new DoFn that with per-runner support is able to emit > multi-windowed values. > On Sun, Dec 11, 2016 at 4:28 AM Amit Sela <amitsel...@gmail.com> wrote: > > > Hi all, > > > > I've been working on migrating the Spark runner to new DoFn and I've > > stumbled upon a couple of cases where OldDoFn is used in a way that > > accessed windowInternals (outputWindowedValue) such as AssignWindowsDoFn. > > > > Since changing windows is no longer the responsibility of DoFn I was > > wondering who and how is this done. > > > > Thanks, > > Amit > > >
