Re: Slides from OSCON

2018-09-04 Thread Matthias Baetens
Hey Holden, just checking if you were able to find something? :) -M On Sat, Aug 25, 2018, 12:07 Holden Karau wrote: > Good question, I’ll take a look tonight :) > > On Sat, Aug 25, 2018 at 6:32 AM Matthias Baetens < > baetensmatth...@gmail.com> wrote: > >> Hey Holden, >> Have you found the video

Re: How to update side inputs.

2018-09-04 Thread Lukasz Cwik
Jose, what Bart is recommending is a path that should work. Bart, what do you mean by conflicting windows? On Mon, Sep 3, 2018 at 11:29 PM Bart Aelterman wrote: > Hi Jose, > > > You could generate a sequence of "ticks" and use that as input to > continuously update your side input. This is wha

Re: Slides from OSCON

2018-09-04 Thread Pablo Estrada
I found this video: https://www.safaribooksonline.com/videos/oscon-2018/9781492026075/9781492026075-video321550 Though it requires a suscription to Safari books. Best -P. On Tue, Sep 4, 2018 at 8:21 AM Matthias Baetens wrote: > Hey Holden, just checking if you were able to find something? :) > -

Re: Slides from OSCON

2018-09-04 Thread Holden Karau
I think I have permission to share it on YouTube, I'll ask the O'Reilly folks when I see them @ Strata next week :) On Tue, Sep 4, 2018 at 10:35 AM Pablo Estrada wrote: > I found this video: > https://www.safaribooksonline.com/videos/oscon-2018/9781492026075/9781492026075-video321550 > Though it

Re: How to update side inputs.

2018-09-04 Thread Bart Aelterman
Hi Lukasz, My solution yields a java.lang.IllegalArgumentException: Attempted to get side input window for GlobalWindow from non-global WindowFn I am now setting a fixedWindow on the ticks (as was shown in the stack overflow example) and a couple of lines later on the Map. I've tried removing one

Re: How to update side inputs.

2018-09-04 Thread Lukasz Cwik
Bart, the error you are hitting is because the other part of the pipeline is operating on a global window. Every time a side input is looked up in the DoFn, the main window (global window in your case) is mapped onto the side input window (a fixed window). There is no logical mapping from global w

Re: How to update side inputs.

2018-09-04 Thread Lukasz Cwik
Sorry, * If this is ok for you, then this is likely the easiest solution but if not you need to guarantee that the computations are happening with the updated side input data, you'll need to modify your triggers and pipeline to be based upon watermarks (suggestion #2). On Tue, Sep 4, 2018 at 1:4