Re: Beam SQL found limitations

2023-05-31 Thread Kenneth Knowles
1. Yes, using state is a better fit than Beam windowing. You will want to use the new feature of annotating the DoFn with @RequiresTimeSortedInput. This will make it so you can be sure you are actually getting the "previous" event. They can arrive in any order without this annotation. You won't be

Re: Beam SQL found limitations

2023-05-31 Thread Wiśniowski Piotr
Hi Kenn, Thanks for clarification. 1. Just to put an example in front - for every event that comes in I need to find corresponding previous event of same user_id and pass previous_event_timestamp in the current event payload down (and also current event becomes previous event for future event