Re: Non-trivial joins examples

2020-05-04 Thread Marcin Kuthan
@Kenneth - thank for your response, for sure I was inspired a lot by earlier discussions on the group and latest documentation updates about Timers: https://beam.apache.org/documentation/programming-guide/#timers In the limitations I forgot to mention about SideInputs, it works quite well for scen

Re: Non-trivial joins examples

2020-05-03 Thread Reza Ardeshir Rokni
A couple of things that are really nice here, 1- Domain specific (CTR in your example). We may find that eventually it's not possible / practical to build out generic joins for all situations. But with the primitives available in Beam and good 'patterns' domain specific joins could be added for di

Re: Non-trivial joins examples

2020-05-01 Thread Jan Lukavský
Interestingly, I'm currently also working on a proposal for generic join semantics. I plan to send a proposal for review, but unfortunately, there are still other things keeping me busy. I take this opportunity to review high-level thoughts, maybe someone can give some points. The general idea

Re: Non-trivial joins examples

2020-05-01 Thread Kenneth Knowles
+dev @beam and some people who I talk about joins with Interesting! It is a lot to take in and fully grok the code, so calling in reinforcements... Generally, I think there's agreement that for a lot of real use cases, you have to roll your own join using the lower level Beam primitives. So I thi