Re: Documenting per-key delivery semantics for various runners

2021-09-24 Thread Kenneth Knowles
Took a look. I definitely agree that something like this is useful, and well-motivated by the use cases you raise. Kenn On Thu, Sep 23, 2021 at 4:30 PM Pablo Estrada wrote: > Hi all, > I've been spending some time thinking about CDC use cases on Beam. One > valuable piece to enable these use ca

Re: Documenting per-key delivery semantics for various runners

2021-09-25 Thread Jan Lukavský
+1 to adding a Pipeline requirement for this, if business logic relies on a specific feature runner might/might not have, then Pipeline should be rejected on runners that do not support it. Do we have a list runners that have or lack this semantics? Just for clarification - sorry my ignorance,

Re: Documenting per-key delivery semantics for various runners

2021-09-29 Thread Jan Lukavský
Hi Pablo, thanks for the motivating examples. I understand the motivation now, but one question comes to mind - we do not mind the case, when in-between the emitting PTransform and the comsuming PTransform is another (grouping) PTransform which _changes the key_? Via the change of key, two el