I posted a PoC PR [1] for fixing deferred side inputs with combiners in the
python SDK. Would someone be willing to take a look at it?

I have it working but could use some feedback on where to take it next. It
looks like bundle processor combiner operations don't currently support
side inputs [2] so I added a conditional in `CombinePerKey` that checks
whether it was instantiated with a side input and if so, use a ParDo-based
version of the combiner so we can piggyback off of the Do operations
implementation of side inputs rather than reimplementing it for the
combiner operation.

[1] https://github.com/apache/beam/pull/30743
[2]
https://github.com/apache/beam/blob/e3fee5156b3515f96dc5ba90ea2fbc6f6be2bd34/sdks/python/apache_beam/runners/worker/operations.py#L1146

Reply via email to