Etienne Chauchot created BEAM-6740:
--------------------------------------
Summary: Combine.globally translation is never called
Key: BEAM-6740
URL: https://issues.apache.org/jira/browse/BEAM-6740
Project: Beam
Issue Type: Bug
Components: runner-spark
Reporter: Etienne Chauchot
SDK translates Combine.Globally as a composite transform composed of:
* Map that assigns Void keys
* Combine.PerKey
As Combine.Perkey uses a spark GBK inside it, the runner adds its own
translation of Combine.Globally to avoid less performant GBK. This translation
should be called in place of entering the composite transform translation.A
pipeline like this:
{code}
PCollection<Integer> input = pipeline.apply(Create.of(1, 2, 3, 4, 5, 6, 7, 8,
9, 10));
input.apply(
Combine.globally(new IntegerCombineFn()));
{code}
is translated as the above composite.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
