Re: ParDo(DoFn) with multiple context.output vs FlatMapElements

2024-01-26 Thread Robert Bradshaw via user
There is no difference; FlatMapElements is implemented in terms of a DoFn that invokes context.output multiple times. And, yes, Dataflow will fuse consecutive operations automatically. So if you have something like ... -> DoFnA -> DoFnB -> GBK -> DoFnC -> ... Dataflow will fuse DoFnA and DoFnB to

ParDo(DoFn) with multiple context.output vs FlatMapElements

2023-12-27 Thread hsy...@gmail.com
Hello I have a question. If I have a transform for each input it will emit 1 or many output (same collection) I can do it with ParDo + DoFun while in processElement method for each input, call context.output multiply times vs doing it with FlatMapElements, is there any difference? Does the dataflo