Re: fliter and flatMap operation VS only a flatMap operation

2020-01-29 Thread Tzu-Li (Gordon) Tai
Hi, If your filter and flatMap operators are chained, then the performance difference should not be noticeable. If a shuffle (i.e. a keyBy operation) occurs after the filter and before the flatMap, then applying the filter first will be more efficient. Cheers, Gordon On Thu, Jan 30, 2020 at

fliter and flatMap operation VS only a flatMap operation

2020-01-29 Thread Soheil Pourbafrani
Hi, In case we need to filter operation followed by a transformation, which one is more efficient in Flink, applying the filter operation first and then a flatMap operation separately OR using only a flatMap operation that internally includes the filter logic, too? best Soheil