[
https://issues.apache.org/jira/browse/FLINK-30242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641636#comment-17641636
]
Aitozi commented on FLINK-30242:
--------------------------------
> which scenario do you meet this problem?
I met this when we ran the batch job with flink-1.15. There is an unnecessary
shuffle between the map node and local hash aggregate, and this will impact the
performance.
It's not an urgent need. We have already added a rule to support the local hash
aggregate and union transpose. And it's a common optimization rule. So, it
would be nice to contribute it upstream.
If you have already done the work, you can take this ticket. If not, I can
prepare a PR for this.
BTW, I post two images to describe our use case problem
Thanks.
> Push localHashAggregate pass the union node
> -------------------------------------------
>
> Key: FLINK-30242
> URL: https://issues.apache.org/jira/browse/FLINK-30242
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
> Reporter: Aitozi
> Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>
> The local hash aggregate after union will have an extra shuffle stage. We can
> swap it with the union node so the local hash aggregate can chain with the
> mapper stage saving the unnecessary shuffle, especially in the batch job.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
