[
https://issues.apache.org/jira/browse/FLINK-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhu Zhu closed FLINK-15325.
---------------------------
Fix Version/s: 1.17.0
Resolution: Done
Done via
c0bfb0b04bb38411c390813fea1ff93f6638f409
76ebeb93257369c136a9eeabd5bacb71a9699968
> Input location preference which affects task distribution may make certain
> job performance worse
> -------------------------------------------------------------------------------------------------
>
> Key: FLINK-15325
> URL: https://issues.apache.org/jira/browse/FLINK-15325
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Affects Versions: 1.16.0, 1.14.6, 1.15.3
> Reporter: Zhu Zhu
> Assignee: Zhu Zhu
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.17.0
>
> Attachments: D58ADB03-7187-46B1-B077-91E5005FD463.png
>
>
> When running TPC-DS jobs in a session cluster, we observed that sometimes
> tasks are not evenly distributed in TMs. The root cause turned out to be that
> the downstream tasks tend to be TM or host local with its input tasks. This
> helps to reduce network shuffle.
> However, in certain cases, like the topology presented in the attached image,
> jamming the input task's TM and machine with downstream tasks would affect
> the performance. In this case, respecting input location preferences is
> causing troubles more than bringing benefits.
> So I'm wondering whether we should introduce a config so that users can
> disable input location preferences?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
