[jira] (FLINK-15325) Input location preference which affects task distribution may make certain job performance worse
[ https://issues.apache.org/jira/browse/FLINK-15325 ] Zhu Zhu deleted comment on FLINK-15325: - was (Author: flink-jira-bot): This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Input location preference which affects task distribution may make certain > job performance worse > - > > Key: FLINK-15325 > URL: https://issues.apache.org/jira/browse/FLINK-15325 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.10.0 >Reporter: Zhu Zhu >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor > Attachments: D58ADB03-7187-46B1-B077-91E5005FD463.png > > > When running TPC-DS jobs in a session cluster, we observed that sometimes > tasks are not evenly distributed in TMs. The root cause turned out to be that > the downstream tasks tend to be TM or host local with its input tasks. This > helps to reduce network shuffle. > However, in certain cases, like the topology presented in the attached image, > jamming the input task's TM and machine with downstream tasks would affect > the performance. In this case, respecting input location preferences is > causing troubles more than bringing benefits. > So I'm wondering whether we should introduce a config so that users can > disable input location preferences? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] (FLINK-15325) Input location preference which affects task distribution may make certain job performance worse
[ https://issues.apache.org/jira/browse/FLINK-15325 ] Zhu Zhu deleted comment on FLINK-15325: - was (Author: flink-jira-bot): This issue was labeled "stale-major" 7 ago and has not received any updates so it is being deprioritized. If this ticket is actually Major, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Input location preference which affects task distribution may make certain > job performance worse > - > > Key: FLINK-15325 > URL: https://issues.apache.org/jira/browse/FLINK-15325 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.10.0 >Reporter: Zhu Zhu >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor > Attachments: D58ADB03-7187-46B1-B077-91E5005FD463.png > > > When running TPC-DS jobs in a session cluster, we observed that sometimes > tasks are not evenly distributed in TMs. The root cause turned out to be that > the downstream tasks tend to be TM or host local with its input tasks. This > helps to reduce network shuffle. > However, in certain cases, like the topology presented in the attached image, > jamming the input task's TM and machine with downstream tasks would affect > the performance. In this case, respecting input location preferences is > causing troubles more than bringing benefits. > So I'm wondering whether we should introduce a config so that users can > disable input location preferences? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] (FLINK-15325) Input location preference which affects task distribution may make certain job performance worse
[ https://issues.apache.org/jira/browse/FLINK-15325 ] Zhu Zhu deleted comment on FLINK-15325: - was (Author: flink-jira-bot): This major issue is unassigned and itself and all of its Sub-Tasks have not been updated for 30 days. So, it has been labeled "stale-major". If this ticket is indeed "major", please either assign yourself or give an update. Afterwards, please remove the label. In 7 days the issue will be deprioritized. > Input location preference which affects task distribution may make certain > job performance worse > - > > Key: FLINK-15325 > URL: https://issues.apache.org/jira/browse/FLINK-15325 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.10.0 >Reporter: Zhu Zhu >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor > Attachments: D58ADB03-7187-46B1-B077-91E5005FD463.png > > > When running TPC-DS jobs in a session cluster, we observed that sometimes > tasks are not evenly distributed in TMs. The root cause turned out to be that > the downstream tasks tend to be TM or host local with its input tasks. This > helps to reduce network shuffle. > However, in certain cases, like the topology presented in the attached image, > jamming the input task's TM and machine with downstream tasks would affect > the performance. In this case, respecting input location preferences is > causing troubles more than bringing benefits. > So I'm wondering whether we should introduce a config so that users can > disable input location preferences? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] (FLINK-15325) Input location preference which affects task distribution may make certain job performance worse
[ https://issues.apache.org/jira/browse/FLINK-15325 ] Zhu Zhu deleted comment on FLINK-15325: - was (Author: flink-jira-bot): I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized. > Input location preference which affects task distribution may make certain > job performance worse > - > > Key: FLINK-15325 > URL: https://issues.apache.org/jira/browse/FLINK-15325 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.10.0 >Reporter: Zhu Zhu >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor > Attachments: D58ADB03-7187-46B1-B077-91E5005FD463.png > > > When running TPC-DS jobs in a session cluster, we observed that sometimes > tasks are not evenly distributed in TMs. The root cause turned out to be that > the downstream tasks tend to be TM or host local with its input tasks. This > helps to reduce network shuffle. > However, in certain cases, like the topology presented in the attached image, > jamming the input task's TM and machine with downstream tasks would affect > the performance. In this case, respecting input location preferences is > causing troubles more than bringing benefits. > So I'm wondering whether we should introduce a config so that users can > disable input location preferences? -- This message was sent by Atlassian Jira (v8.20.10#820010)
