[
https://issues.apache.org/jira/browse/FLINK-32037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhanghao Chen updated FLINK-32037:
----------------------------------
Description:
*Background*
After FLINK-30213, in case of parallelism changes to the JobGraph, as done via
the AdaptiveScheduler or through providing JobVertexId overrides in
PipelineOptions#PARALLELISM_OVERRIDES, when the consumer parallelism doesn't
match the local parallelism, the original ForwardPartitioner will be replaced
with the RebalancePartitioner.
*Problem*
Although the actual partitioner changes to RebalancePartitioner underneath, the
ship strategy seen on the Web UI is still FORWARD.
This is because the fix patch applies when we init StreamTask, and the job
graph is not touched. Web UI uses the JSON plan generated from the job graph
for display, and the ship strategy is get by JobEdge#getShipStrategyName.
was:
*Background*
After FLINK-30213, in case of parallelism changes to the JobGraph, as done via
the AdaptiveScheduler or through providing JobVertexId overrides in
PipelineOptions#PARALLELISM_OVERRIDES, when the consumer parallelism doesn't
match the local parallelism, the original ForwardPartitioner will be replaced
with the RebalancePartitioner.
*Problem*
Although the actual partitioner changes underneath, the ship strategy seen on
the Web UI is still FORWARD.
This is because the fix patch applies when we init StreamTask, and the job
graph is not touched. Web UI uses the JSON plan generated from the job graph
for display, and the ship strategy is get by JobEdge#getShipStrategyName.
> The edge on Web UI is wrong after parallelism changes via parallelism
> overrides or AdaptiveScheduler
> -----------------------------------------------------------------------------------------------------
>
> Key: FLINK-32037
> URL: https://issues.apache.org/jira/browse/FLINK-32037
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination, Runtime / REST
> Affects Versions: 1.17.0
> Reporter: Zhanghao Chen
> Priority: Major
>
> *Background*
> After FLINK-30213, in case of parallelism changes to the JobGraph, as done
> via the AdaptiveScheduler or through providing JobVertexId overrides in
> PipelineOptions#PARALLELISM_OVERRIDES, when the consumer parallelism doesn't
> match the local parallelism, the original ForwardPartitioner will be replaced
> with the RebalancePartitioner.
> *Problem*
> Although the actual partitioner changes to RebalancePartitioner underneath,
> the ship strategy seen on the Web UI is still FORWARD.
> This is because the fix patch applies when we init StreamTask, and the job
> graph is not touched. Web UI uses the JSON plan generated from the job graph
> for display, and the ship strategy is get by JobEdge#getShipStrategyName.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
