[
https://issues.apache.org/jira/browse/FLINK-40269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18100978#comment-18100978
]
Rui Fan edited comment on FLINK-40269 at 8/4/26 10:50 AM:
----------------------------------------------------------
Merged to
master 2.4.0 via 9f867dace15d3d9ae5daa4f68539ff1da05adace
2.3.1 via 5e5e73d09663425ec15e4b6eacb6b94c11be1e16
2.2.2 via: f66c02ef10f40c1aee85fc1a98f3bc47c3d1915f
2.1.4 via: b60607179c7df628db3bcb27d215537e05bfed5a
2.0.3 : 654354dcdaba9eafca1948c8d8a5baff90769e0c
1.20.6 : 59f2705f245353224411f1dbdb65babb1c0541dc
was (Author: fanrui):
Merged to master 2.4.0 via 9f867dace15d3d9ae5daa4f68539ff1da05adace
> Unaligned checkpoint restore may fail after rescale when a two-input task
> reads from the same upstream
> ------------------------------------------------------------------------------------------------------
>
> Key: FLINK-40269
> URL: https://issues.apache.org/jira/browse/FLINK-40269
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
> Reporter: Rui Fan
> Assignee: Rui Fan
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.0.3, 2.2.2, 1.20.6, 2.3.1, 2.4.0, 2.1.4
>
>
> We found a restore failure in Flink 2.3.0 when restoring from an unaligned
> checkpoint after rescaling the job.
> This issue likely exists since the early unaligned-checkpoint-rescaling
> support around Flink 1.13/1.14.
> It can happen when one two-input task has both inputs derived from the same
> upstream stream:
>
> {code:java}
> source
> |-- rebalance() --> CoMap input 1
> |-- keyBy(...) --> CoMap input 2{code}
>
>
> During rescale restore, Flink can confuse the two input edges and route
> channel state with the wrong mapping. The job then fails during restore with
> an error like:
>
> {code:java}
> text Cannot select SubtaskConnectionDescriptor{inputSubtaskIndex=1,
> outputSubtaskIndex=0};
> known channels are [SubtaskConnectionDescriptor{inputSubtaskIndex=0,
> outputSubtaskIndex=0}]{code}
>
>
> Reproducer branch:
> https://github.com/1996fanrui/flink/tree/repro/unaligned-channel-remap
>
> Commit `2711b807112e50b4bdabbd1ca181f800e8968a15` adds a minimal ITCase that
> reproduces the failure 100% before the runtime fix:
> UnalignedCheckpointRescaleSameUpstreamITCase.java
>
> The proposed fix is in commit `9b8560e69818e25ac20e19875aca55bdea208fcc`.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
