[jira] [Commented] (FLINK-35522) The source task may get stuck after a failover occurs in batch jobs
[ https://issues.apache.org/jira/browse/FLINK-35522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17852625#comment-17852625 ] xingbe commented on FLINK-35522: [~zhuzh] Sure, I have opened 2 PRS of 1.18 and 1.19, PTAL~ > The source task may get stuck after a failover occurs in batch jobs > --- > > Key: FLINK-35522 > URL: https://issues.apache.org/jira/browse/FLINK-35522 > Project: Flink > Issue Type: Bug > Components: Runtime / Coordination >Affects Versions: 1.17.2, 1.19.0, 1.18.1, 1.20.0 >Reporter: xingbe >Assignee: xingbe >Priority: Major > Labels: pull-request-available > Fix For: 1.20.0 > > > If the source task does not get assigned a split because the SplitEnumerator > has no more splits, and a failover occurs during the closing process, the > SourceCoordinatorContext will not resend the NoMoreSplit event to the newly > started source task, causing the source vertex to remain stuck indefinitely. > This case may only occur in batch jobs where speculative execution has been > enabled. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-35522) The source task may get stuck after a failover occurs in batch jobs
[ https://issues.apache.org/jira/browse/FLINK-35522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17852611#comment-17852611 ] Zhu Zhu commented on FLINK-35522: - [~xiasun] would you open 2 PRs to backport this change to 1.18 and 1.19? > The source task may get stuck after a failover occurs in batch jobs > --- > > Key: FLINK-35522 > URL: https://issues.apache.org/jira/browse/FLINK-35522 > Project: Flink > Issue Type: Bug > Components: Runtime / Coordination >Affects Versions: 1.17.2, 1.19.0, 1.18.1, 1.20.0 >Reporter: xingbe >Assignee: xingbe >Priority: Major > Labels: pull-request-available > Fix For: 1.20.0 > > > If the source task does not get assigned a split because the SplitEnumerator > has no more splits, and a failover occurs during the closing process, the > SourceCoordinatorContext will not resend the NoMoreSplit event to the newly > started source task, causing the source vertex to remain stuck indefinitely. > This case may only occur in batch jobs where speculative execution has been > enabled. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-35522) The source task may get stuck after a failover occurs in batch jobs
[ https://issues.apache.org/jira/browse/FLINK-35522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17852241#comment-17852241 ] Zhu Zhu commented on FLINK-35522: - Thanks for reporting this problem and volunteering to fix it! [~xiasun] The ticket is assigned to you. > The source task may get stuck after a failover occurs in batch jobs > --- > > Key: FLINK-35522 > URL: https://issues.apache.org/jira/browse/FLINK-35522 > Project: Flink > Issue Type: Bug > Components: Runtime / Coordination >Affects Versions: 1.17.2, 1.19.0, 1.18.1, 1.20.0 >Reporter: xingbe >Assignee: xingbe >Priority: Major > Fix For: 1.20.0 > > > If the source task does not get assigned a split because the SplitEnumerator > has no more splits, and a failover occurs during the closing process, the > SourceCoordinatorContext will not resend the NoMoreSplit event to the newly > started source task, causing the source vertex to remain stuck indefinitely. > This case may only occur in batch jobs where speculative execution has been > enabled. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-35522) The source task may get stuck after a failover occurs in batch jobs
[ https://issues.apache.org/jira/browse/FLINK-35522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17852240#comment-17852240 ] xingbe commented on FLINK-35522: I have a solution to fix it, Could you please assign this ticket to me? [~zhuzh] > The source task may get stuck after a failover occurs in batch jobs > --- > > Key: FLINK-35522 > URL: https://issues.apache.org/jira/browse/FLINK-35522 > Project: Flink > Issue Type: Bug > Components: Runtime / Coordination >Affects Versions: 1.17.2, 1.19.0, 1.18.1, 1.20.0 >Reporter: xingbe >Priority: Major > Fix For: 1.20.0 > > > If the source task does not get assigned a split because the SplitEnumerator > has no more splits, and a failover occurs during the closing process, the > SourceCoordinatorContext will not resend the NoMoreSplit event to the newly > started source task, causing the source vertex to remain stuck indefinitely. > This case may only occur in batch jobs where speculative execution has been > enabled. -- This message was sent by Atlassian Jira (v8.20.10#820010)
