[ https://issues.apache.org/jira/browse/FLINK-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15391849#comment-15391849 ]
Wenlong Lyu commented on FLINK-4256: ------------------------------------ hi, Stephan, we have implemented similar solution before, but simple back tracking and forward cannot work well in situation following: Assuming job graph has A/B/C job vertices, A is connected to C with forward strategy, and B is connected to C all-to-all strategy, when a task of A failed, only one C task will be added to restart node set. I suggesting divide the job graph in maximal connected sub-graphs treating the job graph as an undirected graph, when a job graph is submitted. Besides, when the job graph is in large scale because extracting related nodes according to a given node can be time costly and will be repeatedly used in long running, using sub-graphs can avoid the problem > Fine-grained recovery > --------------------- > > Key: FLINK-4256 > URL: https://issues.apache.org/jira/browse/FLINK-4256 > Project: Flink > Issue Type: Improvement > Components: JobManager > Affects Versions: 1.1.0 > Reporter: Stephan Ewen > Assignee: Stephan Ewen > Fix For: 1.2.0 > > > When a task fails during execution, Flink currently resets the entire > execution graph and triggers complete re-execution from the last completed > checkpoint. This is more expensive than just re-executing the failed tasks. > In many cases, more fine-grained recovery is possible. > The full description and design is in the corresponding FLIP. > https://cwiki.apache.org/confluence/display/FLINK/FLIP-1+%3A+Fine+Grained+Recovery+from+Task+Failures -- This message was sent by Atlassian JIRA (v6.3.4#6332)