[ https://issues.apache.org/jira/browse/FLINK-10205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16604577#comment-16604577 ]
ASF GitHub Bot commented on FLINK-10205: ---------------------------------------- yanghua commented on issue #6657: [FLINK-10205] [JobManager] Batch Job: InputSplit Fault tolerant for DataSource… URL: https://github.com/apache/flink/pull/6657#issuecomment-418776835 @isunjin It seems that you don't need to have permission to edit the wiki right away. You can write the document through Google Doc and give others access permission, and then others can discuss and review it. When the community feels that the design document is ok, it is not too late to edit the wiki. @zentol right? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Batch Job: InputSplit Fault tolerant for DataSourceTask > ------------------------------------------------------- > > Key: FLINK-10205 > URL: https://issues.apache.org/jira/browse/FLINK-10205 > Project: Flink > Issue Type: Sub-task > Components: JobManager > Reporter: JIN SUN > Assignee: JIN SUN > Priority: Major > Labels: pull-request-available > Original Estimate: 168h > Remaining Estimate: 168h > > Today DataSource Task pull InputSplits from JobManager to achieve better > performance, however, when a DataSourceTask failed and rerun, it will not get > the same splits as its previous version. this will introduce inconsistent > result or even data corruption. > Furthermore, if there are two executions run at the same time (in batch > scenario), this two executions should process same splits. > we need to fix the issue to make the inputs of a DataSourceTask > deterministic. The propose is save all splits into ExecutionVertex and > DataSourceTask will pull split from there. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)