[ https://issues.apache.org/jira/browse/FLINK-6364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15992662#comment-15992662 ]
ASF GitHub Bot commented on FLINK-6364: --------------------------------------- Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3801 @gyfora this is already happening in the code: the backend tracks which files are already transferred to hdfs, and a reference counting in the checkpoint coordinator (introduced in FLINK-6014) takes care of deleting SSTable files from the shared HDFS directory that are no longer referenced by any incremental checkpoint. > Implement incremental checkpointing in RocksDBStateBackend > ---------------------------------------------------------- > > Key: FLINK-6364 > URL: https://issues.apache.org/jira/browse/FLINK-6364 > Project: Flink > Issue Type: Sub-task > Components: State Backends, Checkpointing > Reporter: Xiaogang Shi > Assignee: Xiaogang Shi > > {{RocksDBStateBackend}} is well suited for incremental checkpointing because > RocksDB is base on LSM trees, which record updates in new sst files and all > sst files are immutable. By only materializing those new sst files, we can > significantly improve the performance of checkpointing. -- This message was sent by Atlassian JIRA (v6.3.15#6346)