Re: Incremental checkpoint branch

2017-03-03 Thread Shaoxuan Wang
Vinshnu, You can find the latest design discussion for incremental checkpoint in http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-Incremental-Checkpointing-in-Flink-td15931.html @Stefan Richter

Re: Incremental checkpoint branch

2017-03-03 Thread Chen Qin
Hi Vishnu, My best gussing is there are lots of customized "incremental checkpointing" done via patch around rocksdb statebackend and rocksdb checkpoints. http://rocksdb.org/blog/2015/11/10/use-checkpoints-for-efficient-snapshots.html Thanks, Chen On Fri, Mar 3, 2017 at 1:16 PM, Ted Yu

Incremental checkpoint branch

2017-03-03 Thread Vishnu Viswanath
Hi, Can someone point me to the branch where the ongoing work for incremental checkpoint is going on, I would like to try it out even if the work is not complete. I have a use case where the state size increase about ~1gb every 5 minutes. Thanks, Vishnu