Re: flink async snapshots

2016-05-20 Thread Aljoscha Krettek
Thats correct. With the fully async option the checkpoints take longer but you don't impact ongoing processing of elements. With the semi-async method snapshots take a shorter time but during the synchronous part no element processing can happen. On Fri, 20 May 2016 at 15:04 Abhishek Singh

Re: flink async snapshots

2016-05-20 Thread Abhishek Singh
Yes. Thanks for explaining. On Friday, May 20, 2016, Ufuk Celebi wrote: > On Thu, May 19, 2016 at 8:54 PM, Abhishek R. Singh > > wrote: > > If you can take atomic in-memory copies, then it works (at the cost of > > doubling your

Re: flink async snapshots

2016-05-20 Thread Ufuk Celebi
On Thu, May 19, 2016 at 8:54 PM, Abhishek R. Singh wrote: > If you can take atomic in-memory copies, then it works (at the cost of > doubling your instantaneous memory). For larger state (say rocks DB), won’t > you have to stop the world (atomic snapshot) and make

Re: flink async snapshots

2016-05-19 Thread Stavros Kontopoulos
No problem ;) On Thu, May 19, 2016 at 9:54 PM, Abhishek R. Singh < abhis...@tetrationanalytics.com> wrote: > If you can take atomic in-memory copies, then it works (at the cost of > doubling your instantaneous memory). For larger state (say rocks DB), won’t > you have to stop the world (atomic

flink async snapshots

2016-05-19 Thread Abhishek R. Singh
If you can take atomic in-memory copies, then it works (at the cost of doubling your instantaneous memory). For larger state (say rocks DB), won’t you have to stop the world (atomic snapshot) and make a copy? Doesn’t that make it synchronous, instead of background/async? Sorry Stravros - for