Re: [EXTERNAL] difference between checkpoints & savepoints

Richter > *Date: *Monday, August 14, 2017 at 7:47 AM > *To: *Raja Aravapalli > *Cc: *"user@flink.apache.org" > *Subject: *Re: [EXTERNAL] difference between checkpoints & savepoints > > > > Just noticed that I forgot to include also a reference to the >

Re: [EXTERNAL] difference between checkpoints & savepoints

Thanks very much for the detailed explanation Stefan. Regards, Raja. From: Stefan Richter Date: Monday, August 14, 2017 at 7:47 AM To: Raja Aravapalli Cc: "user@flink.apache.org" Subject: Re: [EXTERNAL] difference between checkpoints & savepoints Just noticed that I forgot t

Re: [EXTERNAL] difference between checkpoints & savepoints

Just noticed that I forgot to include also a reference to the documentation about externalized checkpoints: https://ci.apache.org/projects/flink/flink-docs-master/ops/state/checkpoints.html > Am 14.08.2017 um 1

Re: [EXTERNAL] difference between checkpoints & savepoints

Hi, > > Also, in the same line, can someone detail the difference between State > Backend & External checkpoint? > Those are two very different things. If we talk about state backends in Flink, we mean the entity that is responsible for storing and managing the state inside an operator. Th

Re: [EXTERNAL] Re: difference between checkpoints & savepoints

sday, August 10, 2017 at 11:38 AM To: Henri Heiskanen Cc: "user@flink.apache.org" Subject: [EXTERNAL] Re: difference between checkpoints & savepoints I most of the things you are asking for are already there: you can configure checkpoint interval + externalized cp, the backend, an

Re: difference between checkpoints & savepoints

I most of the things you are asking for are already there: you can configure checkpoint interval + externalized cp, the backend, and the location for savepoints and externalized checkpoints. You can restart from savepoints and externalized checkpoints from the CLI. One point I am not entirely su

Re: difference between checkpoints & savepoints

Hi, But I still need to resolve the latest checkpoint and pass that as an argument. My question still is that why all this can not be handled by Flink core? Why not just have parameters enable savepoints, location of savepoints and state backend and system would then automatically do checkpoints /

Re: difference between checkpoints & savepoints

Hi, but I think this is exactly the case for externalized checkpoints. Periodic savepoints are problematic because, their lifecycle is meant to be under the control of the user and Flink can not make any assumptions when they can be dropped. So in the conservative scenario, savepoints would qui

Re: difference between checkpoints & savepoints

Hi, It would be super helpful if Flink would provide out of the box functionality for writing automatic savepoints and then starting from the latest savepoint. If external checkpoints would support rescaling then 1st requirement is met, but one would still need to e.g. find the latest checkpoint f

Re: difference between checkpoints & savepoints

Hi, I would explain the main conceptual difference as follows: - Checkpoints are periodically triggered by the system for fault tolerance. They are used to automatically recover from failures. Because of their automatic and periodical nature, they should be lightweight to produce and will rest

difference between checkpoints & savepoints

Hi, Can someone please help me understand the difference between Flink's Checkpoints & Savepoints. While I read the documentation, couldn't understand the difference! :s Thanks a lot. Regards, Raja.