Hi Vasyl, thx for your reply, I’ll check > On 10 Jun 2019, at 14:22, Vasyl Bervetskyi <vb...@softserveinc.com> wrote: > > Hi Rinat, > > Savepoint need to be triggered when you want to create point in time which > you want to use in future to revert back your state, also you could cancel > job with savepoint which makes sure that you won’t lose any data during > canceling job. > > About your question that flink delete metadata file: I am using Flink 1.8.0 > and Flink deletes metadata file when I cancel job with savepoint, It seems > Flink doesn’t see any reason to keep checkpoint cause you created savepoint. > When you just cancel job without savepoint flink shouldn’t delete metadata > file from checkpoint. > From: Rinat <r.shari...@cleverdata.ru> > Sent: Wednesday, June 5, 2019 4:36 PM > To: user <user@flink.apache.org> > Subject: [Flink 1.6.1] _metadata file in retained checkpoint > > Hi mates, I’m trying to configure my job to retain checkpoints on it’s > cancellation and got some troubles. > > I got the following args why not to use savepoints: > > 1. we already have all the job state on DFS in checkpoints directory > 2. I can multiply size of the state on 2, when stopping the job, because the > state is already stored in checkpoints dir, and I’ll save it one more time > into savepoint dir > 3. creation of checkpoints is incremental and savepoints - is not, so in my > case (10 Tb state) the process of savepoint creation will took too long time > 4. As I know, we can rescale or job from retained checkpoints > > I've configured my job to retain checkpoints on job cancelation and found an > interesting issue - _metadata file is removed, when job is cancelled. > > So, I couldn’t restore my job from the retained checkpoint, is it an expected > behaviour ? If so, what is wrong ? > > > Sincerely yours, > Rinat Sharipov > Software Engineer at 1DMP CORE Team > > email: r.shari...@cleverdata.ru <mailto:a.totma...@cleverdata.ru> > mobile: +7 (925) 416-37-26 > > CleverDATA > make your data clever
Sincerely yours, Rinat Sharipov Software Engineer at 1DMP CORE Team email: r.shari...@cleverdata.ru <mailto:a.totma...@cleverdata.ru> mobile: +7 (925) 416-37-26 CleverDATA make your data clever
