Re: Discard checkpoint files through a single recursive call

2021-06-18 Thread Piotr Nowojski
nk you so much! > Jiahui > -- > *From:* Yun Tang > *Sent:* Tuesday, June 15, 2021 10:27 PM > *To:* Guowei Ma ; Jiahui Jiang < > qzhzm173...@hotmail.com> > *Cc:* user@flink.apache.org > *Subject:* Re: Discard checkpoint files through a single recursive call > &g

Re: Discard checkpoint files through a single recursive call

2021-06-15 Thread Jiahui Jiang
From: Yun Tang Sent: Tuesday, June 15, 2021 10:27 PM To: Guowei Ma ; Jiahui Jiang Cc: user@flink.apache.org Subject: Re: Discard checkpoint files through a single recursive call Hi Jiang, Please take a look at FLINK-17860 and FLINK-13856 for previous discussion of this problem. [1] https

Re: Discard checkpoint files through a single recursive call

2021-06-15 Thread Yun Tang
, June 16, 2021 8:40 To: Jiahui Jiang Cc: user@flink.apache.org Subject: Re: Discard checkpoint files through a single recursive call hi, Jiang I am afraid of misunderstanding what you mean, so can you elaborate on how you want to change it? For example, which interface or class do you want to add

Re: Discard checkpoint files through a single recursive call

2021-06-15 Thread Guowei Ma
hi, Jiang I am afraid of misunderstanding what you mean, so can you elaborate on how you want to change it? For example, which interface or class do you want to add a method to? Although I am not a state expert, as far as I know, due to incremental checkpoints, when CompleteCheckpoint is

Discard checkpoint files through a single recursive call

2021-06-14 Thread Jiahui Jiang
Hello Flink! We are building an infrastructure where we implement our own CompletedCheckpointStore. The read and write to the external storage location of these checkpoints are through HTTP calls to an external service. Recently we noticed some checkpoint file cleanup performance issue when