Yes, that’s correct. There shouldn’t be any data loss. Stop with savepoint is a solution to make sure, that if you are stopping a job (either permanently or temporarily) that all of the results are published/committed to external systems before you actually stop the job.
If you just cancel/kill/crash a job, in some rare cases (if a checkpoint was completing at the time cluster was crashing), some records might not be committed before the cancellation/kill/crash happened. Also note that doesn’t mean there is a data loss, just those records will be published once you restore your job from a checkpoint. If you want to stop the job permanently, that might not happen, hence we need stop with savepoint. Piotrek > On 28 Feb 2020, at 15:02, Kaymak, Tobias <tobias.kay...@ricardo.ch> wrote: > > Thank you! For understanding the matter: When I have a streaming pipeline > (reading from Kafka, writing somewhere) and I click "cancel" and after that I > restart the pipeline - I should not expect any data to be lost - is that > correct? > > Best, > Tobias > > On Fri, Feb 28, 2020 at 2:51 PM Piotr Nowojski <pi...@ververica.com > <mailto:pi...@ververica.com>> wrote: > Thanks for confirming that Yadong. I’ve created a ticket for that [1]. > > Piotrek > > [1] https://issues.apache.org/jira/browse/FLINK-16340 > <https://issues.apache.org/jira/browse/FLINK-16340> > >> On 28 Feb 2020, at 14:32, Yadong Xie <vthink...@gmail.com >> <mailto:vthink...@gmail.com>> wrote: >> >> Hi >> >> 1. the old stop button was removed in flink 1.9.0 since it could not work >> properly as I know >> 2. if we have the feature of the stop with savepoint, we could add it to the >> web UI, but it may still need some work on the rest API to support the new >> feature >> >> >> Best, >> Yadong >> >> >> Piotr Nowojski <pi...@ververica.com <mailto:pi...@ververica.com>> >> 于2020年2月28日周五 下午8:49写道： >> Hi, >> >> I’m not sure. Maybe Yadong (CC) will know more, but to the best of my >> knowledge and research: >> >> 1. In Flink 1.9 we switched from the old webUI to a new one, that probably >> explains the difference you are seeing. >> 2. The “Stop” button in the old webUI, was not working properly - that was >> not stop with savepoint, as stop with savepoint is a relatively new feature. >> 3. Now that we have stop with savepoint (it can be used from CLI as you >> wrote), probably we could expose this feature in the new UI as well, unless >> it’s already exposed somewhere? Yadong, do you know an answer for that? >> >> Piotrek >> >>> On 27 Feb 2020, at 13:31, Kaymak, Tobias <tobias.kay...@ricardo.ch >>> <mailto:tobias.kay...@ricardo.ch>> wrote: >>> >>> Hello, >>> >>> before Flink 1.9 I was able to "Stop" a streaming pipeline - after clicking >>> that button in the webinterface it performed a clean shutdown. Now with >>> Flink 1.9 I just see the option to cancel it. >>> >>> However, using the commandline flink stop -d >>> 266c5b38cf9d8e61a398a0bef4a1b350 still does the trick. So the functionality >>> is there. >>> >>> Has the button been removed on purpose? >>> >>> Best, >>> Tobias >> > > > > -- > > Tobias Kaymak > Data Engineer > Data Intelligence > > tobias.kay...@ricardo.ch <mailto:tobias.kay...@ricardo.ch> > www.ricardo.ch <http://www.ricardo.ch/> > Theilerstrasse 1a, 6300 Zug >
