Hi Alexey, stop() is a protected method. So I tried cancel() instead. Are they different in behaviour?
Regards, Mani On Mon, 21 Sep, 2020, 5:27 PM Alexey Romanenko, <aromanenko....@gmail.com> wrote: > This is how Spark runner handle this: > > > https://github.com/apache/beam/blob/3e6f7b77add44b2b7b1a0ef4afd631642b7d0b59/runners/spark/src/main/java/org/apache/beam/runners/spark/SparkPipelineResult.java#L173 > > On 21 Sep 2020, at 18:17, Luke Cwik <lc...@google.com> wrote: > > +user <u...@beam.apache.org> > > On Mon, Sep 21, 2020 at 9:16 AM Luke Cwik <lc...@google.com> wrote: > >> You need the "sources" to stop and advance the watermark to infinity and >> have that propagate through the entire pipeline. There are propoosals for >> pipeline drain[1] and also for snapshot and update[2] for Apache Beam. We >> would love contributions in this space. >> >> Max shared some more details about how Flink users typically do this[3], >> does that apply to Spark? >> >> 1: >> https://docs.google.com/document/d/1NExwHlj-2q2WUGhSO4jTu8XGhDPmm3cllSN8IMmWci8 >> 2: >> https://docs.google.com/document/d/1UWhnYPgui0gUYOsuGcCjLuoOUlGA4QaY91n8p3wz9MY >> 3: >> https://lists.apache.org/thread.html/864eb7b4e7192706074059eef1e116146382552fa885dd6054ef4988%40%3Cuser.beam.apache.org%3E >> >> On Mon, Sep 21, 2020 at 7:43 AM Sunny, Mani Kolbe <sun...@dnb.com> wrote: >> >>> Forgot to mention, we are using spark runner. >>> >>> >>> >>> *From:* Sunny, Mani Kolbe <sun...@dnb.com> >>> *Sent:* Monday, September 21, 2020 12:33 PM >>> *To:* dev@beam.apache.org >>> *Subject:* How to gracefully stop a beam application >>> >>> >>> >>> *CAUTION:* This email originated from outside of D&B. Please do not >>> click links or open attachments unless you recognize the sender and know >>> the content is safe. >>> >>> >>> >>> Hello Beam community, >>> >>> >>> >>> When you are running a Beam application in full stream mode, it is >>> continuously running. What is the recommended way to stop it gracefully for >>> say maintenance/upgrades etc? When I say gracefully, I mean (1) without >>> data loss and (2) application existing with exit 0 code. >>> >>> >>> >>> Regards, >>> >>> Mani >>> >> >
