Well, I think cancel() should work since it will call stop() in the end for CANCELLED state.
> On 21 Sep 2020, at 18:57, Mani Kolbe <maniko...@gmail.com> wrote: > > Hi Alexey, > > stop() is a protected method. So I tried cancel() instead. Are they different > in behaviour? > > Regards, > Mani > > On Mon, 21 Sep, 2020, 5:27 PM Alexey Romanenko, <aromanenko....@gmail.com > <mailto:aromanenko....@gmail.com>> wrote: > This is how Spark runner handle this: > > https://github.com/apache/beam/blob/3e6f7b77add44b2b7b1a0ef4afd631642b7d0b59/runners/spark/src/main/java/org/apache/beam/runners/spark/SparkPipelineResult.java#L173 > > <https://github.com/apache/beam/blob/3e6f7b77add44b2b7b1a0ef4afd631642b7d0b59/runners/spark/src/main/java/org/apache/beam/runners/spark/SparkPipelineResult.java#L173> > >> On 21 Sep 2020, at 18:17, Luke Cwik <lc...@google.com >> <mailto:lc...@google.com>> wrote: >> >> +user <mailto:u...@beam.apache.org> >> >> On Mon, Sep 21, 2020 at 9:16 AM Luke Cwik <lc...@google.com >> <mailto:lc...@google.com>> wrote: >> You need the "sources" to stop and advance the watermark to infinity and >> have that propagate through the entire pipeline. There are propoosals for >> pipeline drain[1] and also for snapshot and update[2] for Apache Beam. We >> would love contributions in this space. >> >> Max shared some more details about how Flink users typically do this[3], >> does that apply to Spark? >> >> 1: >> https://docs.google.com/document/d/1NExwHlj-2q2WUGhSO4jTu8XGhDPmm3cllSN8IMmWci8 >> >> <https://docs.google.com/document/d/1NExwHlj-2q2WUGhSO4jTu8XGhDPmm3cllSN8IMmWci8> >> 2: >> https://docs.google.com/document/d/1UWhnYPgui0gUYOsuGcCjLuoOUlGA4QaY91n8p3wz9MY >> >> <https://docs.google.com/document/d/1UWhnYPgui0gUYOsuGcCjLuoOUlGA4QaY91n8p3wz9MY> >> 3: >> https://lists.apache.org/thread.html/864eb7b4e7192706074059eef1e116146382552fa885dd6054ef4988%40%3Cuser.beam.apache.org%3E >> >> <https://lists.apache.org/thread.html/864eb7b4e7192706074059eef1e116146382552fa885dd6054ef4988%40%3Cuser.beam.apache.org%3E> >> On Mon, Sep 21, 2020 at 7:43 AM Sunny, Mani Kolbe <sun...@dnb.com >> <mailto:sun...@dnb.com>> wrote: >> Forgot to mention, we are using spark runner. >> >> >> >> From: Sunny, Mani Kolbe <sun...@dnb.com <mailto:sun...@dnb.com>> >> Sent: Monday, September 21, 2020 12:33 PM >> To: dev@beam.apache.org <mailto:dev@beam.apache.org> >> Subject: How to gracefully stop a beam application >> >> >> >> CAUTION: This email originated from outside of D&B. Please do not click >> links or open attachments unless you recognize the sender and know the >> content is safe. >> >> >> >> Hello Beam community, >> >> >> >> When you are running a Beam application in full stream mode, it is >> continuously running. What is the recommended way to stop it gracefully for >> say maintenance/upgrades etc? When I say gracefully, I mean (1) without data >> loss and (2) application existing with exit 0 code. >> >> >> >> Regards, >> >> Mani >> >
