Hi if you're interested in stopping you're spark application externally, you will probably need a way to communicate with the spark driver (which start and holds a ref to the spark context) this can be done by adding some code to the driver app, for example:
- you can expose a rest api that stop the query and the spark context - if running in client mode you can listen to stdin - you can also listen to an external system (like kafka) Eyal On Tue, Dec 26, 2017 at 10:37 PM, M Singh <mans2si...@yahoo.com.invalid> wrote: > Thanks Diogo. My question is how to gracefully call the stop method while > the streaming application is running in a cluster. > > > > > On Monday, December 25, 2017 5:39 PM, Diogo Munaro Vieira < > diogo.mun...@corp.globo.com> wrote: > > > Hi M Singh! Here I'm using query.stop() > > Em 25 de dez de 2017 19:19, "M Singh" <mans2si...@yahoo.com.invalid> > escreveu: > > Hi: > Are there any patterns/recommendations for gracefully stopping a > structured streaming application ? > Thanks > > > > >
