Thank you.
Would the following approaches to address this problem an overkills?
a. create a ServerSocket in a different thread from the main thread that
created the Spark StreamingContext, and listens to shutdown command there
b. create a web service that wraps around the main thread that
What we did for gracefully shutting down the spark streaming context is
extend a Spark Web UI Tab and perform a
SparkContext.SparkUI.attachTab(custom web ui). However, the custom scala
Web UI extensions needs to be under the package org.apache.spark.ui to get
around with the package access
Your app is the running Spark Streaming system. It would be up to you
to build some mechanism that lets you cause it to call stop() in
response to some signal from you.
On Fri, Sep 12, 2014 at 3:59 PM, stanley wangshua...@yahoo.com wrote:
In spark streaming programming document
In spark streaming programming document
https://spark.apache.org/docs/latest/streaming-programming-guide.html ,
it specifically states how to shut down a spark streaming context:
The existing application is shutdown gracefully (see
StreamingContext.stop(...) or JavaStreamingContext.stop(...)
