I'd like the spark application to be stopped gracefully while received kill
signal, so I add these code:
sys.ShutdownHookThread {
println("Gracefully stopping Spark Streaming Application")
ssc.stop(stopSparkContext = true, stopGracefully = true)
println("Application stopped")
YEs, since 1.4.0, it shuts down streamingContext without gracefully from
shutdown hook.
You can make it shutdown gracefully in that hook by setting the SparkConf
"spark.streaming.stopGracefullyOnShutdown" to "true"
Note to self, document this in the programming guide.
On Wed, Sep 23, 2015 at 3:33
Thanks for the explain. I've opened a PR at
https://github.com/apache/spark/pull/8898
Tathagata Das 于2015年9月24日周四 上午2:44写道：
> YEs, since 1.4.0, it shuts down streamingContext without gracefully from
> shutdown hook.
> You can make it shutdown gracefully in that hook by setting the SparkConf
> "sp
