Hi, I'm working on my companie's system that constructs out of Spark, Zeppelin, Hive and some other technology and wonder regarding to ability to stop contexts.
Working on the test framwork for the system, when run tests someting I would like to create new SparkContext in order to run the tests on 'clean' context. I found it hard to do as, first of all, I couldn't find any way to understand of SparkContext is already stopped. It has private flag for that but its private. Anther problem is that when creating local HiveContext it initialize derby instance. when trying to create new HiveContext it fails cause the DB already exists. Apperantly, there isn't anyway to tell HiveContext to stop and clear its connection to the DB. Essintelly I'm looking for two things: 1. Way to understand if SparkContext stopped already or not. 2. Way to stop/close HiveContext that will close relevant files/connection and release the resources. Thanks, Ophir
