Re: Allow multiple SparkContexts in Unit Testing

Already tried setting spark.driver.allowMultipleContexts to true. But it not successful. I the problem is we have different test suites which of course run in parallel. How do we stop sparkContext after each test suite and start it in the next test suite or is there any way to share sparkContext

Re: Allow multiple SparkContexts in Unit Testing

Priya, If you're trying to get unit tests running local spark contexts, you can just set up your spark context with 'spark.driver.allowMultipleContexts' set to true. Example: def create(seconds : Int, appName : String): StreamingContext = { val master = "local[*]" val conf = new

Re: Allow multiple SparkContexts in Unit Testing

Are you trying to speed up tests where each test suite uses single SparkContext ? You may want to read: https://issues.apache.org/jira/browse/SPARK-2243 Cheers On Wed, Nov 4, 2015 at 4:59 AM, Priya Ch wrote: > Hello All, > > How to use multiple Spark Context in

Allow multiple SparkContexts in Unit Testing

Hello All, How to use multiple Spark Context in executing multiple test suite of spark code ??? Can some one throw light on this ?