try setting in SparkConf.set( 'spark.driver.allowMultipleContexts' , true) On 30 November 2014 at 17:37, lokeshkumar [via Apache Spark User List] < ml-node+s1001560n20037...@n3.nabble.com> wrote:
> Hi Forum, > > Is it not possible to run multiple SparkContexts concurrently without > stopping the other one in the spark 1.3.0. > I have been trying this out and getting the below error. > > Caused by: org.apache.spark.SparkException: Only one SparkContext may be > running in this JVM (see SPARK-2243). To ignore this error, set > spark.driver.allowMultipleContexts = true. The currently running > SparkContext was created at: > > According to this, its not possible to create unless we specify the option > spark.driver.allowMultipleContexts = true. > > So is there a way to create multiple concurrently running SparkContext in > same JVM or should we trigger Driver processes in different JVMs to do the > same? > > Also please let me know where the option > 'spark.driver.allowMultipleContexts' to be set? I have set it in > spark-env.sh SPARK_MASTER_OPTS but no luck. > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://apache-spark-user-list.1001560.n3.nabble.com/Multiple-SparkContexts-in-same-Driver-JVM-tp20037.html > To start a new topic under Apache Spark User List, email > ml-node+s1001560n1...@n3.nabble.com > To unsubscribe from Apache Spark User List, click here > <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=aG5haGFrQHd5bnlhcmRncm91cC5jb218MXwtMTgxOTE5MTkyOQ==> > . > NAML > <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- Regards, Harihar Nahak BigData Developer Wynyard Email:hna...@wynyardgroup.com | Extn: 8019 ----- --Harihar -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Multiple-SparkContexts-in-same-Driver-JVM-tp20037p20055.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
