Pranay: Please take a look at the Redirector class inside: ./launcher/src/test/java/org/apache/spark/launcher/SparkLauncherSuite.java
Cheers On Sun, Aug 30, 2015 at 11:25 AM, Pranay Tonpay <pranay.ton...@impetus.co.in > wrote: > yes, the context is being closed at the end. > ------------------------------ > *From:* Akhil Das <ak...@sigmoidanalytics.com> > *Sent:* Sunday, August 30, 2015 9:03 AM > *To:* Pranay Tonpay > *Cc:* user@spark.apache.org > *Subject:* Re: spark-submit issue > > Did you try putting a sc.stop at the end of your pipeline? > > Thanks > Best Regards > > On Thu, Aug 27, 2015 at 6:41 PM, pranay <pranay.ton...@impetus.co.in> > wrote: > >> I have a java program that does this - (using Spark 1.3.1 ) Create a >> command >> string that uses "spark-submit" in it ( with my Class file etc ), and i >> store this string in a temp file somewhere as a shell script Using >> Runtime.exec, i execute this script and wait for its completion, using >> process.waitFor Doing ps -ef shows me SparkSubmitDriverBootstrapper , the >> script running my class ... parent child relationship.. >> >> The job gets triggered on spark-cluster and gets over but >> SparkSubmitDriverBootstrapper still shows up, due to this the >> process.waitFor never comes out and i can't detect the execution end... >> >> If i run the /temp file independently. things work file... only when i >> trigger /temp scrict inside Runtime.exec , this issue occurs... Any >> comments >> ? >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/spark-submit-issue-tp24474.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> For additional commands, e-mail: user-h...@spark.apache.org >> >> > > ------------------------------ > > > > > > > NOTE: This message may contain information that is confidential, > proprietary, privileged or otherwise protected by law. The message is > intended solely for the named addressee. If received in error, please > destroy and notify the sender. Any use of this email is prohibited when > received in error. Impetus does not represent, warrant and/or guarantee, > that the integrity of this communication has been maintained nor that the > communication is free of errors, virus, interception or interference. >
