Sounds like you are in Yarn-Cluster mode. I created a JIRA SPARK-3913 <https://issues.apache.org/jira/browse/SPARK-3913> and PR https://github.com/apache/spark/pull/2786
is this what you looking for ? Chester On Sat, May 2, 2015 at 10:32 PM, Yijie Shen <henry.yijies...@gmail.com> wrote: > Hi, > > I’ve posted this problem in user@spark but find no reply, therefore moved > to dev@spark, sorry for duplication. > > I am wondering if it is possible to submit, monitor & kill spark > applications from another service. > > I have wrote a service this: > > parse user commands > translate them into understandable arguments to an already prepared > Spark-SQL application > submit the application along with arguments to Spark Cluster > using spark-submit from ProcessBuilder > run generated applications' driver in cluster mode. > The above 4 steps has been finished, but I have difficulties in these two: > > Query about the applications status, for example, the percentage > completion. > Kill queries accordingly > What I find in spark standalone documentation suggest kill application > using: > > ./bin/spark-class org.apache.spark.deploy.Client kill <master url> <driver > ID> > > And should find > the driver ID through the standalone Master web UI at > http://<master url>:8080. > > Are there any programmatically methods I could get the driverID submitted > by my `ProcessBuilder` and query status about the query? > > Any Suggestions? > > — > Best Regards! > Yijie Shen
