Don't have a Spark cluster up to verify this, but try port 8080. http://spark-master-ip:8080/api/v1/applications.
But glad to hear you're getting somewhere, best of luck. On Wed, Nov 30, 2016 at 9:59 PM, Carl Ballantyne <carl.ballant...@guvera.com > wrote: > Hmmm getting closer I think. > > I thought this was only for Mesos and Yarn clusters (from reading the > documentation). I tried anyway and initially received Connection Refused. > So I ran ./start-history-server.sh. This was on the Spark Master instance. > > I now get 404 not found. > > Nothing in the log file for the history server indicates there was a > problem. > > I will keep digging around. Thanks for your help so far Miguel. > > On 1/12/2016 3:33 PM, Miguel Morales wrote: > > Try hitting: http://<server-url>:18080/api/v1 > > Then hit /applications. > > That should give you a list of running spark jobs on a given server. > > On Wed, Nov 30, 2016 at 9:30 PM, Carl Ballantyne<carl.ballant...@guvera.com> > <carl.ballant...@guvera.com> wrote: > > Yes I was looking at this. But it says I need to access the driver - > http://<driver-node>:4040. > > I don't have a running driver Spark instance since I am submitting jobs to > Spark using the SparkLauncher class. Or maybe I am missing something obvious. > Apologies if so. > > > > > On 1/12/2016 3:21 PM, Miguel Morales wrote: > > Check the Monitoring and Instrumentation API: > http://spark.apache.org/docs/latest/monitoring.html > > On Wed, Nov 30, 2016 at 9:20 PM, Carl Ballantyne <carl.ballant...@guvera.com> > <carl.ballant...@guvera.com> wrote: > > Hi All, > > I want to get the running applications for my Spark Standalone cluster in > JSON format. The same information displayed on the web UI on port 8080 ... > but in JSON. > > Is there an easy way to do this? It seems I need to scrap the HTML page in > order to get this information. > > The reason I want to know this information is so I can ensure the Spark > cluster does not get too many jobs submitted at once. A Stand alone cluster > processes jobs FIFO. I would prefer to just send back a message to the user > telling them to try later then submit a job which has to wait for other jobs > to finish before starting. > > Any help appreciated. Thanks. > > Cheers, > Carl > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > > > -- > Carl Ballantyne > Lead Reporting Developer > Guvera Operations Pty Ltd. > Suite 1b, 58 Kingston Drive > Helensvale, QLD, 4212 > Australia > PO Box 3330 > Helensvale Town Centre, QLD, 4212 > Phone +61 (0) 7 5578 8987 <+61%207%205578%208987> > Email carl.ballant...@guvera.com > Web www.guveralimited.com > > > -- > *Carl Ballantyne* > Lead Reporting Developer > *Guvera Operations Pty Ltd.* > Suite 1b, 58 Kingston Drive > Helensvale, QLD, 4212 > Australia > PO Box 3330 > Helensvale Town Centre, QLD, 4212 > *Phone *+61 (0) 7 5578 8987 <+61%207%205578%208987> > *Email *carl.ballant...@guvera.com > *Web *www.guveralimited.com > > >
