Hmmm getting closer I think.
I thought this was only for Mesos and Yarn clusters (from reading the
documentation). I tried anyway and initially received Connection
Refused. So I ran ./start-history-server.sh. This was on the Spark
Master instance.
I now get 404 not found.
Nothing in the log file for the history server indicates there was a
problem.
I will keep digging around. Thanks for your help so far Miguel.
On 1/12/2016 3:33 PM, Miguel Morales wrote:
Try hitting: http://<server-url>:18080/api/v1
Then hit /applications.
That should give you a list of running spark jobs on a given server.
On Wed, Nov 30, 2016 at 9:30 PM, Carl Ballantyne
<carl.ballant...@guvera.com> wrote:
Yes I was looking at this. But it says I need to access the driver -
http://<driver-node>:4040.
I don't have a running driver Spark instance since I am submitting jobs to
Spark using the SparkLauncher class. Or maybe I am missing something obvious.
Apologies if so.
On 1/12/2016 3:21 PM, Miguel Morales wrote:
Check the Monitoring and Instrumentation API:
http://spark.apache.org/docs/latest/monitoring.html
On Wed, Nov 30, 2016 at 9:20 PM, Carl Ballantyne <carl.ballant...@guvera.com>
wrote:
Hi All,
I want to get the running applications for my Spark Standalone cluster in JSON
format. The same information displayed on the web UI on port 8080 ... but in
JSON.
Is there an easy way to do this? It seems I need to scrap the HTML page in
order to get this information.
The reason I want to know this information is so I can ensure the Spark cluster
does not get too many jobs submitted at once. A Stand alone cluster processes
jobs FIFO. I would prefer to just send back a message to the user telling them
to try later then submit a job which has to wait for other jobs to finish
before starting.
Any help appreciated. Thanks.
Cheers,
Carl
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscr...@spark.apache.org
--
Carl Ballantyne
Lead Reporting Developer
Guvera Operations Pty Ltd.
Suite 1b, 58 Kingston Drive
Helensvale, QLD, 4212
Australia
PO Box 3330
Helensvale Town Centre, QLD, 4212
Phone +61 (0) 7 5578 8987
Email carl.ballant...@guvera.com
Web www.guveralimited.com
--
*Carl Ballantyne*
Lead Reporting Developer
*Guvera Operations Pty Ltd.*
Suite 1b, 58 Kingston Drive
Helensvale, QLD, 4212
Australia
PO Box 3330
Helensvale Town Centre, QLD, 4212
*Phone *+61 (0) 7 5578 8987
*Email *carl.ballant...@guvera.com <mailto:carl.ballant...@guvera.com>
*Web *www.guveralimited.com <http://www.guveralimited.com/>
