Re: Spark Standalone Cluster - Running applications in JSON format

2016-11-30 Thread Carl Ballantyne
PM, Miguel Morales wrote: Don't have a Spark cluster up to verify this, but try port 8080. http://spark-master-ip:8080/api/v1/applications. But glad to hear you're getting somewhere, best of luck. On Wed, Nov 30, 2016 at 9:59 PM, Carl Ballantyne <carl.ballant...@guvera.com <mailto:carl.b

Re: Spark Standalone Cluster - Running applications in JSON format

2016-11-30 Thread Carl Ballantyne
server. On Wed, Nov 30, 2016 at 9:30 PM, Carl Ballantyne <carl.ballant...@guvera.com> wrote: Yes I was looking at this. But it says I need to access the driver - http://:4040. I don't have a running driver Spark instance since I am submitting jobs to Spark using the SparkLauncher class. Or m

Re: Spark Standalone Cluster - Running applications in JSON format

2016-11-30 Thread Carl Ballantyne
Morales wrote: Check the Monitoring and Instrumentation API: http://spark.apache.org/docs/latest/monitoring.html On Wed, Nov 30, 2016 at 9:20 PM, Carl Ballantyne <carl.ballant...@guvera.com <mailto:carl.ballant...@guvera.com>> wrote: Hi All, I want to get the running applica

Spark Standalone Cluster - Running applications in JSON format

2016-11-30 Thread Carl Ballantyne
Hi All, I want to get the running applications for my Spark Standalone cluster in JSON format. The same information displayed on the web UI on port 8080 ... but in JSON. Is there an easy way to do this? It seems I need to scrap the HTML page in order to get this information. The reason I