Re: Spark Standalone Cluster - Running applications in JSON format

8080 is just the normal web UI. Which is the information I want, ie Running Applications, but in HTML format. I want it in JSON so I don't have to be scraping and parsing HTML. From my understanding api/v1/applications should do the trick ... except it doesn't. Ah well. On 1/12/2016 4:00

Re: Spark Standalone Cluster - Running applications in JSON format

Don't have a Spark cluster up to verify this, but try port 8080. http://spark-master-ip:8080/api/v1/applications. But glad to hear you're getting somewhere, best of luck. On Wed, Nov 30, 2016 at 9:59 PM, Carl Ballantyne wrote: > Hmmm getting closer I think. > > I

Re: Spark Standalone Cluster - Running applications in JSON format

Hmmm getting closer I think. I thought this was only for Mesos and Yarn clusters (from reading the documentation). I tried anyway and initially received Connection Refused. So I ran ./start-history-server.sh. This was on the Spark Master instance. I now get 404 not found. Nothing in the

Re: Spark Standalone Cluster - Running applications in JSON format

Try hitting: http://:18080/api/v1 Then hit /applications. That should give you a list of running spark jobs on a given server. On Wed, Nov 30, 2016 at 9:30 PM, Carl Ballantyne wrote: > > Yes I was looking at this. But it says I need to access the driver - >

Re: Spark Standalone Cluster - Running applications in JSON format

Yes I was looking at this. But it says I need to access the driver - |http://:4040.| I don't have a running driver Spark instance since I am submitting jobs to Spark using the SparkLauncher class. Or maybe I am missing something obvious. Apologies if so. On 1/12/2016 3:21 PM, Miguel

Re: Spark Standalone Cluster - Running applications in JSON format

Check the Monitoring and Instrumentation API: http://spark.apache.org/docs/latest/monitoring.html On Wed, Nov 30, 2016 at 9:20 PM, Carl Ballantyne wrote: > Hi All, > > I want to get the running applications for my Spark Standalone cluster in > JSON format. The same

Spark Standalone Cluster - Running applications in JSON format

Hi All, I want to get the running applications for my Spark Standalone cluster in JSON format. The same information displayed on the web UI on port 8080 ... but in JSON. Is there an easy way to do this? It seems I need to scrap the HTML page in order to get this information. The reason I