Re: Spark streaming completed batches statistics

2016-12-07 Thread Richard Startin
Is there any way to get this information as CSV/JSON? https://docs.databricks.com/_images/CompletedBatches.png [https://docs.databricks.com/_images/CompletedBatches.png] https://richardstartin.com/ From: Richard Startin Sent: 05 December 2016 15:55 To: user@

Re: Spark streaming completed batches statistics

2016-12-07 Thread map reduced
Have you checked http://spark.apache.org/docs/latest/monitoring.html#rest-api ? KP On Wed, Dec 7, 2016 at 11:43 AM, Richard Startin wrote: > Is there any way to get this information as CSV/JSON? > > > https://docs.databricks.com/_images/CompletedBatches.png > > > https://richardstartin.com/ > >

Re: Spark streaming completed batches statistics

2016-12-07 Thread Richard Startin
Ok it looks like I could reconstruct the logic in the Spark UI from the /jobs resource. Thanks. https://richardstartin.com/ From: map reduced Sent: 07 December 2016 19:49 To: Richard Startin Cc: user@spark.apache.org Subject: Re: Spark streaming completed

Re: Spark streaming completed batches statistics

2016-12-07 Thread map reduced
hardstartin.com/ > > > -- > *From:* map reduced > *Sent:* 07 December 2016 19:49 > *To:* Richard Startin > *Cc:* user@spark.apache.org > *Subject:* Re: Spark streaming completed batches statistics > > Have you checked http://spark.apach