Hi Andy! Yes Spark UI provides a lot of interesting informations for debugging purposes.
Here I’m trying to integrate a simple progress monitoring in my app ui. I’m typically running a few “jobs” (or rather actions), and I’d like to be able to display the progress of each of those in my ui. I don’t really see how i could do that using SparkListener for the moment … Thanks for your help! Cheers! Pierre Borckmans Software team RealImpact Analytics | Brussels Office www.realimpactanalytics.com | pierre.borckm...@realimpactanalytics.com FR +32 485 91 87 31 | Skype pierre.borckmans On 22 May 2014, at 16:58, andy petrella [via Apache Spark User List] <ml-node+s1001560n6258...@n3.nabble.com> wrote: > SparkListener offers good stuffs. > But I also completed it with another metrics stuffs on my own that use Akka > to aggregate metrics from anywhere I'd like to collect them (without any deps > on ganglia yet on Codahale). > However, this was useful to gather some custom metrics (from within the tasks > then) not really to collect overall monitoring information about the spark > thingies themselves. > For that Spark UI offers already a pretty good insight no? > > Cheers, > > aℕdy ℙetrella > about.me/noootsab > > > > > On Thu, May 22, 2014 at 4:51 PM, Pierre B <[hidden email]> wrote: > Is there a simple way to monitor the overall progress of an action using > SparkListener or anything else? > > I see that one can name an RDD... Could that be used to determine which > action triggered a stage, ... ? > > > Thanks > > Pierre > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Use-SparkListener-to-get-overall-progress-of-an-action-tp6256.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > > > If you reply to this email, your message will be added to the discussion > below: > http://apache-spark-user-list.1001560.n3.nabble.com/Use-SparkListener-to-get-overall-progress-of-an-action-tp6256p6258.html > To unsubscribe from Use SparkListener to get overall progress of an action, > click here. > NAML -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Use-SparkListener-to-get-overall-progress-of-an-action-tp6256p6259.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
