This is something we are interested as well. We are planning to investigate more on this. If someone has suggestions, we would love to hear.
Chester Sent from my iPad On May 22, 2014, at 8:02 AM, Pierre B <pierre.borckm...@realimpactanalytics.com> wrote: > Hi Andy! > > Yes Spark UI provides a lot of interesting informations for debugging > purposes. > > Here I’m trying to integrate a simple progress monitoring in my app ui. > > I’m typically running a few “jobs” (or rather actions), and I’d like to be > able to display the progress of each of those in my ui. > > I don’t really see how i could do that using SparkListener for the moment … > > Thanks for your help! > > Cheers! > > > > > Pierre Borckmans > Software team > > RealImpact Analytics | Brussels Office > www.realimpactanalytics.com | [hidden email] > > FR +32 485 91 87 31 | Skype pierre.borckmans > > > > > > > On 22 May 2014, at 16:58, andy petrella [via Apache Spark User List] <[hidden > email]> wrote: > >> SparkListener offers good stuffs. >> But I also completed it with another metrics stuffs on my own that use Akka >> to aggregate metrics from anywhere I'd like to collect them (without any >> deps on ganglia yet on Codahale). >> However, this was useful to gather some custom metrics (from within the >> tasks then) not really to collect overall monitoring information about the >> spark thingies themselves. >> For that Spark UI offers already a pretty good insight no? >> >> Cheers, >> >> aℕdy ℙetrella >> about.me/noootsab >> >> >> >> >> On Thu, May 22, 2014 at 4:51 PM, Pierre B <<a >> href="x-msg://7/user/SendEmail.jtp?type=node&node=6258&i=0" >> target="_top" rel="nofollow" link="external">[hidden email]> wrote: >> Is there a simple way to monitor the overall progress of an action using >> SparkListener or anything else? >> >> I see that one can name an RDD... Could that be used to determine which >> action triggered a stage, ... ? >> >> >> Thanks >> >> Pierre >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/Use-SparkListener-to-get-overall-progress-of-an-action-tp6256.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> >> >> >> If you reply to this email, your message will be added to the discussion >> below: >> http://apache-spark-user-list.1001560.n3.nabble.com/Use-SparkListener-to-get-overall-progress-of-an-action-tp6256p6258.html >> To unsubscribe from Use SparkListener to get overall progress of an action, >> click here. >> NAML > > > View this message in context: Re: Use SparkListener to get overall progress > of an action > Sent from the Apache Spark User List mailing list archive at Nabble.com.
