SparkListener offers good stuffs. But I also completed it with another metrics stuffs on my own that use Akka to aggregate metrics from anywhere I'd like to collect them (without any deps on ganglia yet on Codahale). However, this was useful to gather some custom metrics (from within the tasks then) not really to collect overall monitoring information about the spark thingies themselves. For that Spark UI offers already a pretty good insight no?
Cheers, aℕdy ℙetrella about.me/noootsab [image: aℕdy ℙetrella on about.me] <http://about.me/noootsab> On Thu, May 22, 2014 at 4:51 PM, Pierre B < pierre.borckm...@realimpactanalytics.com> wrote: > Is there a simple way to monitor the overall progress of an action using > SparkListener or anything else? > > I see that one can name an RDD... Could that be used to determine which > action triggered a stage, ... ? > > > Thanks > > Pierre > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Use-SparkListener-to-get-overall-progress-of-an-action-tp6256.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >
