+dev mailing list Time series analysis on metrics becomes quite useful when running spark jobs using a workflow manager like oozie.
Would love to take this up if the community thinks its worthwhile. On Tue, Feb 23, 2016 at 2:59 PM, Karan Kumar <karankumar1...@gmail.com> wrote: > HI > > Spark at the moment uses application ID to report metrics. I was thinking > that if we can create an option to export metrics on a user-controlled key. > This will allow us to do time series analysis on counters by dumping these > counters in a DB such as graphite. > > One of the approaches I had in mind was allowing a user to set a property > via the spark client. If that property is set, use the property value to > report metrics else use the current implementation > <https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/metrics/MetricsSystem.scala>of > reporting metrics on appid. > > Thoughts? > > -- > Thanks > Karan > -- Thanks Karan
