Re: Get spark metrics in code

2016-09-11 Thread Steve Loughran
> On 9 Sep 2016, at 13:20, Han JU wrote: > > Hi, > > I'd like to know if there's a possibility to get spark's metrics from code. > For example > > val sc = new SparkContext(conf) > val result = myJob(sc, ...) > result.save(...) > > val gauge =

Get spark metrics in code

2016-09-09 Thread Han JU
Hi, I'd like to know if there's a possibility to get spark's metrics from code. For example val sc = new SparkContext(conf) val result = myJob(sc, ...) result.save(...) val gauge = MetricSystem.getGauge("org.apahce.spark") println(gauge.getValue) // or send to to internal