I'm also pretty interested how to create custom Sinks in Spark. I'm using it with Ganglia and the normal metrics from JVM source do show up. I tried to create my own metric based on Issac's code, but does not show up in Ganglia. Does anyone know where is the problem? Here's the code snippet:
class AccumulatorSource(accumulator: Accumulator[Long], name: String) extends Source { val sourceName = "accumulator.metrics" val metricRegistry = new MetricRegistry() metricRegistry.register(MetricRegistry.name("accumulator", name), new Gauge[Long] { override def getValue: Long = { return accumulator.value; }}); } and then in the main: val longAccumulator = sc.accumulator[Long](0); val accumulatorMetrics = new AccumulatorSource(longAccumulator , "counters.accumulator"); SparkEnv.get.metricsSystem.registerSource(accumulatorMetrics); -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Executor-metrics-in-spark-application-tp188p10385.html Sent from the Apache Spark User List mailing list archive at Nabble.com.