vHi, I've been exploring the metrics exposed by Spark and I'm wondering whether there's a way to register job-specific metrics that could be exposed through the existing metrics system.
Would there be an example somewhere? BTW, documentation about how the metrics work could be improved. I found out about the default servlet and the metrics/json/ endpoint on the code. I could not find any reference to that on the dedicated doc page [1]. Probably something I could contribute if there's nobody on that at the moment. -kr, Gerard. [1] http://spark.apache.org/docs/1.1.0/monitoring.html#Metrics