Has anyone tried pushing Spark metrics into elasticsearch? We have other metrics, eg some runtime information, going into ES and would like to be able to combine this with the Spark metrics for visualization with Kibana.
I experimented with a new sink using ES's ElasticsearchReporter for the Coda Hale metrics but have a few issues with default mappings. Has anyone already implemented this before I start to dig deeper? Cheers,
