Fwd: Elasticsearch sink for metrics

2016-01-18 Thread Pete Robbins
The issue I had was with the ElasticsearchReporter and how it maps eg a Gauge in JSON. The "value" was typed to whatever the first Guage was, eg int, which caused issues with some of my other guages which were double. As I say I've just started looking at this and was wanting to see if this was

Elasticsearch sink for metrics

2016-01-15 Thread Pete Robbins
Has anyone tried pushing Spark metrics into elasticsearch? We have other metrics, eg some runtime information, going into ES and would like to be able to combine this with the Spark metrics for visualization with Kibana. I experimented with a new sink using ES's ElasticsearchReporter for the Coda

Re: Elasticsearch sink for metrics

2016-01-15 Thread Nick Pentreath
I haven't come across anything, but could you provide more detail on what issues you're encountering? On Fri, Jan 15, 2016 at 11:09 AM, Pete Robbins wrote: > Has anyone tried pushing Spark metrics into elasticsearch? We have other > metrics, eg some runtime information,