Fwd: Elasticsearch sink for metrics

[Pete Robbins]

The issue I had was with the ElasticsearchReporter and how it maps eg a
Gauge in JSON. The "value" was typed to whatever the first Guage was, eg
int, which caused issues with some of my other guages which were double.

As I say I've just started looking at this and was wanting to see if this
was already implemented before continuing.

On 15 January 2016 at 09:18, Nick Pentreath 
wrote:

> I haven't come across anything, but could you provide more detail on what
> issues you're encountering?
>
>
>
> On Fri, Jan 15, 2016 at 11:09 AM, Pete Robbins 
> wrote:
>
>> Has anyone tried pushing Spark metrics into elasticsearch? We have other
>> metrics, eg some runtime information, going into ES and would like to be
>> able to combine this with the Spark metrics for visualization with Kibana.
>>
>> I experimented with a new sink using ES's ElasticsearchReporter for the
>> Coda Hale metrics but have a few issues with default mappings.
>>
>> Has anyone already implemented this before I start to dig deeper?
>>
>> Cheers,
>>
>>
>>
>

Elasticsearch sink for metrics

[Pete Robbins]

Has anyone tried pushing Spark metrics into elasticsearch? We have other
metrics, eg some runtime information, going into ES and would like to be
able to combine this with the Spark metrics for visualization with Kibana.

I experimented with a new sink using ES's ElasticsearchReporter for the
Coda Hale metrics but have a few issues with default mappings.

Has anyone already implemented this before I start to dig deeper?

Cheers,

Re: Elasticsearch sink for metrics

[Nick Pentreath]

I haven't come across anything, but could you provide more detail on what
issues you're encountering?



On Fri, Jan 15, 2016 at 11:09 AM, Pete Robbins  wrote:

> Has anyone tried pushing Spark metrics into elasticsearch? We have other
> metrics, eg some runtime information, going into ES and would like to be
> able to combine this with the Spark metrics for visualization with Kibana.
>
> I experimented with a new sink using ES's ElasticsearchReporter for the
> Coda Hale metrics but have a few issues with default mappings.
>
> Has anyone already implemented this before I start to dig deeper?
>
> Cheers,
>
>
>