Re: Monitor incoming data rates

[Magnus Bäck]

On Thursday, April 09, 2015 at 05:49 CEST,
 Bernie Carolan  wrote:

> I have alerts configured on my Logstash - Elasticsearch setup which
> perform regular queries to see what state the cluster is in etc.
> Recently I had a situation where Logstash was running OK and ES
> cluster was in Green state, but there was no data going into ES.
> Is there a way to monitor this, e.g if the incoming data rate to ES
> drops below 100 events a minute.
> I can't do it by Logstash metrics because that side of it was running
> normally, i.e. no change in data rates.

How can Logstash be running normally when it halts the pipeline when one
output isn't able to accept messages? Are you saying that Elasticsearch
accepts messages but just drops them on the floor?

-- 
Magnus Bäck| Software Engineer, Development Tools
magnus.b...@sonymobile.com | Sony Mobile Communications

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/20150409061248.GA13186%40seldlx20533.corpusers.net.
For more options, visit https://groups.google.com/d/optout.

Re: Monitor incoming data rates

[Mark Walkom]

Marvel monitors these sorts of things out of the box.

But maybe _cluster/stats and keep an eye on docs.count? What is monitoring
things now?

On 9 April 2015 at 13:49, Bernie Carolan  wrote:

> I have alerts configured on my Logstash - Elasticsearch setup which
> perform regular queries to see what state the cluster is in etc.
>
> Recently I had a situation where Logstash was running OK and ES cluster
> was in Green state, but there was no data going into ES.
> Is there a way to monitor this, e.g if the incoming data rate to ES drops
> below 100 events a minute.
>
> I can't do it by Logstash metrics because that side of it was running
> normally, i.e. no change in data rates.
>
>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/2458951a-5c82-4003-841a-d634aa1c1dfd%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8xhf2bKbH73eY-EMkoUXbShbBe%3Db%2Bg9n%2BaCyzoBqOvXw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Monitor incoming data rates

[Bernie Carolan]

I have alerts configured on my Logstash - Elasticsearch setup which perform 
regular queries to see what state the cluster is in etc.

Recently I had a situation where Logstash was running OK and ES cluster was 
in Green state, but there was no data going into ES.
Is there a way to monitor this, e.g if the incoming data rate to ES drops 
below 100 events a minute.

I can't do it by Logstash metrics because that side of it was running 
normally, i.e. no change in data rates.



-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/2458951a-5c82-4003-841a-d634aa1c1dfd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.