Re: Best monitoring tool for Kafka in production

[Gerard Klijs]

Not that I have anything against paying for monitoring, or against
Confluent, but you will need your consumers to be using kafka 1.10 is you
want to make most out of the confluent solution. We currently are using
zabbix, it's free, and it has complete functionality in one product. It
does can be a but verbose, and the graphs don't look very fancy.

On Thu, Jun 2, 2016 at 1:16 PM Michael Noll  wrote:

> Hafsa,
>
> since you specifically asked about non-free Kafka monitoring options as
> well:  As of version 3.0.0, the Confluent Platform provides a commercial
> monitoring tool for Kafka called Confluent Control Center.  (Disclaimer: I
> work for Confluent.)
>
> Quoting from the product page at
> http://www.confluent.io/product/control-center:
>
> "Know where your messages are at every step between source and destination.
> Identify slow brokers, delivery failures, and sleuth the truth out of
> unexpected latency in your network. Confluent Control Center delivers
> end-to-end stream monitoring. Unlike other monitoring tools, this one is
> purpose-built for your Kafka environment. Instead of identifying the
> throughput in your data center or other “clocks and cables” types of
> monitors, it tracks messages."
>
> Best wishes,
> Michael
>
>
>
>
> On Wed, May 25, 2016 at 12:42 PM, Hafsa Asif 
> wrote:
>
> > Hello,
> >
> > What is the best monitoring tool for Kafka in production, preferable free
> > tool? If there is no free tool, then please mention non-free efficient
> > monitoring tools also.
> >
> > We are feeling so much problem without monitoring tool. Sometimes brokers
> > goes down or consumer is not working, we are not informed.
> >
> > Best Regards,
> > Hafsa
> >
>

Re: Best monitoring tool for Kafka in production

[Michael Noll]

Hafsa,

since you specifically asked about non-free Kafka monitoring options as
well:  As of version 3.0.0, the Confluent Platform provides a commercial
monitoring tool for Kafka called Confluent Control Center.  (Disclaimer: I
work for Confluent.)

Quoting from the product page at
http://www.confluent.io/product/control-center:

"Know where your messages are at every step between source and destination.
Identify slow brokers, delivery failures, and sleuth the truth out of
unexpected latency in your network. Confluent Control Center delivers
end-to-end stream monitoring. Unlike other monitoring tools, this one is
purpose-built for your Kafka environment. Instead of identifying the
throughput in your data center or other “clocks and cables” types of
monitors, it tracks messages."

Best wishes,
Michael




On Wed, May 25, 2016 at 12:42 PM, Hafsa Asif 
wrote:

> Hello,
>
> What is the best monitoring tool for Kafka in production, preferable free
> tool? If there is no free tool, then please mention non-free efficient
> monitoring tools also.
>
> We are feeling so much problem without monitoring tool. Sometimes brokers
> goes down or consumer is not working, we are not informed.
>
> Best Regards,
> Hafsa
>

Re: Best monitoring tool for Kafka in production

[Paul Leclercq]

Hello there,

Pretty efficient way to monitor consumers and producers :
https://github.com/shunfei/DCMonitor
You will need to install https://prometheus.io/ as DCMonitor is based on it.


2016-05-27 11:24 GMT+02:00 Florian Hussonnois :

> Hi,
>
> You can also use InfluxDB instead of Graphite. InfluxDB has a Graphite
> plugin so that you can still use the graphite metrics reporter.
> InfluxDB can be queried through REST APIs. This is very convenient to
> implement some alerts on kafka metrics.
>
> 2016-05-25 21:45 GMT+02:00 Alex Loddengaard :
>
> > Hi Hafsa,
> >
> > We often see Grafana and Graphite, which are both free. Keep in mind you
> > should monitor the system's metrics and Kafka's JMX metrics.
> >
> > Alex
> >
> > On Wed, May 25, 2016 at 3:42 AM, Hafsa Asif 
> > wrote:
> >
> > > Hello,
> > >
> > > What is the best monitoring tool for Kafka in production, preferable
> free
> > > tool? If there is no free tool, then please mention non-free efficient
> > > monitoring tools also.
> > >
> > > We are feeling so much problem without monitoring tool. Sometimes
> brokers
> > > goes down or consumer is not working, we are not informed.
> > >
> > > Best Regards,
> > > Hafsa
> > >
> >
>
>
>
> --
> Florian HUSSONNOIS
>



-- 
Paul Leclercq
Data engineer

  
 

Re: Best monitoring tool for Kafka in production

[Alex Loddengaard]

Hi Hafsa,

We often see Grafana and Graphite, which are both free. Keep in mind you
should monitor the system's metrics and Kafka's JMX metrics.

Alex

On Wed, May 25, 2016 at 3:42 AM, Hafsa Asif 
wrote:

> Hello,
>
> What is the best monitoring tool for Kafka in production, preferable free
> tool? If there is no free tool, then please mention non-free efficient
> monitoring tools also.
>
> We are feeling so much problem without monitoring tool. Sometimes brokers
> goes down or consumer is not working, we are not informed.
>
> Best Regards,
> Hafsa
>