SPM only works for Java consumers or, I guess consumers using the built-in
offset management in kafka
On Tue, Mar 17, 2015 at 11:44 PM, Otis Gospodnetic
otis.gospodne...@gmail.com wrote:
Mathias,
SPM for Kafka will give you Consumer Offsets by Host, Consumer Id, Topic,
and Partition, and
Hi Lance,
I tried Kafka Offset Monitor a while back, but it didn't play especially
nice with a lot of topics / partitions (we currently have around 1400
topics and 4000 partitions in total). Might be possible to make it work a
bit better, but not sure it would be the best way to do alerting.
Hi Mathias,
We call bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker via NRPE,
and alert through Nagios.
-Robin
On Tue, Mar 17, 2015 at 2:46 AM, Kasper Mackenhauer Jacobsen
kas...@falconsocial.com wrote:
Hi Mathias,
We're currently using a custom solution that queries kafka and
Good day,
I'm looking into using SimpleConsumer#getOffsetsBefore and offsets
committed in ZooKeeper for monitoring the lag of a consumer group.
Our current use case is that we have a service that is continuously
consuming messages of a large number of topics and persisting the messages
to S3 at
Hey Mathias,
Kafka Offset Monitor will give you a general idea of where your consumer
group(s) are at:
http://quantifind.com/KafkaOffsetMonitor/
However, I'm not sure how useful it will be with a large number of topics
/ turning its output into a script that alerts upon a threshold. Could take
