I'm not sure to follow you:
either mine
- https://github.com/ut0mt8/yakle or
- https://github.com/danielqsj/kafka_exporter or
- https://github.com/redpanda-data/kminion
export consumer-group lag metrics. All works.
best,
--
Raphael Mazelier
On 10/05/2023 22:47, Akshay Kumar wrote:
Hello te
as well.Sent from my Galaxy
> Original message From: Akshay Kumar
> Date: 5/9/23 20:14 (GMT+01:00) To:
> users@kafka.apache.org Subject: Kafka Consumer Lag Monitoring Hello
> team,I am using Zookeeper less Kafka (Kafka Kraft - version 3.3.1). I
> wanted to monitor consume
he consumer lag in
> Apache Kafka. However, Burrow depends on ZooKeeper to access the Kafka
> metadata and track the consumer lag. Therefore, it is not possible to use
> Burrow without ZooKeeper.
>
> Since you are using Kafka version 3.3.1, you can leverage Kafka's built-in
> cons
, you can leverage Kafka's built-in
consumer lag monitoring functionality without the need for any additional
tools or dependencies. Kafka exposes consumer lag metrics through JMX, and
you can use any monitoring system that supports JMX to collect and
visualize these metrics.
Here are the ste
@kafka.apache.org Subject: Kafka Consumer Lag Monitoring Hello team,I am
using Zookeeper less Kafka (Kafka Kraft - version 3.3.1). I wanted to monitor
consumer lag, so I was using Burrow for that, but I am unable to use Burrow
without Zookeeper.Does Burror work without Zookeeper?Or what is the better or
best
Hello team,
I am using Zookeeper less Kafka (Kafka Kraft - version 3.3.1). I wanted to
monitor consumer lag, so I was using Burrow for that, but I am unable to
use Burrow without Zookeeper.
Does Burror work without Zookeeper?
Or what is the better or best way to monitor consumer lag and lag hist
FWIW - solved this by calling '.poll()' with 'enable.auto.commit' set to
false.
On Mon, Dec 5, 2016 at 5:53 AM, Mathieu Fenniak <
mathieu.fenn...@replicon.com> wrote:
> Hi Jon,
>
> Here are some lag monitoring options that are external to the consumer
> app
Hi Jon,
Here are some lag monitoring options that are external to the consumer
application itself; I don't know if these will be appropriate for you. You
can use a command-line tool like kafka-consumer-groups.sh to monitor
consumer group lag externally (
http://kafka.apach
Is there a way to get updated consumer position(s) without subscribing to a
topic? I can achieve this by continually closing / reopening a
KafkaConsumer object but this is problematic as it often times out.
Im getting consumer lag from a combination of
(start) .seekToEnd() (and then) .position()
It took a little while for this to post (aligning with company PR and all),
but there's a detailed description of what Burrow is on the LinkedIn
Engineering Blog now:
http://engineering.linkedin.com/apache-kafka/burrow-kafka-consumer-monitoring-reinvented
-Todd
On Thu, Jun 4, 2015 at 6:43 PM, T
Issue is here: https://github.com/linkedin/Burrow/issues/3
On Fri, Jun 12, 2015 at 11:34 AM, Roger Hoover
wrote:
> Will do. Thanks
>
> Sent from my iPhone
>
> > On Jun 12, 2015, at 10:43 AM, Todd Palino wrote:
> >
> > Can you open an issue on the github page please, and we can investigate
> >
Will do. Thanks
Sent from my iPhone
> On Jun 12, 2015, at 10:43 AM, Todd Palino wrote:
>
> Can you open an issue on the github page please, and we can investigate
> further there?
>
> -Todd
>
> On Fri, Jun 12, 2015 at 10:22 AM, Roger Hoover
> wrote:
>
>> Thanks, Todd. I'm also using ZK 3.
Can you open an issue on the github page please, and we can investigate
further there?
-Todd
On Fri, Jun 12, 2015 at 10:22 AM, Roger Hoover
wrote:
> Thanks, Todd. I'm also using ZK 3.4.6 with no ACLs. I'm surprised and
> currently stumped by this error.
>
> On Fri, Jun 12, 2015 at 9:49 AM, To
Thanks, Todd. I'm also using ZK 3.4.6 with no ACLs. I'm surprised and
currently stumped by this error.
On Fri, Jun 12, 2015 at 9:49 AM, Todd Palino wrote:
> The invalid ACL error is an error that is passed back from Zookeeper. What
> version of Zookeeper are you using, and have you set up ACLs
The invalid ACL error is an error that is passed back from Zookeeper. What
version of Zookeeper are you using, and have you set up ACLs within it? I'm
not able to see this on our ZK (3.4.6 with no ACLs).
-Todd
On Fri, Jun 12, 2015 at 9:34 AM, Roger Hoover
wrote:
> Hi,
>
> I was trying to give b
Hi,
I was trying to give burrow a try and got a ZK error "invalid ACL
specified". Any suggestions on what's going wrong?
1434044348908673512 [Critical] Cannot get ZK notifier lock: zk: invalid ACL
specified
Here's my config:
[general]
logdir=log
logconfig=logging.cfg
pidfile=burrow.pid
c
For mirror maker and our audit application, we've been using
Kafka-committed offsets for some time now. We've got a few other consumers
who are using it, but we haven't actively worked on moving the bulk of them
over. It's been less critical since we put the ZK transaction logs on SSD.
And yeah, t
Hi Todd,
Thanks for open sourcing this, I'm excited to take a look.
It looks like it's specific to offsets stored in kafka (and not zookeeper)
correct? I assume by that that LinkedIn is using the kafka storage now in
production?
Jason
On Thu, Jun 4, 2015 at 9:43 PM, Todd Palino wrote:
> I am
I am very happy to introduce Burrow, an application to provide Kafka
consumer status as a service. Burrow is different than just a "lag checker":
* Multiple Kafka cluster support - Burrow supports any number of Kafka
clusters in a single instance. You can also run multiple copies of Burrow
in para
19 matches
Mail list logo
