Hi Liam,
Thanks for the response.
As we are using Spark Structured Streaming, the commit won't happen at Kafka
Side. For checkpoint, we are using HDFS.
As we are expecting, Kafka-consumer-groups.sh CLI should return LOG-END-OFFSET
with Partition details. However, it didn't display anyth
Hi Ann,
It's common practice in many Spark Streaming apps to store offsets external
to Kafka. Especially when checkpointing is enabled.
Are you sure that the app is committing offsets to Kafka?
Kind regards,
Liam Clarke
On Thu, 9 Jul. 2020, 8:00 am Ann Pricks, wrote:
> Hi Ricardo,
>
> Thanks
Hi Ricardo,
Thanks for your kind response.
As per your suggestion, I have enabled trace and PFB the content of the log
file.
Log File Content:
[2020-07-08 18:48:08,963] INFO Registered kafka:type=kafka.Log4jController
MBean (kafka.utils.Log4jControllerRegistration$)
[2020-07-08 18:48:09,244]
users@kafka.apache.org" , Ann Pricks
Subject: Re: Consumer Groups Describe is not working
Ann,
You can try execute the CLI `kafka-consumer-groups` with TRACE enabled to dig a
little deeper in the problem. In order to do this you need to:
1. Make a copy of your `$KAFKA_HOME/etc/kafka/tools-log4j.
ay, 3 July 2020 at 4:10 PM
To: "users@kafka.apache.org"
Subject: Consumer Groups Describe is not working
Hi Team,
Today, In our production cluster, we faced an issue with Kafka (Old offsets
was getting pulled from spark streaming application) and couldn'
afka.apache.org"
Subject: Consumer Groups Describe is not working
Hi Team,
Today, In our production cluster, we faced an issue with Kafka (Old offsets was
getting pulled from spark streaming application) and couldn't debug the issue
using kafka_consumer_group.sh CLI.
Whenever we execute
Hi Team,
Any update on this.
Regards,
Pricks
From: Ann Pricks
Date: Friday, 3 July 2020 at 4:10 PM
To: "users@kafka.apache.org"
Subject: Consumer Groups Describe is not working
Hi Team,
Today, In our production cluster, we faced an issue with Kafka (Old offsets was
getting p
Hi Team,
Today, In our production cluster, we faced an issue with Kafka (Old offsets was
getting pulled from spark streaming application) and couldn't debug the issue
using kafka_consumer_group.sh CLI.
Whenever we execute the below command to list the consumer groups, it is
working fine. Howev
