Hi Dev team, Could you Please let me know if any updates on my query. Thanks, Santhosh Aditya
From: Marigowda, Santhosh Aditya Sent: Wednesday, January 31, 2024 8:50 AM To: [email protected] Cc: Jain, Ankit <[email protected]>; Namboodiri, Vishnu <[email protected]>; Mudlapur, Rajesh <[email protected]>; Reddy, Thanmai <[email protected]> Subject: RE: Kafka consumer group crashing and not able to consume once service is up Thanks Matthias for your response. Sorry, my bad please find the updated Image. · We have 1 topic and 4 partitions, Each consumer is pointing to the topic. There is no partition assignment specific to the consumer group from our software code. · Please let me know if you need Kafka server logs or consumer service logs, Post restart of our service no logs are printed in onMessage method. · We are using consumer.subscribe(topic) [cid:[email protected]] Thanks, Santhosh Aditya From: Matthias J. Sax <[email protected]<mailto:[email protected]>> Sent: Wednesday, January 31, 2024 12:13 AM To: [email protected]<mailto:[email protected]> Subject: Re: Kafka consumer group crashing and not able to consume once service is up I am a not sure if I can follow completely. From the figures you show, you have a topic with 4 partitions, and 4 consumer groups. Thus, each consumer group should read all 4 partitions, but the figure indicate that each group would read a single I am a not sure if I can follow completely. From the figures you show, you have a topic with 4 partitions, and 4 consumer groups. Thus, each consumer group should read all 4 partitions, but the figure indicate that each group would read a single partition only? Can you clarify? Are you using `consumer.subscribe` or `consumer.assign`? In general, might be good too collect some INFO (or DEBUG) level logs for the crashing service after restart to see what it's doing. -Matthias On 1/30/24 7:17 AM, Marigowda, Santhosh Aditya wrote: > Hi Kafka Dev Team, > > Could you please help us with our problem. > > In our POC, if one of the kafka consumer(Service) shuts down or crashes > then post restart of service none of the messages are getting consumed > by the crashed Service. > > Other services are consuming without any issues. > > One of service crash/Shutdown > > If we rename the Kafka consumer group name and start the service then > messages start consuming. > > Consumer configuration : > > { > > delay:1000 > > timeout:0 > > topic-name= test > > handlers=["Listener"] > > source="kafka-consumer" > > enable_auto_commit="false" > > group="Consumer-Group-1" > > } > > local-kafka-consumer = { > > server= > "{kafka-hostname}:{kafka-port}" > > > deserializer="org.apache.kafka.common.serialization.StringDeserializer" > > auto_offset_reset="latest" > > enable_auto_commit="true" > > maxRequestSize=20971520 > > } > > Thanks, > > Santhosh Aditya >
