RE: How to Identify Consumers of a Topic?

[Jillian Cocklin]

Thanks Derar,

I'll check that out and see if it gives enough information about the consumer 
to track it.

Thanks!
Jillian

-Original Message-
From: Derar Alassi [mailto:derar.ala...@gmail.com] 
Sent: Monday, August 08, 2016 3:35 PM
To: users@kafka.apache.org
Subject: Re: How to Identify Consumers of a Topic?

I use kafka-consumer-offset-checker.sh to check offsets of consumers and along 
that you get which consumer is attached to each partition.

On Mon, Aug 8, 2016 at 3:12 PM, Jillian Cocklin < jillian.cock...@danalinc.com> 
wrote:

> Hello,
>
> Our team is using Kafka for the first time and are in the testing 
> phase of getting a new product ready, which uses Kafka as the 
> communications backbone.  Basically, a processing unit will consume a 
> message from a topic, do the processing, then produce the output to another 
> topic.
> Messages get passed back and forth between processors until done.
>
> We had an issue last week where an outdated processor was "stealing"
> messages from a topic, doing incorrect (outdated) processing, and 
> putting it in the next topic.  We could not find the rogue processor 
> (aka consumer).  We shut down all known consumers of that topic, and 
> it was still happening.  We finally gave up and renamed the topic to 
> get around the issue.
>
> Is there a Kafka tool we could have used to find the connected 
> consumer in that consumer group?  Maybe by name or by IP?
>
> Thanks,
> Jillian
>
>

How to Identify Consumers of a Topic?

[Jillian Cocklin]

Hello,

Our team is using Kafka for the first time and are in the testing phase of 
getting a new product ready, which uses Kafka as the communications backbone.  
Basically, a processing unit will consume a message from a topic, do the 
processing, then produce the output to another topic.  Messages get passed back 
and forth between processors until done.

We had an issue last week where an outdated processor was "stealing" messages 
from a topic, doing incorrect (outdated) processing, and putting it in the next 
topic.  We could not find the rogue processor (aka consumer).  We shut down all 
known consumers of that topic, and it was still happening.  We finally gave up 
and renamed the topic to get around the issue.

Is there a Kafka tool we could have used to find the connected consumer in that 
consumer group?  Maybe by name or by IP?

Thanks,
Jillian