Sorry for not being clear. Kafka ships with a tool, that allows to get a dump of topic content in human readable form (ie, what is store on disk in a topic).
You can call it via `bin/kafka-run-class.sh kafka.tools.DumpLogSegments` and need to point it to the corresponding segment files. For example: /tmp/kafka-logs/__consumer_offsets-1/00000000000000000000.log with /tmp/kafka-logs being whatever you configured for `log.dir` for your broker. Than it's topic-name and segment files are numbered and end with .log -Matthias On 2/16/18 12:57 PM, Russell Teabeault wrote: > Matthias, > > I will try to get you the broker logs. As far as the partitions of > __consumer_offsets topic, I am not quite clear on what you want me to > provide. Are you wanting the output of > > ./kafka-topics.sh --zookeeper zk-host --describe --topic __consumer_offsets > > -russ > > On Fri, Feb 16, 2018 at 1:45 PM, Matthias J. Sax <[email protected]> > wrote: > >> Ok. Let us know if bouncing works. >> >> Sounds like a bug thought, that the consumer group is not cleared. It >> would be helpful to get the broker logs (of the node that hosts the >> coordinator). Can you also provide a dump of the corresponding >> partitions of __consumer_offsets topic? >> >> Thanks a lot. >> >> -Matthias >> >> >> On 2/16/18 12:35 PM, Russell Teabeault wrote: >>> Matthias, >>> >>> The session.timeout.ms is set to 10000. It has been in this "weird" >> state >>> now for going on 24 hours. >>> We are going to try to bounce the group coordinator. >>> >>> Thanks, >>> -russ >>> >>> On Fri, Feb 16, 2018 at 12:18 PM, Matthias J. Sax <[email protected] >>> >>> wrote: >>> >>>> Hi, >>>> >>>> this "weird" state can happen. Usually, a consumer sends a "leave group >>>> request" to inform the broker it's shutting down. However, this is a >>>> send-and-forget approach and if the message is lost, the broker might >>>> think that a stopped consumer is still alive. For this reason, the >>>> broker should eventually time out consumers and thus clean-up the >>>> consumer group. >>>> >>>> How long did you wait? >>>> >>>> Did you modify the `session.timeout.ms` config? >>>> >>>> As a last resort, bouncing the broker that hosts the consumer group >>>> coordinator should resolve the issue. >>>> >>>> -Matthias >>>> >>>> On 2/15/18 10:09 PM, Russell Teabeault wrote: >>>>> It appears that kafka and/or kafka streams has gotten itself into a >> weird >>>>> state. My KS app is not running. I try to reset it using the reset >>>> command >>>>> line tool and I get: >>>>> >>>>> ERROR: java.lang.IllegalStateException: Consumer group >>>>> 'dp-consumption-aggregation-processor-staging2' is still active. Make >>>> sure >>>>> to stop all running application instances before running the reset >> tool. >>>>> java.lang.IllegalStateException: Consumer group >>>>> 'dp-consumption-aggregation-processor-staging2' is still active. Make >>>> sure >>>>> to stop all running application instances before running the reset >> tool. >>>>> >>>>> But as I said the application is not running. So I ran >>>>> >>>>> ./kafka-consumer-groups.sh --bootstrap-server >>>>> smf1-abi-09-sr1.prod.twitter.com:9092 --group >>>>> dp-consumption-aggregation-processor-staging2 --describe >>>>> >>>>> and the first line is >>>>> >>>>> Warning: Consumer group 'dp-consumption-aggregation- >> processor-staging2' >>>> is >>>>> rebalancing. >>>>> >>>>> How do I get it out of this state and how did I get it into this state? >>>>> >>>>> Thanks! >>>>> >>>> >>>> >>> >>> >> >> > >
signature.asc
Description: OpenPGP digital signature
