Hi All, Sorry for the miscommunication. I'm not using 0.8. I'm using latest available flink-kafka client. I don't see my app registered as a consumer group. I wanted to know if there is a way to monitor Kafka offsets.
Thanks, Sandeep On Apr 23, 2017 9:38 AM, "Stephan Ewen" <se...@apache.org> wrote: > Since it is something special to Kafka 0.8, it could be implemented in a > simple addition to the ZooKeeperOffsetHandler used by the > FlinkKafkaConsumer08. > > Would you be willing to contribute this? That would certainly help > speeding up the resolution of the issue... > > > On Fri, Apr 21, 2017 at 2:33 AM, Tzu-Li (Gordon) Tai <tzuli...@apache.org> > wrote: > >> One additional note: >> >> In FlinkKafkaConsumer 0.9+, the current read offset should already exist >> in Flink metrics. >> See https://issues.apache.org/jira/browse/FLINK-4186. >> >> But yes, this is still missing for 0.8, so you need to directly query ZK >> for this. >> >> Cheers, >> Gordon >> >> >> On 21 April 2017 at 8:28:09 AM, Tzu-Li (Gordon) Tai (tzuli...@apache.org) >> wrote: >> >> Hi Sandeep, >> >> It isn’t fixed yet, so I think external tools like the Kafka offset >> checker still won’t work. >> If you’re using 08 and is currently stuck with this issue, you can still >> directly query ZK to get the offsets. >> >> I think for FlinkKafkaConsumer09 the offset is exposed to Flink's metric >> system using Kafka’s own returned metrics, but for 08 this is still missing. >> >> There is this JIRA [1] that aims at exposing consumer lag across all >> Kafka consumer versions to Flink metrics. Perhaps it would make sense to >> also generally expose the offset for all Kafka consumer versions to Flink >> metrics as well. >> >> - Gordon >> >> [1] https://issues.apache.org/jira/browse/FLINK-6109 >> >> >> On 19 April 2017 at 5:11:11 AM, sandeep6 (vr1meghash...@gmail.com) wrote: >> >> Is this fixed now? If not, is there any way to monitor kafka offset that >> is >> being processed by Flink? This should be a use case for everyone who uses >> Flink with Kafka. >> >> >> >> -- >> View this message in context: http://apache-flink-user-maili >> ng-list-archive.2336050.n4.nabble.com/Flink-Kafka-Consume >> r-Behaviour-tp8257p12663.html >> Sent from the Apache Flink User Mailing List archive. mailing list >> archive at Nabble.com. >> >> >
