That thread is describing my exact experience. The last reply is in June. Has anything changed? Are we supposed to stop trusting JMX and try to use borrow or integrate with Consumer Offset Checker? Shouldn't the documentation reflect this matter as I am sure everyone will try to use JMX first since the docs say that is how to monitor.
On Fri, Nov 20, 2015 at 2:32 AM, Prabhjot Bharaj <prabhbha...@gmail.com> wrote: > Hi Allen, > > I was referring to one of the issues here: > http://search-hadoop.com/m/uyzND1XVyK12UNtd32/kafka+orphaned/v=threaded > This linked thread discusses one of such issues where consumer lag was not > reported correctly. > > Regards, > Prabhjot > > On Sun, Nov 15, 2015 at 7:04 AM, allen chan <allen.michael.c...@gmail.com> > wrote: > > > I believe producers / brokers / and consumers has been restarted at > > different times. > > What do you think the issue is? > > > > On Sat, Nov 14, 2015 at 2:52 AM, Prabhjot Bharaj <prabhbha...@gmail.com> > > wrote: > > > > > Hi, > > > > > > Did any of your brokers/consumers undergo a restart between the time > > after > > > you had started the consumption and until you see this issue ? > > > > > > Thanks, > > > Prabhjot > > > > > > > > > > > > On Sat, Nov 14, 2015 at 5:53 AM, allen chan < > > allen.michael.c...@gmail.com> > > > wrote: > > > > > > > I also looked at this metric in JMX and it is also 0 > > > > > > > > > > *kafka.consumer:type=ConsumerFetcherManager,name=MaxLag,clientId=logstash* > > > > > > > > On Fri, Nov 13, 2015 at 4:06 PM, allen chan < > > > allen.michael.c...@gmail.com> > > > > wrote: > > > > > > > > > Hi All, > > > > > > > > > > I am comparing the output from kafka.tools.ConsumerOffsetChecker vs > > JMX > > > > > > > > > > > > > > > (kafka.server:type=FetcherLagMetrics,name=ConsumerLag,clientId=logstash,topic=logstash_fdm,partition=*) > > > > > and they do not match. > > > > > > > > > > ConsumerOffsetChecker is showing ~60 Lag per partition and JMX > shows > > 0 > > > > for > > > > > all partitions. > > > > > > > > > > I am running kafka_2.11-0.8.2.2 and using jconsole to connect to > JMX. > > > > > > > > > > Has anyone seen this issue before? I am trying to use JMX to pull > > stats > > > > > into monitoring system > > > > > > > > > > > > > > > -- > > > > > Allen Michael Chan > > > > > > > > > > > > > > > > > > > > > -- > > > > Allen Michael Chan > > > > > > > > > > > > > > > > -- > > > --------------------------------------------------------- > > > "There are only 10 types of people in the world: Those who understand > > > binary, and those who don't" > > > > > > > > > > > -- > > Allen Michael Chan > > > > > > -- > --------------------------------------------------------- > "There are only 10 types of people in the world: Those who understand > binary, and those who don't" > -- Allen Michael Chan
