I can observe the same thing: - Lag values read through the Kafka consumer JMX is 0 - Lag values read through kafka-run-class.sh kafka.tools.ConsumerOffsetChecker is on average 200K-400K
When the Lag value in the Kafka consumer JMX is high (for example 5M), ConsumerOffsetChecker shows a matching number. I am running kafka_2.10-0.8.2.1 Osama -----Original Message----- From: allen chan [mailto:allen.michael.c...@gmail.com] Sent: Saturday, November 14, 2015 8:34 PM To: users@kafka.apache.org Subject: Re: consumer offset tool and JMX metrics do not match I believe producers / brokers / and consumers has been restarted at different times. What do you think the issue is? On Sat, Nov 14, 2015 at 2:52 AM, Prabhjot Bharaj <prabhbha...@gmail.com> wrote: > Hi, > > Did any of your brokers/consumers undergo a restart between the time > after you had started the consumption and until you see this issue ? > > Thanks, > Prabhjot > > > > On Sat, Nov 14, 2015 at 5:53 AM, allen chan > <allen.michael.c...@gmail.com> > wrote: > > > I also looked at this metric in JMX and it is also 0 > > > *kafka.consumer:type=ConsumerFetcherManager,name=MaxLag,clientId=logst > ash* > > > > On Fri, Nov 13, 2015 at 4:06 PM, allen chan < > allen.michael.c...@gmail.com> > > wrote: > > > > > Hi All, > > > > > > I am comparing the output from kafka.tools.ConsumerOffsetChecker > > > vs JMX > > > > > > (kafka.server:type=FetcherLagMetrics,name=ConsumerLag,clientId=logstas > h,topic=logstash_fdm,partition=*) > > > and they do not match. > > > > > > ConsumerOffsetChecker is showing ~60 Lag per partition and JMX > > > shows 0 > > for > > > all partitions. > > > > > > I am running kafka_2.11-0.8.2.2 and using jconsole to connect to JMX. > > > > > > Has anyone seen this issue before? I am trying to use JMX to pull > > > stats into monitoring system > > > > > > > > > -- > > > Allen Michael Chan > > > > > > > > > > > -- > > Allen Michael Chan > > > > > > -- > --------------------------------------------------------- > "There are only 10 types of people in the world: Those who understand > binary, and those who don't" > -- Allen Michael Chan -------------------------------------------------------------------------------- NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies; do not disclose, use or act upon the information; and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers. If you cannot access these links, please notify us by reply message and we will send the contents to you. By messaging with Morgan Stanley you consent to the foregoing.
