Thanks for your response Joel. I am currently trying out JMXTrans to get the stats from MBean and I can read attributes fine but it doesn't support JMX Operations yet. What tool do you use for your reporting? Is there another tool that supports JMX operations so that I can use the getOffsetLag operation?
Thanks, Xuyen -----Original Message----- From: Joel Koshy [mailto:jjkosh...@gmail.com] Sent: Friday, January 24, 2014 5:48 PM To: users@kafka.apache.org Subject: Re: Calcuate Consumer lag from JMX beans in Kafka 0.7.2 > kafka.logs.eventdata-0 > Attributes > Name // Name of partition > Size //Is this the > current number of messages? Size -> in bytes > NumberofSegments // Don't know what this is Each partition contains multiple segment (files) on disk. > CurrentOffset //Is this the current offset of > a consumer? No - it is the (byte) offset up to which we have flushed. It is unrelated to the current consumption point of consumers. > NumAppendedMessages // Don't know what this is Simple counter of number of messages sent so far to this partition. > Can someone please tell me what these attributes mean? I am trying to find > out if I can use these values to calculate the consumer lag ie: Size - > CurrentOffset and report it to a chart or dashboard somewhere for real-time > analysis. So the above would be insufficient to measure consumer lag. You can use the consumer offset checker tool and extract the lag from its output. There is also an mbean operation on the consumers (getOffsetLag) that you can use on a per-partition basis. Thanks, Joel
