Checkout the Consumer API http://kafka.apache.org/0100/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html
and search for the method "seekToEnd" Here's the "text" from the API Doc - seekToEnd public void seekToEnd(Collection<TopicPartition> partitions) Seek to the last offset for each of the given partitions. This function evaluates lazily, seeking to the final offset in all partitions only when poll(long) or position(TopicPartition) are called. If no partition is provided, seek to the final offset for all of the currently assigned partitions. Specified by: seekToEnd in interface Consumer<K,V> See Also: seekToEnd(Collection) -----Original Message----- From: Todd Palino [mailto:[email protected]] Sent: Wednesday, July 06, 2016 10:36 AM To: [email protected] Subject: Re: Last offset in all partitions We do this through our monitoring agents by pulling it as a metric from the LogEndOffset beans. By putting it into our metrics system we get a mapping of timestamp to offset for every partition with (currently) 60 second granularity. Useful for offset resets and other tasks. -Todd On Wednesday, July 6, 2016, Kristoffer Sjögren <[email protected]> wrote: > Hi > > Is there a way to get the last offset written by all partitions of a > topic programmatically using the 0.10.0.0 API? > > At the moment I use KafkaConsumer.seekToEnd as seen in this gist[1] > but maybe there is a better, more efficient, way to do it? > > Cheers, > -Kristoffer > > [1] > https://gist.github.com/krisskross/a49e462bedb89505e372672cd81129ab > -- *Todd Palino* Staff Site Reliability Engineer Data Infrastructure Streaming linkedin.com/in/toddpalino This email and any files included with it may contain privileged, proprietary and/or confidential information that is for the sole use of the intended recipient(s). Any disclosure, copying, distribution, posting, or use of the information contained in or attached to this email is prohibited unless permitted by the sender. If you have received this email in error, please immediately notify the sender via return email, telephone, or fax and destroy this original transmission and its included files without reading or saving it in any manner. Thank you.
