FWIW - solved this by calling '.poll()' with 'enable.auto.commit' set to false.
On Mon, Dec 5, 2016 at 5:53 AM, Mathieu Fenniak < [email protected]> wrote: > Hi Jon, > > Here are some lag monitoring options that are external to the consumer > application itself; I don't know if these will be appropriate for you. You > can use a command-line tool like kafka-consumer-groups.sh to monitor > consumer group lag externally ( > http://kafka.apache.org/documentation.html#basic_ops_consumer_group), you > can use Kafka JMX metrics to monitor lag ( > http://kafka.apache.org/documentation.html#monitoring), or you can use an > external tool like Burrow (https://github.com/linkedin/Burrow). > > Mathieu > > > > On Mon, Dec 5, 2016 at 4:47 AM, Jon Yeargers <[email protected]> > wrote: > > > Is there a way to get updated consumer position(s) without subscribing > to a > > topic? I can achieve this by continually closing / reopening a > > KafkaConsumer object but this is problematic as it often times out. > > > > Im getting consumer lag from a combination of > > > > (start) .seekToEnd() (and then) .position() > > > > and > > > > (end) .committed() > > > > but if I wait and call this set of functions again I get the same values. > > > > How do I refresh this connection without interfering with the actual > > consumer group position(s)? > > >
