For my use case, I need to figure out the lag within the Java consumer
itself that is consuming some topic. Ideally, the consumer application
would monitor the lag every minute or so and take some action on its own if
consumer falls behind (i.e. spin up more threads to process records - my
use case
NM. 0.11 KafkaConsumer seems to have added "endOffsets" API!
On Thu, Oct 12, 2017 at 3:31 PM, Manan G wrote:
> For my use case, I need to figure out the lag within the Java consumer
> itself that is consuming some topic. Ideally, the consumer application
> would monitor the lag every minute or s
So I have the same use case as the original poster and had the same issue
with the older 0.10.x clients and not being able to determine the tail
offsets even tho the fetch response contains the HW mark.
>From what I could understand by tracing through the 0.11.0 consumer code,
it makes additional
Hi,
You can get lag as a metric here:
https://kafka.apache.org/0100/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#metrics()
.
BR,
Stas.
2017-10-13 2:08 GMT+02:00 Stephen Powis :
> So I have the same use case as the original poster and had the same issue
> with the older 0.10.x cl
