So I have the same use case as the original poster and had the same issue
with the older 0.10.x clients and not being able to determine the tail
offsets even tho the fetch response contains the HW mark.
>From what I could understand by tracing through the 0.11.0 consumer code,
it makes additional
NM. 0.11 KafkaConsumer seems to have added "endOffsets" API!
On Thu, Oct 12, 2017 at 3:31 PM, Manan G wrote:
> For my use case, I need to figure out the lag within the Java consumer
> itself that is consuming some topic. Ideally, the consumer application
> would monitor the lag every minute or s
Hi list,
I need to backfill a kafka-0.10.1 topic with some historical data (time
series) coming from a DB.
How can I populate data and still leverage on the self-expiring (retention)
capabilities?
Is there a best practice to bulk load historical data?
Thanks,
For my use case, I need to figure out the lag within the Java consumer
itself that is consuming some topic. Ideally, the consumer application
would monitor the lag every minute or so and take some action on its own if
consumer falls behind (i.e. spin up more threads to process records - my
use case
Yes, this is the Jira ticket about this issue:
https://issues.apache.org/jira/browse/KAFKA-5600
On Wed, Oct 11, 2017 at 5:47 PM, Vincent Dautremont <
vincent.dautrem...@olamobile.com.invalid> wrote:
> I would also like to know the related Jira ticket if any, to check that
> what I experience the
See inline.
On Thu, Oct 12, 2017 at 6:43 AM, Vahid S Hashemian <
vahidhashem...@us.ibm.com> wrote:
>
>
> [2017-10-11 21:45:11,642] FATAL (kafka.Kafka$)
> java.lang.IllegalArgumentException: Unknown signal: HUP
> at sun.misc.Signal.(Unknown Source)
> at kafka.Kafka$.registerHandler
