For Fluffka, I created a wrapping function:
IterStatus timedHasNext() {
try {
long startTime = System.currentTimeMillis();
it.hasNext();
long endTime = System.currentTimeMillis();
return new IterStatus(true,endTime-startTime);
} catch (ConsumerTimeoutException e) {
return new IterStatus(false,consumerTimeout);
}
}
IterStatus is basically a pair of whether there is data and how long we waited.
Calling this in a loop like that:
while (timeWaited < timeUpperLimit) {
iterStatus = timedHasNext();
if (iterStatus.hasData()) {
.... // handling data
}
timeWaited += iterStatus.getWaitTime();
}
Allows us to read all messages that are available within a certain
amount of time.
You can do other cool stuff (exit after certain number of messages,
when you waited X amount of time with 0 new data, etc).
Gwen
On Mon, Sep 15, 2014 at 8:54 AM, Neha Narkhede <[email protected]> wrote:
> There isn't a very clean way to do this in the 0.8.x high level consumer.
> You can configure consumer.timeout.ms to a certain value so that the
> consumer's blocking iterator returns if no message arrives for
> consumer.timeout.ms.
>
> Thanks
> Neha
>
> On Mon, Sep 15, 2014 at 6:33 AM, Ivan Balashov <[email protected]> wrote:
>
>> Hi,
>>
>> Is it possible to read all available messages with HLC in a
>> non-blocking way? E.g. read all messages and not wait for more
>> messages to appear in the topic.
>>
>> As far as I understand, currently one has to keep high-level consumer
>> in a separate thread until it is shut down explicitly, but how can one
>> check if all available messages are in fact consumed?
>>
>> Thanks,
>>
