Re: KafkaConsumer poll(timeout) doesn't seem to work as expected

2016-07-23 Thread Ewen Cheslack-Postava
Hey Josh, There's no guarantee that a poll() will return data. It might send a request, but if it takes longer than the timeout to retrieve some data from the brokers, then the call will have no data to return. This doesn't indicate anything is wrong, just that no data has been returned yet. poll

KafkaConsumer poll(timeout) doesn't seem to work as expected

2016-07-13 Thread Josh Goodrich
The poll(timeout) method of the Java KafkaConsumer API doesn’t behave the way you would think. If you create a new Consumer with a groupId that has been seen before, even if there are new events in the topic if you issue a poll(0) it never returns any records. I find I have to put in a loop of 2