Take a look at the consumer.timeout.ms setting if you don't want the iterator to block indefinitely.
And a better long term solution is to switch to the new consumer, but that obviously requires much more significant code changes. The new consumer API is a single-threaded poll-based API where you can always specify timeouts to the consumer's poll() method (though it currently has some limitations to how it enforces that timeout). -Ewen On Wed, Mar 2, 2016 at 11:24 AM, Oleg Zhurakousky < ozhurakou...@hortonworks.com> wrote: > Guys > > We have a consumer deadlock and here is the relevant dump: > > "Timer-Driven Process Thread-10" Id=76 TIMED_WAITING on > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@39775787 > at sun.misc.Unsafe.park(Native Method) > at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > at > java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) > at > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:65) > at > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33) > at > kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66) > at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58) > . . . . . > > What worries me is the fact that ‘hasNext’ is essentially a blocking > operation. I can’t seem to find a single reason when it would be useful, > hence I am calling it a bug, but hopefully someone can clarify. > Kafka version is 0.8.* > > Cheers > Oleg > > -- Thanks, Ewen
