Hi,
I use Kafka 0.8 high level consumer reads message from topic
stream, 3 replica and 10 paritions.
When I use 10 threads read the stream and runing for some time (one
hour or one day),
some threads block at m_stream.iterator().hasNext(), but the parition
still has lots of messages.
I check consumer's fetch.message.max.bytes and broker's
message.max.bytes, there is no
message size bigger than these values.
The consumer configure is
props.put("zookeeper.session.timeout.ms", "4000");
props.put("zookeeper.sync.time.ms", "200");
props.put("auto.commit.interval.ms", "1000");
Please give me some option about how to avoid consumer block.
Is there some configure parameter can fix this problem.
Thanks!
XiaoTian