[
https://issues.apache.org/jira/browse/KAFKA-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15628551#comment-15628551
]
Yiquan Zhou commented on KAFKA-4348:
------------------------------------
Yes it's very likely the same issue as KAFKA-3135. I can reproduce with the
code from that issue as well. I've tested with a larger value of
receive.buffer.bytes (~64K), it did workaround the issue. So we can close the
issue as duplicated and I'll follow the other one. Thank you both for your help.
> On Mac OS, KafkaConsumer.poll returns 0 when there are still messages on
> Kafka server
> -------------------------------------------------------------------------------------
>
> Key: KAFKA-4348
> URL: https://issues.apache.org/jira/browse/KAFKA-4348
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Affects Versions: 0.9.0.0, 0.9.0.1, 0.10.0.1
> Environment: Mac OS X EI Capitan, Java 1.8.0_111
> Reporter: Yiquan Zhou
> Labels: consumer, mac, polling
>
> Steps to reproduce:
> 1. start the zookeeper and kafka server using the default properties from the
> distribution:
> $ bin/zookeeper-server-start.sh config/zookeeper.properties
> $ bin/kafka-server-start.sh config/server.properties
> 2. create a Kafka consumer using the Java API KafkaConsumer.poll(long
> timeout). It polls the records from the server every second (timeout set to
> 1000) and prints the number of records polled. The code can be found here:
> https://gist.github.com/yiquanzhou/a94569a2c4ec8992444c83f3c393f596
> 3. use bin/kafka-verifiable-producer.sh to generate some messages:
> $ bin/kafka-verifiable-producer.sh --topic connect-test --max-messages 200000
> --broker-list localhost:9092
> wait until all 200k messages are generated and sent to the server.
> 4. Run the consumer Java code. In the output console of the consumer, we can
> see that the consumer starts to poll some records, then it polls 0 records
> for several seconds before polling some more. like this:
> polled 27160 records
> polled 0 records
> polled 0 records
> polled 0 records
> polled 0 records
> polled 0 records
> polled 26886 records
> polled 26886 records
> polled 0 records
> polled 0 records
> polled 0 records
> polled 0 records
> polled 0 records
> polled 26701 records
> polled 26214 records
> The bug slows down the consumption of messages a lot. And in our use case,
> the consumer wrongly assumes that all messages are read from the topic.
> It is only reproducible on Mac OS X but neither on Linux nor Windows.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
