[ https://issues.apache.org/jira/browse/KAFKA-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480703#comment-16480703 ]
Ismael Juma commented on KAFKA-6873: ------------------------------------ Thanks for the report. What is the message format version and was compression being used? > Broker is not returning data including requested offset > ------------------------------------------------------- > > Key: KAFKA-6873 > URL: https://issues.apache.org/jira/browse/KAFKA-6873 > Project: Kafka > Issue Type: Bug > Components: consumer > Affects Versions: 1.1.0 > Environment: ubuntu > Reporter: Adam Dratwinski > Priority: Blocker > > After upgrading Kafka to 1.1.0 from 0.9.x I experience issues related with > broker returning incomplete responses. This happens for my all log compacted > topics. I am using Golang client (Sarama). > I debugged the issue and found that for some requests brokers return > FetchResponse with all messages having offsets lower then requested. For > example, I request for offset 1078831, I get FetchResponse with only one > message having offset 1078830, which produces missing blocks error. If I > request the next offset (1078832), then I get a block with many messages, > starting with much higher offset (e.g 1083813). There is a big gap in offsets > between these records, probably because I am using log compacted topics, but > all expected messages are there. > Sarama client treats this as consumer error: > {quote}kafka: response did not contain all the expected topic/partition blocks > {quote} > For build-in java client this issue is not happening. Looks like it is less > restrict regarding the data order, and when the offset is missing in the > returned block, it just simply request the next offset. > I reported this issue at Shopify/sarama Github project (see > [https://github.com/Shopify/sarama/issues/1087)], where I got response, that > this seems to be Kafka bug, as according to the documentation, in this > situation broker should never return only messages having lower offsets. -- This message was sent by Atlassian JIRA (v7.6.3#76005)