[
https://issues.apache.org/jira/browse/KAFKA-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16038950#comment-16038950
]
Magnus Reftel commented on KAFKA-5390:
--------------------------------------
Yes and no =) It's good to know that batches either succeed or fail as a whole,
but in order to use batching in a producer, I'd have to have some way of
knowing whether a new batch has been started. I could try to duplicate the code
in the producer to see if there's enough space in the current batch for one
more message, but that seems very brittle to me, since it depends both on the
details of the serialization of messages and the linger time. Could the
producer API be extended with a send method that fails if it would require a
new batch, or something along those lines, so that producers know when to wait
for ACK before continuing?
> First records in batch rejected but others accepted when rolling log
> --------------------------------------------------------------------
>
> Key: KAFKA-5390
> URL: https://issues.apache.org/jira/browse/KAFKA-5390
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.10.2.1
> Reporter: Magnus Reftel
> Attachments: kafka-sequence-2.tar.gz, kafka-sequence.tar.gz
>
>
> When sending a sequence of records in a batch right when the broker needs to
> roll a new segment, it's possible for the first few records to fail, while
> other records in the batch are accepted. If records have dependencies on
> earlier records, e.g. in the case of a sequence of events in an event-sourced
> system, then a producer cannot use the batching functionality, since it then
> risks consumers receiving a record without first receiving the records it
> depends on.
> See attached testcase (kafka-sequence.tar.gz).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
