[
https://issues.apache.org/jira/browse/KAFKA-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16038969#comment-16038969
]
Ismael Juma commented on KAFKA-5390:
------------------------------------
If you want to do manual batching, you can simply increase `batch.size` to a
large number and use `flush` whenever you want to create a batch. Be aware that
you need to ensure that the batch size is not larger than `max.request.size`
(which you can increase as well, of course).
You also need to configure the brokers appropriately: max.message.size,
log.segment.bytes, replica.fetch.max.bytes, replica.fetch.response.max.bytes.
And the consumer: fetch.max.bytes, max.partition.fetch.bytes.
> First records in batch rejected but others accepted when rolling log
> --------------------------------------------------------------------
>
> Key: KAFKA-5390
> URL: https://issues.apache.org/jira/browse/KAFKA-5390
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.10.2.1
> Reporter: Magnus Reftel
> Attachments: kafka-sequence-2.tar.gz, kafka-sequence.tar.gz
>
>
> When sending a sequence of records in a batch right when the broker needs to
> roll a new segment, it's possible for the first few records to fail, while
> other records in the batch are accepted. If records have dependencies on
> earlier records, e.g. in the case of a sequence of events in an event-sourced
> system, then a producer cannot use the batching functionality, since it then
> risks consumers receiving a record without first receiving the records it
> depends on.
> See attached testcase (kafka-sequence.tar.gz).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
