[
https://issues.apache.org/jira/browse/KAFKA-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16038927#comment-16038927
]
Ismael Juma commented on KAFKA-5390:
------------------------------------
`Log` has the following code:
{code}
// check messages set size may be exceed config.segmentSize
if (validRecords.sizeInBytes > config.segmentSize) {
throw new RecordBatchTooLargeException("Message batch size is %d
bytes which exceeds the maximum configured segment size of %d."
.format(validRecords.sizeInBytes, config.segmentSize))
}
......
val segment = maybeRoll(messagesSize = validRecords.sizeInBytes,
maxTimestampInMessages = appendInfo.maxTimestamp,
maxOffsetInMessages = appendInfo.lastOffset)
val logOffsetMetadata = LogOffsetMetadata(
messageOffset = appendInfo.firstOffset,
segmentBaseOffset = segment.baseOffset,
relativePositionInSegment = segment.size)
segment.append(firstOffset = appendInfo.firstOffset,
largestOffset = appendInfo.lastOffset,
largestTimestamp = appendInfo.maxTimestamp,
shallowOffsetOfMaxTimestamp = appendInfo.offsetOfMaxTimestamp,
records = validRecords)
{code}
So, not clear how your claim is possible. Looking at the callback results, it
looks to me like the first batch got rejected and the second was accepted. This
could happen if the internal batch size calculation was slightly different than
yours and you produced 2 batches instead of 1. The second batch would be much
smaller, and it was accepted. Does that make sense to you?
> First records in batch rejected but others accepted when rolling log
> --------------------------------------------------------------------
>
> Key: KAFKA-5390
> URL: https://issues.apache.org/jira/browse/KAFKA-5390
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.10.2.1
> Reporter: Magnus Reftel
> Attachments: kafka-sequence-2.tar.gz, kafka-sequence.tar.gz
>
>
> When sending a sequence of records in a batch right when the broker needs to
> roll a new segment, it's possible for the first few records to fail, while
> other records in the batch are accepted. If records have dependencies on
> earlier records, e.g. in the case of a sequence of events in an event-sourced
> system, then a producer cannot use the batching functionality, since it then
> risks consumers receiving a record without first receiving the records it
> depends on.
> See attached testcase (kafka-sequence.tar.gz).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
