[ https://issues.apache.org/jira/browse/KAFKA-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arkadiusz Firus resolved KAFKA-3788. ------------------------------------ Resolution: Not A Problem > Potential message lost when switching to new segment > ---------------------------------------------------- > > Key: KAFKA-3788 > URL: https://issues.apache.org/jira/browse/KAFKA-3788 > Project: Kafka > Issue Type: Bug > Components: log > Affects Versions: 0.9.0.0, 0.9.0.1, 0.10.0.0 > Reporter: Arkadiusz Firus > Assignee: Jay Kreps > Priority: Minor > Labels: easyfix > Original Estimate: 1h > Remaining Estimate: 1h > > If a new segment is needed method roll() from class kafka.log.Log is invoked. > It prepares new segment and schedules _asynchronous_ flush of the previous > segment. > Asynchronous call can lead to a problematic situation. As far as I know > neither Linux nor Windows guarantees that the order of files persisted to > disk will be the same as the order of writes to files. This means that > records from the new segment can be flushed before the old ones which in case > of power outage can lead to gaps between records. > Changing asynchronous invocation to synchronous one will solve the problem > because we have guarantee that all records from the previous segment will be > persisted to hard drive before we write any record to the new segment. > I am guessing that asynchronous invocation was chosen to increase performance > but switching between segments is not so often. So it is not a big gain. -- This message was sent by Atlassian JIRA (v6.3.4#6332)