[
https://issues.apache.org/jira/browse/KAFKA-7061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581836#comment-16581836
]
Bertus Greeff commented on KAFKA-7061:
--------------------------------------
This enhancement would solve a problem that is hard and inefficient to solve on
the consumer.
In typical systems with a single primary that is at any time the source of
truth there is the problem that secondaries (in the Paxos sense) may think they
are still primaries and produce data into a Kafka topic when they no longer
have the latest state.
Its trivial in such a system to have an ever increasing number for the actual
primary that can be assigned to the Kafka producer timestamp.
If log compaction would honor this timestamp as described in the associated
KIP, there would be no risk that the actual latest state is lost during
compaction.
I would very much like this KIP to be implemented. (at least the timestamp
part).
> Enhanced log compaction
> -----------------------
>
> Key: KAFKA-7061
> URL: https://issues.apache.org/jira/browse/KAFKA-7061
> Project: Kafka
> Issue Type: Improvement
> Components: core
> Affects Versions: 2.0.0
> Reporter: Luis Cabral
> Priority: Major
> Fix For: 2.1.0
>
>
> Enhance log compaction to support more than just offset comparison, so the
> insertion order isn't dictating which records to keep.
> Default behavior is kept as it was, with the enhanced approached having to be
> purposely activated.
> The enhanced compaction is done either via the record timestamp, by settings
> the new configuration as "timestamp" or via the record headers by setting
> this configuration to anything other than the default "offset" or the
> reserved "timestamp".
> See
> [KIP-280|https://cwiki.apache.org/confluence/display/KAFKA/KIP-280%3A+Enhanced+log+compaction]
> for more details.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
