Regarding "broker-agnostic of headers": there are some KIPs from Streams to use headers for internal purposes as well, e.g. KIP-258 and KIP-213 (I admit there may be a conflict with user space, but practically I think it is very rare). So I think we are very likely going to make Kafka internals to be "headers-aware" anyways.
Regarding the general API: I think it is a good idea in general, but it may still have limits: note that right now our KIP enforce a header type to be long, and we have a very careful discussion about the fall-back policy if header does not have the specified key or if the value is not long-typed; but if we enforce long type version in the interface, it would require users trying to customizing their compaction logic (think: based on some value payload field) to transform their fields to long as well. So I feel we can still go with the current proposed approach, and only consider this general API if we observe it does have a general usage requirement. By that time we can still extend the config values of "log.cleaner.compaction. strategy" to "offset, timestamp, header, myFuncName". @Bertus Thanks for your feedback, I believe the proposed config is indeed for both global (for the whole broker) and per-topic, Luís can confirm if this is the case, and update the wiki page to make it clear. Guozhang On Thu, Aug 16, 2018 at 9:09 AM, Bertus Greeff < bgre...@microsoft.com.invalid> wrote: > I'm interested to know the status of this KIP. I see that the status is > "Voting". How long does this normally take? > > We want to use Kafka and this KIP provides exactly the log compaction > logic that we want for many of our projects. > > One piece of feedback that I have is that log.cleaner.compaction.strategy > and log.cleaner.compaction.strategy.header needs to be per topic. The > text of the KIP makes it sound that the config is only available for all > topics but this makes no sense. Different topics will need different > strategies and/or headers. > > From the KIP: > Provide the configuration for the individual topics > None of the configurations for log compaction are available at topic > level, so adding it there is not a part of this KIP > > > > On 2018/04/05 08:44:00, Luís Cabral <l...@yahoo.com.INVALID> wrote: > > Hello all,> > > Starting a discussion for this feature.> > > KIP-280 : https://cwiki.apache.org/confluence/display/KAFKA/KIP-280% > 3A+Enhanced+log+compactionPull-4822 : https://github.com/apache/kafk > a/pull/4822> > > > Kind Regards,Luís> > -- -- Guozhang
