Ivan, I think log.cleaner.delete.retention.ms does just that?
"The amount of time to retain delete tombstone markers for log compacted topics. This setting also gives a bound on the time in which a consumer must complete a read if they begin from offset 0 to ensure that they get a valid snapshot of the final stage (otherwise delete tombstones may be collected before they complete their scan). This setting can be overridden on a per-topic basis (see the per-topic configuration section)." http://kafka.apache.org/documentation.html#brokerconfigs -James > On Mar 2, 2015, at 8:57 AM, Ivan Balashov <ibalas...@gmail.com> wrote: > > Svante, > > Not sure if I understand your suggestion correctly, but I do think > that enabling retention for deleted values would make a useful > addition to the "compact" policy. Otherwise some data is bound to be > hanging around not used. > > Guozhang, could this potentially deserve a feature request? > > Thanks, > > > 2015-03-02 19:40 GMT+03:00 svante karlsson <s...@csi.se>: >> Wouldn't it be rather simple to add a retention time on "deleted" items ie >> keys with null value for topics that are compacted? >> >> The retention time would then be set to some "large" time to allow all >> consumers to understand that a previous k/v is being deleted.
