Sean McKibben created KAFKA-4046:
------------------------------------
Summary: Expose Cleaner Offset for Compacted Topics
Key: KAFKA-4046
URL: https://issues.apache.org/jira/browse/KAFKA-4046
Project: Kafka
Issue Type: Improvement
Components: metrics, offset manager
Affects Versions: 0.10.0.1, 0.10.0.0, 0.9.0.1
Reporter: Sean McKibben
Priority: Minor
Certain use cases, such as a streaming distinct view on data, could make use of
topic compaction to retrieve a snapshot of unique messages from a compacted
topic. Currently, the point to which compaction has progressed is not exposed
for such use cases, though it appears to be readily available to the brokers.
Without that information being exposed to consumers for such use cases,
techniques such as pigging https://en.wikipedia.org/wiki/Pigging need to be
implemented on an ad-hoc basis to discover the approximate offset to which
compaction has progressed.
Simple API for retrieving the position of compaction for a topic partition
would be useful both operationally and for use cases which need the general
uniqueness guarantees offered by topic compaction.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
