Malthe Borch created KAFKA-14873:
------------------------------------
Summary: Pluggable storage for Kafka Connect internal topics
Key: KAFKA-14873
URL: https://issues.apache.org/jira/browse/KAFKA-14873
Project: Kafka
Issue Type: Improvement
Reporter: Malthe Borch
The Kafka Connect framework relies on compacted topics to store config, offset
and status information for each connector.
This conflates two kinds of data, control and content, which some people
disagree with. Notably, [Azure Event
Hub|https://learn.microsoft.com/en-us/azure/event-hubs/log-compaction] does not
(or _did not_, because there's currently a preview release out which does have
support for compacted topics albeit only at the more expensive premium tiers).
In some deployments, it may be desirable to use a different backend for these
control settings (which essentially take a key/value form), for example [Azure
Table
Storage|https://learn.microsoft.com/en-us/rest/api/storageservices/table-service-rest-api]
– basically any key/value store that provides the Write-If-Matches primitive
to update a key only if the current value matches a known value.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
