[
https://issues.apache.org/jira/browse/KAFKA-3595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438666#comment-15438666
]
ASF GitHub Bot commented on KAFKA-3595:
---------------------------------------
GitHub user dguy opened a pull request:
https://github.com/apache/kafka/pull/1792
KAFKA-3595: window stores use compact,delete config for changelogs
changelogs of window stores now configure cleanup.policy=compact,delete
with retention.ms set to window maintainMs +
StreamsConfig.WINDOW_STORE_CHANGE_LOG_ADDITIONAL_RETENTION_MS_CONFIG
StoreChangeLogger produces messages with context.timestamp().
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dguy/kafka kafka-3595
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/1792.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1792
----
commit c7a13b09efc3695e1d60b8375fa16993c9f9a06e
Author: Damian Guy <[email protected]>
Date: 2016-08-12T14:42:09Z
window stores use compact,delete config for changelogs
----
> Add capability to specify replication compact option for stream store
> ---------------------------------------------------------------------
>
> Key: KAFKA-3595
> URL: https://issues.apache.org/jira/browse/KAFKA-3595
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Affects Versions: 0.10.1.0
> Reporter: Henry Cai
> Assignee: Damian Guy
> Priority: Minor
> Labels: user-experience
> Fix For: 0.10.1.0
>
>
> Currently state store replication always go through a compact kafka topic.
> For some state stores, e.g. JoinWindow, there are no duplicates in the store,
> there is not much benefit using a compacted topic.
> The problem of using compacted topic is the records can stay in kafka broker
> forever. In my use case, my key is ad_id, it's incrementing all the time, not
> bounded, I am worried the disk space on broker for that topic will go forever.
> I think we either need the capability to purge the compacted records on
> broker, or allow us to specify different compact option for state store
> replication.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
