[
https://issues.apache.org/jira/browse/SAMZA-918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15458715#comment-15458715
]
Tommy Becker commented on SAMZA-918:
------------------------------------
Is there a reason this can't be done at this point? What is the minimum Kafka
version Samza supports? We'd like to explore compressing topics but this
behavior in Samza is preventing us from doing so.
> Disable check to turn off compression for changelog topics
> ----------------------------------------------------------
>
> Key: SAMZA-918
> URL: https://issues.apache.org/jira/browse/SAMZA-918
> Project: Samza
> Issue Type: Improvement
> Reporter: Jagadish
> Assignee: Jagadish
>
> There was a Kafka bug that would cause the log cleaner thread on the broker
> to crash when it was compacting compressed topics. Since, changelog topics
> are log-compacted, we have enabled the override to explicitly disable
> compaction if a k-v store topic is compressed. This kafka bug
> https://issues.apache.org/jira/browse/KAFKA-1374 is fixed as of kafka
> 0.9.0.2. We must ensure that we disable that check as we move to kafka
> 0.9.0.2.
> Enabling compression has the following benefit:
> 1. When sending large messages (the limit of a kafka message is typically
> 1M), with compression and batching, the message sizes get reduced
> significantly. This helps Samza customers to write larger data to their K-V
> stores (because the size of the write you can do to a kafka-changelog backed
> store is dependent on the size of the Kafka message)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
