Jagadish created SAMZA-918:
------------------------------
Summary: Disable check to turn off compression for changelog topics
Key: SAMZA-918
URL: https://issues.apache.org/jira/browse/SAMZA-918
Project: Samza
Issue Type: Improvement
Reporter: Jagadish
Assignee: Jagadish
There was a Kafka bug that would cause the log cleaner thread on the broker to
crash when it was compacting compressed topics. Since, changelog topics are
log-compacted, we have enabled the override to explicitly disable compaction if
a k-v store topic is compressed. This kafka bug
https://issues.apache.org/jira/browse/KAFKA-1374 is fixed as of kafka 0.9.0.2.
We must ensure that we disable that check as we move to kafka 0.9.0.2.
Enabling compression has the following benefit:
1. When sending large messages (the limit of a kafka message is typically 1M),
with compression and batching, the message sizes get reduced significantly.
This helps Samza customers to write larger data to their K-V stores (because
the size of the write you can do to a kafka-changelog backed store is dependent
on the size of the Kafka message)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
