[
https://issues.apache.org/jira/browse/SAMZA-586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Riccomini updated SAMZA-586:
----------------------------------
Attachment: SAMZA-586-0.patch
Attaching patch. RB at:
https://reviews.apache.org/r/32127
# Added StorageConfig.isChangelogSystem to check if a system has a changelog
attached.
# Wrote KafkaSystemFactory.getInjectedProducerProperties, which uses
isChangelogSystem to force compression.type=none if the system has at least one
changelog system attached.
# Wrote tests for both.
# Noticed that KafkaCheckpointManagerFactory is still using compression.code
instead of compression.type. This was missed during SAMZA-227. The [new
producer config|http://kafka.apache.org/documentation.html#newproducerconfigs]
use the {{compression.type}} key. Updated this.
> LoggedStore can send compressed messages to log-compacted Kafka topic
> ---------------------------------------------------------------------
>
> Key: SAMZA-586
> URL: https://issues.apache.org/jira/browse/SAMZA-586
> Project: Samza
> Issue Type: Bug
> Components: kafka
> Affects Versions: 0.9.0
> Reporter: Chris Riccomini
> Fix For: 0.9.0
>
> Attachments: SAMZA-586-0.patch
>
>
> I think we might have an issue with changelogs that use Kafka as its backing
> store. The Kafka topics are log-compacted. If the system is configured to use
> compression (systems.<sysname>.producer.compression.type=gzip), I believe
> that the LoggedStore will send compressed messages to the log-compacted
> topic. The LoggedStore shares the normal producer based on system name. Due
> to KAFKA-1374, this will cause the log compaction thread to die, or at the
> very least, not compress the topic.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
