Eazhilan Nagarajan created KAFKA-10728:
------------------------------------------
Summary: Mirroring data without decompressing with MirrorMaker 2.0
Key: KAFKA-10728
URL: https://issues.apache.org/jira/browse/KAFKA-10728
Project: Kafka
Issue Type: Improvement
Components: mirrormaker
Reporter: Eazhilan Nagarajan
Hello,
I use MirrorMaker 2.0 to copy data across two Kafka clusters and it's all
working fine. Recently we enabled compressing while producing data into any
topic which had a very positive impact on the storage and other resources but
while mirroring, the data seems to be decompressed at the target Kafka cluster.
I tried enabling compression using the below config in MM2, the data at the
target cluster is compressed now, the decompress and re-compress continues to
happen and it eats up a lot of resources unnecessarily.
{noformat}
- alias: my-passive-cluster
authentication:
passwordSecret:
password: password
secretName: passive-cluster-secret
type: scram-sha-512
username: user-1
bootstrapServers: my-passive-cluster.com:443
config:
config.storage.replication.factor: 3
offset.storage.replication.factor: 3
status.storage.replication.factor: 3
producer.compression.type: gzip{noformat}
I found couple of Jira issues talking about it but I don't know if the shallow
iterator option is available now.
https://issues.apache.org/jira/browse/KAFKA-732,
https://issues.apache.org/jira/browse/KAFKA-845
Kindly let me if this is currently available or if it'll be available in the
future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
