[jira] [Comment Edited] (KAFKA-10424) MirrorMaker 2.0 does not replicates topic's "clean.policy"
[ https://issues.apache.org/jira/browse/KAFKA-10424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182710#comment-17182710 ] Mikhail Grinfeld edited comment on KAFKA-10424 at 8/24/20, 6:25 AM: I can see the same behaviour with KTable topics generated by KStreams app :( was (Author: grinfeld): I can see the same behaviour with KTable topics generated by KStreams app, but in case I create topic with "*kafka-topics.sh --bootstrap-server localhost:9092 --create --topic my-topic --partitions 1 --replication-factor 1 --config max.message.bytes=64000 --config flush.messages=1*" the topic not replicated at all. I'll check my configuration, maybe I have some buggy conf in my app :( > MirrorMaker 2.0 does not replicates topic's "clean.policy" > -- > > Key: KAFKA-10424 > URL: https://issues.apache.org/jira/browse/KAFKA-10424 > Project: Kafka > Issue Type: Bug > Components: mirrormaker >Affects Versions: 2.5.0 >Reporter: Mikhail Grinfeld >Priority: Major > > I needed to replicate schema-registry "_schemas" topic. > data was replicated successfully and everything looked good, but new > schema-registry started with warning that replicated topic's cleanup.policy > is not "compact" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-10424) MirrorMaker 2.0 does not replicates topic's "clean.policy"
[ https://issues.apache.org/jira/browse/KAFKA-10424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182710#comment-17182710 ] Mikhail Grinfeld commented on KAFKA-10424: -- I can see the same behaviour with KTable topics generated by KStreams app, but in case I create topic with "*kafka-topics.sh --bootstrap-server localhost:9092 --create --topic my-topic --partitions 1 --replication-factor 1 --config max.message.bytes=64000 --config flush.messages=1*" the topic not replicated at all. I'll check my configuration, maybe I have some buggy conf in my app :( > MirrorMaker 2.0 does not replicates topic's "clean.policy" > -- > > Key: KAFKA-10424 > URL: https://issues.apache.org/jira/browse/KAFKA-10424 > Project: Kafka > Issue Type: Bug > Components: mirrormaker >Affects Versions: 2.5.0 >Reporter: Mikhail Grinfeld >Priority: Major > > I needed to replicate schema-registry "_schemas" topic. > data was replicated successfully and everything looked good, but new > schema-registry started with warning that replicated topic's cleanup.policy > is not "compact" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (KAFKA-10424) MirrorMaker 2.0 does not replicates topic's "clean.policy"
[ https://issues.apache.org/jira/browse/KAFKA-10424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182581#comment-17182581 ] Mikhail Grinfeld edited comment on KAFKA-10424 at 8/23/20, 6:29 AM: I checked again and it never copied cleanup.policy. was (Author: grinfeld): Thx for quick comment. I don't use prefixes, since I migrate data to new servers and only switch between broker.ids on client machines. Maybe, really, the reason is absent of prefixes. I'll try to debug and check it. Thx. > MirrorMaker 2.0 does not replicates topic's "clean.policy" > -- > > Key: KAFKA-10424 > URL: https://issues.apache.org/jira/browse/KAFKA-10424 > Project: Kafka > Issue Type: Bug > Components: mirrormaker >Affects Versions: 2.5.0 >Reporter: Mikhail Grinfeld >Priority: Major > > I needed to replicate schema-registry "_schemas" topic. > data was replicated successfully and everything looked good, but new > schema-registry started with warning that replicated topic's cleanup.policy > is not "compact" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-10424) MirrorMaker 2.0 does not replicates topic's "clean.policy"
[ https://issues.apache.org/jira/browse/KAFKA-10424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182581#comment-17182581 ] Mikhail Grinfeld commented on KAFKA-10424: -- Thx for quick comment. I don't use prefixes, since I migrate data to new servers and only switch between broker.ids on client machines. Maybe, really, the reason is absent of prefixes. I'll try to debug and check it. Thx. > MirrorMaker 2.0 does not replicates topic's "clean.policy" > -- > > Key: KAFKA-10424 > URL: https://issues.apache.org/jira/browse/KAFKA-10424 > Project: Kafka > Issue Type: Bug > Components: mirrormaker >Affects Versions: 2.5.0 >Reporter: Mikhail Grinfeld >Priority: Major > > I needed to replicate schema-registry "_schemas" topic. > data was replicated successfully and everything looked good, but new > schema-registry started with warning that replicated topic's cleanup.policy > is not "compact" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KAFKA-10424) MirrorMaker 2.0 does not replicates topic's "clean.policy"
Mikhail Grinfeld created KAFKA-10424: Summary: MirrorMaker 2.0 does not replicates topic's "clean.policy" Key: KAFKA-10424 URL: https://issues.apache.org/jira/browse/KAFKA-10424 Project: Kafka Issue Type: Bug Components: mirrormaker Affects Versions: 2.5.0 Reporter: Mikhail Grinfeld I needed to replicate schema-registry "_schemas" topic. data was replicated successfully and everything looked good, but new schema-registry started with warning that replicated topic's cleanup.policy is not "compact" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KAFKA-9482) mirror maker 2.0 doesn't replicates (create topic) created automatically by producing messages
Mikhail Grinfeld created KAFKA-9482:
---
Summary: mirror maker 2.0 doesn't replicates (create topic)
created automatically by producing messages
Key: KAFKA-9482
URL: https://issues.apache.org/jira/browse/KAFKA-9482
Project: Kafka
Issue Type: Bug
Components: mirrormaker
Affects Versions: 2.4.0
Reporter: Mikhail Grinfeld
I have 2 kafka (3 brokers each) clusters and MirrorMaker instance. (built with
docker and docker-compose)
Both cluster is defined to create topics automatically (no
auto.create.topics.enable in conf and default, at least as it appears in docs,
is true).
If topic created on source cluster before MirrorMaker starts, everything works
as expected: producig messages to source cluster, causes replication to
destination cluster.
If topic doesn't exist on source cluster, when starting to produce messages, it
created in source cluster, but not in destination - and no replication
performed.
Using following mm2.properties:
{code:java}
# mm2.propertiesclusters=src,dest
src.bootstrap.servers=kafka-1:9092,kafka-2:19092,kafka-3:29092
dest.bootstrap.servers=kafka-4:39092,kafka-5:49092,kafka-6:59092
src->dest.enabled=true
src->dest.topics=.*
{code}
and running MirrorMaker with
{code:java}
connect-mirror-maker /etc/mm2.properties --clusters src dest
{code}
Note:
when I am using Kafka-Streams to read from initial topic, there are few KTable
topics created automatically by Kafka-Stream - and these topics created OK (of
course, when initial topic created at the beginning)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
