[
https://issues.apache.org/jira/browse/KAFKA-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16590425#comment-16590425
]
John Wilkinson edited comment on KAFKA-7314 at 8/23/18 4:09 PM:
----------------------------------------------------------------
Here are the docker commands I am using to start the kafka and zk instances:
-----
docker run -d \
--net=confluent \
--name=zookeeper \
-e ZOOKEEPER_CLIENT_PORT=2181 \
confluentinc/cp-zookeeper:latest
docker run -d \
--net=confluent \
--name=kafka \
-h `hostname` \
-p 9092:9092 \
-p 9094:9094 \
-e KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181 \
-e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092 \
-e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 \
confluentinc/cp-kafka:latest
-----
What I mean when I say the value of {{bootstrap.servers}} doesn't matter is
that regardless of what you put in them, mirrormaker consumes and publishes to
localhost. However, if you put unresolvable garbage in those values,
mirrormaker will fail to start with a DNS resolution error. So mirrormaker
appears to be reading those values, but then not using them.
This is easily 100% reproducible, just launch those two nodes in the docker
containers on two different hosts and try to use mirrormaker on them.
was (Author: jwilkinson):
Here are the docker commands I am using to start the kafka and zk instances:
{{docker run -d \
--net=confluent \
--name=zookeeper \
-e ZOOKEEPER_CLIENT_PORT=2181 \
confluentinc/cp-zookeeper:latest}}
{{docker run -d \
--net=confluent \
--name=kafka \
-h `hostname` \
-p 9092:9092 \
-p 9094:9094 \
-e KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181 \
-e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092 \
-e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 \
confluentinc/cp-kafka:latest}}
What I mean when I say the value of {{bootstrap.servers}} doesn't matter is
that regardless of what you put in them, mirrormaker consumes and publishes to
localhost. However, if you put unresolvable garbage in those values,
mirrormaker will fail to start with a DNS resolution error. So mirrormaker
appears to be reading those values, but then not using them.
This is easily 100% reproducible, just launch those two nodes in the docker
containers on two different hosts and try to use mirrormaker on them.
> MirrorMaker example in documentation does not work
> --------------------------------------------------
>
> Key: KAFKA-7314
> URL: https://issues.apache.org/jira/browse/KAFKA-7314
> Project: Kafka
> Issue Type: Bug
> Components: mirrormaker
> Reporter: John Wilkinson
> Priority: Critical
>
> Kafka MirrorMaker as described in the documentation
> [here|https://kafka.apache.org/documentation/#basic_ops_mirror_maker] does
> not work. Instead of pulling messages from the consumer-defined
> {{bootstrap.servers}} and pushing to the producer-defined
> {{bootstrap.servers}}, it consumes and producers on the same topic on the
> same host repeatedly.
> To replicate, set up two instances of kafka following
> [this|https://docs.confluent.io/current/installation/docker/docs/installation/recipes/single-node-client.html]
> guide. The schema registry and rest proxy are unnecessary.
> [Here|https://hub.docker.com/r/confluentinc/cp-kafka/] is the DockerHub page
> for the image. The Kafka version is 2.0.0.
> Using those two kafka instances, go {{docker exec}} into one and set up the
> {{consumer.properties}} and the {{producer.properties}} following the
> MirrorMaker guide.
> Oddly, if you put in garbage unresolvable server addresses in the config,
> there will be an error, despite the configs not getting used.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
