Alex, Are you sure you have data continually being sent to the topic in source cluster after you bring up MM? By default auto.offset.reset=largest in MM consumer config which means MM only fetches the largest offset if the consumer group has no initial offset in zookeeper.
You can have MM print more log by changing the log level in config/tools-log4j.properties On Sun, Feb 15, 2015 at 8:39 AM, Alex Melville <amelvi...@g.hmc.edu> wrote: > Hi Kafka'ers, > > > I am trying to get the Mirrormaker working with two separate clusters, one > as the source and the other as the target. The topic I'm trying to copy > over exists on both the source and target clusters. Here are the relevant > entries in my consumer and producer properties files, which I'm specifying > the command I run to start the MM: > > *mirrorconsumer.properties:* > zookeeper.connect=ad-0104:2181 > zookeeper.connection.timeout.ms=6000 > group.id=test-consumer-group > > > *mirrorproducer.properties:* > metadata.broker.list=ba-02:9092,ba-03:9092 > producer.type=sync > compression.codec=none > serializer.class=kafka.serializer.DefaultEncoder > > > Then I run the following command: > bin/kafka-run-class.sh kafka.tools.MirrorMaker --consumer.config > ../config/mirrorconsumer.properties --producer.config > ../config/mirrorproducer.properties --whitelist consolemm > > so consolemm is the topic I'm trying to copy over. I've created consolemm > and have used to console-consumer to verify that there are messages in the > topic. > > When I run this command... nothing happens. The process keeps running and > prints nothing to the Terminal. If I look in the output of the zookeeper on > the source cluster I get only the following: > > [2015-02-15 00:34:06,102] INFO Accepted socket connection from / > 10.7.162.75:42819 (org.apache.zookeeper.server.NIOServerCnxnFactory) > [2015-02-15 00:34:06,104] INFO Client attempting to establish new session > at /10.7.162.75:42819 (org.apache.zookeeper.server.ZooKeeperServer) > [2015-02-15 00:34:06,106] INFO Established session 0x14b668b0fbe0033 with > negotiated timeout 6000 for client /10.7.162.75:42819 > (org.apache.zookeeper.server.ZooKeeperServer) > > > and when I look at the output of one of the brokers on the source cluster I > get: > > [2015-02-15 00:32:14,382] INFO Closing socket connection to /10.7.162.75. > (kafka.network.Processor) > > and there is no output on the zookeeper on the target cluster. > > > > Any advice on what is causing MM to not properly copy over data to the > target cluster would be extremely helpful. > > -Alex > -- Regards, Tao
