Updated the doc in 0.8.1. Thanks,
Jun On Mon, Mar 24, 2014 at 2:44 PM, Guozhang Wang <wangg...@gmail.com> wrote: > Hi Churu, > > The documentations needs to be updated. > > Basically, you need to specify the producer and consumer config files as > part of the parameters: > > bin/kafka-run-class.sh kafka.tools.MirrorMaker > Missing required argument "[consumer.config]" > Option > Description > ------ > ----------- > --blacklist <Java regex (String)> Blacklist of topics to > mirror. > *--consumer.config <config file> Consumer config to consume from > a * > source cluster. You may > specify > multiple of > these. > --help Print this > message. > --num.producers <Integer: Number of Number of producer instances > (default: > producers> > 1) > --num.streams <Integer: Number of Number of consumption > streams. > threads> (default: > 1) > > *--producer.config <config file> Embedded producer > config. *--queue.size <Integer: Queue size in Number of > messages that are buffered > terms of number of messages> between the consumer and > producer > (default: > 10000) > --whitelist <Java regex (String)> Whitelist of topics to > mirror. > > > The configs of producer/consumer can be found here: > > http://kafka.apache.org/documentation.html#configuration > > For most of them you can just go with the defaults, but for some of them > such as the metadata.broker.list and zookeeper.connect string you are > required to specify the value. > > Guozhang > > > On Mon, Mar 24, 2014 at 2:29 PM, Churu Tang <ct...@rubiconproject.com > >wrote: > > > Hi, > > > > 1. In documentation 0.8.1, the mirror maker command is: > > bin/kafka-run-class.sh kafka.tools.MirrorMaker --zookeeper > > zk_host:port/chroot > > --consumer.config consumer-1.properties --consumer.config > > consumer-2.properties > > --producer.config producer.properties --whitelist my-topic > > > > However, the zookeeper option seems not allowed. > > [kafka_2.10-0.8.1]]$ bin/kafka-run-class.sh kafka.tools.MirrorMaker > > --zookeeper localhost:2181 --consumer.config config/consumer1.properties > > --producer.config config/producer.properties --whitelist="test" > > Exception in thread "main" joptsimple.UnrecognizedOptionException: > > 'zookeeper' is not a recognized option > > at > > joptsimple.OptionException.unrecognizedOption(OptionException.java:93) > > at > > joptsimple.OptionParser.handleLongOptionToken(OptionParser.java:402) > > at > > joptsimple.OptionParserState$2.handleArgument(OptionParserState.java:55) > > at joptsimple.OptionParser.parse(OptionParser.java:392) > > at kafka.tools.MirrorMaker$.main(MirrorMaker.scala:90) > > at kafka.tools.MirrorMaker.main(MirrorMaker.scala) > > > > 2. After I remove the zookeeper option, the command seems still not work. > > I have ensured that the configuration files will direct the consumer to > the > > source cluster's ZooKeeper, and the producer to the mirror cluster's > > ZooKeeper. The following is the output for both target and source > clusters. > > Is there any other thing that I need to watch out? Thank you very much! > > > > At target side: > > [kafka_2.10-0.8.1]$ bin/kafka-run-class.sh kafka.tools.MirrorMaker > > --consumer.config config/consumer1.properties --producer.config > > config/producer.properties --whitelist="test" > > SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". > > SLF4J: Defaulting to no-operation (NOP) logger implementation > > SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for > further > > details. > > > > At source zookeeper side: > > [2014-03-24 14:12:19,285] INFO Accepted socket connection from / > > 10.96.120.31:60102 (org.apache.zookeeper.server.NIOServerCnxn) > > [2014-03-24 14:12:19,291] INFO Client attempting to establish new session > > at /10.96.120.31:60102 (org.apache.zookeeper.server.NIOServerCnxn) > > [2014-03-24 14:12:19,295] INFO Established session 0x144f5e0cb510003 with > > negotiated timeout 6000 for client /10.96.120.31:60102 > (org.apache.zookeeper.server.NIOServerCnxn) > > > > At source broker side: > > [2014-03-24 14:12:20,386] INFO Closing socket connection to / > 10.96.120.31. > > (kafka.network.Processor) > > > > Cheers, > > Churu > > > > > > > > > -- > -- Guozhang >
