You need to set auto.offset.reset=smallest to mirror data from beginning On Mon, 24 Oct 2016 at 17:07 ZHU Hua B <hua.b....@alcatel-lucent.com> wrote:
> Hi, > > > Thanks for your info! > > > Before I launch mirror maker first time, there is a topic include some > messages, which have been produced and consumed on source Kafka cluster. > Then I launch mirror maker first time on target cluster, but this topic did > not mirror from source cluster. > > So I keep mirror maker running on target cluster, and produced a new > message to this topic on source cluster, this time the topic mirror from > source cluster to target, which include the new message only, if this > behavior is correct? Thanks! > > > > > > > Best Regards > > Johnny > > > -----Original Message----- > From: Manikumar [mailto:manikumar.re...@gmail.com] > Sent: 2016年10月24日 16:56 > To: users@kafka.apache.org > Subject: Re: Mirror multi-embedded consumer's configuration > > Yes, it should mirror again. To mirror the topic from beginning, you may > have to start with different consumer group, > > On Mon, Oct 24, 2016 at 12:39 PM, ZHU Hua B <hua.b....@alcatel-lucent.com> > wrote: > > > Hi, > > > > > > Many thanks for your confirm! > > > > I have another question, if I deleted a mirrored topic on the target > > Kafka cluster, if Kafka mirror maker could mirror the same topic again > > from source cluster when I launch mirror maker next time? Thanks! > > > > > > > > > > > > > > Best Regards > > > > Johnny > > > > > > -----Original Message----- > > From: Manikumar [mailto:manikumar.re...@gmail.com] > > Sent: 2016年10月24日 15:03 > > To: users@kafka.apache.org > > Subject: Re: Mirror multi-embedded consumer's configuration > > > > I think we need to update the docs. This behavior got changed longtime > ago. > > we need to pass only one consumer configuration. This config will be > > applicable for all the consumer threads/instances. > > > > On Mon, Oct 24, 2016 at 11:43 AM, ZHU Hua B > > <hua.b....@alcatel-lucent.com> > > wrote: > > > > > Hi, > > > > > > > > > Because from some Kafka wiki I saw "At minimum, the mirror maker > > > takes one or more consumer configurations, a producer configuration > > > and either a whitelist or a blacklist", but the test failed, so I > > > want to know if Kafka mirror really support more consumer > > > configurations at > > minimum? Thanks! > > > > > > > > > > > > > > > Best Regards > > > > > > Johnny > > > > > > > > > -----Original Message----- > > > From: Manikumar [mailto:manikumar.re...@gmail.com] > > > Sent: 2016年10月24日 13:48 > > > To: users@kafka.apache.org > > > Subject: Re: Mirror multi-embedded consumer's configuration > > > > > > why are you passing "consumer.config" twice? > > > > > > On Mon, Oct 24, 2016 at 11:07 AM, ZHU Hua B > > > <hua.b....@alcatel-lucent.com> > > > wrote: > > > > > > > Hi, > > > > > > > > > > > > The version of Kafka I used is 0.10.0.0. Thanks! > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best Regards > > > > > > > > Johnny > > > > > > > > -----Original Message----- > > > > From: Guozhang Wang [mailto:wangg...@gmail.com] > > > > Sent: 2016年10月24日 12:22 > > > > To: users@kafka.apache.org > > > > Subject: Re: Mirror multi-embedded consumer's configuration > > > > > > > > Which version are you using for the MM? > > > > > > > > Guozhang > > > > > > > > On Thu, Oct 20, 2016 at 10:13 PM, ZHU Hua B > > > > <hua.b....@alcatel-lucent.com> > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > > > > > > Anybody could help to answer below question? Thanks! > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best Regards > > > > > > > > > > Johnny > > > > > > > > > > From: ZHU Hua B > > > > > Sent: 2016年10月19日 16:22 > > > > > To: 'users@kafka.apache.org' > > > > > Subject: Mirror multi-embedded consumer's configuration > > > > > > > > > > Hi, > > > > > > > > > > > > > > > I launch Kafka mirror maker with multi-embedded consumer's > > > > > configuration but failed as below, what’s the mean of “you asked > > > > > for only one”, is there an option control it? Thanks! > > > > > > > > > > # bin/kafka-mirror-maker.sh --consumer.config > > > > > config/consumer-1.properties --consumer.config > > > > > config/consumer-2.properties --num.streams 2 --producer.config > > > > config/producer.properties --whitelist '.*' > > > > > [2016-10-19 16:00:14,183] ERROR Exception when starting mirror > maker. > > > > > (kafka.tools.MirrorMaker$) > > > > > joptsimple.MultipleArgumentsForOptionException: Found multiple > > > > > arguments for option consumer.config, but you asked for only one > > > > > at joptsimple.OptionSet.valueOf(OptionSet.java:179) > > > > > at kafka.tools.MirrorMaker$.main(MirrorMaker.scala:235) > > > > > at kafka.tools.MirrorMaker.main(MirrorMaker.scala) > > > > > Exception in thread "main" java.lang.NullPointerException > > > > > at kafka.tools.MirrorMaker$.main(MirrorMaker.scala:286) > > > > > at kafka.tools.MirrorMaker.main(MirrorMaker.scala) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best Regards > > > > > > > > > > Johnny > > > > > > > > > > > > > > > > > > > > > > -- > > > > -- Guozhang > > > > > > > > > >
