Hi,
It is OK now, thanks for your help! Best Regards Johnny -----Original Message----- From: tao xiao [mailto:xiaotao...@gmail.com] Sent: 2016年10月24日 17:43 To: users@kafka.apache.org Subject: Re: Mirror multi-embedded consumer's configuration auto.offset.reset=smallest takes effect only when there is no offset committed prior or offset being requested is out of range. I believe your mirror maker has committed offset therefore auto.offset.reset=smallest wont take effect. You can create a new consumer group and try again On Mon, 24 Oct 2016 at 17:25 ZHU Hua B <hua.b....@alcatel-lucent.com> wrote: Hi, Thanks for your info! I set auto.offset.reset=smallest in consumer.properties on target cluster and retest, it still failed. Seems the topic with original messages could not mirror to target cluster while running mirror maker first time, unless produce a new message to this topic to trigger mirror. Thanks! Best Regards Johnny -----Original Message----- From: tao xiao [mailto:xiaotao...@gmail.com] Sent: 2016年10月24日 17:10 To: users@kafka.apache.org Subject: Re: Mirror multi-embedded consumer's configuration You need to set auto.offset.reset=smallest to mirror data from beginning On Mon, 24 Oct 2016 at 17:07 ZHU Hua B <hua.b....@alcatel-lucent.com> wrote: > Hi, > > > Thanks for your info! > > > Before I launch mirror maker first time, there is a topic include some > messages, which have been produced and consumed on source Kafka cluster. > Then I launch mirror maker first time on target cluster, but this > topic did not mirror from source cluster. > > So I keep mirror maker running on target cluster, and produced a new > message to this topic on source cluster, this time the topic mirror > from source cluster to target, which include the new message only, if > this behavior is correct? Thanks! > > > > > > > Best Regards > > Johnny > > > -----Original Message----- > From: Manikumar [mailto:manikumar.re...@gmail.com] > Sent: 2016年10月24日 16:56 > To: users@kafka.apache.org > Subject: Re: Mirror multi-embedded consumer's configuration > > Yes, it should mirror again. To mirror the topic from beginning, you > may have to start with different consumer group, > > On Mon, Oct 24, 2016 at 12:39 PM, ZHU Hua B > <hua.b....@alcatel-lucent.com> > wrote: > > > Hi, > > > > > > Many thanks for your confirm! > > > > I have another question, if I deleted a mirrored topic on the target > > Kafka cluster, if Kafka mirror maker could mirror the same topic > > again from source cluster when I launch mirror maker next time? Thanks! > > > > > > > > > > > > > > Best Regards > > > > Johnny > > > > > > -----Original Message----- > > From: Manikumar [mailto:manikumar.re...@gmail.com] > > Sent: 2016年10月24日 15:03 > > To: users@kafka.apache.org > > Subject: Re: Mirror multi-embedded consumer's configuration > > > > I think we need to update the docs. This behavior got changed > > longtime > ago. > > we need to pass only one consumer configuration. This config will be > > applicable for all the consumer threads/instances. > > > > On Mon, Oct 24, 2016 at 11:43 AM, ZHU Hua B > > <hua.b....@alcatel-lucent.com> > > wrote: > > > > > Hi, > > > > > > > > > Because from some Kafka wiki I saw "At minimum, the mirror maker > > > takes one or more consumer configurations, a producer > > > configuration and either a whitelist or a blacklist", but the test > > > failed, so I want to know if Kafka mirror really support more > > > consumer configurations at > > minimum? Thanks! > > > > > > > > > > > > > > > Best Regards > > > > > > Johnny > > > > > > > > > -----Original Message----- > > > From: Manikumar [mailto:manikumar.re...@gmail.com] > > > Sent: 2016年10月24日 13:48 > > > To: users@kafka.apache.org > > > Subject: Re: Mirror multi-embedded consumer's configuration > > > > > > why are you passing "consumer.config" twice? > > > > > > On Mon, Oct 24, 2016 at 11:07 AM, ZHU Hua B > > > <hua.b....@alcatel-lucent.com> > > > wrote: > > > > > > > Hi, > > > > > > > > > > > > The version of Kafka I used is 0.10.0.0. Thanks! > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best Regards > > > > > > > > Johnny > > > > > > > > -----Original Message----- > > > > From: Guozhang Wang [mailto:wangg...@gmail.com] > > > > Sent: 2016年10月24日 12:22 > > > > To: users@kafka.apache.org > > > > Subject: Re: Mirror multi-embedded consumer's configuration > > > > > > > > Which version are you using for the MM? > > > > > > > > Guozhang > > > > > > > > On Thu, Oct 20, 2016 at 10:13 PM, ZHU Hua B > > > > <hua.b....@alcatel-lucent.com> > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > > > > > > Anybody could help to answer below question? Thanks! > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best Regards > > > > > > > > > > Johnny > > > > > > > > > > From: ZHU Hua B > > > > > Sent: 2016年10月19日 16:22 > > > > > To: 'users@kafka.apache.org' > > > > > Subject: Mirror multi-embedded consumer's configuration > > > > > > > > > > Hi, > > > > > > > > > > > > > > > I launch Kafka mirror maker with multi-embedded consumer's > > > > > configuration but failed as below, what’s the mean of “you > > > > > asked for only one”, is there an option control it? Thanks! > > > > > > > > > > # bin/kafka-mirror-maker.sh --consumer.config > > > > > config/consumer-1.properties --consumer.config > > > > > config/consumer-2.properties --num.streams 2 --producer.config > > > > config/producer.properties --whitelist '.*' > > > > > [2016-10-19 16:00:14,183] ERROR Exception when starting mirror > maker. > > > > > (kafka.tools.MirrorMaker$) > > > > > joptsimple.MultipleArgumentsForOptionException: Found multiple > > > > > arguments for option consumer.config, but you asked for only one > > > > > at joptsimple.OptionSet.valueOf(OptionSet.java:179) > > > > > at kafka.tools.MirrorMaker$.main(MirrorMaker.scala:235) > > > > > at kafka.tools.MirrorMaker.main(MirrorMaker.scala) > > > > > Exception in thread "main" java.lang.NullPointerException > > > > > at kafka.tools.MirrorMaker$.main(MirrorMaker.scala:286) > > > > > at kafka.tools.MirrorMaker.main(MirrorMaker.scala) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best Regards > > > > > > > > > > Johnny > > > > > > > > > > > > > > > > > > > > > > -- > > > > -- Guozhang > > > > > > > > > >
