Hi Richard, thanks for the response .. i was able to get this to work, by adding auto.offset.reset=earliest in consumer.properties. Atleast the count matched, i'll check on the contents to ensure there are no duplicates.
However, when i set auto.offset.reset=latest, i did not see any effect. When would one need to set -> auto.offset.reset=latest, (as opposed to auto.offset.reset=earliest) ? On Wed, Jun 21, 2017 at 9:05 PM, Richard Shaw <rich...@aggress.net> wrote: > Karan, have you got auto.offset.reset in your consumer.properties? > > https://kafka.apache.org/documentation/#newconsumerconfigs > > On Thu, Jun 22, 2017 at 2:00 AM, karan alang <karan.al...@gmail.com> > wrote: > > > Hi All, > > > > I've 2 Kafka clusters (Kafka 10) & I'm trying to test the MirrorMaker > > functionality. > > > > Here is what i did : > > > > 1) I have identical topics Topic1 on 2 Kafka clusters - Cluster1 & > Cluster2 > > 2) On Cluster1, I publish 100 messages on Topic1 > > > > 3) I've 2 consumers reading messages from the 2 topics on Cluster1 & > > Cluster2 > > > > 4) I start MirrorMaker AFTER THE MESSAGES HAVE BEEN PUBLISHED ON Cluster1 > > > > $KAFKA10_HOME1/bin/kafka-run-class.sh kafka.tools.MirrorMaker > > --consumer.config $KAFKA10_HOME1/config/mmConsumer.config --num.streams > 3 > > --producer.config $KAFKA10_HOME1/config/mmProducer.config > > --whitelist="mmtopic1" --abort.on.send.failure true > > > > > > I expected that Once the MirrorMaker was started, the 100 messages > > published on Cluster1, would be published on Cluster2 > > > > However that is not happening... > > > > What needs to be done to enable this ? > > >
