You may need to look into the consumer metrics and producer metrics to identify the root cause. metrics in kafka.consumer and kafka.producer categories will help you find out the problems.
This link gives instruction how to read the metrics http://kafka.apache.org/documentation.html#monitoring On Wed, Apr 8, 2015 at 3:39 AM, nitin sharma <kumarsharma.ni...@gmail.com> wrote: > hi, > > sorry for late response. ... i have been able to fix the issue .. problem > was in my approach. I got confused between my source and target system > while defining consumer & producer property file .. it is fixed now > > Now new issue.. the rate at which data is migrated is very very slow... i > mean it took 5 min to copy only 15Kb.. :( .. here are the property for > producer and consumer.. there is no network latency between Source and > Destination clusters as such. > > > #### Producer ########### > metadata.broker.list=<broker1IP>:9092,<broker2IP>:9092 > serializer.class=kafka.serializer.DefaultEncoder > auto.create.topics.enable=true > request.required.acks=1 > request.required.acks=1 > producer.type=async > batch.num.messages=3000 > queue.buffering.max.ms=5000 > queue.buffering.max.messages=100000 > queue.enqueue.timeout.ms=-1 > socket.send.buffer.bytes=5282880 > > ######### Consumer ########### > zookeeper.connect=<zk1hostname>:2181,<zk2hostname>:2181,<zk3hostname>:2181 > group.id=KafkaMaker > auto.create.topics.enable=true > socket.receive.buffer.bytes=5243880 > zookeeper.connection.timeout.ms=1000000 > num.consumer.fetchers=20 > fetch.message.max.bytes=5243880 > > > > > Regards, > Nitin Kumar Sharma. > > > On Tue, Mar 31, 2015 at 12:36 PM, tao xiao <xiaotao...@gmail.com> wrote: > > > Can you attach your mirror maker log? > > > > On Wed, Apr 1, 2015 at 12:28 AM, nitin sharma < > kumarsharma.ni...@gmail.com > > > > > wrote: > > > > > i tried with auto.offset.reset=smallest, but still not working.. > > > > > > there is data in my source cluster > > > > > > Regards, > > > Nitin Kumar Sharma. > > > > > > > > > On Mon, Mar 30, 2015 at 10:30 PM, tao xiao <xiaotao...@gmail.com> > wrote: > > > > > > > Do you have data sending to *testtopic? *By default mirror maker only > > > > consumes data being sent after it taps into the topic. you need to > keep > > > > sending data to the topic after mirror maker connection is > established. > > > If > > > > you want to change the behavior you can set > auto.offset.reset=smallest > > so > > > > that any new mirror maker coming to the topic will start from the > > > smallest > > > > offset > > > > > > > > On Tue, Mar 31, 2015 at 3:53 AM, nitin sharma < > > > kumarsharma.ni...@gmail.com > > > > > > > > > wrote: > > > > > > > > > thanks Xiao > > > > > > > > > > I tried MirrorMaker option in my test environment but failed. I am > > not > > > > > able to see the log getting copied to destination cluster. I see in > > the > > > > log > > > > > of MirrorMaker process that connection is successfully established > > > > between > > > > > source and destination cluster but still not sure what i causing > the > > > > problem > > > > > > > > > > Env. Setup ==> > > > > > > > > > > I). Source Cluster (Qenv02) -- i have 2 > broker(Qenv02kf01,Qenv02kf02) > > > and > > > > > 3 zk(Qenv02zk01,Qenv02zk02 and Qenv02zk03). > > > > > Destination Clustern (Qenv05) -- i have 2 broker > > > (Qenv05kf01,Qenv05kf02) > > > > > and 3 zk(Qenv05zk01,Qenv05zk02 and Qenv05zk03). > > > > > > > > > > II). i have kept consumer and producer properties file in one of > the > > > > > source kafka broker config folder. > > > > > > > > > > III).i have executed following command from the same kafka broker > to > > > > > start the process.. log are attached : > > > > > > > > > > /app/kafka/bin/kafka-run-class.sh kafka.tools.MirrorMaker > > > > -consumer.config > > > > > /app/kafka/config/consumer1.properties --num.streams=2 > > > --producer.config > > > > > /app/kafka/config/producer1.properties --whitelist testtopic > > > > > > > > > > > > > > > IV). I tried Consumer offset tracker tool also, while Mirror Maker > > > > running > > > > > . I tried by launching second session of same broker where mirror > > maker > > > > is > > > > > running. I got error message that "*NoNode for > > > > > /consumers/KafkaMaker/offsets/testtopic/0*" .Complete log attached. > > > > > > > > > > > > > > > Regards, > > > > > Nitin Kumar Sharma. > > > > > > > > > > > > > > > On Thu, Mar 26, 2015 at 11:24 AM, tao xiao <xiaotao...@gmail.com> > > > wrote: > > > > > > > > > >> Both consumer-1 and consumer-2 are properties of source clusters > > > mirror > > > > >> maker transfers data from. Mirror maker is designed to be able to > > > > consume > > > > >> data from N sources (N >= 1) and transfer data to one destination > > > > cluster. > > > > >> You are free to supply as many consumer properties as you want to > > > > instruct > > > > >> mirror maker where to consumer data from. > > > > >> > > > > >> On Thu, Mar 26, 2015 at 9:50 PM, nitin sharma < > > > > >> kumarsharma.ni...@gmail.com> > > > > >> wrote: > > > > >> > > > > >> > thanks Mayuresh and Jiangjie for your response. > > > > >> > > > > > >> > I have actually not understood Mirror maker clearly and hence > bit > > > > >> skeptical > > > > >> > if i will be able to execute it effectively. > > > > >> > > > > > >> > Online i hv seen the following command to be execute, but not > > > > understood > > > > >> > what is consumer-1 & -2.properties here? do i need to copy from > > my > > > > >> > consumer code? also, any reason why to provide consumer > property? > > > > >> > > > > > >> > bin/kafka-run-class.sh kafka.tools.MirrorMaker --consumer.config > > > > >> > consumer-1.properties --consumer.config consumer-2.properties > > > > >> > --producer.config producer.properties --whitelist my-topic > > > > >> > > > > > >> > > > > > >> > Regards, > > > > >> > Nitin Kumar Sharma. > > > > >> > > > > > >> > > > > > >> > On Wed, Mar 25, 2015 at 8:57 PM, Mayuresh Gharat < > > > > >> > gharatmayures...@gmail.com > > > > >> > > wrote: > > > > >> > > > > > >> > > You can use the Mirror maker to move data from one data center > > to > > > > >> other > > > > >> > and > > > > >> > > once all the data has been moved you can shut down the source > > data > > > > >> center > > > > >> > > by doing a controlled shutdown. > > > > >> > > > > > > >> > > Thanks, > > > > >> > > > > > > >> > > Mayuresh > > > > >> > > > > > > >> > > On Wed, Mar 25, 2015 at 2:35 PM, Jiangjie Qin > > > > >> <j...@linkedin.com.invalid > > > > >> > > > > > > >> > > wrote: > > > > >> > > > > > > >> > > > If you want to do a seamless migration. I think a better way > > is > > > to > > > > >> > build > > > > >> > > a > > > > >> > > > cross datacenter Kafka cluster temporarily. So the process > is: > > > > >> > > > 1. Add several new Kafka brokers in your new datacenter and > > add > > > > >> them to > > > > >> > > > the old cluster. > > > > >> > > > 2. Use replica assignment tool to reassign all the > partitions > > to > > > > >> > brokers > > > > >> > > > in new datacenter. > > > > >> > > > 3. Perform controlled shutdown on the brokers in old > > datacenter. > > > > >> > > > > > > > >> > > > Jiangjie (Becket) Qin > > > > >> > > > > > > > >> > > > On 3/25/15, 2:01 PM, "nitin sharma" < > > > kumarsharma.ni...@gmail.com> > > > > >> > wrote: > > > > >> > > > > > > > >> > > > >Hi Team, > > > > >> > > > > > > > > >> > > > >in my project, we have built a new datacenter for Kafka > > brokers > > > > and > > > > >> > > wants > > > > >> > > > >to migrate from current datacenter to new one. > > > > >> > > > > > > > > >> > > > >Switching producers and consumers wont be a problem > provided > > > New > > > > >> > > > >Datacenter > > > > >> > > > >has all the messages of existing Datacenter. > > > > >> > > > > > > > > >> > > > > > > > > >> > > > >i have only 1 topic with 2 partition that need to be > > > migrated... > > > > >> that > > > > >> > > too > > > > >> > > > >it is only 1 time activity. > > > > >> > > > > > > > > >> > > > >Kindly suggest the best way to deal with this situation. > > > > >> > > > > > > > > >> > > > > > > > > >> > > > >Regards, > > > > >> > > > >Nitin Kumar Sharma. > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > >> > > -- > > > > >> > > -Regards, > > > > >> > > Mayuresh R. Gharat > > > > >> > > (862) 250-7125 > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > >> > > > > >> -- > > > > >> Regards, > > > > >> Tao > > > > >> > > > > > > > > > > > > > > > > > > > > > > -- > > > > Regards, > > > > Tao > > > > > > > > > > > > > > > -- > > Regards, > > Tao > > > -- Regards, Tao
