No, I missed that.
thanks, svante 2015-01-07 6:44 GMT+01:00 Jun Rao <j...@confluent.io>: > Did you set offsets.storage to kafka in the consumer of mirror maker? > > Thanks, > > Jun > > On Mon, Jan 5, 2015 at 3:49 PM, svante karlsson <s...@csi.se> wrote: > > > I'm using 0.82beta and I'm trying to push data with the mirrormaker tool > > from several remote sites to two datacenters. I'm testing this from a > node > > containing zk, broker and mirrormaker and the data is pushed to a > "normal" > > cluster. 3 zk and 4 brokers with replication. > > > > While the configuration seems straight forward things are a bit shaky. > > > > First, if something goes wrong on either consumer or producer side it > stops > > transmitting data without dying which makes it harder to run under a > > supervisor (upstart in my case) > > > > I start pushing data to site 02 and mirrormaker pushes it to my > datacenter > > - so far all good... > > > > But, when I run. (on my remote site) > > kafka-consumer-offset-checker.sh > > --group mirrormaker.from.site_02.to.site_ktv --zookeeper 192.168.0.106 > > > > Could not fetch offset for [saka.test.ext_datastream,7] due to > > kafka.common.NotCoordinatorForConsumerException. > > Could not fetch offset for [saka.test.ext_datastream,4] due to > > kafka.common.NotCoordinatorForConsumerException. > > Could not fetch offset for [saka.test.ext_datastream,2] due to > > kafka.common.NotCoordinatorForConsumerException. > > Could not fetch offset for [saka.test.ext_datastream,5] due to > > kafka.common.NotCoordinatorForConsumerException. > > Could not fetch offset for [saka.test.ext_datastream,6] due to > > kafka.common.NotCoordinatorForConsumerException. > > Could not fetch offset for [saka.test.ext_datastream,3] due to > > kafka.common.NotCoordinatorForConsumerException. > > Could not fetch offset for [saka.test.ext_datastream,1] due to > > kafka.common.NotCoordinatorForConsumerException. > > Could not fetch offset for [saka.test.ext_datastream,0] due to > > kafka.common.NotCoordinatorForConsumerException. > > > > Group Topic Pid Offset > logSize > > Lag Owner > > mirrormaker.from.site_02.to.site_ktv saka.test.ext_datastream 0 > > unknown 9310560 unknown > > mirrormaker.from.site_02.to.site_ktv_kafka-4-1420497015616-725cd1b5-0 > > mirrormaker.from.site_02.to.site_ktv saka.test.ext_datastream 1 > > unknown 9313497 unknown > > mirrormaker.from.site_02.to.site_ktv_kafka-4-1420497015616-725cd1b5-0 > > mirrormaker.from.site_02.to.site_ktv saka.test.ext_datastream 2 > > unknown 9323623 unknown > > mirrormaker.from.site_02.to.site_ktv_kafka-4-1420497015616-725cd1b5-0 > > mirrormaker.from.site_02.to.site_ktv saka.test.ext_datastream 3 > > unknown 9334005 unknown > > mirrormaker.from.site_02.to.site_ktv_kafka-4-1420497015616-725cd1b5-0 > > mirrormaker.from.site_02.to.site_ktv saka.test.ext_datastream 4 > > unknown 9324176 unknown > > mirrormaker.from.site_02.to.site_ktv_kafka-4-1420497015616-725cd1b5-0 > > mirrormaker.from.site_02.to.site_ktv saka.test.ext_datastream 5 > > unknown 9336504 unknown > > mirrormaker.from.site_02.to.site_ktv_kafka-4-1420497015616-725cd1b5-0 > > mirrormaker.from.site_02.to.site_ktv saka.test.ext_datastream 6 > > unknown 9316139 unknown > > mirrormaker.from.site_02.to.site_ktv_kafka-4-1420497015616-725cd1b5-0 > > mirrormaker.from.site_02.to.site_ktv saka.test.ext_datastream 7 > > unknown 9318770 unknown > > mirrormaker.from.site_02.to.site_ktv_kafka-4-1420497015616-725cd1b5-0 > > > > > > Since I can push data between my sites something is working but what can > be > > the case for NotCoordinatorForConsumerException on a single node cluster? > > > > > > Finally the __consumer_offsets topic should probably be masked out from > > mirroring without using whitelists or blacklists > > > > > > thanks > > /svante > > >
