I think this exception is logged by the consumer when there is a leadership
change for a partition (which might be caused by one of the brokers going
down or losing their session with zookeeper). Typically the consumer will
recover after the cluster finds a new leader and the consumer updates its
metadata. Are you not seeing that?
On Mon, Jun 22, 2015 at 5:05 PM, Srividhya Anantharamakrishnan <
srivid...@hedviginc.com> wrote:
> ok, let me make it a little more clear.
>
> Datacenter A has 3 nodes, each acting as a broker, publishing messages to
> one of the nodes that has zookeeper running.
>
> Datacenter B has the same set up.
>
> Now, I am trying to publish message from one of the nodes in A to the ZK in
> A and make one of the nodes in B consume the message by connecting to A's
> ZK.
>
>
>
> On Mon, Jun 22, 2015 at 4:25 PM, Jason Gustafson <ja...@confluent.io>
> wrote:
>
> > Hi Srividhya,
> >
> > I'm a little confused about your setup. You have both clusters pointed to
> > the same zookeeper, right? You don't appear to be using the zookeeper
> > chroot option, so I think they would just form a single cluster.
> >
> > -Jason
> >
> > On Mon, Jun 22, 2015 at 3:50 PM, Srividhya Anantharamakrishnan <
> > srivid...@hedviginc.com> wrote:
> >
> > > Hi,
> > >
> > > I have the following Kafka Set up - Two 3-node clusters A, B, where
> each
> > > node is acting as a broker and is connected by on ZK running in one
> node
> > in
> > > cluster A.
> > >
> > > I was able to publish messages from cluster A and could subscribe to
> > > consuming from both A and B clusters.
> > >
> > > However, I am suddenly running into the following issue:
> > >
> > > [ConsumerFetcherThread-group1_<NodeName>-1435011736956-12cac90d-0-2]
> > ERROR
> > > [kafka.consumer.ConsumerFetcherThread] (Logging.scala:97) -
> > > [ConsumerFetcherThread-group1_<NodeName>-1435011736956-12cac90d-0-2],
> > Error
> > > for partition [TestTop,0] to broker 2:class
> > > kafka.common.NotLeaderForPartitionException
> > >
> > > I did ensure that all the 3 brokers are running. I have the following
> > > consumer properties:
> > >
> > > Properties props = new Properties();
> > >
> > > props.put("zookeeper.connect", "192.168.111.203:2181");
> > >
> > > props.put("group.id", "group1");
> > >
> > > props.put("zookeeper.session.timeout.ms", "6000");
> > >
> > > props.put("zookeeper.sync.time.ms", "2000");
> > >
> > > props.put("auto.commit.enable", "true");
> > >
> > > props.put("auto.commit.interval.ms", "1000");
> > >
> > > props.put("auto.offset.reset", "smallest");
> > >
> > > props.put("consumer.timeout.ms", "120000");
> > >
> > > props.put("backoff.increment.ms", "1500");
> > > Any pointers would really help!
> > >
> > > Thanks!
> > >
> >
>
