RE: leader update partitions fail with KeeperErrorCode = BadVersion,kafka version=0.8.1.1

2015-06-02 Thread chenlax
i create a topic with 72 partitions 2 replicas,then increased to 108,and the 
cluster is run ok.
some days later i find the topic has 2 partition which ISR only include 
leader,check follow log-segment with partitions,the log-segment does not later.

and i can not find more useful logs from kafka logs or zk logs.

if restart leader,the follow will become leader,and pre-leader whill back as 
follow.

Thanks,
Lax


> Date: Mon, 1 Jun 2015 16:14:15 -0400
> Subject: Re: leader update partitions fail with KeeperErrorCode = 
> BadVersion,kafka version=0.8.1.1
> From: j...@squareup.com
> To: users@kafka.apache.org
> 
> I've seen this problem now too with 0.8.2.1.  It happened after we had a
> disk failure (but the server failed to shutdown:  KAFKA-).  After that
> happened, subsequently, several ISR's underwent I think 'unclean leader
> election', but I'm not 100% sure. But I did see lots of those same error
> messages: "Cached zkVersion [X] not equal to that in zookeeper, skip
> updating ISR...".
> 
> So, I don't know that the issue was fixed in 0.8.1.1.
> 
> Can you describe the circumstances for the errors you saw?
> 
> Jason
> 
> On Fri, May 29, 2015 at 12:17 AM, chenlax  wrote:
> 
> > kafka version =0.8.1.1
> >
> >  i get the error log as follow:
> >
> > INFO Partition [Topic_Beacon_1,10] on broker 4: Shrinking ISR for
> > partition [Topic_Beacon_1,10] from 4,7 to 4 (kafka.cluster.Partition)
> > ERROR Conditional update of path
> > /brokers/topics/Topic_Beacon_1/partitions/10/state with data
> > {"controller_epoch":16,"leader":4,"version":1,"leader_epoch":7,"isr":[4]}
> > and expecte
> > d version 5032 failed due to
> > org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode =
> > BadVersion for /brokers/topics/Topic_Beacon_1/partitions/10/state
> > (kafka.utils.ZkUtils$)
> > INFO Partition [Topic_Beacon_1,10] on broker 4: Cached zkVersion [5032]
> > not equal to that in zookeeper, skip updating ISR (kafka.cluster.Partition)
> >
> >
> > Only restart broker,the issue will fix, why the broker can not back to ISR
> > for partition?
> >
> > and i find the same issue,
> > http://mail-archives.apache.org/mod_mbox/kafka-users/201404.mbox/%3CCAHwHRrW_vKsSpoAnhEqQUZVBT5_Djx3qbixsH8=6hAe=vg4...@mail.gmail.com%3E
> >
> > it point out kafka_0.8.1.1 fix the bug,so i want to know, what causes this
> > problem.
> >
> >
> > Thanks,
> > Lax
> >
  

Re: leader update partitions fail with KeeperErrorCode = BadVersion,kafka version=0.8.1.1

2015-06-01 Thread Jason Rosenberg
I've seen this problem now too with 0.8.2.1.  It happened after we had a
disk failure (but the server failed to shutdown:  KAFKA-).  After that
happened, subsequently, several ISR's underwent I think 'unclean leader
election', but I'm not 100% sure. But I did see lots of those same error
messages: "Cached zkVersion [X] not equal to that in zookeeper, skip
updating ISR...".

So, I don't know that the issue was fixed in 0.8.1.1.

Can you describe the circumstances for the errors you saw?

Jason

On Fri, May 29, 2015 at 12:17 AM, chenlax  wrote:

> kafka version =0.8.1.1
>
>  i get the error log as follow:
>
> INFO Partition [Topic_Beacon_1,10] on broker 4: Shrinking ISR for
> partition [Topic_Beacon_1,10] from 4,7 to 4 (kafka.cluster.Partition)
> ERROR Conditional update of path
> /brokers/topics/Topic_Beacon_1/partitions/10/state with data
> {"controller_epoch":16,"leader":4,"version":1,"leader_epoch":7,"isr":[4]}
> and expecte
> d version 5032 failed due to
> org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode =
> BadVersion for /brokers/topics/Topic_Beacon_1/partitions/10/state
> (kafka.utils.ZkUtils$)
> INFO Partition [Topic_Beacon_1,10] on broker 4: Cached zkVersion [5032]
> not equal to that in zookeeper, skip updating ISR (kafka.cluster.Partition)
>
>
> Only restart broker,the issue will fix, why the broker can not back to ISR
> for partition?
>
> and i find the same issue,
> http://mail-archives.apache.org/mod_mbox/kafka-users/201404.mbox/%3CCAHwHRrW_vKsSpoAnhEqQUZVBT5_Djx3qbixsH8=6hAe=vg4...@mail.gmail.com%3E
>
> it point out kafka_0.8.1.1 fix the bug,so i want to know, what causes this
> problem.
>
>
> Thanks,
> Lax
>


leader update partitions fail with KeeperErrorCode = BadVersion,kafka version=0.8.1.1

2015-05-28 Thread chenlax
kafka version =0.8.1.1

 i get the error log as follow:

INFO Partition [Topic_Beacon_1,10] on broker 4: Shrinking ISR for partition 
[Topic_Beacon_1,10] from 4,7 to 4 (kafka.cluster.Partition)
ERROR Conditional update of path 
/brokers/topics/Topic_Beacon_1/partitions/10/state with data 
{"controller_epoch":16,"leader":4,"version":1,"leader_epoch":7,"isr":[4]} and 
expecte
d version 5032 failed due to 
org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode = 
BadVersion for /brokers/topics/Topic_Beacon_1/partitions/10/state 
(kafka.utils.ZkUtils$)
INFO Partition [Topic_Beacon_1,10] on broker 4: Cached zkVersion [5032] not 
equal to that in zookeeper, skip updating ISR (kafka.cluster.Partition)


Only restart broker,the issue will fix, why the broker can not back to ISR for 
partition?

and i find the same issue, 
http://mail-archives.apache.org/mod_mbox/kafka-users/201404.mbox/%3CCAHwHRrW_vKsSpoAnhEqQUZVBT5_Djx3qbixsH8=6hAe=vg4...@mail.gmail.com%3E

it point out kafka_0.8.1.1 fix the bug,so i want to know, what causes this 
problem.


Thanks,
Lax