I could be totally wrong, but I seem to recall that delete wasn't fully implemented in 0.8.x?
On Fri, Jul 21, 2017 at 10:10 AM, Carl Haferd <chaf...@groupon.com.invalid> wrote: > Chris, > > You could first check to make sure that delete.topic.enable is true and try > deleting again if not. If that doesn't work with 0.8.1.1 you might need to > manually remove the topic's log files from the configured log.dirs folder > on each broker in addition to removing the topic's zookeeper path. > > Carl > > On Thu, Jul 20, 2017 at 10:06 AM, Chris Neal <cwn...@gmail.com> wrote: > > > Hi all, > > > > I have a weird situation here. I have deleted a few topics on my 0.8.1.1 > > cluster (old, I know...). The deletes succeeded according to the > > controller.log: > > > > [2017-07-20 16:40:31,175] INFO [TopicChangeListener on Controller 1]: New > > topics: [Set()], deleted topics: > > [Set(perf_doorway-supplier-adapter-uat_raw)], new partition replica > > assignment [Map()] > > (kafka.controller.PartitionStateMachine$TopicChangeListener) > > [2017-07-20 16:40:33,507] INFO [TopicChangeListener on Controller 1]: New > > topics: [Set()], deleted topics: > > [Set(perf_doorway-supplier-scheduler-uat_raw)], new partition replica > > assignment [Map()] > > (kafka.controller.PartitionStateMachine$TopicChangeListener) > > [2017-07-20 16:40:36,504] INFO [TopicChangeListener on Controller 1]: New > > topics: [Set()], deleted topics: [Set(perf_gocontent-uat_raw)], new > > partition replica assignment [Map()] > > (kafka.controller.PartitionStateMachine$TopicChangeListener) > > [2017-07-20 16:40:38,290] INFO [TopicChangeListener on Controller 1]: New > > topics: [Set()], deleted topics: [Set(perf_goplatform-uat_raw)], new > > partition replica assignment [Map()] > > (kafka.controller.PartitionStateMachine$TopicChangeListener) > > > > I query Zookeeper and the path is not there under /brokers/topics as > well. > > > > But, one of the nodes in my cluster continues to try and use them: > > > > [2017-07-20 17:04:36,723] ERROR Conditional update of path > > /brokers/topics/perf_doorway-supplier-scheduler-uat_raw/ > partitions/3/state > > with data > > {"controller_epoch":34,"leader":1,"version":1,"leader_ > > epoch":2,"isr":[1,0]} > > and expected version 69 failed due to > > org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = > > NoNode for > > /brokers/topics/perf_doorway-supplier-scheduler-uat_raw/ > partitions/3/state > > (kafka.utils.ZkUtils$) > > [2017-07-20 17:04:36,723] INFO Partition > > [perf_doorway-supplier-scheduler-uat_raw,3] on broker 1: Cached > zkVersion > > [69] not equal to that in zookeeper, skip updating ISR > > (kafka.cluster.Partition) > > [2017-07-20 17:04:36,723] INFO Partition > > [perf_doorway-supplier-scheduler-uat_raw,3] on broker 1: Cached > zkVersion > > [69] not equal to that in zookeeper, skip updating ISR > > (kafka.cluster.Partition) > > [2017-07-20 17:04:36,764] INFO Partition [perf_goplatform-uat_raw,2] on > > broker 1: Shrinking ISR for partition [perf_goplatform-uat_raw,2] from > 1,0 > > to 1 (kafka.cluster.Partition) > > [2017-07-20 17:04:36,764] INFO Partition [perf_goplatform-uat_raw,2] on > > broker 1: Shrinking ISR for partition [perf_goplatform-uat_raw,2] from > 1,0 > > to 1 (kafka.cluster.Partition) > > [2017-07-20 17:04:36,765] ERROR Conditional update of path > > /brokers/topics/perf_goplatform-uat_raw/partitions/2/state with data > > {"controller_epoch":34,"leader":1,"version":1,"leader_ > epoch":2,"isr":[1]} > > and expected version 70 failed due to > > org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = > > NoNode for /brokers/topics/perf_goplatform-uat_raw/partitions/2/state > > (kafka.utils.ZkUtils$) > > [2017-07-20 17:04:36,765] ERROR Conditional update of path > > /brokers/topics/perf_goplatform-uat_raw/partitions/2/state with data > > {"controller_epoch":34,"leader":1,"version":1,"leader_ > epoch":2,"isr":[1]} > > and expected version 70 failed due to > > org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = > > NoNode for /brokers/topics/perf_goplatform-uat_raw/partitions/2/state > > (kafka.utils.ZkUtils$) > > [2017-07-20 17:04:36,765] INFO Partition [perf_goplatform-uat_raw,2] on > > broker 1: Cached zkVersion [70] not equal to that in zookeeper, skip > > updating ISR (kafka.cluster.Partition) > > [2017-07-20 17:04:36,765] INFO Partition [perf_goplatform-uat_raw,2] on > > broker 1: Cached zkVersion [70] not equal to that in zookeeper, skip > > updating ISR (kafka.cluster.Partition) > > [2017-07-20 17:04:36,981] INFO Partition [perf_gocontent-uat_raw,1] on > > broker 1: Shrinking ISR for partition [perf_gocontent-uat_raw,1] from 1,0 > > to 1 (kafka.cluster.Partition) > > [2017-07-20 17:04:36,981] INFO Partition [perf_gocontent-uat_raw,1] on > > broker 1: Shrinking ISR for partition [perf_gocontent-uat_raw,1] from 1,0 > > to 1 (kafka.cluster.Partition) > > [2017-07-20 17:04:36,988] ERROR Conditional update of path > > /brokers/topics/perf_gocontent-uat_raw/partitions/1/state with data > > {"controller_epoch":34,"leader":1,"version":1,"leader_ > epoch":4,"isr":[1]} > > and expected version 90 failed due to > > org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = > > NoNode for /brokers/topics/perf_gocontent-uat_raw/partitions/1/state > > (kafka.utils.ZkUtils$) > > [2017-07-20 17:04:36,988] ERROR Conditional update of path > > /brokers/topics/perf_gocontent-uat_raw/partitions/1/state with data > > {"controller_epoch":34,"leader":1,"version":1,"leader_ > epoch":4,"isr":[1]} > > and expected version 90 failed due to > > org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = > > NoNode for /brokers/topics/perf_gocontent-uat_raw/partitions/1/state > > (kafka.utils.ZkUtils$) > > [2017-07-20 17:04:36,988] INFO Partition [perf_gocontent-uat_raw,1] on > > broker 1: Cached zkVersion [90] not equal to that in zookeeper, skip > > updating ISR (kafka.cluster.Partition) > > [2017-07-20 17:04:36,988] INFO Partition [perf_gocontent-uat_raw,1] on > > broker 1: Cached zkVersion [90] not equal to that in zookeeper, skip > > updating ISR (kafka.cluster.Partition) > > > > I've tried a rolling restart of the cluster to see if that fixed it, but > it > > did not. > > > > Can someone please help me out here? I'm not sure how I can get things > > back in sync. > > > > Thank you so much for your time. > > Chris > > >