Just to add (in case the platoform is Windows)
For Windows based cluster implementation, log/topic cleanup doesn't work
out of the box. Users are more or less aware of it, and doing their own
maintenance as workaround.
If you have issues on Topic deletion not working properly on Windows (i.e.
with topic deletion enable and all other settings). then you have to
manually delete the files.
On 21 July 2017 at 16:53, Chris Neal <cwn...@gmail.com> wrote:
> @Carl,
>
> There is nothing under /admin/delete_topics other than
>
> []
>
> And nothing under /admin other than delete_topics :)
>
> The topics DO exist, however, under /config/topics! We may be on to
> something. I will remove them here and see if that clears it up.
>
> Thanks so much for all the help!
> Chris
>
> On Thu, Jul 20, 2017 at 10:37 PM, Chris Neal <cwn...@gmail.com> wrote:
>
> > Thanks again for the replies. VERY much appreciated. I'll check both
> > /admin/delete_topics and /config/topics.
> >
> > Chris
> >
> > On Thu, Jul 20, 2017 at 9:22 PM, Carl Haferd <chaf...@groupon.com.invalid
> >
> > wrote:
> >
> >> If delete normally works, there would hopefully be some log entries when
> >> it
> >> fails. Are there any unusual zookeeper entries in the
> >> /admin/delete_topics
> >> path or in the other /admin folders?
> >>
> >> Does the topic name still exist in zookeeper under /config/topics? If
> so,
> >> that should probably deleted as well.
> >>
> >> Carl
> >>
> >> On Thu, Jul 20, 2017 at 6:42 PM, Chris Neal <cwn...@gmail.com> wrote:
> >>
> >> > Delete is definitely there. The delete worked fine, based on the fact
> >> that
> >> > there is nothing in Zookeeper, and that the controller reported that
> the
> >> > delete was successful, it's just something seems to have gotten out of
> >> > sync.
> >> >
> >> > delete.topic.enabled is true. I've successfully deleted topics in the
> >> > past, so I know it *should* work. :)
> >> >
> >> > I also had already checked in Zookeeper, and there is no directory for
> >> the
> >> > topics under /brokers/topics.... Very strange indeed.
> >> >
> >> > If I just remove the log directories from the filesystem, is that
> >> enough to
> >> > get the broker to stop asking about the topics? I would guess there
> >> would
> >> > need to be more than just that, but I could be wrong.
> >> >
> >> > Thanks guys for the suggestions though!
> >> >
> >> > On Thu, Jul 20, 2017 at 8:19 PM, Stephen Powis <spo...@salesforce.com
> >
> >> > wrote:
> >> >
> >> > > I could be totally wrong, but I seem to recall that delete wasn't
> >> fully
> >> > > implemented in 0.8.x?
> >> > >
> >> > > On Fri, Jul 21, 2017 at 10:10 AM, Carl Haferd
> >> > <chaf...@groupon.com.invalid
> >> > > >
> >> > > wrote:
> >> > >
> >> > > > Chris,
> >> > > >
> >> > > > You could first check to make sure that delete.topic.enable is
> true
> >> and
> >> > > try
> >> > > > deleting again if not. If that doesn't work with 0.8.1.1 you
> might
> >> > need
> >> > > to
> >> > > > manually remove the topic's log files from the configured log.dirs
> >> > folder
> >> > > > on each broker in addition to removing the topic's zookeeper path.
> >> > > >
> >> > > > Carl
> >> > > >
> >> > > > On Thu, Jul 20, 2017 at 10:06 AM, Chris Neal <cwn...@gmail.com>
> >> wrote:
> >> > > >
> >> > > > > Hi all,
> >> > > > >
> >> > > > > I have a weird situation here. I have deleted a few topics on
> my
> >> > > 0.8.1.1
> >> > > > > cluster (old, I know...). The deletes succeeded according to
> the
> >> > > > > controller.log:
> >> > > > >
> >> > > > > [2017-07-20 16:40:31,175] INFO [TopicChangeListener on
> Controller
> >> 1]:
> >> > > New
> >> > > > > topics: [Set()], deleted topics:
> >> > > > > [Set(perf_doorway-supplier-adapter-uat_raw)], new partition
> >> replica
> >> > > > > assignment [Map()]
> >> > > > > (kafka.controller.PartitionStateMachine$TopicChangeListener)
> >> > > > > [2017-07-20 16:40:33,507] INFO [TopicChangeListener on
> Controller
> >> 1]:
> >> > > New
> >> > > > > topics: [Set()], deleted topics:
> >> > > > > [Set(perf_doorway-supplier-scheduler-uat_raw)], new partition
> >> > replica
> >> > > > > assignment [Map()]
> >> > > > > (kafka.controller.PartitionStateMachine$TopicChangeListener)
> >> > > > > [2017-07-20 16:40:36,504] INFO [TopicChangeListener on
> Controller
> >> 1]:
> >> > > New
> >> > > > > topics: [Set()], deleted topics: [Set(perf_gocontent-uat_raw)],
> >> new
> >> > > > > partition replica assignment [Map()]
> >> > > > > (kafka.controller.PartitionStateMachine$TopicChangeListener)
> >> > > > > [2017-07-20 16:40:38,290] INFO [TopicChangeListener on
> Controller
> >> 1]:
> >> > > New
> >> > > > > topics: [Set()], deleted topics: [Set(perf_goplatform-uat_raw)]
> ,
> >> new
> >> > > > > partition replica assignment [Map()]
> >> > > > > (kafka.controller.PartitionStateMachine$TopicChangeListener)
> >> > > > >
> >> > > > > I query Zookeeper and the path is not there under
> /brokers/topics
> >> as
> >> > > > well.
> >> > > > >
> >> > > > > But, one of the nodes in my cluster continues to try and use
> them:
> >> > > > >
> >> > > > > [2017-07-20 17:04:36,723] ERROR Conditional update of path
> >> > > > > /brokers/topics/perf_doorway-supplier-scheduler-uat_raw/
> >> > > > partitions/3/state
> >> > > > > with data
> >> > > > > {"controller_epoch":34,"leader":1,"version":1,"leader_
> >> > > > > epoch":2,"isr":[1,0]}
> >> > > > > and expected version 69 failed due to
> >> > > > > org.apache.zookeeper.KeeperException$NoNodeException:
> >> > KeeperErrorCode
> >> > > =
> >> > > > > NoNode for
> >> > > > > /brokers/topics/perf_doorway-supplier-scheduler-uat_raw/
> >> > > > partitions/3/state
> >> > > > > (kafka.utils.ZkUtils$)
> >> > > > > [2017-07-20 17:04:36,723] INFO Partition
> >> > > > > [perf_doorway-supplier-scheduler-uat_raw,3] on broker 1: Cached
> >> > > > zkVersion
> >> > > > > [69] not equal to that in zookeeper, skip updating ISR
> >> > > > > (kafka.cluster.Partition)
> >> > > > > [2017-07-20 17:04:36,723] INFO Partition
> >> > > > > [perf_doorway-supplier-scheduler-uat_raw,3] on broker 1: Cached
> >> > > > zkVersion
> >> > > > > [69] not equal to that in zookeeper, skip updating ISR
> >> > > > > (kafka.cluster.Partition)
> >> > > > > [2017-07-20 17:04:36,764] INFO Partition
> >> [perf_goplatform-uat_raw,2]
> >> > on
> >> > > > > broker 1: Shrinking ISR for partition
> [perf_goplatform-uat_raw,2]
> >> > from
> >> > > > 1,0
> >> > > > > to 1 (kafka.cluster.Partition)
> >> > > > > [2017-07-20 17:04:36,764] INFO Partition
> >> [perf_goplatform-uat_raw,2]
> >> > on
> >> > > > > broker 1: Shrinking ISR for partition
> [perf_goplatform-uat_raw,2]
> >> > from
> >> > > > 1,0
> >> > > > > to 1 (kafka.cluster.Partition)
> >> > > > > [2017-07-20 17:04:36,765] ERROR Conditional update of path
> >> > > > > /brokers/topics/perf_goplatform-uat_raw/partitions/2/state with
> >> data
> >> > > > > {"controller_epoch":34,"leader":1,"version":1,"leader_
> >> > > > epoch":2,"isr":[1]}
> >> > > > > and expected version 70 failed due to
> >> > > > > org.apache.zookeeper.KeeperException$NoNodeException:
> >> > KeeperErrorCode
> >> > > =
> >> > > > > NoNode for /brokers/topics/perf_goplatform-uat_raw/partitions/
> >> > 2/state
> >> > > > > (kafka.utils.ZkUtils$)
> >> > > > > [2017-07-20 17:04:36,765] ERROR Conditional update of path
> >> > > > > /brokers/topics/perf_goplatform-uat_raw/partitions/2/state with
> >> data
> >> > > > > {"controller_epoch":34,"leader":1,"version":1,"leader_
> >> > > > epoch":2,"isr":[1]}
> >> > > > > and expected version 70 failed due to
> >> > > > > org.apache.zookeeper.KeeperException$NoNodeException:
> >> > KeeperErrorCode
> >> > > =
> >> > > > > NoNode for /brokers/topics/perf_goplatform-uat_raw/partitions/
> >> > 2/state
> >> > > > > (kafka.utils.ZkUtils$)
> >> > > > > [2017-07-20 17:04:36,765] INFO Partition
> >> [perf_goplatform-uat_raw,2]
> >> > on
> >> > > > > broker 1: Cached zkVersion [70] not equal to that in zookeeper,
> >> skip
> >> > > > > updating ISR (kafka.cluster.Partition)
> >> > > > > [2017-07-20 17:04:36,765] INFO Partition
> >> [perf_goplatform-uat_raw,2]
> >> > on
> >> > > > > broker 1: Cached zkVersion [70] not equal to that in zookeeper,
> >> skip
> >> > > > > updating ISR (kafka.cluster.Partition)
> >> > > > > [2017-07-20 17:04:36,981] INFO Partition
> >> [perf_gocontent-uat_raw,1]
> >> > on
> >> > > > > broker 1: Shrinking ISR for partition [perf_gocontent-uat_raw,1]
> >> from
> >> > > 1,0
> >> > > > > to 1 (kafka.cluster.Partition)
> >> > > > > [2017-07-20 17:04:36,981] INFO Partition
> >> [perf_gocontent-uat_raw,1]
> >> > on
> >> > > > > broker 1: Shrinking ISR for partition [perf_gocontent-uat_raw,1]
> >> from
> >> > > 1,0
> >> > > > > to 1 (kafka.cluster.Partition)
> >> > > > > [2017-07-20 17:04:36,988] ERROR Conditional update of path
> >> > > > > /brokers/topics/perf_gocontent-uat_raw/partitions/1/state with
> >> data
> >> > > > > {"controller_epoch":34,"leader":1,"version":1,"leader_
> >> > > > epoch":4,"isr":[1]}
> >> > > > > and expected version 90 failed due to
> >> > > > > org.apache.zookeeper.KeeperException$NoNodeException:
> >> > KeeperErrorCode
> >> > > =
> >> > > > > NoNode for /brokers/topics/perf_gocontent
> >> -uat_raw/partitions/1/state
> >> > > > > (kafka.utils.ZkUtils$)
> >> > > > > [2017-07-20 17:04:36,988] ERROR Conditional update of path
> >> > > > > /brokers/topics/perf_gocontent-uat_raw/partitions/1/state with
> >> data
> >> > > > > {"controller_epoch":34,"leader":1,"version":1,"leader_
> >> > > > epoch":4,"isr":[1]}
> >> > > > > and expected version 90 failed due to
> >> > > > > org.apache.zookeeper.KeeperException$NoNodeException:
> >> > KeeperErrorCode
> >> > > =
> >> > > > > NoNode for /brokers/topics/perf_gocontent
> >> -uat_raw/partitions/1/state
> >> > > > > (kafka.utils.ZkUtils$)
> >> > > > > [2017-07-20 17:04:36,988] INFO Partition
> >> [perf_gocontent-uat_raw,1]
> >> > on
> >> > > > > broker 1: Cached zkVersion [90] not equal to that in zookeeper,
> >> skip
> >> > > > > updating ISR (kafka.cluster.Partition)
> >> > > > > [2017-07-20 17:04:36,988] INFO Partition
> >> [perf_gocontent-uat_raw,1]
> >> > on
> >> > > > > broker 1: Cached zkVersion [90] not equal to that in zookeeper,
> >> skip
> >> > > > > updating ISR (kafka.cluster.Partition)
> >> > > > >
> >> > > > > I've tried a rolling restart of the cluster to see if that fixed
> >> it,
> >> > > but
> >> > > > it
> >> > > > > did not.
> >> > > > >
> >> > > > > Can someone please help me out here? I'm not sure how I can get
> >> > things
> >> > > > > back in sync.
> >> > > > >
> >> > > > > Thank you so much for your time.
> >> > > > > Chris
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>
