Re: delete topic causing spikes in fetch/metadata requests

[Zakee]

These errors are possibly caused by delete topic (esp. using v0.8.2.x), refer 
below issues.

https://issues.apache.org/jira/browse/KAFKA-2231
https://issues.apache.org/jira/browse/KAFKA-1194

The replica fetchers still querying their leaders for partitions of deleted 
topic. May need a restart of all brokers and possibly ZK (cleanup of data 
directory) to get rid of these errors. I had the same issue (using 0.10.1.0) 
which claims the issue above is fixed. However, rolling restart of all Zk 
instances only helped get rid of the error in my case.

Thanks.

> On Oct 16, 2016, at 8:37 AM, sunil kalva  wrote:
> 
> Hi
> Can you guys help me with this issue
> 
> On Oct 12, 2016 10:35 PM, "sunil kalva"  wrote:
> 
>> 
>> We are using kafka 0.8.2.2 (client and server), when ever we delete a
>> topic we see lot of errors in broker logs like below, and there is also a
>> spike in fetch/metadata requests. Can i correlate these errors with topic
>> delete or its a known issue. Since there is spike in metadata requests and
>> fetch requests broker throughput has comedown.
>> 
>> 
>> 
>> 
>> --
>> [2016-10-12 16:04:55,054] ERROR [Replica Manager on Broker 4]: Error when
>> processing fetch request for partition [xyz,0] offset 161946645 from
>> consumer with correlation id 0. Possible cause: Request for offset
>> 161946645 but we only have log segments in the range 185487049 to
>> 202816546. (kafka.server.ReplicaManager)
>> [2016-10-12 16:04:55,056] ERROR [Replica Manager on Broker 4]: Error when
>> processing fetch request for partition [xyz,0] offset 161946645 from
>> consumer with correlation id 0. Possible cause: Request for offset
>> 161946645 but we only have log segments in the range 185487049 to
>> 202816546. (kafka.server.ReplicaManager)
>> [2016-10-12 16:04:55,057] ERROR [Replica Manager on Broker 4]: Error when
>> processing fetch request for partition [xyz,0] offset 161946645 from
>> consumer with correlation id 0. Possible cause: Request for offset
>> 161946645 but we only have log segments in the range 185487049 to
>> 202816546. (kafka.server.ReplicaManager)
>> [2016-10-12 16:04:55,059] ERROR [Replica Manager on Broker 4]: Error when
>> processing fetch request for partition [xyz,0] offset 161946645 from
>> consumer with correlation id 0. Possible cause: Request for offset
>> 161946645 but we only have log segments in the range 185487049 to
>> 202816546. (kafka.server.ReplicaManager)
>> [2016-10-12 16:04:55,060] ERROR [Replica Manager on Broker 4]: Error when
>> processing fetch request for partition [xyz,0] offset 161946645 from
>> consumer with correlation id 0. Possible cause: Request for offset
>> 161946645 but we only have log segments in the range 185487049 to
>> 202816546. (kafka.server.ReplicaManager)
>> [2016-10-12 16:04:55,062] ERROR [Replica Manager on Broker 4]: Error when
>> processing fetch request for partition [xyz,0] offset 161946645 from
>> consumer with correlation id 0. Possible cause: Request for offset
>> 161946645 but we only have log segments in the range 185487049 to
>> 202816546. (kafka.server.ReplicaManager)
>> [2016-10-12 16:04:55,064] ERROR [Replica Manager on Broker 4]: Error when
>> processing fetch request for partition [xyz,0] offset 161946645 from
>> consumer with correlation id 0. Possible cause: Request for offset
>> 161946645 but we only have log segments in the range 185487049 to
>> 202816546. (kafka.server.ReplicaManager)
>> [2016-10-12 16:04:55,065] ERROR [Replica Manager on Broker 4]: Error when
>> processing fetch request for partition [xyz,0] offset 161946645 from
>> consumer with correlation id 0. Possible cause: Request for offset
>> 161946645 but we only have log segments in the range 185487049 to
>> 202816546. (kafka.server.ReplicaManager)
>> [2016-10-12 16:04:55,067] ERROR [Replica Manager on Broker 4]: Error when
>> processing fetch request for partition [xyz,0] offset 161946645 from
>> consumer with correlation id 0. Possible cause: Request for offset
>> 161946645 but we only have log segments in the range 185487049 to
>> 202816546. (kafka.server.ReplicaManager)
>> [2016-10-12 16:04:55,068] ERROR [Replica Manager on Broker 4]: Error when
>> processing fetch request for partition [xyz,0] offset 161946645 from
>> consumer with correlation id 0. Possible cause: Request for offset
>> 161946645 but we only have log segments in the range 185487049 to
>> 202816546. (kafka.server.ReplicaManager)
>> [2016-10-12 16:04:55,070] ERROR [Replica Manager on Broker 4]: Error when
>> processing fetch request for partition [xyz,0] offset 161946645 from
>> consumer with correlation id 0. Possible cause: Request for offset
>> 161946645 but we only have log segments in the range 185487049 

Re: delete topic causing spikes in fetch/metadata requests

[sunil kalva]

Hi
Can you guys help me with this issue

On Oct 12, 2016 10:35 PM, "sunil kalva"  wrote:

>
> We are using kafka 0.8.2.2 (client and server), when ever we delete a
> topic we see lot of errors in broker logs like below, and there is also a
> spike in fetch/metadata requests. Can i correlate these errors with topic
> delete or its a known issue. Since there is spike in metadata requests and
> fetch requests broker throughput has comedown.
>
> 
> 
> 
> --
> [2016-10-12 16:04:55,054] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,056] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,057] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,059] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,060] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,062] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,064] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,065] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,067] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,068] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,070] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,072] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,073] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 

Re: Delete topic and logs

[Feroze Daud]

unsubscribe 

On Saturday, February 20, 2016 9:22 AM, Neelesh  wrote:
 

 I searched for this and could not get a definitive answer- when do logs get
deleted , after a topic is deleted? What happens if I delete a topic and
recreate it immediately?

So far I know that deleting a topic marks it for deleted in zk, and some
sweeper eventually deletes the topic meta and logs. It's quite not clear if
there is a state in between that may  reattach old logs to the newly
created topic with the same name?
Thank you!


  

Re: Delete topic using kafka-topics.sh also not working (or takes forever)

[JIEFU GONG]

How long do you have your retention configuration set for under
server.properties? Is it under 30 hours?

On Thu, Jul 16, 2015 at 10:44 PM, Ashish Singh asi...@cloudera.com wrote:

 Hi Marina,

 Is this reproducible? Can you reproduce this with a smaller number of
 events? Did you see any error in broker /server logs?

 On Thu, Jul 16, 2015 at 5:42 PM, Marina ppi...@yahoo.com.invalid wrote:

  A variation on the posted question - as a new thread so that not to
  high-jack the original one...
  How long does it usually take to finally delete the topic? I tried to
  delete the topic via
  /bin/kafka-topics.sh --delete --zookeeper localhost:2181 --topic mytopic
  and saw that it was marked for deletion. However,  about 30 hours
 later,
  the topic is still showing up in the list of topics:
  = bin/kafka-topics.sh --list --zookeeper localhost:2181
  __consumer_offsets
  mytopic
 
  The topic did have about 150 million events as Log size, but none were
  unprocessed (Lag == 0).
  As a test , I tried to create a new topic , and delete it right away with
  no events posted - and in that case it was deleted for good right away.
  I'm using kafka 1.8.2.1
  thanks!MArina
 
From: Stevo Slavić ssla...@gmail.com
   To: users@kafka.apache.org
   Sent: Thursday, July 16, 2015 6:56 PM
   Subject: Re: Delete topic using Admintools is not working
 
  Hello Sivananda,
 
  Calling AdminUtils.deleteTopic just requests topic to be deleted - it
 does
  not actually delete topic immediately. Requests for topic deletion get
  saved in ZooKeeper as a node (named by topic name), under
  /admin/delete_topics node.
 
  If brokers in the cluster are configured with topic deletion enabled, and
  if they are running, they will notice requests to delete the topic,
 perform
  actual deletion (all partitions on all brokers, lead and replicas, as
 well
  as deletion of topic metadata in zookeeper), and also delete saved
 request
  for topic deletion (that node under /admin/delete_topics). This broker
 side
  part of topic deletion process is only working in Kafka 0.8.2.x.
 
  From your stack trace it seems you've requested topic deletion at least
  twice, while topic has not been deleted yet, so zk client used by admin
  utils, throws exception signalling that the node (request for topic
  deletion) already exists.
 
  If topic is not getting deleted even after a while I'd suggest that you
  check version, configuration and running status of your Kafka cluster.
 
  Kind regards,
  Stevo Slavic.
 
 
 
  On Fri, Jul 17, 2015 at 12:40 AM, Sivananda Reddy 
 sivananda2...@gmail.com
  
  wrote:
 
   Hi,
  
   As per the jira https://issues.apache.org/jira/browse/KAFKA-1737, I
 have
   set ZKStringSerializer, while creating the ZkClient and still the
 delete
   topic function call is not working:
  
   Version: kafka_2.11-0.8.2.1
  
   *My code looks like this*:
   ZkClient zkClient = new ZkClient(localhost:2181, 6000, 6000,
   ZKStringSerializer$.MODULE$);
   AdminUtils.deleteTopic(zkClient, testTopic17);
  
   *I am getting the following exception's*:
   Exception in thread main
   org.I0Itec.zkclient.exception.ZkNodeExistsException:
   org.apache.zookeeper.KeeperException$NodeExistsException:
  KeeperErrorCode =
   NodeExists for /admin/delete_topics/testTopic17
  at
   org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:55)
  at
 org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:685)
  at org.I0Itec.zkclient.ZkClient.create(ZkClient.java:304)
  at org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:243)
  at kafka.utils.ZkUtils$.createPersistentPath(ZkUtils.scala:306)
  at kafka.admin.AdminUtils$.deleteTopic(AdminUtils.scala:159)
  at kafka.admin.AdminUtils.deleteTopic(AdminUtils.scala)
  at Test.deleteTopic(Test.java:98)
  at Test.main(Test.java:76)
   Caused by: org.apache.zookeeper.KeeperException$NodeExistsException:
   KeeperErrorCode = NodeExists for /admin/delete_topics/testTopic17
  at
   org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
  at
  org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
  at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
  at org.I0Itec.zkclient.ZkConnection.create(ZkConnection.java:87)
  at org.I0Itec.zkclient.ZkClient$1.call(ZkClient.java:308)
  at org.I0Itec.zkclient.ZkClient$1.call(ZkClient.java:304)
  at
 org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675)
  ... 7 more
  
   *The corresponding zookeeper logs*:
   [2015-07-16 15:31:14,383] INFO Accepted socket connection from /
   127.0.0.1:23363 (org.apache.zookeeper.server.NIOServerCnxnFactory)
   [2015-07-16 15:31:14,385] INFO Client attempting to establish new
 session
   at /127.0.0.1:23363 (org.apache.zookeeper.server.ZooKeeperServer)
   [2015-07-16 15:31:14,391] INFO Established session 0x14e98d2ce98000c
 with
   negotiated timeout 6000 for client /127.0.0.1:23363

Re: Delete topic using kafka-topics.sh also not working (or takes forever)

[Marina]

Jeifu, Ashish,Thanks for your response. 

1. the log retention was set to 24 hrs:log.retention.hours=24

so, it seems like the topic was still there even after the retention period has 
passed.
2. I was not able to reproduce the issue with a small amount of data - up to a 
few dozen events. But I see this consistently when I have more than 100M 
events. I know it is a wide range :) - so I will try to ramp it up gradually, I 
just have to write a couple of scripts for that.
thanks!Marina


  From: JIEFU GONG jg...@berkeley.edu
 To: users@kafka.apache.org 
 Sent: Friday, July 17, 2015 2:56 PM
 Subject: Re: Delete topic using kafka-topics.sh also not working (or takes 
forever)
   
How long do you have your retention configuration set for under
server.properties? Is it under 30 hours?



On Thu, Jul 16, 2015 at 10:44 PM, Ashish Singh asi...@cloudera.com wrote:

 Hi Marina,

 Is this reproducible? Can you reproduce this with a smaller number of
 events? Did you see any error in broker /server logs?

 On Thu, Jul 16, 2015 at 5:42 PM, Marina ppi...@yahoo.com.invalid wrote:

  A variation on the posted question - as a new thread so that not to
  high-jack the original one...
  How long does it usually take to finally delete the topic? I tried to
  delete the topic via
  /bin/kafka-topics.sh --delete --zookeeper localhost:2181 --topic mytopic
  and saw that it was marked for deletion. However,  about 30 hours
 later,
  the topic is still showing up in the list of topics:
  = bin/kafka-topics.sh --list --zookeeper localhost:2181
  __consumer_offsets
  mytopic
 
  The topic did have about 150 million events as Log size, but none were
  unprocessed (Lag == 0).
  As a test , I tried to create a new topic , and delete it right away with
  no events posted - and in that case it was deleted for good right away.
  I'm using kafka 1.8.2.1
  thanks!MArina
 
       From: Stevo Slavić ssla...@gmail.com
   To: users@kafka.apache.org
   Sent: Thursday, July 16, 2015 6:56 PM
   Subject: Re: Delete topic using Admintools is not working
 
  Hello Sivananda,
 
  Calling AdminUtils.deleteTopic just requests topic to be deleted - it
 does
  not actually delete topic immediately. Requests for topic deletion get
  saved in ZooKeeper as a node (named by topic name), under
  /admin/delete_topics node.
 
  If brokers in the cluster are configured with topic deletion enabled, and
  if they are running, they will notice requests to delete the topic,
 perform
  actual deletion (all partitions on all brokers, lead and replicas, as
 well
  as deletion of topic metadata in zookeeper), and also delete saved
 request
  for topic deletion (that node under /admin/delete_topics). This broker
 side
  part of topic deletion process is only working in Kafka 0.8.2.x.
 
  From your stack trace it seems you've requested topic deletion at least
  twice, while topic has not been deleted yet, so zk client used by admin
  utils, throws exception signalling that the node (request for topic
  deletion) already exists.
 
  If topic is not getting deleted even after a while I'd suggest that you
  check version, configuration and running status of your Kafka cluster.
 
  Kind regards,
  Stevo Slavic.
 
 
 
  On Fri, Jul 17, 2015 at 12:40 AM, Sivananda Reddy 
 sivananda2...@gmail.com
  
  wrote:
 
   Hi,
  
   As per the jira https://issues.apache.org/jira/browse/KAFKA-1737, I
 have
   set ZKStringSerializer, while creating the ZkClient and still the
 delete
   topic function call is not working:
  
   Version: kafka_2.11-0.8.2.1
  
   *My code looks like this*:
   ZkClient zkClient = new ZkClient(localhost:2181, 6000, 6000,
   ZKStringSerializer$.MODULE$);
   AdminUtils.deleteTopic(zkClient, testTopic17);
  
   *I am getting the following exception's*:
   Exception in thread main
   org.I0Itec.zkclient.exception.ZkNodeExistsException:
   org.apache.zookeeper.KeeperException$NodeExistsException:
  KeeperErrorCode =
   NodeExists for /admin/delete_topics/testTopic17
      at
   org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:55)
      at
 org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:685)
      at org.I0Itec.zkclient.ZkClient.create(ZkClient.java:304)
      at org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:243)
      at kafka.utils.ZkUtils$.createPersistentPath(ZkUtils.scala:306)
      at kafka.admin.AdminUtils$.deleteTopic(AdminUtils.scala:159)
      at kafka.admin.AdminUtils.deleteTopic(AdminUtils.scala)
      at Test.deleteTopic(Test.java:98)
      at Test.main(Test.java:76)
   Caused by: org.apache.zookeeper.KeeperException$NodeExistsException:
   KeeperErrorCode = NodeExists for /admin/delete_topics/testTopic17
      at
   org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
      at
  org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
      at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
      at org.I0Itec.zkclient.ZkConnection.create(ZkConnection.java:87

Re: Delete topic using kafka-topics.sh also not working (or takes forever)

[Ashish Singh]

Hi Marina,

Is this reproducible? Can you reproduce this with a smaller number of
events? Did you see any error in broker /server logs?

On Thu, Jul 16, 2015 at 5:42 PM, Marina ppi...@yahoo.com.invalid wrote:

 A variation on the posted question - as a new thread so that not to
 high-jack the original one...
 How long does it usually take to finally delete the topic? I tried to
 delete the topic via
 /bin/kafka-topics.sh --delete --zookeeper localhost:2181 --topic mytopic
 and saw that it was marked for deletion. However,  about 30 hours later,
 the topic is still showing up in the list of topics:
 = bin/kafka-topics.sh --list --zookeeper localhost:2181
 __consumer_offsets
 mytopic

 The topic did have about 150 million events as Log size, but none were
 unprocessed (Lag == 0).
 As a test , I tried to create a new topic , and delete it right away with
 no events posted - and in that case it was deleted for good right away.
 I'm using kafka 1.8.2.1
 thanks!MArina

   From: Stevo Slavić ssla...@gmail.com
  To: users@kafka.apache.org
  Sent: Thursday, July 16, 2015 6:56 PM
  Subject: Re: Delete topic using Admintools is not working

 Hello Sivananda,

 Calling AdminUtils.deleteTopic just requests topic to be deleted - it does
 not actually delete topic immediately. Requests for topic deletion get
 saved in ZooKeeper as a node (named by topic name), under
 /admin/delete_topics node.

 If brokers in the cluster are configured with topic deletion enabled, and
 if they are running, they will notice requests to delete the topic, perform
 actual deletion (all partitions on all brokers, lead and replicas, as well
 as deletion of topic metadata in zookeeper), and also delete saved request
 for topic deletion (that node under /admin/delete_topics). This broker side
 part of topic deletion process is only working in Kafka 0.8.2.x.

 From your stack trace it seems you've requested topic deletion at least
 twice, while topic has not been deleted yet, so zk client used by admin
 utils, throws exception signalling that the node (request for topic
 deletion) already exists.

 If topic is not getting deleted even after a while I'd suggest that you
 check version, configuration and running status of your Kafka cluster.

 Kind regards,
 Stevo Slavic.



 On Fri, Jul 17, 2015 at 12:40 AM, Sivananda Reddy sivananda2...@gmail.com
 
 wrote:

  Hi,
 
  As per the jira https://issues.apache.org/jira/browse/KAFKA-1737, I have
  set ZKStringSerializer, while creating the ZkClient and still the delete
  topic function call is not working:
 
  Version: kafka_2.11-0.8.2.1
 
  *My code looks like this*:
  ZkClient zkClient = new ZkClient(localhost:2181, 6000, 6000,
  ZKStringSerializer$.MODULE$);
  AdminUtils.deleteTopic(zkClient, testTopic17);
 
  *I am getting the following exception's*:
  Exception in thread main
  org.I0Itec.zkclient.exception.ZkNodeExistsException:
  org.apache.zookeeper.KeeperException$NodeExistsException:
 KeeperErrorCode =
  NodeExists for /admin/delete_topics/testTopic17
 at
  org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:55)
 at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:685)
 at org.I0Itec.zkclient.ZkClient.create(ZkClient.java:304)
 at org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:243)
 at kafka.utils.ZkUtils$.createPersistentPath(ZkUtils.scala:306)
 at kafka.admin.AdminUtils$.deleteTopic(AdminUtils.scala:159)
 at kafka.admin.AdminUtils.deleteTopic(AdminUtils.scala)
 at Test.deleteTopic(Test.java:98)
 at Test.main(Test.java:76)
  Caused by: org.apache.zookeeper.KeeperException$NodeExistsException:
  KeeperErrorCode = NodeExists for /admin/delete_topics/testTopic17
 at
  org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
 at
 org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
 at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
 at org.I0Itec.zkclient.ZkConnection.create(ZkConnection.java:87)
 at org.I0Itec.zkclient.ZkClient$1.call(ZkClient.java:308)
 at org.I0Itec.zkclient.ZkClient$1.call(ZkClient.java:304)
 at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675)
 ... 7 more
 
  *The corresponding zookeeper logs*:
  [2015-07-16 15:31:14,383] INFO Accepted socket connection from /
  127.0.0.1:23363 (org.apache.zookeeper.server.NIOServerCnxnFactory)
  [2015-07-16 15:31:14,385] INFO Client attempting to establish new session
  at /127.0.0.1:23363 (org.apache.zookeeper.server.ZooKeeperServer)
  [2015-07-16 15:31:14,391] INFO Established session 0x14e98d2ce98000c with
  negotiated timeout 6000 for client /127.0.0.1:23363
  (org.apache.zookeeper.server.ZooKeeperServer)
  [2015-07-16 15:31:14,510] INFO Got user-level KeeperException when
  processing sessionid:0x14e98d2ce98000c type:create cxid:0x1 zxid:0x963
  txntype:-1 reqpath:n/a Error Path:/admin/delete_topics/testTopic17
  Error:KeeperErrorCode = NodeExists for /admin

Re: Delete topic using Admintools is not working

[Stevo Slavić]

Hello Sivananda,

Calling AdminUtils.deleteTopic just requests topic to be deleted - it does
not actually delete topic immediately. Requests for topic deletion get
saved in ZooKeeper as a node (named by topic name), under
/admin/delete_topics node.

If brokers in the cluster are configured with topic deletion enabled, and
if they are running, they will notice requests to delete the topic, perform
actual deletion (all partitions on all brokers, lead and replicas, as well
as deletion of topic metadata in zookeeper), and also delete saved request
for topic deletion (that node under /admin/delete_topics). This broker side
part of topic deletion process is only working in Kafka 0.8.2.x.

From your stack trace it seems you've requested topic deletion at least
twice, while topic has not been deleted yet, so zk client used by admin
utils, throws exception signalling that the node (request for topic
deletion) already exists.

If topic is not getting deleted even after a while I'd suggest that you
check version, configuration and running status of your Kafka cluster.

Kind regards,
Stevo Slavic.

On Fri, Jul 17, 2015 at 12:40 AM, Sivananda Reddy sivananda2...@gmail.com
wrote:

 Hi,

 As per the jira https://issues.apache.org/jira/browse/KAFKA-1737, I have
 set ZKStringSerializer, while creating the ZkClient and still the delete
 topic function call is not working:

 Version: kafka_2.11-0.8.2.1

 *My code looks like this*:
 ZkClient zkClient = new ZkClient(localhost:2181, 6000, 6000,
 ZKStringSerializer$.MODULE$);
 AdminUtils.deleteTopic(zkClient, testTopic17);

 *I am getting the following exception's*:
 Exception in thread main
 org.I0Itec.zkclient.exception.ZkNodeExistsException:
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
 NodeExists for /admin/delete_topics/testTopic17
 at
 org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:55)
 at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:685)
 at org.I0Itec.zkclient.ZkClient.create(ZkClient.java:304)
 at org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:243)
 at kafka.utils.ZkUtils$.createPersistentPath(ZkUtils.scala:306)
 at kafka.admin.AdminUtils$.deleteTopic(AdminUtils.scala:159)
 at kafka.admin.AdminUtils.deleteTopic(AdminUtils.scala)
 at Test.deleteTopic(Test.java:98)
 at Test.main(Test.java:76)
 Caused by: org.apache.zookeeper.KeeperException$NodeExistsException:
 KeeperErrorCode = NodeExists for /admin/delete_topics/testTopic17
 at
 org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
 at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
 at org.I0Itec.zkclient.ZkConnection.create(ZkConnection.java:87)
 at org.I0Itec.zkclient.ZkClient$1.call(ZkClient.java:308)
 at org.I0Itec.zkclient.ZkClient$1.call(ZkClient.java:304)
 at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675)
 ... 7 more

 *The corresponding zookeeper logs*:
 [2015-07-16 15:31:14,383] INFO Accepted socket connection from /
 127.0.0.1:23363 (org.apache.zookeeper.server.NIOServerCnxnFactory)
 [2015-07-16 15:31:14,385] INFO Client attempting to establish new session
 at /127.0.0.1:23363 (org.apache.zookeeper.server.ZooKeeperServer)
 [2015-07-16 15:31:14,391] INFO Established session 0x14e98d2ce98000c with
 negotiated timeout 6000 for client /127.0.0.1:23363
 (org.apache.zookeeper.server.ZooKeeperServer)
 [2015-07-16 15:31:14,510] INFO Got user-level KeeperException when
 processing sessionid:0x14e98d2ce98000c type:create cxid:0x1 zxid:0x963
 txntype:-1 reqpath:n/a Error Path:/admin/delete_topics/testTopic17
 Error:KeeperErrorCode = NodeExists for /admin/delete_topics/testTopic17
 (org.apache.zookeeper.server.PrepRequestProcessor)
 [2015-07-16 15:31:14,842] WARN caught end of stream exception
 (org.apache.zookeeper.server.NIOServerCnxn)
 EndOfStreamException: Unable to read additional data from client sessionid
 0x14e98d2ce98000c, likely client has closed socket
 at
 org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
 at

 org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
 at java.lang.Thread.run(Thread.java:744)
 [2015-07-16 15:31:14,843] INFO Closed socket connection for client /
 127.0.0.1:23363 which had sessionid 0x14e98d2ce98000c
 (org.apache.zookeeper.server.NIOServerCnxn)
 [2015-07-16 15:31:21,000] INFO Expiring session 0x14e98d2ce98000c, timeout
 of 6000ms exceeded (org.apache.zookeeper.server.ZooKeeperServer)
 [2015-07-16 15:31:21,001] INFO Processed session termination for sessionid:
 0x14e98d2ce98000c (org.apache.zookeeper.server.PrepRequestProcessor)

 Could someone please let me know if I am missing anything?

 Thank you,
 Siva.

Re: Delete topic using Admintools is not working

[Gwen Shapira]

Looks like you try to delete a topic that is already in the process of
getting deleted:
NodeExists for /admin/delete_topics/testTopic17

(We can improve the error messages for sure, or maybe even catch the
exception and ignore it)

Gwen

On Thu, Jul 16, 2015 at 3:40 PM, Sivananda Reddy
sivananda2...@gmail.com wrote:
 Hi,

 As per the jira https://issues.apache.org/jira/browse/KAFKA-1737, I have
 set ZKStringSerializer, while creating the ZkClient and still the delete
 topic function call is not working:

 Version: kafka_2.11-0.8.2.1

 *My code looks like this*:
 ZkClient zkClient = new ZkClient(localhost:2181, 6000, 6000,
 ZKStringSerializer$.MODULE$);
 AdminUtils.deleteTopic(zkClient, testTopic17);

 *I am getting the following exception's*:
 Exception in thread main
 org.I0Itec.zkclient.exception.ZkNodeExistsException:
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
 NodeExists for /admin/delete_topics/testTopic17
 at org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:55)
 at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:685)
 at org.I0Itec.zkclient.ZkClient.create(ZkClient.java:304)
 at org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:243)
 at kafka.utils.ZkUtils$.createPersistentPath(ZkUtils.scala:306)
 at kafka.admin.AdminUtils$.deleteTopic(AdminUtils.scala:159)
 at kafka.admin.AdminUtils.deleteTopic(AdminUtils.scala)
 at Test.deleteTopic(Test.java:98)
 at Test.main(Test.java:76)
 Caused by: org.apache.zookeeper.KeeperException$NodeExistsException:
 KeeperErrorCode = NodeExists for /admin/delete_topics/testTopic17
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
 at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
 at org.I0Itec.zkclient.ZkConnection.create(ZkConnection.java:87)
 at org.I0Itec.zkclient.ZkClient$1.call(ZkClient.java:308)
 at org.I0Itec.zkclient.ZkClient$1.call(ZkClient.java:304)
 at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675)
 ... 7 more

 *The corresponding zookeeper logs*:
 [2015-07-16 15:31:14,383] INFO Accepted socket connection from /
 127.0.0.1:23363 (org.apache.zookeeper.server.NIOServerCnxnFactory)
 [2015-07-16 15:31:14,385] INFO Client attempting to establish new session
 at /127.0.0.1:23363 (org.apache.zookeeper.server.ZooKeeperServer)
 [2015-07-16 15:31:14,391] INFO Established session 0x14e98d2ce98000c with
 negotiated timeout 6000 for client /127.0.0.1:23363
 (org.apache.zookeeper.server.ZooKeeperServer)
 [2015-07-16 15:31:14,510] INFO Got user-level KeeperException when
 processing sessionid:0x14e98d2ce98000c type:create cxid:0x1 zxid:0x963
 txntype:-1 reqpath:n/a Error Path:/admin/delete_topics/testTopic17
 Error:KeeperErrorCode = NodeExists for /admin/delete_topics/testTopic17
 (org.apache.zookeeper.server.PrepRequestProcessor)
 [2015-07-16 15:31:14,842] WARN caught end of stream exception
 (org.apache.zookeeper.server.NIOServerCnxn)
 EndOfStreamException: Unable to read additional data from client sessionid
 0x14e98d2ce98000c, likely client has closed socket
 at
 org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
 at
 org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
 at java.lang.Thread.run(Thread.java:744)
 [2015-07-16 15:31:14,843] INFO Closed socket connection for client /
 127.0.0.1:23363 which had sessionid 0x14e98d2ce98000c
 (org.apache.zookeeper.server.NIOServerCnxn)
 [2015-07-16 15:31:21,000] INFO Expiring session 0x14e98d2ce98000c, timeout
 of 6000ms exceeded (org.apache.zookeeper.server.ZooKeeperServer)
 [2015-07-16 15:31:21,001] INFO Processed session termination for sessionid:
 0x14e98d2ce98000c (org.apache.zookeeper.server.PrepRequestProcessor)

 Could someone please let me know if I am missing anything?

 Thank you,
 Siva.

Re: Delete topic using kafka-topics.sh also not working (or takes forever)

[Marina]

A variation on the posted question - as a new thread so that not to high-jack 
the original one...
How long does it usually take to finally delete the topic? I tried to delete 
the topic via 
/bin/kafka-topics.sh --delete --zookeeper localhost:2181 --topic mytopic
and saw that it was marked for deletion. However,  about 30 hours later, the 
topic is still showing up in the list of topics:
= bin/kafka-topics.sh --list --zookeeper localhost:2181
__consumer_offsets
mytopic

The topic did have about 150 million events as Log size, but none were 
unprocessed (Lag == 0).
As a test , I tried to create a new topic , and delete it right away with no 
events posted - and in that case it was deleted for good right away.
I'm using kafka 1.8.2.1
thanks!MArina

  From: Stevo Slavić ssla...@gmail.com
 To: users@kafka.apache.org 
 Sent: Thursday, July 16, 2015 6:56 PM
 Subject: Re: Delete topic using Admintools is not working
   
Hello Sivananda,

Calling AdminUtils.deleteTopic just requests topic to be deleted - it does
not actually delete topic immediately. Requests for topic deletion get
saved in ZooKeeper as a node (named by topic name), under
/admin/delete_topics node.

If brokers in the cluster are configured with topic deletion enabled, and
if they are running, they will notice requests to delete the topic, perform
actual deletion (all partitions on all brokers, lead and replicas, as well
as deletion of topic metadata in zookeeper), and also delete saved request
for topic deletion (that node under /admin/delete_topics). This broker side
part of topic deletion process is only working in Kafka 0.8.2.x.

From your stack trace it seems you've requested topic deletion at least
twice, while topic has not been deleted yet, so zk client used by admin
utils, throws exception signalling that the node (request for topic
deletion) already exists.

If topic is not getting deleted even after a while I'd suggest that you
check version, configuration and running status of your Kafka cluster.

Kind regards,
Stevo Slavic.



On Fri, Jul 17, 2015 at 12:40 AM, Sivananda Reddy sivananda2...@gmail.com
wrote:

 Hi,

 As per the jira https://issues.apache.org/jira/browse/KAFKA-1737, I have
 set ZKStringSerializer, while creating the ZkClient and still the delete
 topic function call is not working:

 Version: kafka_2.11-0.8.2.1

 *My code looks like this*:
 ZkClient zkClient = new ZkClient(localhost:2181, 6000, 6000,
 ZKStringSerializer$.MODULE$);
 AdminUtils.deleteTopic(zkClient, testTopic17);

 *I am getting the following exception's*:
 Exception in thread main
 org.I0Itec.zkclient.exception.ZkNodeExistsException:
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
 NodeExists for /admin/delete_topics/testTopic17
    at
 org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:55)
    at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:685)
    at org.I0Itec.zkclient.ZkClient.create(ZkClient.java:304)
    at org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:243)
    at kafka.utils.ZkUtils$.createPersistentPath(ZkUtils.scala:306)
    at kafka.admin.AdminUtils$.deleteTopic(AdminUtils.scala:159)
    at kafka.admin.AdminUtils.deleteTopic(AdminUtils.scala)
    at Test.deleteTopic(Test.java:98)
    at Test.main(Test.java:76)
 Caused by: org.apache.zookeeper.KeeperException$NodeExistsException:
 KeeperErrorCode = NodeExists for /admin/delete_topics/testTopic17
    at
 org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
    at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
    at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
    at org.I0Itec.zkclient.ZkConnection.create(ZkConnection.java:87)
    at org.I0Itec.zkclient.ZkClient$1.call(ZkClient.java:308)
    at org.I0Itec.zkclient.ZkClient$1.call(ZkClient.java:304)
    at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675)
    ... 7 more

 *The corresponding zookeeper logs*:
 [2015-07-16 15:31:14,383] INFO Accepted socket connection from /
 127.0.0.1:23363 (org.apache.zookeeper.server.NIOServerCnxnFactory)
 [2015-07-16 15:31:14,385] INFO Client attempting to establish new session
 at /127.0.0.1:23363 (org.apache.zookeeper.server.ZooKeeperServer)
 [2015-07-16 15:31:14,391] INFO Established session 0x14e98d2ce98000c with
 negotiated timeout 6000 for client /127.0.0.1:23363
 (org.apache.zookeeper.server.ZooKeeperServer)
 [2015-07-16 15:31:14,510] INFO Got user-level KeeperException when
 processing sessionid:0x14e98d2ce98000c type:create cxid:0x1 zxid:0x963
 txntype:-1 reqpath:n/a Error Path:/admin/delete_topics/testTopic17
 Error:KeeperErrorCode = NodeExists for /admin/delete_topics/testTopic17
 (org.apache.zookeeper.server.PrepRequestProcessor)
 [2015-07-16 15:31:14,842] WARN caught end of stream exception
 (org.apache.zookeeper.server.NIOServerCnxn)
 EndOfStreamException: Unable to read additional data from client sessionid
 0x14e98d2ce98000c, likely client has

Re: Delete topic pending

[Sriharsha Chintalapani]

May be thats a bug. We should probably alert the user saying the topic doesn’t 
exist rather than go and add it to the zookeeper for deletion trigger. Also to 
delete any topic you need to set “delete.topic.enable” to true

-- 
Harsha


On May 18, 2015 at 10:16:46 AM, Dillian Murphey (crackshotm...@gmail.com) wrote:

If a broker doesn't have the topic, and I run delete topic, that topic will  
be in a pending delete state forever.  

What am I doing wrong here?  

Also, what if I have data loss and I just want to delete the dang topic  
form zookeeper directly with non of this pending stuff.  

Thanks  

Re: Delete topic pending

[Jiangjie Qin]

Hmm, which Kafka version are you running? From the code it looks we should
have already ignored the deletion of a topic if it does not exist.

Jiangjie (Becket) Qin

On 5/18/15, 10:15 AM, Dillian Murphey crackshotm...@gmail.com wrote:

If a broker doesn't have the topic, and I run delete topic, that topic
will
be in a pending delete state forever.

What am I doing wrong here?

Also, what if I have data loss and I just want to delete the dang topic
form zookeeper directly with non of this pending stuff.

Thanks

Re: Delete topic / Recreate = No leader

[马哲超]

I've came accross the same issue with kafka 0.8.10.

2015-05-01 2:51 GMT+08:00 Dillian Murphey crackshotm...@gmail.com:

 I am trying to reproduce this. But if I create a topic, then delete it,
 then re-create it, no leader is getting assigned.

 I can still produce/consume messages (via command line, basic testing).

 Is there some additional cleanup I need to do?

 Thanks for your time!

RE: Delete topic / Recreate = No leader

[Aditya Auradkar]

I believe there were some issues with delete topic in 0.8.10 (other correct me 
if I am wrong).
Can you try with the most recent release ?

Thanks,
Aditya


From: 马哲超 [mazhechaomaill...@gmail.com]
Sent: Sunday, May 03, 2015 7:51 PM
To: users@kafka.apache.org
Subject: Re: Delete topic / Recreate = No leader

I've came accross the same issue with kafka 0.8.10.

2015-05-01 2:51 GMT+08:00 Dillian Murphey crackshotm...@gmail.com:

 I am trying to reproduce this. But if I create a topic, then delete it,
 then re-create it, no leader is getting assigned.

 I can still produce/consume messages (via command line, basic testing).

 Is there some additional cleanup I need to do?

 Thanks for your time!

RE: Delete Topic in 0.8.2

[Hema Bhatia]

Grr.. delete.topic.enable=true was wiped out at kafka restart from 
server.properties. I have it working now.. thanks.

-Original Message-
From: Harsha [mailto:ka...@harsha.io]
Sent: Sunday, March 01, 2015 12:53 PM
To: users@kafka.apache.org
Subject: Re: Delete Topic in 0.8.2


Hi Hema, Can you attach controller.log and state-change.log. Image is not 
showing up at least for me. Can you also give us details on how big the cluster 
is and topic's partitions and replication-factor and any steps on reproducing 
this. Thanks, Harsha


On Sun, Mar 1, 2015, at 12:40 PM, Hema Bhatia wrote:
 I upgraded my kafka server to 0.8.2 and client to use 0.8.2 as well..





 I am trying to test delete topic feature and I see that delete topic
 do not work consistently.


 I saw it working fine a first few times but after a while I saw that
 deleting topics add them to delete_topic node in admin folder but does
 not remove it from topics in brokers.


 Also, to make things simple, no other application server was running,
 and no consumer created for these topics. Just create topic followed
 by delete topic.





 Anything I am missing?





 Here is the snapshot of createTopic followed by deleteTopic:


















 This message is private and confidential. If you have received it in
 error, please notify the sender and remove it from your system.


This message is private and confidential. If you have received it in error, 
please notify the sender and remove it from your system.

Re: Delete Topic in 0.8.2

[Harsha]

Hi Hema, Can you attach controller.log and state-change.log. Image is
not showing up at least for me. Can you also give us details on how big
the cluster is and topic's partitions and replication-factor and any
steps on reproducing this. Thanks, Harsha


On Sun, Mar 1, 2015, at 12:40 PM, Hema Bhatia wrote:
 I upgraded my kafka server to 0.8.2 and client to use 0.8.2 as well..





 I am trying to test delete topic feature and I see that delete topic
 do not work consistently.


 I saw it working fine a first few times but after a while I saw that
 deleting topics add them to delete_topic node in admin folder but does
 not remove it from topics in brokers.


 Also, to make things simple, no other application server was running,
 and no consumer created for these topics. Just create topic followed
 by delete topic.





 Anything I am missing?





 Here is the snapshot of createTopic followed by deleteTopic:


















 This message is private and confidential. If you have received it in
 error, please notify the sender and remove it from your system.

Re: Delete topic functionality can't use in 0.8.1

[Stephen Boesch]

Ryco, you are correct: delete topic is a new feature for 0.8.2

2015-02-09 19:53 GMT-08:00 Ryco Xiao ryco.x...@appcoachs.com:

 when I exec the delete command,return information is below:
 It mark the kafka-topic.sh not support the delete parameter.
 my package is compiled by myself.


 ​

Re: Delete topic API in 0.8.2

[Harsha]

Sumit,
   You can use AdminUtils.deleteTopic(zkClient, topicName) . This
   will initiate the deleteTopic process by zookeeper notification
   to KafkaController.deleteTopicManager.  It deletes log files
   along with zookeeper topic path as Timothy mentioned.
-Harsha

On Fri, Jan 23, 2015, at 12:18 PM, Timothy Chen wrote:
 I believe that's the only way it's supported from the CLI.
 
 Delete topic actually fully removes the topic from the cluster, which
 also includes cleaning the logs and removing it from zookeeper (once
 it is fully deleted).
 
 Tim
 
 On Fri, Jan 23, 2015 at 12:13 PM, Sumit Rangwala
 sumitrangw...@gmail.com wrote:
  I am trying to find if there is a supported API to delete topic (from
  within my code) specifically in 0.8.2. One method that I can think of is
  calling kafka.admin.TopicCommand.main with the same parameters as one gives
  on the command line. Is this the recommended way or is there a better way
  of doing it?
 
  Furthermore, are the details of deleting a topic in 0.8.2 documented, since
  I would like to understand if delete topic only deletes the topic in
  zookeeper or it also cleans the logs in kafka broker as well.
 
 
  Sumit

Re: Delete topic API in 0.8.2

[Timothy Chen]

I believe that's the only way it's supported from the CLI.

Delete topic actually fully removes the topic from the cluster, which
also includes cleaning the logs and removing it from zookeeper (once
it is fully deleted).

Tim

On Fri, Jan 23, 2015 at 12:13 PM, Sumit Rangwala
sumitrangw...@gmail.com wrote:
 I am trying to find if there is a supported API to delete topic (from
 within my code) specifically in 0.8.2. One method that I can think of is
 calling kafka.admin.TopicCommand.main with the same parameters as one gives
 on the command line. Is this the recommended way or is there a better way
 of doing it?

 Furthermore, are the details of deleting a topic in 0.8.2 documented, since
 I would like to understand if delete topic only deletes the topic in
 zookeeper or it also cleans the logs in kafka broker as well.


 Sumit

Re: Delete topic API in 0.8.2

[Sumit Rangwala]

Thanks Harsha, exactly what I was looking for.

Sumit


On Fri, Jan 23, 2015 at 12:24 PM, Harsha ka...@harsha.io wrote:

 Sumit,
You can use AdminUtils.deleteTopic(zkClient, topicName) . This
will initiate the deleteTopic process by zookeeper notification
to KafkaController.deleteTopicManager.  It deletes log files
along with zookeeper topic path as Timothy mentioned.
 -Harsha

 On Fri, Jan 23, 2015, at 12:18 PM, Timothy Chen wrote:
  I believe that's the only way it's supported from the CLI.
 
  Delete topic actually fully removes the topic from the cluster, which
  also includes cleaning the logs and removing it from zookeeper (once
  it is fully deleted).
 
  Tim
 
  On Fri, Jan 23, 2015 at 12:13 PM, Sumit Rangwala
  sumitrangw...@gmail.com wrote:
   I am trying to find if there is a supported API to delete topic (from
   within my code) specifically in 0.8.2. One method that I can think of
 is
   calling kafka.admin.TopicCommand.main with the same parameters as one
 gives
   on the command line. Is this the recommended way or is there a better
 way
   of doing it?
  
   Furthermore, are the details of deleting a topic in 0.8.2 documented,
 since
   I would like to understand if delete topic only deletes the topic in
   zookeeper or it also cleans the logs in kafka broker as well.
  
  
   Sumit

Re: Delete topic

[Armando Martinez Briones]

I think that it's for securiry.

El 14 de enero de 2015, 12:44, Manikumar Reddy ku...@nmsworks.co.in
escribió:

 I think now we should delete this config property and allow topic deletion
 in 0.8.2
  Yep, you need to set delete.topic.enable=true.

 Forgot that step :)

 2015-01-14 10:16 GMT-08:00 Jayesh Thakrar j_thak...@yahoo.com.invalid:
  Does one also need to set the config parameter delete.topic.enable to
 true ?I am using 8.2 beta and I had to set it to true to enable topic
 deletion.
From: Armando Martinez Briones arma...@tralix.com
   To: users@kafka.apache.org
   Sent: Wednesday, January 14, 2015 11:33 AM
   Subject: Re: Delete topic
 
  thanks Gwen Shapira ;)
 
  El 14 de enero de 2015, 11:31, Gwen Shapira gshap...@cloudera.com
  escribió:
 
  At the moment, the best way would be:
 
  * Wait about two weeks
  * Upgrade to 0.8.2
  * Use kafka-topic.sh --delete
 
  :)
 
  2015-01-14 9:26 GMT-08:00 Armando Martinez Briones arma...@tralix.com
 :
   Hi.
  
   What is the best way to delete a topic into production environment?
  
   --
   [image: Tralix][image: 1]José Armando Martínez Briones
   *Arquitecto de software*
   Tralix México
   Tel: +52 442 161 1002 ext. 2920
   www.tralix.com.mx
   https://twitter.com/#!/tralix
   http://www.facebook.com/pages/Tralix/47723192646*TRALIX MÉXICO S.
 DE
  R.L.
   DE C.V* como responsable del tratamiento de sus datos personales, hace
 de
   su conocimiento que la información obtenida por este medio es tratada
 de
   forma estrictamente confidencial por lo que recabaremos y trataremos
 de
  sus
   datos según los lineamientos de nuestro Aviso de Privacidad
   http://tralix.com.mx/politicasdeprivacidad/avisodeprivacidad.html.
  Para
   conocer la versión completa podrá hacerlo a través de la página Aviso
 de
   privacidad. Al leer el presente Aviso de Privacidad y no manifestar su
   oposición al tratamiento de sus Datos Personales, se entiende que
 usted
   conoció la versión completa y acepta los términos del mismo, siempre
 de
   conformidad a la Ley Federal de Protección de Datos Personales en
  Posesión
   de los Particulares.
 
 
 
 
 
 
  --
  [image: Tralix][image: 1]José Armando Martínez Briones
  *Arquitecto de software*
  Tralix México
  Tel: +52 442 161 1002  ext. 2920
  www.tralix.com.mx
  https://twitter.com/#!/tralix
  http://www.facebook.com/pages/Tralix/47723192646*TRALIX MÉXICO S. DE
 R.L.
  DE C.V* como responsable del tratamiento de sus datos personales, hace de
  su conocimiento que la información obtenida por este medio es tratada de
  forma estrictamente confidencial por lo que recabaremos y trataremos de
 sus
  datos según los lineamientos de nuestro Aviso de Privacidad
  http://tralix.com.mx/politicasdeprivacidad/avisodeprivacidad.html.
 Para
  conocer la versión completa podrá hacerlo a través de la página Aviso de
  privacidad. Al leer el presente Aviso de Privacidad y no manifestar su
  oposición al tratamiento de sus Datos Personales, se entiende que usted
  conoció la versión completa y acepta los términos del mismo, siempre de
  conformidad a la Ley Federal de Protección de Datos Personales en
 Posesión
  de los Particulares.
 
 




-- 
[image: Tralix][image: 1]José Armando Martínez Briones
*Arquitecto de software*
Tralix México
Tel: +52 442 161 1002  ext. 2920
www.tralix.com.mx
https://twitter.com/#!/tralix
http://www.facebook.com/pages/Tralix/47723192646*TRALIX MÉXICO S. DE R.L.
DE C.V* como responsable del tratamiento de sus datos personales, hace de
su conocimiento que la información obtenida por este medio es tratada de
forma estrictamente confidencial por lo que recabaremos y trataremos de sus
datos según los lineamientos de nuestro Aviso de Privacidad
http://tralix.com.mx/politicasdeprivacidad/avisodeprivacidad.html. Para
conocer la versión completa podrá hacerlo a través de la página Aviso de
privacidad. Al leer el presente Aviso de Privacidad y no manifestar su
oposición al tratamiento de sus Datos Personales, se entiende que usted
conoció la versión completa y acepta los términos del mismo, siempre de
conformidad a la Ley Federal de Protección de Datos Personales en Posesión
de los Particulares.

Re: Delete topic

[Manikumar Reddy]

I think now we should delete this config property and allow topic deletion
in 0.8.2
 Yep, you need to set delete.topic.enable=true.

Forgot that step :)

2015-01-14 10:16 GMT-08:00 Jayesh Thakrar j_thak...@yahoo.com.invalid:
 Does one also need to set the config parameter delete.topic.enable to
true ?I am using 8.2 beta and I had to set it to true to enable topic
deletion.
   From: Armando Martinez Briones arma...@tralix.com
  To: users@kafka.apache.org
  Sent: Wednesday, January 14, 2015 11:33 AM
  Subject: Re: Delete topic

 thanks Gwen Shapira ;)

 El 14 de enero de 2015, 11:31, Gwen Shapira gshap...@cloudera.com
 escribió:

 At the moment, the best way would be:

 * Wait about two weeks
 * Upgrade to 0.8.2
 * Use kafka-topic.sh --delete

 :)

 2015-01-14 9:26 GMT-08:00 Armando Martinez Briones arma...@tralix.com:
  Hi.
 
  What is the best way to delete a topic into production environment?
 
  --
  [image: Tralix][image: 1]José Armando Martínez Briones
  *Arquitecto de software*
  Tralix México
  Tel: +52 442 161 1002 ext. 2920
  www.tralix.com.mx
  https://twitter.com/#!/tralix
  http://www.facebook.com/pages/Tralix/47723192646*TRALIX MÉXICO S. DE
 R.L.
  DE C.V* como responsable del tratamiento de sus datos personales, hace
de
  su conocimiento que la información obtenida por este medio es tratada
de
  forma estrictamente confidencial por lo que recabaremos y trataremos de
 sus
  datos según los lineamientos de nuestro Aviso de Privacidad
  http://tralix.com.mx/politicasdeprivacidad/avisodeprivacidad.html.
 Para
  conocer la versión completa podrá hacerlo a través de la página Aviso
de
  privacidad. Al leer el presente Aviso de Privacidad y no manifestar su
  oposición al tratamiento de sus Datos Personales, se entiende que usted
  conoció la versión completa y acepta los términos del mismo, siempre de
  conformidad a la Ley Federal de Protección de Datos Personales en
 Posesión
  de los Particulares.






 --
 [image: Tralix][image: 1]José Armando Martínez Briones
 *Arquitecto de software*
 Tralix México
 Tel: +52 442 161 1002  ext. 2920
 www.tralix.com.mx
 https://twitter.com/#!/tralix
 http://www.facebook.com/pages/Tralix/47723192646*TRALIX MÉXICO S. DE
R.L.
 DE C.V* como responsable del tratamiento de sus datos personales, hace de
 su conocimiento que la información obtenida por este medio es tratada de
 forma estrictamente confidencial por lo que recabaremos y trataremos de
sus
 datos según los lineamientos de nuestro Aviso de Privacidad
 http://tralix.com.mx/politicasdeprivacidad/avisodeprivacidad.html. Para
 conocer la versión completa podrá hacerlo a través de la página Aviso de
 privacidad. Al leer el presente Aviso de Privacidad y no manifestar su
 oposición al tratamiento de sus Datos Personales, se entiende que usted
 conoció la versión completa y acepta los términos del mismo, siempre de
 conformidad a la Ley Federal de Protección de Datos Personales en Posesión
 de los Particulares.

Re: Delete topic

[Gwen Shapira]

At the moment, the best way would be:

* Wait about two weeks
* Upgrade to 0.8.2
* Use kafka-topic.sh --delete

:)

2015-01-14 9:26 GMT-08:00 Armando Martinez Briones arma...@tralix.com:
 Hi.

 What is the best way to delete a topic into production environment?

 --
 [image: Tralix][image: 1]José Armando Martínez Briones
 *Arquitecto de software*
 Tralix México
 Tel: +52 442 161 1002 ext. 2920
 www.tralix.com.mx
 https://twitter.com/#!/tralix
 http://www.facebook.com/pages/Tralix/47723192646*TRALIX MÉXICO S. DE R.L.
 DE C.V* como responsable del tratamiento de sus datos personales, hace de
 su conocimiento que la información obtenida por este medio es tratada de
 forma estrictamente confidencial por lo que recabaremos y trataremos de sus
 datos según los lineamientos de nuestro Aviso de Privacidad
 http://tralix.com.mx/politicasdeprivacidad/avisodeprivacidad.html. Para
 conocer la versión completa podrá hacerlo a través de la página Aviso de
 privacidad. Al leer el presente Aviso de Privacidad y no manifestar su
 oposición al tratamiento de sus Datos Personales, se entiende que usted
 conoció la versión completa y acepta los términos del mismo, siempre de
 conformidad a la Ley Federal de Protección de Datos Personales en Posesión
 de los Particulares.

Re: Delete topic

[Jayesh Thakrar]

Does one also need to set the config parameter delete.topic.enable to true ?I 
am using 8.2 beta and I had to set it to true to enable topic deletion.
  From: Armando Martinez Briones arma...@tralix.com
 To: users@kafka.apache.org 
 Sent: Wednesday, January 14, 2015 11:33 AM
 Subject: Re: Delete topic
   
thanks Gwen Shapira ;)

El 14 de enero de 2015, 11:31, Gwen Shapira gshap...@cloudera.com
escribió:

 At the moment, the best way would be:

 * Wait about two weeks
 * Upgrade to 0.8.2
 * Use kafka-topic.sh --delete

 :)

 2015-01-14 9:26 GMT-08:00 Armando Martinez Briones arma...@tralix.com:
  Hi.
 
  What is the best way to delete a topic into production environment?
 
  --
  [image: Tralix][image: 1]José Armando Martínez Briones
  *Arquitecto de software*
  Tralix México
  Tel: +52 442 161 1002 ext. 2920
  www.tralix.com.mx
  https://twitter.com/#!/tralix
  http://www.facebook.com/pages/Tralix/47723192646*TRALIX MÉXICO S. DE
 R.L.
  DE C.V* como responsable del tratamiento de sus datos personales, hace de
  su conocimiento que la información obtenida por este medio es tratada de
  forma estrictamente confidencial por lo que recabaremos y trataremos de
 sus
  datos según los lineamientos de nuestro Aviso de Privacidad
  http://tralix.com.mx/politicasdeprivacidad/avisodeprivacidad.html.
 Para
  conocer la versión completa podrá hacerlo a través de la página Aviso de
  privacidad. Al leer el presente Aviso de Privacidad y no manifestar su
  oposición al tratamiento de sus Datos Personales, se entiende que usted
  conoció la versión completa y acepta los términos del mismo, siempre de
  conformidad a la Ley Federal de Protección de Datos Personales en
 Posesión
  de los Particulares.






-- 
[image: Tralix][image: 1]José Armando Martínez Briones
*Arquitecto de software*
Tralix México
Tel: +52 442 161 1002  ext. 2920
www.tralix.com.mx
https://twitter.com/#!/tralix
http://www.facebook.com/pages/Tralix/47723192646*TRALIX MÉXICO S. DE R.L.
DE C.V* como responsable del tratamiento de sus datos personales, hace de
su conocimiento que la información obtenida por este medio es tratada de
forma estrictamente confidencial por lo que recabaremos y trataremos de sus
datos según los lineamientos de nuestro Aviso de Privacidad
http://tralix.com.mx/politicasdeprivacidad/avisodeprivacidad.html. Para
conocer la versión completa podrá hacerlo a través de la página Aviso de
privacidad. Al leer el presente Aviso de Privacidad y no manifestar su
oposición al tratamiento de sus Datos Personales, se entiende que usted
conoció la versión completa y acepta los términos del mismo, siempre de
conformidad a la Ley Federal de Protección de Datos Personales en Posesión
de los Particulares.

  

Re: Delete topic

[Armando Martinez Briones]

thanks Jayesh,  i'm working with the version 8.1.1 because is the version
on production environment.

El 14 de enero de 2015, 12:16, Jayesh Thakrar j_thak...@yahoo.com.invalid
escribió:

 Does one also need to set the config parameter delete.topic.enable to
 true ?I am using 8.2 beta and I had to set it to true to enable topic
 deletion.
   From: Armando Martinez Briones arma...@tralix.com
  To: users@kafka.apache.org
  Sent: Wednesday, January 14, 2015 11:33 AM
  Subject: Re: Delete topic

 thanks Gwen Shapira ;)

 El 14 de enero de 2015, 11:31, Gwen Shapira gshap...@cloudera.com
 escribió:

  At the moment, the best way would be:
 
  * Wait about two weeks
  * Upgrade to 0.8.2
  * Use kafka-topic.sh --delete
 
  :)
 
  2015-01-14 9:26 GMT-08:00 Armando Martinez Briones arma...@tralix.com:
   Hi.
  
   What is the best way to delete a topic into production environment?
  
   --
   [image: Tralix][image: 1]José Armando Martínez Briones
   *Arquitecto de software*
   Tralix México
   Tel: +52 442 161 1002 ext. 2920
   www.tralix.com.mx
   https://twitter.com/#!/tralix
   http://www.facebook.com/pages/Tralix/47723192646*TRALIX MÉXICO S. DE
  R.L.
   DE C.V* como responsable del tratamiento de sus datos personales, hace
 de
   su conocimiento que la información obtenida por este medio es tratada
 de
   forma estrictamente confidencial por lo que recabaremos y trataremos de
  sus
   datos según los lineamientos de nuestro Aviso de Privacidad
   http://tralix.com.mx/politicasdeprivacidad/avisodeprivacidad.html.
  Para
   conocer la versión completa podrá hacerlo a través de la página Aviso
 de
   privacidad. Al leer el presente Aviso de Privacidad y no manifestar su
   oposición al tratamiento de sus Datos Personales, se entiende que usted
   conoció la versión completa y acepta los términos del mismo, siempre de
   conformidad a la Ley Federal de Protección de Datos Personales en
  Posesión
   de los Particulares.


 



 --
 [image: Tralix][image: 1]José Armando Martínez Briones
 *Arquitecto de software*
 Tralix México
 Tel: +52 442 161 1002  ext. 2920
 www.tralix.com.mx
 https://twitter.com/#!/tralix
 http://www.facebook.com/pages/Tralix/47723192646*TRALIX MÉXICO S. DE
 R.L.
 DE C.V* como responsable del tratamiento de sus datos personales, hace de
 su conocimiento que la información obtenida por este medio es tratada de
 forma estrictamente confidencial por lo que recabaremos y trataremos de sus
 datos según los lineamientos de nuestro Aviso de Privacidad
 http://tralix.com.mx/politicasdeprivacidad/avisodeprivacidad.html. Para
 conocer la versión completa podrá hacerlo a través de la página Aviso de
 privacidad. Al leer el presente Aviso de Privacidad y no manifestar su
 oposición al tratamiento de sus Datos Personales, se entiende que usted
 conoció la versión completa y acepta los términos del mismo, siempre de
 conformidad a la Ley Federal de Protección de Datos Personales en Posesión
 de los Particulares.






-- 
[image: Tralix][image: 1]José Armando Martínez Briones
*Arquitecto de software*
Tralix México
Tel: +52 442 161 1002  ext. 2920
www.tralix.com.mx
https://twitter.com/#!/tralix
http://www.facebook.com/pages/Tralix/47723192646*TRALIX MÉXICO S. DE R.L.
DE C.V* como responsable del tratamiento de sus datos personales, hace de
su conocimiento que la información obtenida por este medio es tratada de
forma estrictamente confidencial por lo que recabaremos y trataremos de sus
datos según los lineamientos de nuestro Aviso de Privacidad
http://tralix.com.mx/politicasdeprivacidad/avisodeprivacidad.html. Para
conocer la versión completa podrá hacerlo a través de la página Aviso de
privacidad. Al leer el presente Aviso de Privacidad y no manifestar su
oposición al tratamiento de sus Datos Personales, se entiende que usted
conoció la versión completa y acepta los términos del mismo, siempre de
conformidad a la Ley Federal de Protección de Datos Personales en Posesión
de los Particulares.

Re: Delete topic

[Gwen Shapira]

Yep, you need to set delete.topic.enable=true.

Forgot that step :)

2015-01-14 10:16 GMT-08:00 Jayesh Thakrar j_thak...@yahoo.com.invalid:
 Does one also need to set the config parameter delete.topic.enable to true 
 ?I am using 8.2 beta and I had to set it to true to enable topic deletion.
   From: Armando Martinez Briones arma...@tralix.com
  To: users@kafka.apache.org
  Sent: Wednesday, January 14, 2015 11:33 AM
  Subject: Re: Delete topic

 thanks Gwen Shapira ;)

 El 14 de enero de 2015, 11:31, Gwen Shapira gshap...@cloudera.com
 escribió:

 At the moment, the best way would be:

 * Wait about two weeks
 * Upgrade to 0.8.2
 * Use kafka-topic.sh --delete

 :)

 2015-01-14 9:26 GMT-08:00 Armando Martinez Briones arma...@tralix.com:
  Hi.
 
  What is the best way to delete a topic into production environment?
 
  --
  [image: Tralix][image: 1]José Armando Martínez Briones
  *Arquitecto de software*
  Tralix México
  Tel: +52 442 161 1002 ext. 2920
  www.tralix.com.mx
  https://twitter.com/#!/tralix
  http://www.facebook.com/pages/Tralix/47723192646*TRALIX MÉXICO S. DE
 R.L.
  DE C.V* como responsable del tratamiento de sus datos personales, hace de
  su conocimiento que la información obtenida por este medio es tratada de
  forma estrictamente confidencial por lo que recabaremos y trataremos de
 sus
  datos según los lineamientos de nuestro Aviso de Privacidad
  http://tralix.com.mx/politicasdeprivacidad/avisodeprivacidad.html.
 Para
  conocer la versión completa podrá hacerlo a través de la página Aviso de
  privacidad. Al leer el presente Aviso de Privacidad y no manifestar su
  oposición al tratamiento de sus Datos Personales, se entiende que usted
  conoció la versión completa y acepta los términos del mismo, siempre de
  conformidad a la Ley Federal de Protección de Datos Personales en
 Posesión
  de los Particulares.






 --
 [image: Tralix][image: 1]José Armando Martínez Briones
 *Arquitecto de software*
 Tralix México
 Tel: +52 442 161 1002  ext. 2920
 www.tralix.com.mx
 https://twitter.com/#!/tralix
 http://www.facebook.com/pages/Tralix/47723192646*TRALIX MÉXICO S. DE R.L.
 DE C.V* como responsable del tratamiento de sus datos personales, hace de
 su conocimiento que la información obtenida por este medio es tratada de
 forma estrictamente confidencial por lo que recabaremos y trataremos de sus
 datos según los lineamientos de nuestro Aviso de Privacidad
 http://tralix.com.mx/politicasdeprivacidad/avisodeprivacidad.html. Para
 conocer la versión completa podrá hacerlo a través de la página Aviso de
 privacidad. Al leer el presente Aviso de Privacidad y no manifestar su
 oposición al tratamiento de sus Datos Personales, se entiende que usted
 conoció la versión completa y acepta los términos del mismo, siempre de
 conformidad a la Ley Federal de Protección de Datos Personales en Posesión
 de los Particulares.

Re: delete topic ?

[Timothy Chen]

Is this the latest master? I've added the delete option in trunk, but
it's not in any release yet.

We used to have the delete option flag but I believe we removed it
that's why the documentation difference.

Tim

On Wed, Aug 6, 2014 at 10:53 PM, Shlomi Hazan shl...@viber.com wrote:
 if the answer is pointing out the 'chroot', as a word, it makes no
 difference. the result is the same:

 kafka/bin/kafka-topics.sh --zookeeper localhost:2181/chroot --delete
 --topic topic-3

 gives the same:

 Command must include exactly one action: --list, --describe, --create or
 --alter...

 or should I write something instead of chroot?



 On Wed, Jun 18, 2014 at 2:06 PM, Shlomi Hazan shl...@viber.com wrote:

 Hi,

 Doing some evaluation testing, and accidently create a queue with wrong
 replication factor.

 Trying to delete as in:

 kafka_2.10-0.8.1.1/bin/kafka-topics.sh --zookeeper localhost:2181 --delete
 --topic replicated-topic

 Yeilded:

 Command must include exactly one action: --list, --describe, --create or
 –alter

 Event though this page (https://kafka.apache.org/documentation.html) says:



 And finally deleting a topic:

   bin/kafka-topics.sh --zookeeper zk_host:port/chroot --delete --topic 
 my_topic_name

 WARNING: Delete topic functionality is beta in 0.8.1. Please report any
 bugs that you encounter on themailing list %20us...@kafka.apache.org or
 JIRA https://issues.apache.org/jira/browse/KAFKA.

 Kafka does not currently support reducing the number of partitions for a
 topic or changing the replication factor.

 What should I do?

 Shlomi

Re: delete topic ?

[Timothy Chen]

Hi Gwen,

That is a very confusing error message for sure, feel free to file a
jira for both the experience cases.

But in general how delete topic works is that it creates a entry in
the delete_topic zk path, and the leader has a delete topic thread
that watches that path and starts the topic deletion once it receives
the message. It then requires rounds of coordination among all the
brokers that has partitions for the topic to delete all the
partitions, then finally delete the topic from zk.

Therefore once the deletion finishes it will also deleted from zk. The
topic command can definitely however join the topic list with the
delete topic list and mark the ones being deleted with a special
status.

Tim

On Wed, Aug 6, 2014 at 11:20 PM, Gwen Shapira gshap...@cloudera.com wrote:
 Hi Timothy,

 While we are on the subject, few questions/comments (based on the
 trunk implementation of delete topic command):

 * After deleting a topic, I still see it when listing topics. Is the
 expected behavior? Should it disappear after some time?
 * When does the actual deletion gets triggered?
 * If I try to delete a topic twice I get a pretty confusing exception
 (Node exists from zkclient). It will be nice to catch this and say
 Topic is being deleted or something to this effect.
 * Even nicer if list topics command will mark topics as being deleted.

 I'll probably open a separate Jira for the nice behavior, but
 interested in hearing your thoughts.

 Gwen

 On Wed, Aug 6, 2014 at 11:01 PM, Timothy Chen tnac...@gmail.com wrote:
 Is this the latest master? I've added the delete option in trunk, but
 it's not in any release yet.

 We used to have the delete option flag but I believe we removed it
 that's why the documentation difference.

 Tim

 On Wed, Aug 6, 2014 at 10:53 PM, Shlomi Hazan shl...@viber.com wrote:
 if the answer is pointing out the 'chroot', as a word, it makes no
 difference. the result is the same:

 kafka/bin/kafka-topics.sh --zookeeper localhost:2181/chroot --delete
 --topic topic-3

 gives the same:

 Command must include exactly one action: --list, --describe, --create or
 --alter...

 or should I write something instead of chroot?



 On Wed, Jun 18, 2014 at 2:06 PM, Shlomi Hazan shl...@viber.com wrote:

 Hi,

 Doing some evaluation testing, and accidently create a queue with wrong
 replication factor.

 Trying to delete as in:

 kafka_2.10-0.8.1.1/bin/kafka-topics.sh --zookeeper localhost:2181 --delete
 --topic replicated-topic

 Yeilded:

 Command must include exactly one action: --list, --describe, --create or
 –alter

 Event though this page (https://kafka.apache.org/documentation.html) says:



 And finally deleting a topic:

   bin/kafka-topics.sh --zookeeper zk_host:port/chroot --delete --topic 
 my_topic_name

 WARNING: Delete topic functionality is beta in 0.8.1. Please report any
 bugs that you encounter on themailing list %20us...@kafka.apache.org or
 JIRA https://issues.apache.org/jira/browse/KAFKA.

 Kafka does not currently support reducing the number of partitions for a
 topic or changing the replication factor.

 What should I do?

 Shlomi

Re: delete topic ?

[Jason Rosenberg]

Since the deletion stuff is now in trunk, would be compatible to issue the
command from a jar built from trunk, against a running 0.8.1.1 cluster?  Or
does the cluster also have to be running trunk?  (I'm guessing it does :)).

I have some topics I'd like to delete, but don't want to wait for 0.8.2
(but will probably have to, I'm guessing).

Jason


On Thu, Aug 7, 2014 at 2:53 AM, Timothy Chen tnac...@gmail.com wrote:

 Hi Gwen,

 That is a very confusing error message for sure, feel free to file a
 jira for both the experience cases.

 But in general how delete topic works is that it creates a entry in
 the delete_topic zk path, and the leader has a delete topic thread
 that watches that path and starts the topic deletion once it receives
 the message. It then requires rounds of coordination among all the
 brokers that has partitions for the topic to delete all the
 partitions, then finally delete the topic from zk.

 Therefore once the deletion finishes it will also deleted from zk. The
 topic command can definitely however join the topic list with the
 delete topic list and mark the ones being deleted with a special
 status.

 Tim

 On Wed, Aug 6, 2014 at 11:20 PM, Gwen Shapira gshap...@cloudera.com
 wrote:
  Hi Timothy,
 
  While we are on the subject, few questions/comments (based on the
  trunk implementation of delete topic command):
 
  * After deleting a topic, I still see it when listing topics. Is the
  expected behavior? Should it disappear after some time?
  * When does the actual deletion gets triggered?
  * If I try to delete a topic twice I get a pretty confusing exception
  (Node exists from zkclient). It will be nice to catch this and say
  Topic is being deleted or something to this effect.
  * Even nicer if list topics command will mark topics as being deleted.
 
  I'll probably open a separate Jira for the nice behavior, but
  interested in hearing your thoughts.
 
  Gwen
 
  On Wed, Aug 6, 2014 at 11:01 PM, Timothy Chen tnac...@gmail.com wrote:
  Is this the latest master? I've added the delete option in trunk, but
  it's not in any release yet.
 
  We used to have the delete option flag but I believe we removed it
  that's why the documentation difference.
 
  Tim
 
  On Wed, Aug 6, 2014 at 10:53 PM, Shlomi Hazan shl...@viber.com wrote:
  if the answer is pointing out the 'chroot', as a word, it makes no
  difference. the result is the same:
 
  kafka/bin/kafka-topics.sh --zookeeper localhost:2181/chroot --delete
  --topic topic-3
 
  gives the same:
 
  Command must include exactly one action: --list, --describe, --create
 or
  --alter...
 
  or should I write something instead of chroot?
 
 
 
  On Wed, Jun 18, 2014 at 2:06 PM, Shlomi Hazan shl...@viber.com
 wrote:
 
  Hi,
 
  Doing some evaluation testing, and accidently create a queue with
 wrong
  replication factor.
 
  Trying to delete as in:
 
  kafka_2.10-0.8.1.1/bin/kafka-topics.sh --zookeeper localhost:2181
 --delete
  --topic replicated-topic
 
  Yeilded:
 
  Command must include exactly one action: --list, --describe, --create
 or
  –alter
 
  Event though this page (https://kafka.apache.org/documentation.html)
 says:
 
 
 
  And finally deleting a topic:
 
bin/kafka-topics.sh --zookeeper zk_host:port/chroot --delete
 --topic my_topic_name
 
  WARNING: Delete topic functionality is beta in 0.8.1. Please report
 any
  bugs that you encounter on themailing list %20us...@kafka.apache.org
 or
  JIRA https://issues.apache.org/jira/browse/KAFKA.
 
  Kafka does not currently support reducing the number of partitions
 for a
  topic or changing the replication factor.
 
  What should I do?
 
  Shlomi
 

Re: delete topic ?

[Gwen Shapira]

Looking at the delete topic patch, it looks like there were
significant modifications in the controller code to support that, so I
think you are out of luck.

(https://reviews.apache.org/r/20745)

On Thu, Aug 7, 2014 at 8:18 AM, Jason Rosenberg j...@squareup.com wrote:
 Since the deletion stuff is now in trunk, would be compatible to issue the
 command from a jar built from trunk, against a running 0.8.1.1 cluster?  Or
 does the cluster also have to be running trunk?  (I'm guessing it does :)).

 I have some topics I'd like to delete, but don't want to wait for 0.8.2
 (but will probably have to, I'm guessing).

 Jason


 On Thu, Aug 7, 2014 at 2:53 AM, Timothy Chen tnac...@gmail.com wrote:

 Hi Gwen,

 That is a very confusing error message for sure, feel free to file a
 jira for both the experience cases.

 But in general how delete topic works is that it creates a entry in
 the delete_topic zk path, and the leader has a delete topic thread
 that watches that path and starts the topic deletion once it receives
 the message. It then requires rounds of coordination among all the
 brokers that has partitions for the topic to delete all the
 partitions, then finally delete the topic from zk.

 Therefore once the deletion finishes it will also deleted from zk. The
 topic command can definitely however join the topic list with the
 delete topic list and mark the ones being deleted with a special
 status.

 Tim

 On Wed, Aug 6, 2014 at 11:20 PM, Gwen Shapira gshap...@cloudera.com
 wrote:
  Hi Timothy,
 
  While we are on the subject, few questions/comments (based on the
  trunk implementation of delete topic command):
 
  * After deleting a topic, I still see it when listing topics. Is the
  expected behavior? Should it disappear after some time?
  * When does the actual deletion gets triggered?
  * If I try to delete a topic twice I get a pretty confusing exception
  (Node exists from zkclient). It will be nice to catch this and say
  Topic is being deleted or something to this effect.
  * Even nicer if list topics command will mark topics as being deleted.
 
  I'll probably open a separate Jira for the nice behavior, but
  interested in hearing your thoughts.
 
  Gwen
 
  On Wed, Aug 6, 2014 at 11:01 PM, Timothy Chen tnac...@gmail.com wrote:
  Is this the latest master? I've added the delete option in trunk, but
  it's not in any release yet.
 
  We used to have the delete option flag but I believe we removed it
  that's why the documentation difference.
 
  Tim
 
  On Wed, Aug 6, 2014 at 10:53 PM, Shlomi Hazan shl...@viber.com wrote:
  if the answer is pointing out the 'chroot', as a word, it makes no
  difference. the result is the same:
 
  kafka/bin/kafka-topics.sh --zookeeper localhost:2181/chroot --delete
  --topic topic-3
 
  gives the same:
 
  Command must include exactly one action: --list, --describe, --create
 or
  --alter...
 
  or should I write something instead of chroot?
 
 
 
  On Wed, Jun 18, 2014 at 2:06 PM, Shlomi Hazan shl...@viber.com
 wrote:
 
  Hi,
 
  Doing some evaluation testing, and accidently create a queue with
 wrong
  replication factor.
 
  Trying to delete as in:
 
  kafka_2.10-0.8.1.1/bin/kafka-topics.sh --zookeeper localhost:2181
 --delete
  --topic replicated-topic
 
  Yeilded:
 
  Command must include exactly one action: --list, --describe, --create
 or
  –alter
 
  Event though this page (https://kafka.apache.org/documentation.html)
 says:
 
 
 
  And finally deleting a topic:
 
bin/kafka-topics.sh --zookeeper zk_host:port/chroot --delete
 --topic my_topic_name
 
  WARNING: Delete topic functionality is beta in 0.8.1. Please report
 any
  bugs that you encounter on themailing list %20us...@kafka.apache.org
 or
  JIRA https://issues.apache.org/jira/browse/KAFKA.
 
  Kafka does not currently support reducing the number of partitions
 for a
  topic or changing the replication factor.
 
  What should I do?
 
  Shlomi
 

Re: delete topic ?

[Timothy Chen]

Hi Jason,

You do want to wait for the next release as a lot of stability fixes
are going into that.

Tim

On Thu, Aug 7, 2014 at 10:25 AM, Gwen Shapira gshap...@cloudera.com wrote:
 Looking at the delete topic patch, it looks like there were
 significant modifications in the controller code to support that, so I
 think you are out of luck.

 (https://reviews.apache.org/r/20745)

 On Thu, Aug 7, 2014 at 8:18 AM, Jason Rosenberg j...@squareup.com wrote:
 Since the deletion stuff is now in trunk, would be compatible to issue the
 command from a jar built from trunk, against a running 0.8.1.1 cluster?  Or
 does the cluster also have to be running trunk?  (I'm guessing it does :)).

 I have some topics I'd like to delete, but don't want to wait for 0.8.2
 (but will probably have to, I'm guessing).

 Jason


 On Thu, Aug 7, 2014 at 2:53 AM, Timothy Chen tnac...@gmail.com wrote:

 Hi Gwen,

 That is a very confusing error message for sure, feel free to file a
 jira for both the experience cases.

 But in general how delete topic works is that it creates a entry in
 the delete_topic zk path, and the leader has a delete topic thread
 that watches that path and starts the topic deletion once it receives
 the message. It then requires rounds of coordination among all the
 brokers that has partitions for the topic to delete all the
 partitions, then finally delete the topic from zk.

 Therefore once the deletion finishes it will also deleted from zk. The
 topic command can definitely however join the topic list with the
 delete topic list and mark the ones being deleted with a special
 status.

 Tim

 On Wed, Aug 6, 2014 at 11:20 PM, Gwen Shapira gshap...@cloudera.com
 wrote:
  Hi Timothy,
 
  While we are on the subject, few questions/comments (based on the
  trunk implementation of delete topic command):
 
  * After deleting a topic, I still see it when listing topics. Is the
  expected behavior? Should it disappear after some time?
  * When does the actual deletion gets triggered?
  * If I try to delete a topic twice I get a pretty confusing exception
  (Node exists from zkclient). It will be nice to catch this and say
  Topic is being deleted or something to this effect.
  * Even nicer if list topics command will mark topics as being deleted.
 
  I'll probably open a separate Jira for the nice behavior, but
  interested in hearing your thoughts.
 
  Gwen
 
  On Wed, Aug 6, 2014 at 11:01 PM, Timothy Chen tnac...@gmail.com wrote:
  Is this the latest master? I've added the delete option in trunk, but
  it's not in any release yet.
 
  We used to have the delete option flag but I believe we removed it
  that's why the documentation difference.
 
  Tim
 
  On Wed, Aug 6, 2014 at 10:53 PM, Shlomi Hazan shl...@viber.com wrote:
  if the answer is pointing out the 'chroot', as a word, it makes no
  difference. the result is the same:
 
  kafka/bin/kafka-topics.sh --zookeeper localhost:2181/chroot --delete
  --topic topic-3
 
  gives the same:
 
  Command must include exactly one action: --list, --describe, --create
 or
  --alter...
 
  or should I write something instead of chroot?
 
 
 
  On Wed, Jun 18, 2014 at 2:06 PM, Shlomi Hazan shl...@viber.com
 wrote:
 
  Hi,
 
  Doing some evaluation testing, and accidently create a queue with
 wrong
  replication factor.
 
  Trying to delete as in:
 
  kafka_2.10-0.8.1.1/bin/kafka-topics.sh --zookeeper localhost:2181
 --delete
  --topic replicated-topic
 
  Yeilded:
 
  Command must include exactly one action: --list, --describe, --create
 or
  –alter
 
  Event though this page (https://kafka.apache.org/documentation.html)
 says:
 
 
 
  And finally deleting a topic:
 
bin/kafka-topics.sh --zookeeper zk_host:port/chroot --delete
 --topic my_topic_name
 
  WARNING: Delete topic functionality is beta in 0.8.1. Please report
 any
  bugs that you encounter on themailing list %20us...@kafka.apache.org
 or
  JIRA https://issues.apache.org/jira/browse/KAFKA.
 
  Kafka does not currently support reducing the number of partitions
 for a
  topic or changing the replication factor.
 
  What should I do?
 
  Shlomi
 

Re: delete topic ?

[Neha Narkhede]

Kafka allows increasing the replication factor of a topic. You can read
about it here
http://kafka.apache.org/081/documentation.html#basic_ops_increase_replication_factor.
We do not support reducing the number of partitions, so you either have to
create a new topic or delete the existing one. We fixed a number of bugs in
delete topic but those fixes are on trunk. You can give it a spin.


On Wed, Jun 18, 2014 at 4:06 AM, Shlomi Hazan shl...@viber.com wrote:

 Hi,

 Doing some evaluation testing, and accidently create a queue with wrong
 replication factor.

 Trying to delete as in:

 kafka_2.10-0.8.1.1/bin/kafka-topics.sh --zookeeper localhost:2181 --delete
 --topic replicated-topic

 Yeilded:

 Command must include exactly one action: --list, --describe, --create or
 -alter

 Event though this page (https://kafka.apache.org/documentation.html) says:



 And finally deleting a topic:

   bin/kafka-topics.sh --zookeeper zk_host:port/chroot --delete --topic
 my_topic_name

 WARNING: Delete topic functionality is beta in 0.8.1. Please report any
 bugs
 that you encounter on the mailto:%20us...@kafka.apache.org mailing list
 or
 https://issues.apache.org/jira/browse/KAFKA JIRA.

 Kafka does not currently support reducing the number of partitions for a
 topic or changing the replication factor.

 What should I do?

 Shlomi

Re: delete topic ?

[Mark Roberts]

When we were in testing phase, we would either create a new topic with the 
correct details or shut the cluster down and hard kill the topic in zookeeper + 
local disk.  In prod we have the cluster configured via configuration 
management and auto create turned off.

The ability to delete a topic in a live, running kafka cluster is tricky, and 
the implementations of it have been subtly incorrect (and therefore dangerous). 
I know that there is work happening around that, but haven't kept up with the 
status of it.  Maybe in 8.2? It sounds conceptually simpler to implement with 
the new metadata API.

-Mark

 On Jun 18, 2014, at 4:06, Shlomi Hazan shl...@viber.com wrote:
 
 Hi,
 
 Doing some evaluation testing, and accidently create a queue with wrong
 replication factor.
 
 Trying to delete as in:
 
 kafka_2.10-0.8.1.1/bin/kafka-topics.sh --zookeeper localhost:2181 --delete
 --topic replicated-topic
 
 Yeilded:
 
 Command must include exactly one action: --list, --describe, --create or
 -alter
 
 Event though this page (https://kafka.apache.org/documentation.html) says:
 
 
 
 And finally deleting a topic:
 
 bin/kafka-topics.sh --zookeeper zk_host:port/chroot --delete --topic
 my_topic_name
 
 WARNING: Delete topic functionality is beta in 0.8.1. Please report any bugs
 that you encounter on the mailto:%20us...@kafka.apache.org mailing list or
 https://issues.apache.org/jira/browse/KAFKA JIRA.
 
 Kafka does not currently support reducing the number of partitions for a
 topic or changing the replication factor.
 
 What should I do?
 
 Shlomi
 

Re: delete topic ?

[hsy...@gmail.com]

I'm using 0.8.1.1
I use DeleteTopicCommand to delete topic
args[0] = --topic;
args[1] = the topic you want to delete
args[2] = --zookeeper;
args[3] = kafkaZookeepers;
DeleteTopicCommand.main(args);

You can write your own script to delete the topic, I guess. And I think it
only deletes the entry in zookeeper

Best,
Siyuan



On Wed, Jun 18, 2014 at 9:13 AM, Mark Roberts wiz...@gmail.com wrote:

 When we were in testing phase, we would either create a new topic with the
 correct details or shut the cluster down and hard kill the topic in
 zookeeper + local disk.  In prod we have the cluster configured via
 configuration management and auto create turned off.

 The ability to delete a topic in a live, running kafka cluster is tricky,
 and the implementations of it have been subtly incorrect (and therefore
 dangerous). I know that there is work happening around that, but haven't
 kept up with the status of it.  Maybe in 8.2? It sounds conceptually
 simpler to implement with the new metadata API.

 -Mark

  On Jun 18, 2014, at 4:06, Shlomi Hazan shl...@viber.com wrote:
 
  Hi,
 
  Doing some evaluation testing, and accidently create a queue with wrong
  replication factor.
 
  Trying to delete as in:
 
  kafka_2.10-0.8.1.1/bin/kafka-topics.sh --zookeeper localhost:2181
 --delete
  --topic replicated-topic
 
  Yeilded:
 
  Command must include exactly one action: --list, --describe, --create or
  -alter
 
  Event though this page (https://kafka.apache.org/documentation.html)
 says:
 
 
 
  And finally deleting a topic:
 
  bin/kafka-topics.sh --zookeeper zk_host:port/chroot --delete --topic
  my_topic_name
 
  WARNING: Delete topic functionality is beta in 0.8.1. Please report any
 bugs
  that you encounter on the mailto:%20us...@kafka.apache.org mailing
 list or
  https://issues.apache.org/jira/browse/KAFKA JIRA.
 
  Kafka does not currently support reducing the number of partitions for a
  topic or changing the replication factor.
 
  What should I do?
 
  Shlomi
 

Re: delete topic ?

[Timothy Chen]

Yes the existing delete topic command just cleans up the topic entry in zk, but 
not really deleting the topic from the cluster.

I have a patch that enables kafka-topics.sh to delete topic but not sure if 
it's merged to trunk.

Tim

 On Jun 18, 2014, at 1:39 PM, hsy...@gmail.com hsy...@gmail.com wrote:
 
 I'm using 0.8.1.1
 I use DeleteTopicCommand to delete topic
args[0] = --topic;
args[1] = the topic you want to delete
args[2] = --zookeeper;
args[3] = kafkaZookeepers;
DeleteTopicCommand.main(args);
 
 You can write your own script to delete the topic, I guess. And I think it
 only deletes the entry in zookeeper
 
 Best,
 Siyuan
 
 
 
 On Wed, Jun 18, 2014 at 9:13 AM, Mark Roberts wiz...@gmail.com wrote:
 
 When we were in testing phase, we would either create a new topic with the
 correct details or shut the cluster down and hard kill the topic in
 zookeeper + local disk.  In prod we have the cluster configured via
 configuration management and auto create turned off.
 
 The ability to delete a topic in a live, running kafka cluster is tricky,
 and the implementations of it have been subtly incorrect (and therefore
 dangerous). I know that there is work happening around that, but haven't
 kept up with the status of it.  Maybe in 8.2? It sounds conceptually
 simpler to implement with the new metadata API.
 
 -Mark
 
 On Jun 18, 2014, at 4:06, Shlomi Hazan shl...@viber.com wrote:
 
 Hi,
 
 Doing some evaluation testing, and accidently create a queue with wrong
 replication factor.
 
 Trying to delete as in:
 
 kafka_2.10-0.8.1.1/bin/kafka-topics.sh --zookeeper localhost:2181
 --delete
 --topic replicated-topic
 
 Yeilded:
 
 Command must include exactly one action: --list, --describe, --create or
 -alter
 
 Event though this page (https://kafka.apache.org/documentation.html)
 says:
 
 
 
 And finally deleting a topic:
 
 bin/kafka-topics.sh --zookeeper zk_host:port/chroot --delete --topic
 my_topic_name
 
 WARNING: Delete topic functionality is beta in 0.8.1. Please report any
 bugs
 that you encounter on the mailto:%20us...@kafka.apache.org mailing
 list or
 https://issues.apache.org/jira/browse/KAFKA JIRA.
 
 Kafka does not currently support reducing the number of partitions for a
 topic or changing the replication factor.
 
 What should I do?
 
 Shlomi
 

Re: Delete Topic - BadVersionException

[Jun Rao]

Delete topic doesn't quite work yet and we will try to fix it in the next
release. https://issues.apache.org/jira/browse/KAFKA-1397

Thanks,

Jun


On Thu, Apr 24, 2014 at 9:49 AM, Drew Goya d...@gradientx.com wrote:

 Just tried my first topic delete today and it looks like something went
 wrong on the controller.  I issued the command on a test topic and shortly
 after that a describe looked like:

 Topic:TimeoutQueueTest PartitionCount:256 ReplicationFactor:3 Configs:
 Topic: TimeoutQueueTest Partition: 0 Leader: -1 Replicas: 9,14,15 Isr:
 Topic: TimeoutQueueTest Partition: 1 Leader: -1 Replicas: 10,15,1 Isr:
 Topic: TimeoutQueueTest Partition: 2 Leader: -1 Replicas: 11,1,2 Isr:
 Topic: TimeoutQueueTest Partition: 3 Leader: -1 Replicas: 12,2,3 Isr:
 Topic: TimeoutQueueTest Partition: 4 Leader: -1 Replicas: 13,3,4 Isr:
 Topic: TimeoutQueueTest Partition: 5 Leader: -1 Replicas: 14,4,5 Isr:

 It stayed that way for quite a while so I hit zookeeper and went looking
 for who was the controller, I found these in that brokers logs:

 [2014-04-24 16:27:42,498] ERROR Conditional update of path
 /brokers/topics/TimeoutQueueTest/partitions/170/state with data

 {controller_epoch:18,leader:2,version:1,leader_epoch:14,isr:[2,14]}
 and expected version 30 failed due to
 org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode =
 BadVersion for /brokers/topics/TimeoutQueueTest/partitions/170/state
 (kafka.utils.ZkUtils$)
 [2014-04-24 16:27:42,504] ERROR Conditional update of path
 /brokers/topics/TimeoutQueueTest/partitions/113/state with data

 {controller_epoch:18,leader:2,version:1,leader_epoch:4,isr:[2,15,14]}
 and expected version 17 failed due to
 org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode =
 BadVersion for /brokers/topics/TimeoutQueueTest/partitions/113/state
 (kafka.utils.ZkUtils$)

 Any ideas?