Hi, Jason, I am not sure that we should delay the 0.8.2 release. The reasons are (1) There are other features such as the new java producer and Kafka-based offset management that are potentially useful to people. (2) It may take some time to completely fix the issue with deleting topic since it involves wire protocol changes.
What we can do is probably just to document the limitation of delete topic in 0.8.2.0 in our site. Thanks, Jun On Mon, Jan 26, 2015 at 11:05 AM, Jason Rosenberg <j...@squareup.com> wrote: > I think this shortcoming should probably delay release of 0.8.2 until > resolved, no? > > On Mon, Jan 26, 2015 at 2:01 PM, Guozhang Wang <wangg...@gmail.com> wrote: > > > It then seems to me that delete-topic will not actually work "smoothly" > > until create topic request is added since it is too much to require > people > > to turn off their clients while deleting topics. In this case shall we > make > > it clear in the release docs of in 0.8.2, or even still mark it as > > not-supported? > > > > Guozhang > > > > On Mon, Jan 26, 2015 at 8:14 AM, Harsha <ka...@harsha.io> wrote: > > > > > Jun, > > > I made an attempt at fixing that issue as part of this JIRA > > > https://issues.apache.org/jira/browse/KAFKA-1507 . > > > As Jay pointed out there should be admin api if there is more info on > > > this api I am interested in adding/fixing this issue. > > > Thanks, > > > Harsha > > > > > > On Mon, Jan 26, 2015, at 07:28 AM, Jun Rao wrote: > > > > Yes, that's the issue. Currently, topics can be auto-created on > > > > TopicMetadataRequest, which can be issued from both the producer and > > the > > > > consumer. To prevent that, you would need to stop the producer and > the > > > > consumer before deleting a topic. We plan to address this issue once > we > > > > have a separate request for creating topics. > > > > > > > > Thanks, > > > > > > > > Jun > > > > > > > > On Mon, Jan 26, 2015 at 7:21 AM, Harsha <ka...@harsha.io> wrote: > > > > > > > > > There could be another case where if you have > > auto.create.topics.enable > > > > > to set to true ( its true by default) . Any TopicMetadataRequest > can > > > > > recreate topics. So if you issued a delete topic command and you > have > > > > > producers running or consumers? too which is issuing a > > > > > TopicMetadataRequest than the topic will be recreated. > > > > > -Harsha > > > > > > > > > > On Sun, Jan 25, 2015, at 11:26 PM, Jason Rosenberg wrote: > > > > > > cversion did change (incremented by 2) when I issue the delete > > > command. > > > > > > > > > > > > From the logs on the conroller broker (also the leader for the > > > topic), it > > > > > > looks like the delete proceeds, and then the topic gets recreated > > > > > > immediately (highlighted in yellow). It appears maybe it’s due > to a > > > > > > consumer client app trying to consume the topic. Also, the > consumer > > > is > > > > > > not > > > > > > yet updated to 0.8.2 (it’s using 0.8.1.1), perhaps that’s part of > > the > > > > > > problem? > > > > > > > > > > > > > > > > > > 2015-01-26 07:02:14,281 INFO > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace] > > > > > > controller.PartitionStateMachine$DeleteTopicsListener - > > > > > > [DeleteTopicsListener on 6]: Starting topic deletion for topics > > > > > > mytopic > > > > > > 2015-01-26 07:02:14,282 INFO [delete-topics-thread-6] > > > > > > controller.TopicDeletionManager$DeleteTopicsThread - > > > > > > [delete-topics-thread-6], Handling deletion for topics mytopic > > > > > > 2015-01-26 07:02:14,286 INFO [delete-topics-thread-6] > > > > > > controller.TopicDeletionManager$DeleteTopicsThread - > > > > > > [delete-topics-thread-6], Deletion of topic mytopic (re)started > > > > > > 2015-01-26 07:02:14,286 INFO [delete-topics-thread-6] > > > > > > controller.TopicDeletionManager - [Topic Deletion Manager 6], > Topic > > > > > > deletion callback for mytopic > > > > > > 2015-01-26 07:02:14,289 INFO [delete-topics-thread-6] > > > > > > controller.TopicDeletionManager - [Topic Deletion Manager 6], > > > > > > Partition deletion callback for [mytopic,0] > > > > > > 2015-01-26 07:02:14,295 INFO [delete-topics-thread-6] > > > > > > controller.ReplicaStateMachine - [Replica state machine on > > controller > > > > > > 6]: Invoking state change to OfflineReplica for replicas > > > > > > > > > > > > > > > > > [Topic=mytopic,Partition=0,Replica=7],[Topic=mytopic,Partition=0,Replica=6] > > > > > > 2015-01-26 07:02:14,303 INFO [delete-topics-thread-6] > > > > > > controller.KafkaController - [Controller 6]: New leader and ISR > for > > > > > > partition [mytopic,0] is {"leader":6,"leader_epoch":1,"isr":[6]} > > > > > > 2015-01-26 07:02:14,312 INFO [delete-topics-thread-6] > > > > > > controller.KafkaController - [Controller 6]: New leader and ISR > for > > > > > > partition [mytopic,0] is {"leader":-1,"leader_epoch":2,"isr":[]} > > > > > > 2015-01-26 07:02:14,313 INFO [delete-topics-thread-6] > > > > > > controller.ReplicaStateMachine - [Replica state machine on > > controller > > > > > > 6]: Invoking state change to ReplicaDeletionStarted for replicas > > > > > > > > > > > > > > > > > [Topic=mytopic,Partition=0,Replica=7],[Topic=mytopic,Partition=0,Replica=6] > > > > > > 2015-01-26 07:02:14,313 INFO [kafka-request-handler-5] > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager on broker > 6] > > > > > > Removed fetcher for partitions [mytopic,0] > > > > > > 2015-01-26 07:02:14,313 INFO [kafka-request-handler-7] > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager on broker > 6] > > > > > > Removed fetcher for partitions [mytopic,0] > > > > > > 2015-01-26 07:02:14,313 INFO [kafka-request-handler-7] > > > > > > log.OffsetIndex - Deleting index > > > > > > /mypath/mytopic-0/00000000000000000000.index > > > > > > 2015-01-26 07:02:14,313 INFO [kafka-request-handler-7] > > > log.LogManager > > > > > > - Deleted log for partition [mytopic,0] in /mypath/mytopic-0. > > > > > > 2015-01-26 07:02:14,314 INFO > > [Controller-6-to-broker-6-send-thread] > > > > > > controller.ReplicaStateMachine - [Replica state machine on > > controller > > > > > > 6]: Invoking state change to ReplicaDeletionSuccessful for > replicas > > > > > > [Topic=mytopic,Partition=0,Replica=6] > > > > > > 2015-01-26 07:02:14,314 INFO [delete-topics-thread-6] > > > > > > controller.TopicDeletionManager$DeleteTopicsThread - > > > > > > [delete-topics-thread-6], Handling deletion for topics mytopic > > > > > > 2015-01-26 07:02:14,316 INFO [delete-topics-thread-6] > > > > > > controller.TopicDeletionManager$DeleteTopicsThread - > > > > > > [delete-topics-thread-6], Deletion for replicas 7 for partition > > > > > > [mytopic,0] of topic mytopic in progress > > > > > > 2015-01-26 07:02:14,316 INFO > > [Controller-6-to-broker-7-send-thread] > > > > > > controller.ReplicaStateMachine - [Replica state machine on > > controller > > > > > > 6]: Invoking state change to ReplicaDeletionSuccessful for > replicas > > > > > > [Topic=mytopic,Partition=0,Replica=7] > > > > > > 2015-01-26 07:02:14,316 INFO [delete-topics-thread-6] > > > > > > controller.TopicDeletionManager$DeleteTopicsThread - > > > > > > [delete-topics-thread-6], Handling deletion for topics mytopic > > > > > > 2015-01-26 07:02:14,318 INFO [delete-topics-thread-6] > > > > > > controller.ReplicaStateMachine - [Replica state machine on > > controller > > > > > > 6]: Invoking state change to NonExistentReplica for replicas > > > > > > > > > > > > > > > > > [Topic=mytopic,Partition=0,Replica=6],[Topic=mytopic,Partition=0,Replica=7] > > > > > > 2015-01-26 07:02:14,318 INFO [delete-topics-thread-6] > > > > > > controller.PartitionStateMachine - [Partition state machine on > > > > > > Controller 6]: Invoking state change to OfflinePartition for > > > > > > partitions [mytopic,0] > > > > > > 2015-01-26 07:02:14,318 INFO [delete-topics-thread-6] > > > > > > controller.PartitionStateMachine - [Partition state machine on > > > > > > Controller 6]: Invoking state change to NonExistentPartition for > > > > > > partitions [mytopic,0] > > > > > > 2015-01-26 07:02:14,325 WARN [kafka-request-handler-2] > > > > > > server.ReplicaManager - [Replica Manager on Broker 6]: Fetch > > request > > > > > > with correlation id 2619442 from client ReplicaFetcherThread-0-6 > on > > > > > > partition [mytopic,0] failed due to Partition [mytopic,0] doesn't > > > > > > exist on 6 > > > > > > 2015-01-26 07:02:14,371 INFO [delete-topics-thread-6] > > > > > > controller.TopicDeletionManager$DeleteTopicsThread - > > > > > > [delete-topics-thread-6], Deletion of topic mytopic successfully > > > > > > completed > > > > > > 2015-01-26 07:02:14,376 INFO > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace] > > > > > > controller.PartitionStateMachine$TopicChangeListener - > > > > > > [TopicChangeListener on Controller 6]: New topics: [Set()], > deleted > > > > > > topics: [Set()], new partition replica assignment [Map()] > > > > > > 2015-01-26 07:02:14,403 WARN [request-expiration-task] > > > > > > server.ReplicaManager - [Replica Manager on Broker 6]: Fetch > > request > > > > > > with correlation id 2054626 from client > > > > > > myconsumer-1415333962444-cc78e67e-0-6 on partition [mytopic,0] > > failed > > > > > > due to Partition [mytopic,0] doesn't exist on 6 > > > > > > 2015-01-26 07:02:14,446 INFO [kafka-request-handler-5] > > > > > > admin.AdminUtils$ - Topic creation > > > > > > {"version":1,"partitions":{"0":[7,6]}} > > > > > > 2015-01-26 07:02:14,457 INFO [kafka-request-handler-5] > > > > > > server.KafkaApis - [KafkaApi-6] Auto creation of topic mytopic > > with 1 > > > > > > partitions and replication factor 2 is successful! > > > > > > 2015-01-26 07:02:14,458 INFO [kafka-network-thread-27330-2] > > > > > > network.Processor - Closing socket connection to /10.1.10.4. > > > > > > 2015-01-26 07:02:14,466 INFO > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace] > > > > > > controller.PartitionStateMachine$TopicChangeListener - > > > > > > [TopicChangeListener on Controller 6]: New topics: > [Set(mytopic)], > > > > > > deleted topics: [Set()], new partition replica assignment > > > > > > [Map([mytopic,0] -> List(7, 6))] > > > > > > 2015-01-26 07:02:14,466 INFO > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace] > > > > > > controller.KafkaController - [Controller 6]: New topic creation > > > > > > callback for [mytopic,0] > > > > > > 2015-01-26 07:02:14,466 INFO > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace] > > > > > > controller.KafkaController - [Controller 6]: New partition > creation > > > > > > callback for [mytopic,0] > > > > > > 2015-01-26 07:02:14,466 INFO > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace] > > > > > > controller.PartitionStateMachine - [Partition state machine on > > > > > > Controller 6]: Invoking state change to NewPartition for > partitions > > > > > > [mytopic,0] > > > > > > 2015-01-26 07:02:14,467 INFO > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace] > > > > > > controller.ReplicaStateMachine - [Replica state machine on > > controller > > > > > > 6]: Invoking state change to NewReplica for replicas > > > > > > > > > > > > > > > > > [Topic=mytopic,Partition=0,Replica=7],[Topic=mytopic,Partition=0,Replica=6] > > > > > > 2015-01-26 07:02:14,468 INFO > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace] > > > > > > controller.PartitionStateMachine - [Partition state machine on > > > > > > Controller 6]: Invoking state change to OnlinePartition for > > > partitions > > > > > > [mytopic,0] > > > > > > 2015-01-26 07:02:14,530 INFO > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace] > > > > > > controller.ReplicaStateMachine - [Replica state machine on > > controller > > > > > > 6]: Invoking state change to OnlineReplica for replicas > > > > > > > > > > > > > > > > > [Topic=mytopic,Partition=0,Replica=7],[Topic=mytopic,Partition=0,Replica=6] > > > > > > 2015-01-26 07:02:14,532 INFO [kafka-request-handler-5] log.Log - > > > > > > Completed load of log mytopic-0 with log end offset 0 > > > > > > 2015-01-26 07:02:14,533 INFO [kafka-request-handler-5] > > > log.LogManager > > > > > > - Created log for partition [mytopic,0] in /mypath with > properties > > > > > > {segment.index.bytes -> 10485760, file.delete.delay.ms -> 60000, > > > > > > segment.bytes -> 1073741824, flush.ms -> 9223372036854775807, > > > > > > delete.retention.ms -> 86400000, index.interval.bytes -> 4096, > > > > > > retention.bytes -> 50000000000, min.insync.replicas -> 1, > > > > > > cleanup.policy -> delete, unclean.leader.election.enable -> true, > > > > > > segment.ms -> 604800000, max.message.bytes -> 1000012, > > > flush.messages > > > > > > -> 9223372036854775807, min.cleanable.dirty.ratio -> 0.5, > > > retention.ms > > > > > > -> 86400000, segment.jitter.ms -> 0}. > > > > > > 2015-01-26 07:02:14,540 INFO [kafka-request-handler-5] > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager on broker > 6] > > > > > > Removed fetcher for partitions [mytopic,0] > > > > > > 2015-01-26 07:02:14,540 INFO [kafka-request-handler-5] log.Log - > > > > > > Truncating log mytopic-0 to offset 0. > > > > > > 2015-01-26 07:02:14,547 INFO [kafka-request-handler-5] > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager on broker > 6] > > > > > > Added fetcher for partitions List([[mytopic,0], initOffset 0 to > > > broker > > > > > > id:7,host:mybroker:54321] ) > > > > > > > > > > > > Here’s the log in the other broker with a replica of the topic. > > > > > > > > > > > > 2015-01-26 07:02:14,312 INFO [kafka-request-handler-7] > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager on broker > 7] > > > > > > Removed fetcher for partitions [mytopic,0] > > > > > > 2015-01-26 07:02:14,313 INFO [kafka-request-handler-2] > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager on broker > 7] > > > > > > Removed fetcher for partitions [mytopic,0] > > > > > > 2015-01-26 07:02:14,313 INFO [kafka-request-handler-2] > > > > > > log.OffsetIndex - Deleting index > > > > > > /mypath/mytopic-0/00000000000000000000.index > > > > > > 2015-01-26 07:02:14,313 INFO [kafka-request-handler-2] > > > log.LogManager > > > > > > - Deleted log for partition [mytopic,0] in /mypath/mytopic-0. > > > > > > 2015-01-26 07:02:14,530 INFO [kafka-request-handler-0] > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager on broker > 7] > > > > > > Removed fetcher for partitions [mytopic,0] > > > > > > 2015-01-26 07:02:14,532 INFO [kafka-request-handler-0] log.Log - > > > > > > Completed load of log mytopic-0 with log end offset 0 > > > > > > 2015-01-26 07:02:14,533 INFO [kafka-request-handler-0] > > > log.LogManager > > > > > > - Created log for partition [mytopic,0] in /mypath with > properties > > > > > > {segment.index.bytes -> 10485760, file.delete.delay.ms -> 60000, > > > > > > segment.bytes -> 1073741824, flush.ms -> 9223372036854775807, > > > > > > delete.retention.ms -> 86400000, index.interval.bytes -> 4096, > > > > > > retention.bytes -> 50000000000, min.insync.replicas -> 1, > > > > > > cleanup.policy -> delete, unclean.leader.election.enable -> true, > > > > > > segment.ms -> 604800000, max.message.bytes -> 1000012, > > > flush.messages > > > > > > -> 9223372036854775807, min.cleanable.dirty.ratio -> 0.5, > > > retention.ms > > > > > > -> 86400000, segment.jitter.ms -> 0}. > > > > > > > > > > > > On Mon, Jan 26, 2015 at 1:26 AM, Jun Rao <j...@confluent.io> > wrote: > > > > > > > > > > > > Could you do a get on /admin/deleted_topics and see if its > cversion > > > > > > changes > > > > > > > after issuing the delete topic command? This will tell us > whether > > > the > > > > > > > path /admin/deleted_topics//mytopic > > > > > > > is actually created or not. Anything in the controller log that > > > > > indicates a > > > > > > > delete topic event is fired? > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > On Sun, Jan 25, 2015 at 9:55 PM, Jason Rosenberg < > > j...@squareup.com > > > > > > > > > wrote: > > > > > > > > > > > > > > > yes > > > > > > > > > > > > > > > > On Mon, Jan 26, 2015 at 12:18 AM, Jun Rao <j...@confluent.io> > > > wrote: > > > > > > > > > > > > > > > > > Do you have delete.topic.enable turned on in all brokers? > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > On Sun, Jan 25, 2015 at 7:56 PM, Jason Rosenberg < > > > j...@squareup.com > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > So far, I have been unable to get delete topic to work, > > with > > > > > release > > > > > > > > > > candidate 2 for 0.8.2. > > > > > > > > > > > > > > > > > > > > It worked ok when I ran it in the debugger locally, on a > > > single > > > > > node > > > > > > > > > > instance. But when I run it in our staging environment, > it > > > is not > > > > > > > > > > successfully even marking the topic for delete, for some > > > reason. > > > > > > > > > > > > > > > > > > > > I am setting delete.topic.enable to true (and see the > > > > > confirmation of > > > > > > > > > this > > > > > > > > > > in the startup logs, e.g.): > > > > > > > > > > > > > > > > > > > > INFO [main] utils.VerifiableProperties - Property > > > > > delete.topic.enable > > > > > > > > > > is overridden to true > > > > > > > > > > > > > > > > > > > > I run this command: > > > > > > > > > > > > > > > > > > > > java -cp app.jar kafka.admin.TopicCommand --zookeeper > > > > > > > > > > myzkconnect:12345/mynamespace --delete --topic mytopic > > > > > > > > > > > > > > > > > > > > log4j:WARN No appenders could be found for logger > > > > > > > > > > (org.I0Itec.zkclient.ZkConnection). > > > > > > > > > > log4j:WARN Please initialize the log4j system properly. > > > > > > > > > > log4j:WARN No appenders could be found for logger > > > > > > > > > > (org.I0Itec.zkclient.ZkEventThread). > > > > > > > > > > log4j:WARN See > > > > > http://logging.apache.org/log4j/1.2/faq.html#noconfig > > > > > > > > > > for more info. > > > > > > > > > > log4j:WARN Please initialize the log4j system properly. > > > > > > > > > > log4j:WARN See > > > > > http://logging.apache.org/log4j/1.2/faq.html#noconfig > > > > > > > > > > for more info. > > > > > > > > > > Topic mytopic is marked for deletion. > > > > > > > > > > Note: This will have no impact if delete.topic.enable is > > not > > > set > > > > > to > > > > > > > > true. > > > > > > > > > > > > > > > > > > > > I then do a —list which should at least show the topic > > > marked for > > > > > > > > > deletion: > > > > > > > > > > > > > > > > > > > > java -cp app.jar kafka.admin.TopicCommand --zookeeper > > > > > > > > > > myzkconnect:12345/mynamespace --list --topic mytopic > > > > > > > > > > > > > > > > > > > > log4j:WARN No appenders could be found for logger > > > > > > > > > > (org.I0Itec.zkclient.ZkConnection). > > > > > > > > > > log4j:WARN No appenders could be found for logger > > > > > > > > > > (org.I0Itec.zkclient.ZkEventThread). > > > > > > > > > > log4j:WARN Please initialize the log4j system properly. > > > > > > > > > > log4j:WARN Please initialize the log4j system properly. > > > > > > > > > > log4j:WARN See > > > > > http://logging.apache.org/log4j/1.2/faq.html#noconfig > > > > > > > > > > for more info. > > > > > > > > > > log4j:WARN See > > > > > http://logging.apache.org/log4j/1.2/faq.html#noconfig > > > > > > > > > > for more info. > > > > > > > > > > mytopic > > > > > > > > > > > > > > > > > > > > Note, it doesn’t list it as ‘marked for deletion’. > > > Furthermore, > > > > > > > waiting > > > > > > > > > > multiple hours still doesn’t result in the topic being > > > deleted. > > > > > > > > > > > > > > > > > > > > The topic has 1 partition, 2 replicas, and no data > stored. > > > > > > > > > > > > > > > > > > > > In zookeeper, the /admin/deleted_topics/ path is empty. > > > > > > > > > > > > > > > > > > > > The zookeeper code looks pretty straightforward, but for > > some > > > > > reason > > > > > > > is > > > > > > > > > not > > > > > > > > > > writing the deleted_topics path. We are running > zookeeper > > > 3.4.6. > > > > > > > > > > > > > > > > > > > > Thoughts? > > > > > > > > > > > > > > > > > > > > Jason > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > -- Guozhang > > >
