Hi All, Would like to bump this as the conversation sank a little bit, but more importantly I'd like to validate my plans/ideas on extending the Metadata protocol. I was thinking about two other alternatives, namely: 1. Create a ListTopicUnderDeletion protocol. This however would be unnecessary: it'd have one very narrow functionality which we can't extend. I'd make sense to have a list topics or describe topics protocol where we can list/describe topics under deletion but for normal listing/describing we already use the metadata, so it would be a duplication of functionality. 2. DeleteTopicsResponse could return the topics under deletion if the request's argument list is empty which might make sense at the first look, but actually we'd mix the query functionality with the delete functionality which is counterintuitive.
Even though most clients won't need these "limbo" topics (which are under deletion) in the foreseeable future, it can be considered as part of the cluster state or metadata and to me it makes sense. Also it doesn't have a big overhead in the response size as typically users don't delete topics too often as far as I experienced. I'd be happy to receive some ideas/feedback on this. Cheers, Viktor On Fri, Sep 28, 2018 at 4:51 PM Viktor Somogyi-Vass <viktorsomo...@gmail.com> wrote: > Hi All, > > I made an update to the KIP. Just in short: > Currently KafkaAdminClient.describeTopics() and > KafkaAdminClient.listTopics() uses the Metadata protocol to acquire topic > information. The returned response however won't contain the topics that > are under deletion but couldn't complete yet (for instance because of some > replicas offline), therefore it is not possible to implement the current > command's "marked for deletion" feature. To get around this I introduced > some changes in the Metadata protocol. > > Thanks, > Viktor > > On Fri, Sep 28, 2018 at 4:48 PM Viktor Somogyi-Vass < > viktorsomo...@gmail.com> wrote: > >> Hi Mickael, >> >> Thanks for the feedback, I also think that many customers wanted this for >> a long time. >> >> Cheers, >> Viktor >> >> On Fri, Sep 28, 2018 at 11:45 AM Mickael Maison <mickael.mai...@gmail.com> >> wrote: >> >>> Hi Viktor, >>> Thanks for taking this task! >>> This is a very nice change as it will allow users to use this tool in >>> many Cloud environments where direct zookeeper access is not >>> available. >>> >>> >>> On Thu, Sep 27, 2018 at 10:34 AM Viktor Somogyi-Vass >>> <viktorsomo...@gmail.com> wrote: >>> > >>> > Hi All, >>> > >>> > This is the continuation of the old KIP-375 with the same title: >>> > >>> https://lists.apache.org/thread.html/dc71d08de8cd2f082765be22c9f88bc9f8b39bb8e0929a3a4394e9da@%3Cdev.kafka.apache.org%3E >>> > >>> > The problem there was that two KIPs were created around the same time >>> and I >>> > chose to reorganize mine a bit and give it a new number to avoid >>> > duplication. >>> > >>> > The KIP summary here once again: >>> > >>> > I wrote up a relatively simple KIP about improving the Kafka protocol >>> and >>> > the TopicCommand tool to support the new Java based AdminClient and >>> > hopefully to deprecate the Zookeeper side of it. >>> > >>> > I would be happy to receive some opinions about this. In general I >>> think >>> > this would be an important addition as this is one of the few left but >>> > important tools that still uses direct Zookeeper connection. >>> > >>> > Here is the link for the KIP: >>> > >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-377%3A+TopicCommand+to+use+AdminClient >>> > >>> > Cheers, >>> > Viktor >>> >>
