Hi Colin, Thanks, it makes sense and simplifies this KIP tremendously. I'll move this section to the rejected alternatives with a note that KIP-142 will have this feature. On a similar note: is there a KIP for describe topics protocol or have you been thinking about it? I guess there it's the same problem, we often don't want to forward the entire metadata.
Viktor On Fri, Oct 12, 2018 at 12:03 PM Colin McCabe <cmcc...@apache.org> wrote: > Hi Viktor, > > Thanks for bumping this thread. > > I think we should just focus on transitioning the TopicCommand to use > AdminClient, and talk about protocol changes in a separate KIP. Protocol > changes often involve a lot of discussion. This does mean that we couldn't > implement the "list topics under deletion" feature when using AdminClient > at the moment. We could add a note to the tool output indicating this. > > We should move the protocol discussion to a separate thread. Probably > also look at KIP-142 as well. > > best, > Colin > > > On Tue, Oct 9, 2018, at 07:45, Viktor Somogyi-Vass wrote: > > Hi All, > > > > Would like to bump this as the conversation sank a little bit, but more > > importantly I'd like to validate my plans/ideas on extending the Metadata > > protocol. I was thinking about two other alternatives, namely: > > 1. Create a ListTopicUnderDeletion protocol. This however would be > > unnecessary: it'd have one very narrow functionality which we can't > extend. > > I'd make sense to have a list topics or describe topics protocol where we > > can list/describe topics under deletion but for normal listing/describing > > we already use the metadata, so it would be a duplication of > functionality. > > 2. DeleteTopicsResponse could return the topics under deletion if the > > request's argument list is empty which might make sense at the first > look, > > but actually we'd mix the query functionality with the delete > functionality > > which is counterintuitive. > > > > Even though most clients won't need these "limbo" topics (which are under > > deletion) in the foreseeable future, it can be considered as part of the > > cluster state or metadata and to me it makes sense. Also it doesn't have > a > > big overhead in the response size as typically users don't delete topics > > too often as far as I experienced. > > > > I'd be happy to receive some ideas/feedback on this. > > > > Cheers, > > Viktor > > > > > > On Fri, Sep 28, 2018 at 4:51 PM Viktor Somogyi-Vass < > viktorsomo...@gmail.com> > > wrote: > > > > > Hi All, > > > > > > I made an update to the KIP. Just in short: > > > Currently KafkaAdminClient.describeTopics() and > > > KafkaAdminClient.listTopics() uses the Metadata protocol to acquire > topic > > > information. The returned response however won't contain the topics > that > > > are under deletion but couldn't complete yet (for instance because of > some > > > replicas offline), therefore it is not possible to implement the > current > > > command's "marked for deletion" feature. To get around this I > introduced > > > some changes in the Metadata protocol. > > > > > > Thanks, > > > Viktor > > > > > > On Fri, Sep 28, 2018 at 4:48 PM Viktor Somogyi-Vass < > > > viktorsomo...@gmail.com> wrote: > > > > > >> Hi Mickael, > > >> > > >> Thanks for the feedback, I also think that many customers wanted this > for > > >> a long time. > > >> > > >> Cheers, > > >> Viktor > > >> > > >> On Fri, Sep 28, 2018 at 11:45 AM Mickael Maison < > mickael.mai...@gmail.com> > > >> wrote: > > >> > > >>> Hi Viktor, > > >>> Thanks for taking this task! > > >>> This is a very nice change as it will allow users to use this tool in > > >>> many Cloud environments where direct zookeeper access is not > > >>> available. > > >>> > > >>> > > >>> On Thu, Sep 27, 2018 at 10:34 AM Viktor Somogyi-Vass > > >>> <viktorsomo...@gmail.com> wrote: > > >>> > > > >>> > Hi All, > > >>> > > > >>> > This is the continuation of the old KIP-375 with the same title: > > >>> > > > >>> > https://lists.apache.org/thread.html/dc71d08de8cd2f082765be22c9f88bc9f8b39bb8e0929a3a4394e9da@%3Cdev.kafka.apache.org%3E > > >>> > > > >>> > The problem there was that two KIPs were created around the same > time > > >>> and I > > >>> > chose to reorganize mine a bit and give it a new number to avoid > > >>> > duplication. > > >>> > > > >>> > The KIP summary here once again: > > >>> > > > >>> > I wrote up a relatively simple KIP about improving the Kafka > protocol > > >>> and > > >>> > the TopicCommand tool to support the new Java based AdminClient and > > >>> > hopefully to deprecate the Zookeeper side of it. > > >>> > > > >>> > I would be happy to receive some opinions about this. In general I > > >>> think > > >>> > this would be an important addition as this is one of the few left > but > > >>> > important tools that still uses direct Zookeeper connection. > > >>> > > > >>> > Here is the link for the KIP: > > >>> > > > >>> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-377%3A+TopicCommand+to+use+AdminClient > > >>> > > > >>> > Cheers, > > >>> > Viktor > > >>> > > >> >
