Hi Jun,
Thanks for the response!

30. I think I might have changed this in between. The current version
says:  {"name":
"id", "type": "option[UUID]"}, "doc": topic id}
I have switched to the option type to cover the migration case where a
TopicZNode does not yet have a topic ID.
I understand that due to json, this field is written as a string, so if I
should move the "option[uuid]" to the doc field and the type should be
"string" please let me know.

40. I've added a definition for UUID.
41,42. Fixed

Thank you,
Justine

On Wed, Sep 30, 2020 at 1:15 PM Jun Rao <j...@confluent.io> wrote:

> Hi, Justine,
>
> Thanks for the summary. Just a few minor comments blow.
>
> 30.  {"name": "id", "type": "string", "doc": "version id"}}: The doc should
> say UUID. The issue seems unfixed.
>
> 40. Since UUID is public facing, could you include its definition?
>
> 41. StopReplicaResponse still includes the topic field.
>
> 42. "It is unnecessary to include the name of the topic in the following
> Request/Response calls" It would be useful to include all modified requests
> (e.g. produce) in the list below.
>
> Thanks,
>
> Jun
>
>
> On Wed, Sep 30, 2020 at 10:17 AM Justine Olshan <jols...@confluent.io>
> wrote:
>
> > Hello again,
> >
> > I've taken some time to discuss some of the remaining points brought up
> by
> > the previous emails offline. Here are some of the conclusions.
> >
> > 1. Directory Structure:
> > There was some talk about whether the directory structure should be
> changed
> > to replace all topic names with topic IDs.
> > This will be a useful change, but will prevent downgrades. It will be
> best
> > to wait until a major release, likely alongside KIP-500 changes that will
> > prevent downgrades. I've updated the KIP to include this change with the
> > note about migration and deprecation.
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout
> >
> > 2. Partition Metadata file
> > There was some disagreement about the inclusion of the partition metadata
> > file.
> > This file will be needed to persist the topic ID, especially while we
> still
> > have the old directory structure. Even after the changes, this file can
> be
> > useful for debugging and recovery.
> > Creating a single mapping file from topic names to topic IDs was
> > considered, but ultimately scrapped as it would not be as easy to
> maintain.
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile
> >
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs
> >
> > 3. Produce Protocols
> > After some further discussion, this replacing the topic name with topic
> ID
> > in these protocols has been added to the KIP.
> > This will cut down on the size of the protocol. Since changes to fetch
> are
> > included, it does make sense to update these protocols.
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce
> >
> > 4. KIP-500 Compatibility
> > After some discussion with those involved with KIP-500, it seems best to
> > use a sentinel topic ID for the metadata topic that is used before the
> > first controller election.
> > We had an issue where this metadata topic may not be assigned an ID
> before
> > utilizing the Vote and Fetch protocols. It was decided to reserve a
> unique
> > ID for this topic to be used until the controller could give the topic a
> > unique ID.
> > Having the topic keep the sentinel ID (not be reassigned to a unique ID)
> > was considered, but it seemed like a good idea to leave the option open
> for
> > the metadata topic to have a unique ID in cases where it would need to be
> > differentiated from other clusters' metadata topics. (ex. tiered storage)
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500
> >
> > I've also split up the KIP into sub-tasks on the JIRA. Hopefully this
> will
> > provide a better idea about what tasks we have, and eventually provide a
> > place to see what's done and what is left.
> > If there is a task I am missing, please let me know!
> > https://issues.apache.org/jira/browse/KAFKA-8872
> >
> > Of course, these decisions are not set in stone, and I would love to hear
> > any feedback.
> >
> > Thanks,
> > Justine
> >
> > On Mon, Sep 28, 2020 at 11:38 AM Justine Olshan <jols...@confluent.io>
> > wrote:
> >
> > > Hello all,
> > >
> > > I just wanted to follow up on this discussion. Did we come to an
> > > understanding about the directory structure?
> > >
> > > I think the biggest question here is what is acceptable to leave out
> due
> > > to scope vs. what is considered to be too much tech debt.
> > > This KIP is already pretty large with the number of changes, but it
> also
> > > makes sense to do things correctly, so I'd love to hear everyone's
> > thoughts.
> > >
> > > Thanks,
> > > Justine
> > >
> > > On Fri, Sep 25, 2020 at 8:19 AM Lucas Bradstreet <lu...@confluent.io>
> > > wrote:
> > >
> > >> Hi Ismael,
> > >>
> > >> If you do not store it in a metadata file or in the directory
> structure
> > >> would you then
> > >> require the LeaderAndIsrRequest following startup to give you some
> > notion
> > >> of
> > >> topic name in memory? We would still need this information for the
> older
> > >> protocols, but
> > >> perhaps this is what's meant by tech debt.
> > >>
> > >> Once we're free of the old non-topicID requests then I think you
> > wouldn't
> > >> need to retain the topic name.
> > >> I think the ability to easily look up topic names associated with
> > >> partition
> > >> directories would still be missed
> > >> when diagnosing issues, though maybe it wouldn't be a deal breaker
> with
> > >> the
> > >> right tooling.
> > >>
> > >> Thanks,
> > >>
> > >> Lucas
> > >>
> > >> On Fri, Sep 25, 2020 at 7:55 AM Ismael Juma <ism...@juma.me.uk>
> wrote:
> > >>
> > >> > Hi Lucas,
> > >> >
> > >> > Why would you include the name and id? I think you'd want to remove
> > the
> > >> > name from the directory name right? Jason's suggestion was that if
> you
> > >> > remove the name from the directory, then why would you need the id
> > name
> > >> > mapping file?
> > >> >
> > >> > Ismael
> > >> >
> > >> > On Thu, Sep 24, 2020 at 4:24 PM Lucas Bradstreet <
> lu...@confluent.io>
> > >> > wrote:
> > >> >
> > >> > > > 2. Part of the usage of the file is to have persistent storage
> of
> > >> the
> > >> > > topic
> > >> > > ID and use it to compare with the ID supplied in the LeaderAndIsr
> > >> > Request.
> > >> > > There is some discussion in the KIP about changes to the directory
> > >> > > structure, but I believe directory changes were considered to be
> out
> > >> of
> > >> > > scope when the KIP was written.
> > >> > >
> > >> > >
> > >> > > Yeah, I was hoping to get a better understanding of why it was
> taken
> > >> out
> > >> > of
> > >> > > scope. Perhaps Lucas Bradstreet might have more insight about the
> > >> > decision.
> > >> > > Basically my point is that we have to create additional
> > infrastructure
> > >> > here
> > >> > > to support the name/id mapping, so I wanted to understand if we
> just
> > >> > > consider this a sort of tech debt. It would be useful to cover how
> > we
> > >> > > handle the case when this file gets corrupted. Seems like we just
> > >> have to
> > >> > > trust that it matches whatever the controller tells us and rewrite
> > it?
> > >> > >
> > >> > >
> > >> > > Hi Jason, Justine,
> > >> > >
> > >> > > My thought process is that we will likely need the metadata file
> > >> whether
> > >> > we
> > >> > > rename the directories or not.
> > >> > > Linux supports filenames of up to 255 bytes and that would not be
> > >> enough
> > >> > to
> > >> > > support a directory name
> > >> > >  including both the name and topic ID. Given that fact, it seemed
> > >> > > reasonable to add the metadata file
> > >> > > and leave the directory rename until some time in the future
> > (possibly
> > >> > > never).
> > >> > >
> > >> > > If the file does get corrupted I think you're right. We would
> either
> > >> have
> > >> > > to trust it matches what the controller tells us
> > >> > >  or error out and let an administrator resolve it by checking
> across
> > >> > > replicas for consistency.
> > >> > >
> > >> > > Lucas
> > >> > >
> > >> > >
> > >> > > On Thu, Sep 24, 2020 at 3:41 PM Jason Gustafson <
> ja...@confluent.io
> > >
> > >> > > wrote:
> > >> > >
> > >> > > > Thanks Justine. Responses below:
> > >> > > >
> > >> > > > > 1. Yes, the directory will still be based on the topic names.
> > >> > > > LeaderAndIsrRequest is one of the few requests that will still
> > >> contain
> > >> > > the
> > >> > > > topic name. So I think we have this covered. Sorry for
> confusion.
> > >> > > >
> > >> > > > Ah, you're right. My eyes passed right over the field.
> > >> > > >
> > >> > > > > 2. Part of the usage of the file is to have persistent storage
> > of
> > >> the
> > >> > > > topic
> > >> > > > ID and use it to compare with the ID supplied in the
> LeaderAndIsr
> > >> > > Request.
> > >> > > > There is some discussion in the KIP about changes to the
> directory
> > >> > > > structure, but I believe directory changes were considered to be
> > >> out of
> > >> > > > scope when the KIP was written.
> > >> > > >
> > >> > > > Yeah, I was hoping to get a better understanding of why it was
> > taken
> > >> > out
> > >> > > of
> > >> > > > scope. Perhaps Lucas Bradstreet might have more insight about
> the
> > >> > > decision.
> > >> > > > Basically my point is that we have to create additional
> > >> infrastructure
> > >> > > here
> > >> > > > to support the name/id mapping, so I wanted to understand if we
> > just
> > >> > > > consider this a sort of tech debt. It would be useful to cover
> how
> > >> we
> > >> > > > handle the case when this file gets corrupted. Seems like we
> just
> > >> have
> > >> > to
> > >> > > > trust that it matches whatever the controller tells us and
> rewrite
> > >> it?
> > >> > > >
> > >> > > > > 3. I think this is a good point, but I again I wonder about
> the
> > >> scope
> > >> > > of
> > >> > > > the KIP. Most of the changes mentioned in the KIP are for
> > supporting
> > >> > > topic
> > >> > > > deletion and I believe that is why the produce request was
> listed
> > >> under
> > >> > > > future work.
> > >> > > >
> > >> > > > That's fair. I brought it up since `Fetch` is already included.
> If
> > >> > we've
> > >> > > > got `Metadata` and `Fetch`, seems we may as well do `Produce`
> and
> > >> save
> > >> > an
> > >> > > > extra kip. No strong objection though if you want to leave it
> out.
> > >> > > >
> > >> > > >
> > >> > > > -Jason
> > >> > > >
> > >> > > >
> > >> > > > On Thu, Sep 24, 2020 at 3:26 PM Justine Olshan <
> > >> jols...@confluent.io>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Hi Jason,
> > >> > > > >
> > >> > > > > Thanks for your comments.
> > >> > > > >
> > >> > > > > 1. Yes, the directory will still be based on the topic names.
> > >> > > > > LeaderAndIsrRequest is one of the few requests that will still
> > >> > contain
> > >> > > > the
> > >> > > > > topic name. So I think we have this covered. Sorry for
> > confusion.
> > >> > > > >
> > >> > > > > 2. Part of the usage of the file is to have persistent storage
> > of
> > >> the
> > >> > > > topic
> > >> > > > > ID and use it to compare with the ID supplied in the
> > LeaderAndIsr
> > >> > > > Request.
> > >> > > > > There is some discussion in the KIP about changes to the
> > directory
> > >> > > > > structure, but I believe directory changes were considered to
> be
> > >> out
> > >> > of
> > >> > > > > scope when the KIP was written.
> > >> > > > >
> > >> > > > > 3. I think this is a good point, but I again I wonder about
> the
> > >> scope
> > >> > > of
> > >> > > > > the KIP. Most of the changes mentioned in the KIP are for
> > >> supporting
> > >> > > > topic
> > >> > > > > deletion and I believe that is why the produce request was
> > listed
> > >> > under
> > >> > > > > future work.
> > >> > > > >
> > >> > > > > 4. This sounds like it might be a good solution, but I will
> need
> > >> to
> > >> > > > discuss
> > >> > > > > more with KIP-500 folks to get the details right.
> > >> > > > >
> > >> > > > > Thanks,
> > >> > > > > Justine
> > >> > > > >
> > >> > > > > On Thu, Sep 24, 2020 at 12:30 PM Jason Gustafson <
> > >> ja...@confluent.io
> > >> > >
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > Hi Justine,
> > >> > > > > >
> > >> > > > > > Thanks for picking up this work. I have a few
> > >> questions/comments:
> > >> > > > > >
> > >> > > > > > 1. It sounds like the directory structure is still going to
> be
> > >> > based
> > >> > > on
> > >> > > > > > topic names. Do I have that right? One complication is that
> > the
> > >> > > > > > LeaderAndIsr request does not include the topic name any
> > longer.
> > >> > This
> > >> > > > > means
> > >> > > > > > that a replica must wait for the UpdateMetadata request to
> > >> arrive
> > >> > > with
> > >> > > > > the
> > >> > > > > > topic name mapping before it can create the directory. I
> > wonder
> > >> if
> > >> > it
> > >> > > > > would
> > >> > > > > > be simpler to avoid assumptions on the ordering of
> > >> UpdateMetadata
> > >> > and
> > >> > > > let
> > >> > > > > > LeaderAndIsr include the topic name as well. Feels like we
> are
> > >> not
> > >> > > > saving
> > >> > > > > > that much by excluding it.
> > >> > > > > >
> > >> > > > > > 2. On a related note, it seems that the reason we have the
> > >> > partition
> > >> > > > > > metadata file is because we are not changing the directory
> > >> > structure.
> > >> > > > We
> > >> > > > > > need it so that we remember which directories map to which
> > topic
> > >> > id.
> > >> > > I
> > >> > > > > > think it would be useful to add some detail about why
> changing
> > >> the
> > >> > > > > > directory layout was rejected. Basically trying to
> understand
> > >> how
> > >> > > much
> > >> > > > > > effort we are saving by skipping this step if we have to
> > >> implement
> > >> > > this
> > >> > > > > > additional bookkeeping. One concern I have for example is
> that
> > >> the
> > >> > > > > > partition metadata file gets corrupt and then all bets are
> off
> > >> as
> > >> > far
> > >> > > > as
> > >> > > > > > topic consistency.
> > >> > > > > >
> > >> > > > > > 3. Is it worth adding support now for topic ids in the
> > `Produce`
> > >> > > > request
> > >> > > > > > now? Overhead is mentioned as one of the motivations and the
> > >> best
> > >> > > APIs
> > >> > > > to
> > >> > > > > > get savings from are `Produce` and `Fetch`.
> > >> > > > > >
> > >> > > > > > 4. When it comes to bootstrapping the metadata topic with
> > >> respect
> > >> > to
> > >> > > > > > KIP-500, one suggestion would be to reserve a sentinel topic
> > ID
> > >> > which
> > >> > > > can
> > >> > > > > > be used initially by new brokers. When the first controller
> is
> > >> > > elected,
> > >> > > > > it
> > >> > > > > > can assign a topicId to the metadata topic. This is a bit
> > >> similar
> > >> > to
> > >> > > > how
> > >> > > > > we
> > >> > > > > > were planning to generate the clusterId.
> > >> > > > > >
> > >> > > > > > Thanks,
> > >> > > > > > Jason
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > On Thu, Sep 24, 2020 at 11:10 AM Jun Rao <j...@confluent.io>
> > >> wrote:
> > >> > > > > >
> > >> > > > > > > Hi, Justine,
> > >> > > > > > >
> > >> > > > > > > Thanks for the updated KIP. A few more comments below.
> > >> > > > > > >
> > >> > > > > > > 30.  {"name": "id", "type": "string", "doc": "version
> id"}}:
> > >> The
> > >> > > doc
> > >> > > > > > should
> > >> > > > > > > say UUID.
> > >> > > > > > >
> > >> > > > > > > 31. LeaderAndIsrResponse v5 and StopReplicaResponse v4 :
> It
> > >> seems
> > >> > > > there
> > >> > > > > > is
> > >> > > > > > > no need to add topic_id at partitions level.
> > >> > > > > > >
> > >> > > > > > > 32. Regarding partition metadata file. Perhaps the key can
> > be
> > >> a
> > >> > > > single
> > >> > > > > > > word, sth like the following.
> > >> > > > > > > version: 0
> > >> > > > > > > topic_id: 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
> > >> > > > > > >
> > >> > > > > > > 33. Another tricky thing that I realized is how to support
> > the
> > >> > > > metadata
> > >> > > > > > > topic introduced in KIP-595. It's a bit tricky to assign a
> > >> UUID
> > >> > to
> > >> > > > the
> > >> > > > > > > metadata topic since we have a chicken-and-egg problem.
> The
> > >> > > > controller
> > >> > > > > > > needs to persist the UUID in the metadata topic in order
> to
> > >> > assign
> > >> > > > one
> > >> > > > > > > successfully, but the metadata topic is needed to elect a
> > >> > > controller
> > >> > > > > > > (KIP-631). So, this probably needs a bit more thought.
> > >> > > > > > >
> > >> > > > > > > Jun
> > >> > > > > > >
> > >> > > > > > > On Thu, Sep 24, 2020 at 3:04 AM Ismael Juma <
> > >> ism...@juma.me.uk>
> > >> > > > wrote:
> > >> > > > > > >
> > >> > > > > > > > Also, can we provide more details on how the Partition
> > >> Metadata
> > >> > > > file
> > >> > > > > > will
> > >> > > > > > > > be used?
> > >> > > > > > > >
> > >> > > > > > > > Ismael
> > >> > > > > > > >
> > >> > > > > > > > On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma <
> > >> ism...@juma.me.uk
> > >> > >
> > >> > > > > wrote:
> > >> > > > > > > >
> > >> > > > > > > > > Hi Justine,
> > >> > > > > > > > >
> > >> > > > > > > > > I think we need to update the "Rejected Alternatives"
> > >> section
> > >> > > to
> > >> > > > > take
> > >> > > > > > > > into
> > >> > > > > > > > > account that the proposal now removes the topic name
> > from
> > >> the
> > >> > > > fetch
> > >> > > > > > > > > request. Also, if we are removing it from the Fetch
> > >> request,
> > >> > > does
> > >> > > > > it
> > >> > > > > > > make
> > >> > > > > > > > > sense not to remove it from similar requests like
> > >> > > > > ListOffsetRequest?
> > >> > > > > > > > >
> > >> > > > > > > > > Ismael
> > >> > > > > > > > >
> > >> > > > > > > > > On Thu, Sep 24, 2020 at 2:46 AM David Jacot <
> > >> > > dja...@confluent.io
> > >> > > > >
> > >> > > > > > > wrote:
> > >> > > > > > > > >
> > >> > > > > > > > >> Hi Justine,
> > >> > > > > > > > >>
> > >> > > > > > > > >> Thanks for the KIP. I finally had time to read it :).
> > It
> > >> is
> > >> > a
> > >> > > > > great
> > >> > > > > > > > >> improvement.
> > >> > > > > > > > >>
> > >> > > > > > > > >> I have a few comments/questions:
> > >> > > > > > > > >>
> > >> > > > > > > > >> 1. It seems that the schema of the StopReplicaRequest
> > is
> > >> > > > slightly
> > >> > > > > > > > >> outdated.
> > >> > > > > > > > >> We
> > >> > > > > > > > >> did some changes as part of KIP-570. V3 is already
> > >> organized
> > >> > > by
> > >> > > > > > > topics.
> > >> > > > > > > > >>
> > >> > > > > > > > >> 2. I just want to make sure that I understand the
> > >> > > reconciliation
> > >> > > > > > > > >> logic correctly. When
> > >> > > > > > > > >> an "INCREMENTAL" LeaderAndIsr Request is received,
> the
> > >> > broker
> > >> > > > will
> > >> > > > > > > also
> > >> > > > > > > > >> reconcile
> > >> > > > > > > > >> when the local uuid does not match the uuid in the
> > >> request,
> > >> > > > right?
> > >> > > > > > In
> > >> > > > > > > > this
> > >> > > > > > > > >> case, the
> > >> > > > > > > > >> local log is staged for deletion.
> > >> > > > > > > > >>
> > >> > > > > > > > >> 3. In the documentation of the `
> > >> delete.stale.topic.delay.ms
> > >> > `
> > >> > > > > > config,
> > >> > > > > > > it
> > >> > > > > > > > >> says "When a
> > >> > > > > > > > >> FULL LeaderAndIsrRequest is received..." but I
> suppose
> > it
> > >> > > > applies
> > >> > > > > to
> > >> > > > > > > > both
> > >> > > > > > > > >> types.
> > >> > > > > > > > >>
> > >> > > > > > > > >> Best,
> > >> > > > > > > > >> David
> > >> > > > > > > > >>
> > >> > > > > > > > >> On Thu, Sep 24, 2020 at 1:40 AM Justine Olshan <
> > >> > > > > > jols...@confluent.io>
> > >> > > > > > > > >> wrote:
> > >> > > > > > > > >>
> > >> > > > > > > > >> > Hi Jun,
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > Thanks for the comments. I apologize for some of
> the
> > >> typos
> > >> > > and
> > >> > > > > > > > >> confusion.
> > >> > > > > > > > >> > I’ve updated the KIP to fix some of the issues you
> > >> > > mentioned.
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > 20.2 I’ve changed the type to String.
> > >> > > > > > > > >> > 20.1/3 I’ve updated the TopicZNode to fix
> formatting
> > >> and
> > >> > > > reflect
> > >> > > > > > the
> > >> > > > > > > > >> latest
> > >> > > > > > > > >> > version before this change.
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > 21. You are correct and I’ve removed this line.
> I’ve
> > >> also
> > >> > > > added
> > >> > > > > a
> > >> > > > > > > line
> > >> > > > > > > > >> > mentioning an IBP bump is necessary for migration
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > 22. I think the wording was unclear but your
> summary
> > is
> > >> > what
> > >> > > > was
> > >> > > > > > > > >> intended
> > >> > > > > > > > >> > by this line. I’ve updated to make this point more
> > >> clear.
> > >> > > > > “Remove
> > >> > > > > > > > >> deleted
> > >> > > > > > > > >> > topics from replicas by sending StopReplicaRequest
> V3
> > >> > before
> > >> > > > the
> > >> > > > > > IBP
> > >> > > > > > > > >> bump
> > >> > > > > > > > >> > using the old logic, and using V4 and the new logic
> > >> with
> > >> > > topic
> > >> > > > > IDs
> > >> > > > > > > > after
> > >> > > > > > > > >> > the IBP bump.”
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > 23. I’ve removed the unspecified type from the KIP
> > and
> > >> > > mention
> > >> > > > > > that
> > >> > > > > > > > IBP
> > >> > > > > > > > >> > will be used to handle this request. “IBP will be
> > used
> > >> to
> > >> > > > > > determine
> > >> > > > > > > > >> whether
> > >> > > > > > > > >> > this new form of the request will be used. For
> older
> > >> > > requests,
> > >> > > > > we
> > >> > > > > > > will
> > >> > > > > > > > >> > ignore this field and default to previous
> behavior.”
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > 24. I’ve fixed this typo.
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > 25. I've added a topics at a higher level for
> > >> > > > > LeaderAndIsrResponse
> > >> > > > > > > v5,
> > >> > > > > > > > >> > StopReplicaResponse v4. I've also changed
> > >> > StopReplicaRequest
> > >> > > > v4
> > >> > > > > to
> > >> > > > > > > > have
> > >> > > > > > > > >> > topics at a higher level.
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > 26. I’ve updated forgotten_topics_data--added the
> > >> topic ID
> > >> > > and
> > >> > > > > > > removed
> > >> > > > > > > > >> the
> > >> > > > > > > > >> > topic name
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > 27. I’ve decided on plain text, and I’ve added an
> > >> example.
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > 28. I’ve added this idea to future work.
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > Thanks again for taking a look,
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > Justine
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > On Wed, Sep 23, 2020 at 10:28 AM Jun Rao <
> > >> > j...@confluent.io>
> > >> > > > > > wrote:
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > > Hi, Justine,
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > Thanks for the response. Made another pass. A few
> > >> more
> > >> > > > > comments
> > >> > > > > > > > below.
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > 20. znode schema:
> > >> > > > > > > > >> > > 20.1 It seems that {"name": "version", "type":
> > "int",
> > >> > > "id":
> > >> > > > > > > "UUID",
> > >> > > > > > > > >> > "doc":
> > >> > > > > > > > >> > > "version id"} should be {"name": "version",
> "type":
> > >> > > "int"},
> > >> > > > > > > {"name":
> > >> > > > > > > > >> > "id",
> > >> > > > > > > > >> > > "type": "UUID", "doc": "version id"}.
> > >> > > > > > > > >> > > 20.2 The znode format is JSON which doesn't have
> > UUID
> > >> > > type.
> > >> > > > So
> > >> > > > > > the
> > >> > > > > > > > >> type
> > >> > > > > > > > >> > > probably should be string?
> > >> > > > > > > > >> > > 20.3 Also, the existing format used seems
> outdated.
> > >> It
> > >> > > > should
> > >> > > > > > have
> > >> > > > > > > > the
> > >> > > > > > > > >> > > following format.
> > >> > > > > > > > >> > >     Json.encodeAsBytes(Map(
> > >> > > > > > > > >> > >       "version" -> 2,
> > >> > > > > > > > >> > >       "partitions" ->
> replicaAssignmentJson.asJava,
> > >> > > > > > > > >> > >       "adding_replicas" ->
> > >> > > > > addingReplicasAssignmentJson.asJava,
> > >> > > > > > > > >> > >       "removing_replicas" ->
> > >> > > > > > removingReplicasAssignmentJson.asJava
> > >> > > > > > > > >> > >     ).asJava)
> > >> > > > > > > > >> > >   }
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > 21. Migration: The KIP says "The migration
> process
> > >> can
> > >> > > take
> > >> > > > > > place
> > >> > > > > > > > >> without
> > >> > > > > > > > >> > > an inter-broker protocol bump, as the format
> stored
> > >> in
> > >> > > > > > > > >> > > /brokers/topics/[topic] will be compatible with
> > older
> > >> > > broker
> > >> > > > > > > > >> versions."
> > >> > > > > > > > >> > > However, since we are bumping up the version of
> > >> > > inter-broker
> > >> > > > > > > > >> requests, it
> > >> > > > > > > > >> > > seems that we need to use IBP for migration.
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > 22. The KIP says "Remove deleted topics from
> > >> replicas by
> > >> > > > > sending
> > >> > > > > > > > >> > > StopReplicaRequest V3 for any topics which do not
> > >> > contain
> > >> > > a
> > >> > > > > > topic
> > >> > > > > > > > ID,
> > >> > > > > > > > >> and
> > >> > > > > > > > >> > > V4 for any topics which do contain a topic ID.".
> > >> > However,
> > >> > > if
> > >> > > > > we
> > >> > > > > > > use
> > >> > > > > > > > >> IBP,
> > >> > > > > > > > >> > it
> > >> > > > > > > > >> > > seems that the controller will either send
> > >> > > > StopReplicaRequest
> > >> > > > > V3
> > >> > > > > > > > >> > > or StopReplicaRequest V4, but never mixed V3 and
> V4
> > >> for
> > >> > > > > > different
> > >> > > > > > > > >> topics.
> > >> > > > > > > > >> > > Basically, before the IBP bump, V3 will be used.
> > >> After
> > >> > the
> > >> > > > IBP
> > >> > > > > > > bump,
> > >> > > > > > > > >> > > topicId will be created and V4 will be used.
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > 23. Given that we depend on IBP, do we still need
> > "0
> > >> > > > > > UNSPECIFIED"
> > >> > > > > > > > >> > > in LeaderAndIsr?
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > 24. LeaderAndIsrResponse v5 : It still has the
> > topic
> > >> > > field.
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > 25. LeaderAndIsrResponse v5, StopReplicaResponse
> > v4:
> > >> > Could
> > >> > > > we
> > >> > > > > > use
> > >> > > > > > > > this
> > >> > > > > > > > >> > > opportunity to organize the response in 2 levels,
> > >> first
> > >> > by
> > >> > > > > > topic,
> > >> > > > > > > > >> then by
> > >> > > > > > > > >> > > partition, as most other requests/responses?
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > 26. FetchRequest v13 : Should
> forgotten_topics_data
> > >> use
> > >> > > > > topicId
> > >> > > > > > > too?
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > 27. "This file can either be plain text
> (key/value
> > >> > pairs)
> > >> > > or
> > >> > > > > > > JSON."
> > >> > > > > > > > >> Have
> > >> > > > > > > > >> > we
> > >> > > > > > > > >> > > decided which one to use? Also, it would be
> helpful
> > >> to
> > >> > > > provide
> > >> > > > > > an
> > >> > > > > > > > >> > example.
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > 28. Future improvement: Another future
> improvement
> > >> > > > opportunity
> > >> > > > > > is
> > >> > > > > > > to
> > >> > > > > > > > >> use
> > >> > > > > > > > >> > > topicId in GroupMetadataManager.offsetCommitKey
> in
> > >> the
> > >> > > > > > > offset_commit
> > >> > > > > > > > >> > topic.
> > >> > > > > > > > >> > > This may save some space.
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > Thanks,
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > Jun
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > On Wed, Sep 23, 2020 at 8:50 AM Justine Olshan <
> > >> > > > > > > > jols...@confluent.io>
> > >> > > > > > > > >> > > wrote:
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > > Hi Tom,
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > > > Thanks for the comment. I think this is a
> really
> > >> good
> > >> > > idea
> > >> > > > > and
> > >> > > > > > > it
> > >> > > > > > > > >> has
> > >> > > > > > > > >> > > been
> > >> > > > > > > > >> > > > added to the KIP under the newly added tooling
> > >> > section.
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > > > Thanks again,
> > >> > > > > > > > >> > > > Justine
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom Bentley <
> > >> > > > > > > tbent...@redhat.com>
> > >> > > > > > > > >> > wrote:
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > > > > Hi Justine,
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > > > I know you started the vote thread, but on
> > >> > re-reading
> > >> > > > the
> > >> > > > > > KIP
> > >> > > > > > > I
> > >> > > > > > > > >> > noticed
> > >> > > > > > > > >> > > > > that although the topic id is included in the
> > >> > > > > > MetadataResponse
> > >> > > > > > > > >> it's
> > >> > > > > > > > >> > not
> > >> > > > > > > > >> > > > > surfaced in the output from `kafka-topics.sh
> > >> > > > --describe`.
> > >> > > > > > > Maybe
> > >> > > > > > > > >> that
> > >> > > > > > > > >> > > was
> > >> > > > > > > > >> > > > > intentional because ids are intentionally not
> > >> really
> > >> > > > > > something
> > >> > > > > > > > the
> > >> > > > > > > > >> > user
> > >> > > > > > > > >> > > > > should care deeply about, but it would also
> > make
> > >> > life
> > >> > > > > harder
> > >> > > > > > > for
> > >> > > > > > > > >> > anyone
> > >> > > > > > > > >> > > > > debugging Kafka and this would likely get
> worse
> > >> the
> > >> > > more
> > >> > > > > > topic
> > >> > > > > > > > ids
> > >> > > > > > > > >> > got
> > >> > > > > > > > >> > > > > rolled out across the protocols, clients etc.
> > It
> > >> > seems
> > >> > > > > > likely
> > >> > > > > > > > that
> > >> > > > > > > > >> > > > > `kafka-topics.sh` will eventually need the
> > >> ability
> > >> > to
> > >> > > > show
> > >> > > > > > the
> > >> > > > > > > > id
> > >> > > > > > > > >> of
> > >> > > > > > > > >> > a
> > >> > > > > > > > >> > > > > topic and perhaps find a topic name given an
> > id.
> > >> Is
> > >> > > > there
> > >> > > > > > any
> > >> > > > > > > > >> reason
> > >> > > > > > > > >> > > not
> > >> > > > > > > > >> > > > to
> > >> > > > > > > > >> > > > > implement that in this KIP?
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > > > Many thanks,
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > > > Tom
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > > > On Mon, Sep 21, 2020 at 9:54 PM Justine
> Olshan
> > <
> > >> > > > > > > > >> jols...@confluent.io
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > > > wrote:
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > Hi all,
> > >> > > > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > After thinking about it, I've decided to
> > remove
> > >> > the
> > >> > > > > topic
> > >> > > > > > > name
> > >> > > > > > > > >> from
> > >> > > > > > > > >> > > the
> > >> > > > > > > > >> > > > > > Fetch Request and Response after all. Since
> > >> there
> > >> > > are
> > >> > > > so
> > >> > > > > > > many
> > >> > > > > > > > of
> > >> > > > > > > > >> > > these
> > >> > > > > > > > >> > > > > > requests per second, it is worth removing
> the
> > >> > extra
> > >> > > > > > > > information.
> > >> > > > > > > > >> > I've
> > >> > > > > > > > >> > > > > > updated the KIP to reflect this change.
> > >> > > > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > Please let me know if there is anything
> else
> > we
> > >> > > should
> > >> > > > > > > discuss
> > >> > > > > > > > >> > before
> > >> > > > > > > > >> > > > > > voting.
> > >> > > > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > Thank you,
> > >> > > > > > > > >> > > > > > Justine
> > >> > > > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > On Fri, Sep 18, 2020 at 9:46 AM Justine
> > Olshan
> > >> <
> > >> > > > > > > > >> > jols...@confluent.io
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > > > > > wrote:
> > >> > > > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > > Hi Jun,
> > >> > > > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > > I see what you are saying. For now we can
> > >> remove
> > >> > > the
> > >> > > > > > extra
> > >> > > > > > > > >> > > > information.
> > >> > > > > > > > >> > > > > > > I'll leave the option to add more fields
> to
> > >> the
> > >> > > file
> > >> > > > > in
> > >> > > > > > > the
> > >> > > > > > > > >> > future.
> > >> > > > > > > > >> > > > The
> > >> > > > > > > > >> > > > > > KIP
> > >> > > > > > > > >> > > > > > > has been updated to reflect this change.
> > >> > > > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > > Thanks,
> > >> > > > > > > > >> > > > > > > Justine
> > >> > > > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > > On Fri, Sep 18, 2020 at 8:46 AM Jun Rao <
> > >> > > > > > j...@confluent.io
> > >> > > > > > > >
> > >> > > > > > > > >> > wrote:
> > >> > > > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > >> Hi, Justine,
> > >> > > > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >> Thanks for the reply.
> > >> > > > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >> 13. If the log directory is the source
> of
> > >> > truth,
> > >> > > it
> > >> > > > > > means
> > >> > > > > > > > >> that
> > >> > > > > > > > >> > the
> > >> > > > > > > > >> > > > > > >> redundant info in the metadata file will
> > be
> > >> > > > ignored.
> > >> > > > > > Then
> > >> > > > > > > > the
> > >> > > > > > > > >> > > > question
> > >> > > > > > > > >> > > > > > is
> > >> > > > > > > > >> > > > > > >> why do we need to put the redundant info
> > in
> > >> the
> > >> > > > > > metadata
> > >> > > > > > > > file
> > >> > > > > > > > >> > now?
> > >> > > > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >> Thanks,
> > >> > > > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >> Jun
> > >> > > > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >> On Thu, Sep 17, 2020 at 5:07 PM Justine
> > >> Olshan
> > >> > <
> > >> > > > > > > > >> > > > jols...@confluent.io>
> > >> > > > > > > > >> > > > > > >> wrote:
> > >> > > > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >> > Hi Jun,
> > >> > > > > > > > >> > > > > > >> > Thanks for the quick response!
> > >> > > > > > > > >> > > > > > >> >
> > >> > > > > > > > >> > > > > > >> > 12. I've decided to bump up the
> versions
> > >> on
> > >> > the
> > >> > > > > > > requests
> > >> > > > > > > > >> and
> > >> > > > > > > > >> > > > updated
> > >> > > > > > > > >> > > > > > the
> > >> > > > > > > > >> > > > > > >> > KIP. I think it's good we thoroughly
> > >> > discussed
> > >> > > > the
> > >> > > > > > > > options
> > >> > > > > > > > >> > here,
> > >> > > > > > > > >> > > > so
> > >> > > > > > > > >> > > > > we
> > >> > > > > > > > >> > > > > > >> know
> > >> > > > > > > > >> > > > > > >> > we made a good choice. :)
> > >> > > > > > > > >> > > > > > >> >
> > >> > > > > > > > >> > > > > > >> > 13. This is an interesting situation.
> I
> > >> think
> > >> > > if
> > >> > > > > this
> > >> > > > > > > > does
> > >> > > > > > > > >> > occur
> > >> > > > > > > > >> > > > we
> > >> > > > > > > > >> > > > > > >> should
> > >> > > > > > > > >> > > > > > >> > give a warning. I agree that it's hard
> > to
> > >> > know
> > >> > > > the
> > >> > > > > > > source
> > >> > > > > > > > >> of
> > >> > > > > > > > >> > > truth
> > >> > > > > > > > >> > > > > for
> > >> > > > > > > > >> > > > > > >> sure
> > >> > > > > > > > >> > > > > > >> > since the directory or the file could
> be
> > >> > > manually
> > >> > > > > > > > >> modified. I
> > >> > > > > > > > >> > > > guess
> > >> > > > > > > > >> > > > > > the
> > >> > > > > > > > >> > > > > > >> > directory could be used as the source
> of
> > >> > truth.
> > >> > > > To
> > >> > > > > be
> > >> > > > > > > > >> honest,
> > >> > > > > > > > >> > > I'm
> > >> > > > > > > > >> > > > > not
> > >> > > > > > > > >> > > > > > >> > really sure what happens in kafka when
> > the
> > >> > log
> > >> > > > > > > directory
> > >> > > > > > > > is
> > >> > > > > > > > >> > > > renamed
> > >> > > > > > > > >> > > > > > >> > manually in such a way. I'm also
> > >> wondering if
> > >> > > the
> > >> > > > > > > > >> situation is
> > >> > > > > > > > >> > > > > > >> recoverable
> > >> > > > > > > > >> > > > > > >> > in this scenario.
> > >> > > > > > > > >> > > > > > >> >
> > >> > > > > > > > >> > > > > > >> > Thanks,
> > >> > > > > > > > >> > > > > > >> > Justine
> > >> > > > > > > > >> > > > > > >> >
> > >> > > > > > > > >> > > > > > >> > On Thu, Sep 17, 2020 at 4:28 PM Jun
> Rao
> > <
> > >> > > > > > > > j...@confluent.io>
> > >> > > > > > > > >> > > wrote:
> > >> > > > > > > > >> > > > > > >> >
> > >> > > > > > > > >> > > > > > >> > > Hi, Justine,
> > >> > > > > > > > >> > > > > > >> > >
> > >> > > > > > > > >> > > > > > >> > > Thanks for the reply.
> > >> > > > > > > > >> > > > > > >> > >
> > >> > > > > > > > >> > > > > > >> > > 12. I don't have a strong preference
> > >> > either.
> > >> > > > > > However,
> > >> > > > > > > > if
> > >> > > > > > > > >> we
> > >> > > > > > > > >> > > need
> > >> > > > > > > > >> > > > > IBP
> > >> > > > > > > > >> > > > > > >> > > anyway, maybe it's easier to just
> bump
> > >> up
> > >> > the
> > >> > > > > > version
> > >> > > > > > > > for
> > >> > > > > > > > >> > all
> > >> > > > > > > > >> > > > > inter
> > >> > > > > > > > >> > > > > > >> > broker
> > >> > > > > > > > >> > > > > > >> > > requests and add the topic id field
> > as a
> > >> > > > regular
> > >> > > > > > > > field. A
> > >> > > > > > > > >> > > > regular
> > >> > > > > > > > >> > > > > > >> field
> > >> > > > > > > > >> > > > > > >> > is
> > >> > > > > > > > >> > > > > > >> > > a bit more concise in wire transfer
> > >> than a
> > >> > > > > flexible
> > >> > > > > > > > >> field.
> > >> > > > > > > > >> > > > > > >> > >
> > >> > > > > > > > >> > > > > > >> > > 13. The confusion that I was
> referring
> > >> to
> > >> > is
> > >> > > > > > between
> > >> > > > > > > > the
> > >> > > > > > > > >> > topic
> > >> > > > > > > > >> > > > > name
> > >> > > > > > > > >> > > > > > >> and
> > >> > > > > > > > >> > > > > > >> > > partition number between the log dir
> > and
> > >> > the
> > >> > > > > > metadata
> > >> > > > > > > > >> file.
> > >> > > > > > > > >> > > For
> > >> > > > > > > > >> > > > > > >> example,
> > >> > > > > > > > >> > > > > > >> > if
> > >> > > > > > > > >> > > > > > >> > > the log dir is topicA-1 and the
> > metadata
> > >> > file
> > >> > > > in
> > >> > > > > it
> > >> > > > > > > has
> > >> > > > > > > > >> > topicB
> > >> > > > > > > > >> > > > and
> > >> > > > > > > > >> > > > > > >> > > partition 0 (say due to a bug or
> > manual
> > >> > > > > > > modification),
> > >> > > > > > > > >> which
> > >> > > > > > > > >> > > one
> > >> > > > > > > > >> > > > > do
> > >> > > > > > > > >> > > > > > we
> > >> > > > > > > > >> > > > > > >> > use
> > >> > > > > > > > >> > > > > > >> > > as the source of truth?
> > >> > > > > > > > >> > > > > > >> > >
> > >> > > > > > > > >> > > > > > >> > > Jun
> > >> > > > > > > > >> > > > > > >> > >
> > >> > > > > > > > >> > > > > > >> > > On Thu, Sep 17, 2020 at 3:43 PM
> > Justine
> > >> > > Olshan
> > >> > > > <
> > >> > > > > > > > >> > > > > > jols...@confluent.io>
> > >> > > > > > > > >> > > > > > >> > > wrote:
> > >> > > > > > > > >> > > > > > >> > >
> > >> > > > > > > > >> > > > > > >> > > > Hi Jun,
> > >> > > > > > > > >> > > > > > >> > > > Thanks for the comments.
> > >> > > > > > > > >> > > > > > >> > > >
> > >> > > > > > > > >> > > > > > >> > > > 12. I bumped the
> LeaderAndIsrRequest
> > >> > > because
> > >> > > > I
> > >> > > > > > > > removed
> > >> > > > > > > > >> the
> > >> > > > > > > > >> > > > topic
> > >> > > > > > > > >> > > > > > >> name
> > >> > > > > > > > >> > > > > > >> > > field
> > >> > > > > > > > >> > > > > > >> > > > in the response. It may be
> possible
> > to
> > >> > > avoid
> > >> > > > > > > bumping
> > >> > > > > > > > >> the
> > >> > > > > > > > >> > > > version
> > >> > > > > > > > >> > > > > > >> > without
> > >> > > > > > > > >> > > > > > >> > > > that change, but I may be missing
> > >> > > something.
> > >> > > > > > > > >> > > > > > >> > > > I believe StopReplica is actually
> on
> > >> > > version
> > >> > > > 3
> > >> > > > > > now,
> > >> > > > > > > > but
> > >> > > > > > > > >> > > > because
> > >> > > > > > > > >> > > > > > >> > version 2
> > >> > > > > > > > >> > > > > > >> > > > is flexible, I kept that listed as
> > >> > version
> > >> > > 2
> > >> > > > on
> > >> > > > > > the
> > >> > > > > > > > KIP
> > >> > > > > > > > >> > > page.
> > >> > > > > > > > >> > > > > > >> However,
> > >> > > > > > > > >> > > > > > >> > > you
> > >> > > > > > > > >> > > > > > >> > > > may be right in that we may need
> to
> > >> bump
> > >> > > the
> > >> > > > > > > version
> > >> > > > > > > > on
> > >> > > > > > > > >> > > > > > StopReplica
> > >> > > > > > > > >> > > > > > >> to
> > >> > > > > > > > >> > > > > > >> > > deal
> > >> > > > > > > > >> > > > > > >> > > > with deletion differently as
> > mentioned
> > >> > > > above. I
> > >> > > > > > > don't
> > >> > > > > > > > >> know
> > >> > > > > > > > >> > > if
> > >> > > > > > > > >> > > > I
> > >> > > > > > > > >> > > > > > >> have a
> > >> > > > > > > > >> > > > > > >> > > big
> > >> > > > > > > > >> > > > > > >> > > > preference over used tagged fields
> > or
> > >> > not.
> > >> > > > > > > > >> > > > > > >> > > >
> > >> > > > > > > > >> > > > > > >> > > > 13. I was thinking that in the
> case
> > >> where
> > >> > > the
> > >> > > > > > file
> > >> > > > > > > > and
> > >> > > > > > > > >> the
> > >> > > > > > > > >> > > > > request
> > >> > > > > > > > >> > > > > > >> > topic
> > >> > > > > > > > >> > > > > > >> > > > ids don't match, it means that the
> > >> > broker's
> > >> > > > > > > topic/the
> > >> > > > > > > > >> one
> > >> > > > > > > > >> > in
> > >> > > > > > > > >> > > > the
> > >> > > > > > > > >> > > > > > >> file
> > >> > > > > > > > >> > > > > > >> > has
> > >> > > > > > > > >> > > > > > >> > > > been deleted. In that case, we
> would
> > >> need
> > >> > > to
> > >> > > > > > delete
> > >> > > > > > > > the
> > >> > > > > > > > >> > old
> > >> > > > > > > > >> > > > > topic
> > >> > > > > > > > >> > > > > > >> and
> > >> > > > > > > > >> > > > > > >> > > start
> > >> > > > > > > > >> > > > > > >> > > > receiving the new version. If the
> > >> topic
> > >> > > name
> > >> > > > > were
> > >> > > > > > > to
> > >> > > > > > > > >> > change,
> > >> > > > > > > > >> > > > but
> > >> > > > > > > > >> > > > > > the
> > >> > > > > > > > >> > > > > > >> > ids
> > >> > > > > > > > >> > > > > > >> > > > still matched, the file would also
> > >> need
> > >> > to
> > >> > > > > > update.
> > >> > > > > > > > Am I
> > >> > > > > > > > >> > > > missing
> > >> > > > > > > > >> > > > > a
> > >> > > > > > > > >> > > > > > >> case
> > >> > > > > > > > >> > > > > > >> > > > where the file would be correct
> and
> > >> not
> > >> > the
> > >> > > > > > > request?
> > >> > > > > > > > >> > > > > > >> > > >
> > >> > > > > > > > >> > > > > > >> > > > Thanks,
> > >> > > > > > > > >> > > > > > >> > > > Justine
> > >> > > > > > > > >> > > > > > >> > > >
> > >> > > > > > > > >> > > > > > >> > > > On Thu, Sep 17, 2020 at 3:18 PM
> Jun
> > >> Rao <
> > >> > > > > > > > >> j...@confluent.io
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > > > wrote:
> > >> > > > > > > > >> > > > > > >> > > >
> > >> > > > > > > > >> > > > > > >> > > > > Hi, Justine,
> > >> > > > > > > > >> > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > Thanks for the reply. A couple
> of
> > >> more
> > >> > > > > comments
> > >> > > > > > > > >> below.
> > >> > > > > > > > >> > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > 12. ListOffset and
> OffsetForLeader
> > >> > > > currently
> > >> > > > > > > don't
> > >> > > > > > > > >> > support
> > >> > > > > > > > >> > > > > > >> flexible
> > >> > > > > > > > >> > > > > > >> > > > fields.
> > >> > > > > > > > >> > > > > > >> > > > > So, we have to bump up the
> version
> > >> > number
> > >> > > > and
> > >> > > > > > use
> > >> > > > > > > > >> IBP at
> > >> > > > > > > > >> > > > least
> > >> > > > > > > > >> > > > > > for
> > >> > > > > > > > >> > > > > > >> > > these
> > >> > > > > > > > >> > > > > > >> > > > > two requests. Note that it seems
> > >> 2.7.0
> > >> > > will
> > >> > > > > > > require
> > >> > > > > > > > >> IBP
> > >> > > > > > > > >> > > > anyway
> > >> > > > > > > > >> > > > > > >> > because
> > >> > > > > > > > >> > > > > > >> > > of
> > >> > > > > > > > >> > > > > > >> > > > > changes in KAFKA-10435. Also, it
> > >> seems
> > >> > > that
> > >> > > > > the
> > >> > > > > > > > >> version
> > >> > > > > > > > >> > > for
> > >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequest and
> > StopReplica
> > >> are
> > >> > > > > bumped
> > >> > > > > > > even
> > >> > > > > > > > >> > though
> > >> > > > > > > > >> > > > we
> > >> > > > > > > > >> > > > > > only
> > >> > > > > > > > >> > > > > > >> > > added
> > >> > > > > > > > >> > > > > > >> > > > a
> > >> > > > > > > > >> > > > > > >> > > > > tagged field. But since IBP is
> > >> needed
> > >> > > > anyway,
> > >> > > > > > we
> > >> > > > > > > > may
> > >> > > > > > > > >> > want
> > >> > > > > > > > >> > > to
> > >> > > > > > > > >> > > > > > >> revisit
> > >> > > > > > > > >> > > > > > >> > > the
> > >> > > > > > > > >> > > > > > >> > > > > overall tagged field choice.
> > >> > > > > > > > >> > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > 13. The only downside is the
> > >> potential
> > >> > > > > > confusion
> > >> > > > > > > on
> > >> > > > > > > > >> > which
> > >> > > > > > > > >> > > > one
> > >> > > > > > > > >> > > > > is
> > >> > > > > > > > >> > > > > > >> the
> > >> > > > > > > > >> > > > > > >> > > > source
> > >> > > > > > > > >> > > > > > >> > > > > of truth if they don't match.
> > >> Another
> > >> > > > option
> > >> > > > > is
> > >> > > > > > > to
> > >> > > > > > > > >> > include
> > >> > > > > > > > >> > > > > those
> > >> > > > > > > > >> > > > > > >> > fields
> > >> > > > > > > > >> > > > > > >> > > > in
> > >> > > > > > > > >> > > > > > >> > > > > the metadata file when we
> actually
> > >> > change
> > >> > > > the
> > >> > > > > > > > >> directory
> > >> > > > > > > > >> > > > > > structure.
> > >> > > > > > > > >> > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > Thanks,
> > >> > > > > > > > >> > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > Jun
> > >> > > > > > > > >> > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > On Thu, Sep 17, 2020 at 2:01 PM
> > >> Justine
> > >> > > > > Olshan
> > >> > > > > > <
> > >> > > > > > > > >> > > > > > >> jols...@confluent.io
> > >> > > > > > > > >> > > > > > >> > >
> > >> > > > > > > > >> > > > > > >> > > > > wrote:
> > >> > > > > > > > >> > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > Hello all,
> > >> > > > > > > > >> > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > I've thought some more about
> > >> removing
> > >> > > the
> > >> > > > > > topic
> > >> > > > > > > > >> name
> > >> > > > > > > > >> > > field
> > >> > > > > > > > >> > > > > > from
> > >> > > > > > > > >> > > > > > >> > some
> > >> > > > > > > > >> > > > > > >> > > of
> > >> > > > > > > > >> > > > > > >> > > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > requests. On closer inspection
> > of
> > >> the
> > >> > > > > > > > >> > > requests/responses,
> > >> > > > > > > > >> > > > it
> > >> > > > > > > > >> > > > > > >> seems
> > >> > > > > > > > >> > > > > > >> > > that
> > >> > > > > > > > >> > > > > > >> > > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > internal changes would be much
> > >> larger
> > >> > > > than
> > >> > > > > I
> > >> > > > > > > > >> expected.
> > >> > > > > > > > >> > > > Some
> > >> > > > > > > > >> > > > > > >> > protocols
> > >> > > > > > > > >> > > > > > >> > > > > > involve clients, so they would
> > >> > require
> > >> > > > > > changes
> > >> > > > > > > > too.
> > >> > > > > > > > >> > I'm
> > >> > > > > > > > >> > > > > > thinking
> > >> > > > > > > > >> > > > > > >> > that
> > >> > > > > > > > >> > > > > > >> > > > for
> > >> > > > > > > > >> > > > > > >> > > > > > now, removing the topic name
> > from
> > >> > these
> > >> > > > > > > requests
> > >> > > > > > > > >> and
> > >> > > > > > > > >> > > > > responses
> > >> > > > > > > > >> > > > > > >> are
> > >> > > > > > > > >> > > > > > >> > > out
> > >> > > > > > > > >> > > > > > >> > > > of
> > >> > > > > > > > >> > > > > > >> > > > > > scope.
> > >> > > > > > > > >> > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > I have decided to just keep
> the
> > >> > change
> > >> > > > > > > > >> > > > LeaderAndIsrResponse
> > >> > > > > > > > >> > > > > to
> > >> > > > > > > > >> > > > > > >> > remove
> > >> > > > > > > > >> > > > > > >> > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > topic name, and have updated
> the
> > >> KIP
> > >> > to
> > >> > > > > > reflect
> > >> > > > > > > > >> this
> > >> > > > > > > > >> > > > > change. I
> > >> > > > > > > > >> > > > > > >> have
> > >> > > > > > > > >> > > > > > >> > > > also
> > >> > > > > > > > >> > > > > > >> > > > > > mentioned the other requests
> and
> > >> > > > responses
> > >> > > > > in
> > >> > > > > > > > >> future
> > >> > > > > > > > >> > > work.
> > >> > > > > > > > >> > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > I'm hoping to start the voting
> > >> > process
> > >> > > > > soon,
> > >> > > > > > so
> > >> > > > > > > > >> let me
> > >> > > > > > > > >> > > > know
> > >> > > > > > > > >> > > > > if
> > >> > > > > > > > >> > > > > > >> > there
> > >> > > > > > > > >> > > > > > >> > > is
> > >> > > > > > > > >> > > > > > >> > > > > > anything else we should
> discuss.
> > >> > > > > > > > >> > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > Thank you,
> > >> > > > > > > > >> > > > > > >> > > > > > Justine
> > >> > > > > > > > >> > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > On Tue, Sep 15, 2020 at 3:57
> PM
> > >> > Justine
> > >> > > > > > Olshan
> > >> > > > > > > <
> > >> > > > > > > > >> > > > > > >> > jols...@confluent.io
> > >> > > > > > > > >> > > > > > >> > > >
> > >> > > > > > > > >> > > > > > >> > > > > > wrote:
> > >> > > > > > > > >> > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > > Hello again,
> > >> > > > > > > > >> > > > > > >> > > > > > > To follow up on some of the
> > >> other
> > >> > > > > comments:
> > >> > > > > > > > >> > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > > 10/11) We can remove the
> topic
> > >> name
> > >> > > > from
> > >> > > > > > > these
> > >> > > > > > > > >> > > > > > >> > requests/responses,
> > >> > > > > > > > >> > > > > > >> > > > and
> > >> > > > > > > > >> > > > > > >> > > > > > > that means we will just have
> > to
> > >> > make
> > >> > > a
> > >> > > > > few
> > >> > > > > > > > >> internal
> > >> > > > > > > > >> > > > > changes
> > >> > > > > > > > >> > > > > > to
> > >> > > > > > > > >> > > > > > >> > make
> > >> > > > > > > > >> > > > > > >> > > > > > > partitions accessible by
> topic
> > >> id
> > >> > and
> > >> > > > > > > > partition.
> > >> > > > > > > > >> I
> > >> > > > > > > > >> > can
> > >> > > > > > > > >> > > > > > update
> > >> > > > > > > > >> > > > > > >> the
> > >> > > > > > > > >> > > > > > >> > > KIP
> > >> > > > > > > > >> > > > > > >> > > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > > remove them unless anyone
> > thinks
> > >> > they
> > >> > > > > > should
> > >> > > > > > > > >> stay.
> > >> > > > > > > > >> > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > > 12) Addressed in the
> previous
> > >> > email.
> > >> > > > I've
> > >> > > > > > > > updated
> > >> > > > > > > > >> > the
> > >> > > > > > > > >> > > > KIP
> > >> > > > > > > > >> > > > > to
> > >> > > > > > > > >> > > > > > >> > > include
> > >> > > > > > > > >> > > > > > >> > > > > > > tagged fields for the
> requests
> > >> and
> > >> > > > > > responses.
> > >> > > > > > > > >> (More
> > >> > > > > > > > >> > on
> > >> > > > > > > > >> > > > > that
> > >> > > > > > > > >> > > > > > >> > below)
> > >> > > > > > > > >> > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > > 13) I think part of the idea
> > for
> > >> > > > > including
> > >> > > > > > > this
> > >> > > > > > > > >> > > > > information
> > >> > > > > > > > >> > > > > > >> is to
> > >> > > > > > > > >> > > > > > >> > > > > prepare
> > >> > > > > > > > >> > > > > > >> > > > > > > for future changes. Perhaps
> > the
> > >> > > > directory
> > >> > > > > > > > >> structure
> > >> > > > > > > > >> > > > might
> > >> > > > > > > > >> > > > > > >> change
> > >> > > > > > > > >> > > > > > >> > > from
> > >> > > > > > > > >> > > > > > >> > > > > > > topicName_partitionNumber to
> > >> > > something
> > >> > > > > like
> > >> > > > > > > > >> > > > > > >> > > topicID_partitionNumber.
> > >> > > > > > > > >> > > > > > >> > > > > Then
> > >> > > > > > > > >> > > > > > >> > > > > > > it would be useful to have
> the
> > >> > topic
> > >> > > > name
> > >> > > > > > in
> > >> > > > > > > > the
> > >> > > > > > > > >> > file
> > >> > > > > > > > >> > > > > since
> > >> > > > > > > > >> > > > > > it
> > >> > > > > > > > >> > > > > > >> > > would
> > >> > > > > > > > >> > > > > > >> > > > > not
> > >> > > > > > > > >> > > > > > >> > > > > > be
> > >> > > > > > > > >> > > > > > >> > > > > > > in the directory structure.
> > >> > > Supporting
> > >> > > > > > topic
> > >> > > > > > > > >> renames
> > >> > > > > > > > >> > > > might
> > >> > > > > > > > >> > > > > > be
> > >> > > > > > > > >> > > > > > >> > > easier
> > >> > > > > > > > >> > > > > > >> > > > if
> > >> > > > > > > > >> > > > > > >> > > > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > > other fields are included.
> > Would
> > >> > > there
> > >> > > > be
> > >> > > > > > any
> > >> > > > > > > > >> > > downsides
> > >> > > > > > > > >> > > > to
> > >> > > > > > > > >> > > > > > >> > > including
> > >> > > > > > > > >> > > > > > >> > > > > this
> > >> > > > > > > > >> > > > > > >> > > > > > > information?
> > >> > > > > > > > >> > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > > 14)  Yes, we would need to
> > copy
> > >> the
> > >> > > > > > partition
> > >> > > > > > > > >> > metadata
> > >> > > > > > > > >> > > > > file
> > >> > > > > > > > >> > > > > > in
> > >> > > > > > > > >> > > > > > >> > this
> > >> > > > > > > > >> > > > > > >> > > > > > > process. I've updated the
> KIP
> > to
> > >> > > > include
> > >> > > > > > > this.
> > >> > > > > > > > >> > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > > 15) I believe Lucas meant v1
> > >> and v2
> > >> > > > here.
> > >> > > > > > He
> > >> > > > > > > > was
> > >> > > > > > > > >> > > > referring
> > >> > > > > > > > >> > > > > > to
> > >> > > > > > > > >> > > > > > >> how
> > >> > > > > > > > >> > > > > > >> > > the
> > >> > > > > > > > >> > > > > > >> > > > > > > requests would fall under
> > >> different
> > >> > > IBP
> > >> > > > > and
> > >> > > > > > > > meant
> > >> > > > > > > > >> > that
> > >> > > > > > > > >> > > > > older
> > >> > > > > > > > >> > > > > > >> > > brokers
> > >> > > > > > > > >> > > > > > >> > > > > > would
> > >> > > > > > > > >> > > > > > >> > > > > > > have to use the older
> version
> > of
> > >> > the
> > >> > > > > > request
> > >> > > > > > > > and
> > >> > > > > > > > >> the
> > >> > > > > > > > >> > > > > > existing
> > >> > > > > > > > >> > > > > > >> > topic
> > >> > > > > > > > >> > > > > > >> > > > > > > deletion process. At first,
> it
> > >> > seemed
> > >> > > > > like
> > >> > > > > > > > tagged
> > >> > > > > > > > >> > > fields
> > >> > > > > > > > >> > > > > > would
> > >> > > > > > > > >> > > > > > >> > > > resolve
> > >> > > > > > > > >> > > > > > >> > > > > > > the IBP issue. However, we
> may
> > >> need
> > >> > > IBP
> > >> > > > > for
> > >> > > > > > > > this
> > >> > > > > > > > >> > > request
> > >> > > > > > > > >> > > > > > after
> > >> > > > > > > > >> > > > > > >> > all
> > >> > > > > > > > >> > > > > > >> > > > > since
> > >> > > > > > > > >> > > > > > >> > > > > > > the controller handles the
> > topic
> > >> > > > deletion
> > >> > > > > > > > >> > differently
> > >> > > > > > > > >> > > > > > >> depending
> > >> > > > > > > > >> > > > > > >> > on
> > >> > > > > > > > >> > > > > > >> > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > IBP
> > >> > > > > > > > >> > > > > > >> > > > > > > version. In an older
> version,
> > we
> > >> > > can't
> > >> > > > > just
> > >> > > > > > > > send
> > >> > > > > > > > >> a
> > >> > > > > > > > >> > > > > > StopReplica
> > >> > > > > > > > >> > > > > > >> > > delete
> > >> > > > > > > > >> > > > > > >> > > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > > topic immediately like we'd
> > >> want to
> > >> > > for
> > >> > > > > > this
> > >> > > > > > > > KIP.
> > >> > > > > > > > >> > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > > This makes me wonder if we
> > want
> > >> > > tagged
> > >> > > > > > fields
> > >> > > > > > > > on
> > >> > > > > > > > >> all
> > >> > > > > > > > >> > > the
> > >> > > > > > > > >> > > > > > >> requests
> > >> > > > > > > > >> > > > > > >> > > > after
> > >> > > > > > > > >> > > > > > >> > > > > > > all. Let me know your
> > thoughts!
> > >> > > > > > > > >> > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > > Justine
> > >> > > > > > > > >> > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > > On Tue, Sep 15, 2020 at 1:03
> > PM
> > >> > > Justine
> > >> > > > > > > Olshan
> > >> > > > > > > > <
> > >> > > > > > > > >> > > > > > >> > > jols...@confluent.io
> > >> > > > > > > > >> > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > > wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >> Hi all,
> > >> > > > > > > > >> > > > > > >> > > > > > >> Jun brought up a good point
> > in
> > >> his
> > >> > > > last
> > >> > > > > > > email
> > >> > > > > > > > >> about
> > >> > > > > > > > >> > > > > tagged
> > >> > > > > > > > >> > > > > > >> > fields,
> > >> > > > > > > > >> > > > > > >> > > > and
> > >> > > > > > > > >> > > > > > >> > > > > > >> I've updated the KIP to
> > reflect
> > >> > that
> > >> > > > the
> > >> > > > > > > > >> changes to
> > >> > > > > > > > >> > > > > > requests
> > >> > > > > > > > >> > > > > > >> and
> > >> > > > > > > > >> > > > > > >> > > > > > responses
> > >> > > > > > > > >> > > > > > >> > > > > > >> will be in the form of
> tagged
> > >> > fields
> > >> > > > to
> > >> > > > > > > avoid
> > >> > > > > > > > >> > > changing
> > >> > > > > > > > >> > > > > IBP.
> > >> > > > > > > > >> > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >> > > > > > >> Jun: I plan on sending a
> > >> followup
> > >> > > > email
> > >> > > > > to
> > >> > > > > > > > >> address
> > >> > > > > > > > >> > > some
> > >> > > > > > > > >> > > > > of
> > >> > > > > > > > >> > > > > > >> the
> > >> > > > > > > > >> > > > > > >> > > other
> > >> > > > > > > > >> > > > > > >> > > > > > >> points.
> > >> > > > > > > > >> > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >> > > > > > >> Thanks,
> > >> > > > > > > > >> > > > > > >> > > > > > >> Justine
> > >> > > > > > > > >> > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >> > > > > > >> On Mon, Sep 14, 2020 at
> 4:25
> > PM
> > >> > Jun
> > >> > > > Rao
> > >> > > > > <
> > >> > > > > > > > >> > > > > j...@confluent.io>
> > >> > > > > > > > >> > > > > > >> > wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> Hi, Justine,
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> Thanks for the updated
> KIP.
> > A
> > >> few
> > >> > > > > > comments
> > >> > > > > > > > >> below.
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> 10. LeaderAndIsr Response:
> > Do
> > >> we
> > >> > > need
> > >> > > > > the
> > >> > > > > > > > topic
> > >> > > > > > > > >> > > name?
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> 11. For the changed
> > >> > > request/response,
> > >> > > > > > other
> > >> > > > > > > > >> than
> > >> > > > > > > > >> > > > > > >> LeaderAndIsr,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> UpdateMetadata, Metadata,
> do
> > >> we
> > >> > > need
> > >> > > > to
> > >> > > > > > > > include
> > >> > > > > > > > >> > the
> > >> > > > > > > > >> > > > > topic
> > >> > > > > > > > >> > > > > > >> name?
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> 12. It seems that upgrades
> > >> don't
> > >> > > > > require
> > >> > > > > > > IBP.
> > >> > > > > > > > >> Does
> > >> > > > > > > > >> > > > that
> > >> > > > > > > > >> > > > > > mean
> > >> > > > > > > > >> > > > > > >> > the
> > >> > > > > > > > >> > > > > > >> > > > new
> > >> > > > > > > > >> > > > > > >> > > > > > >>> fields
> > >> > > > > > > > >> > > > > > >> > > > > > >>> in all the
> request/response
> > >> are
> > >> > > added
> > >> > > > > as
> > >> > > > > > > > tagged
> > >> > > > > > > > >> > > fields
> > >> > > > > > > > >> > > > > > >> without
> > >> > > > > > > > >> > > > > > >> > > > > bumping
> > >> > > > > > > > >> > > > > > >> > > > > > up
> > >> > > > > > > > >> > > > > > >> > > > > > >>> the request version? It
> > would
> > >> be
> > >> > > > useful
> > >> > > > > > to
> > >> > > > > > > > make
> > >> > > > > > > > >> > that
> > >> > > > > > > > >> > > > > > clear.
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> 13. Partition Metadata
> file:
> > >> Do
> > >> > we
> > >> > > > need
> > >> > > > > > to
> > >> > > > > > > > >> include
> > >> > > > > > > > >> > > the
> > >> > > > > > > > >> > > > > > topic
> > >> > > > > > > > >> > > > > > >> > name
> > >> > > > > > > > >> > > > > > >> > > > and
> > >> > > > > > > > >> > > > > > >> > > > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> partition id since they
> are
> > >> > implied
> > >> > > > in
> > >> > > > > > the
> > >> > > > > > > > >> > directory
> > >> > > > > > > > >> > > > > name?
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> 14. In the JBOD mode, we
> > >> support
> > >> > > > > moving a
> > >> > > > > > > > >> > > partition's
> > >> > > > > > > > >> > > > > data
> > >> > > > > > > > >> > > > > > >> from
> > >> > > > > > > > >> > > > > > >> > > one
> > >> > > > > > > > >> > > > > > >> > > > > > disk
> > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> another. Will the new
> > >> partition
> > >> > > > > metadata
> > >> > > > > > > file
> > >> > > > > > > > >> be
> > >> > > > > > > > >> > > > copied
> > >> > > > > > > > >> > > > > > >> during
> > >> > > > > > > > >> > > > > > >> > > that
> > >> > > > > > > > >> > > > > > >> > > > > > >>> process?
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> 15. The KIP says "Remove
> > >> deleted
> > >> > > > topics
> > >> > > > > > > from
> > >> > > > > > > > >> > > replicas
> > >> > > > > > > > >> > > > by
> > >> > > > > > > > >> > > > > > >> > sending
> > >> > > > > > > > >> > > > > > >> > > > > > >>> StopReplicaRequest V2 for
> > any
> > >> > > topics
> > >> > > > > > which
> > >> > > > > > > do
> > >> > > > > > > > >> not
> > >> > > > > > > > >> > > > > contain
> > >> > > > > > > > >> > > > > > a
> > >> > > > > > > > >> > > > > > >> > topic
> > >> > > > > > > > >> > > > > > >> > > > ID,
> > >> > > > > > > > >> > > > > > >> > > > > > and
> > >> > > > > > > > >> > > > > > >> > > > > > >>> V3 for any topics which do
> > >> > contain
> > >> > > a
> > >> > > > > > topic
> > >> > > > > > > > >> ID.".
> > >> > > > > > > > >> > > > > However,
> > >> > > > > > > > >> > > > > > it
> > >> > > > > > > > >> > > > > > >> > > seems
> > >> > > > > > > > >> > > > > > >> > > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> updated controller will
> > create
> > >> > all
> > >> > > > > > missing
> > >> > > > > > > > >> topic
> > >> > > > > > > > >> > IDs
> > >> > > > > > > > >> > > > > first
> > >> > > > > > > > >> > > > > > >> > before
> > >> > > > > > > > >> > > > > > >> > > > > doing
> > >> > > > > > > > >> > > > > > >> > > > > > >>> other actions. So, is
> > >> > > > > StopReplicaRequest
> > >> > > > > > V2
> > >> > > > > > > > >> > needed?
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> Jun
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> On Fri, Sep 11, 2020 at
> > 10:31
> > >> AM
> > >> > > John
> > >> > > > > > > > Roesler <
> > >> > > > > > > > >> > > > > > >> > > vvcep...@apache.org
> > >> > > > > > > > >> > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > Thanks, Justine!
> > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > Your response seems
> > >> compelling
> > >> > to
> > >> > > > me.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > -John
> > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > On Fri, 2020-09-11 at
> > 10:17
> > >> > > -0700,
> > >> > > > > > > Justine
> > >> > > > > > > > >> > Olshan
> > >> > > > > > > > >> > > > > wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hello all,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Thanks for continuing
> > the
> > >> > > > > > discussion! I
> > >> > > > > > > > >> have a
> > >> > > > > > > > >> > > few
> > >> > > > > > > > >> > > > > > >> > responses
> > >> > > > > > > > >> > > > > > >> > > to
> > >> > > > > > > > >> > > > > > >> > > > > > your
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > points.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Tom: You are correct
> in
> > >> that
> > >> > > this
> > >> > > > > KIP
> > >> > > > > > > has
> > >> > > > > > > > >> not
> > >> > > > > > > > >> > > > > > mentioned
> > >> > > > > > > > >> > > > > > >> the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > DeleteTopicsRequest. I
> > >> think
> > >> > > that
> > >> > > > > > this
> > >> > > > > > > > >> would
> > >> > > > > > > > >> > be
> > >> > > > > > > > >> > > > out
> > >> > > > > > > > >> > > > > of
> > >> > > > > > > > >> > > > > > >> > scope
> > >> > > > > > > > >> > > > > > >> > > > for
> > >> > > > > > > > >> > > > > > >> > > > > > >>> now, but
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > may be something worth
> > >> adding
> > >> > > in
> > >> > > > > the
> > >> > > > > > > > >> future.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > John: We did consider
> > >> > sequence
> > >> > > > ids,
> > >> > > > > > but
> > >> > > > > > > > >> there
> > >> > > > > > > > >> > > are
> > >> > > > > > > > >> > > > a
> > >> > > > > > > > >> > > > > > few
> > >> > > > > > > > >> > > > > > >> > > reasons
> > >> > > > > > > > >> > > > > > >> > > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> favor
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs. There are
> several
> > >> > cases
> > >> > > > > where
> > >> > > > > > > > topics
> > >> > > > > > > > >> > from
> > >> > > > > > > > >> > > > > > >> different
> > >> > > > > > > > >> > > > > > >> > > > > clusters
> > >> > > > > > > > >> > > > > > >> > > > > > >>> may
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > interact now and in
> the
> > >> > future.
> > >> > > > For
> > >> > > > > > > > >> example,
> > >> > > > > > > > >> > > > Mirror
> > >> > > > > > > > >> > > > > > >> Maker 2
> > >> > > > > > > > >> > > > > > >> > > may
> > >> > > > > > > > >> > > > > > >> > > > > > >>> benefit
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > from being able to
> > detect
> > >> > when
> > >> > > a
> > >> > > > > > > cluster
> > >> > > > > > > > >> being
> > >> > > > > > > > >> > > > > > mirrored
> > >> > > > > > > > >> > > > > > >> is
> > >> > > > > > > > >> > > > > > >> > > > > deleted
> > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > recreated and globally
> > >> unique
> > >> > > > > > > identifiers
> > >> > > > > > > > >> > would
> > >> > > > > > > > >> > > > make
> > >> > > > > > > > >> > > > > > >> > > resolving
> > >> > > > > > > > >> > > > > > >> > > > > > issues
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > easier than sequence
> IDs
> > >> > which
> > >> > > > may
> > >> > > > > > > > collide
> > >> > > > > > > > >> > > between
> > >> > > > > > > > >> > > > > > >> > clusters.
> > >> > > > > > > > >> > > > > > >> > > > > > KIP-405
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > (tiered storage) will
> > also
> > >> > > > benefit
> > >> > > > > > from
> > >> > > > > > > > >> > globally
> > >> > > > > > > > >> > > > > > unique
> > >> > > > > > > > >> > > > > > >> IDs
> > >> > > > > > > > >> > > > > > >> > > as
> > >> > > > > > > > >> > > > > > >> > > > > > shared
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > buckets may be used
> > >> between
> > >> > > > > clusters.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Globally unique IDs
> > would
> > >> > also
> > >> > > > make
> > >> > > > > > > > >> > > functionality
> > >> > > > > > > > >> > > > > like
> > >> > > > > > > > >> > > > > > >> > moving
> > >> > > > > > > > >> > > > > > >> > > > > > topics
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > between disparate
> > clusters
> > >> > > easier
> > >> > > > > in
> > >> > > > > > > the
> > >> > > > > > > > >> > future,
> > >> > > > > > > > >> > > > > > >> simplify
> > >> > > > > > > > >> > > > > > >> > any
> > >> > > > > > > > >> > > > > > >> > > > > > future
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > implementations of
> > backups
> > >> > and
> > >> > > > > > > restores,
> > >> > > > > > > > >> and
> > >> > > > > > > > >> > > more.
> > >> > > > > > > > >> > > > > In
> > >> > > > > > > > >> > > > > > >> > > general,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> unique IDs
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > would ensure that the
> > >> source
> > >> > > > > cluster
> > >> > > > > > > > >> topics do
> > >> > > > > > > > >> > > not
> > >> > > > > > > > >> > > > > > >> conflict
> > >> > > > > > > > >> > > > > > >> > > > with
> > >> > > > > > > > >> > > > > > >> > > > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > destination cluster
> > >> topics.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > If we were to use
> > sequence
> > >> > ids,
> > >> > > > we
> > >> > > > > > > would
> > >> > > > > > > > >> need
> > >> > > > > > > > >> > > > > > >> sufficiently
> > >> > > > > > > > >> > > > > > >> > > > large
> > >> > > > > > > > >> > > > > > >> > > > > > >>> cluster
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > ids to be stored with
> > the
> > >> > topic
> > >> > > > > > > > >> identifiers or
> > >> > > > > > > > >> > > we
> > >> > > > > > > > >> > > > > run
> > >> > > > > > > > >> > > > > > >> the
> > >> > > > > > > > >> > > > > > >> > > risk
> > >> > > > > > > > >> > > > > > >> > > > of
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > collisions. This will
> > >> give up
> > >> > > any
> > >> > > > > > > > >> advantage in
> > >> > > > > > > > >> > > > > > >> compactness
> > >> > > > > > > > >> > > > > > >> > > that
> > >> > > > > > > > >> > > > > > >> > > > > > >>> sequence
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > numbers may bring.
> Given
> > >> > these
> > >> > > > > > > > advantages I
> > >> > > > > > > > >> > > think
> > >> > > > > > > > >> > > > it
> > >> > > > > > > > >> > > > > > >> makes
> > >> > > > > > > > >> > > > > > >> > > > sense
> > >> > > > > > > > >> > > > > > >> > > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> use
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Gokul: This is an
> > >> interesting
> > >> > > > idea,
> > >> > > > > > but
> > >> > > > > > > > >> this
> > >> > > > > > > > >> > is
> > >> > > > > > > > >> > > a
> > >> > > > > > > > >> > > > > > >> breaking
> > >> > > > > > > > >> > > > > > >> > > > > change.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> Out of
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > scope for now, but
> maybe
> > >> > worth
> > >> > > > > > > discussing
> > >> > > > > > > > >> in
> > >> > > > > > > > >> > the
> > >> > > > > > > > >> > > > > > future.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hope this explains
> some
> > of
> > >> > the
> > >> > > > > > > decisions,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Justine
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > On Fri, Sep 11, 2020
> at
> > >> 8:27
> > >> > AM
> > >> > > > > Gokul
> > >> > > > > > > > >> Ramanan
> > >> > > > > > > > >> > > > > > >> Subramanian <
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > gokul24...@gmail.com>
> > >> wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Hi.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks for the KIP.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Have you thought
> about
> > >> > > whether
> > >> > > > it
> > >> > > > > > > makes
> > >> > > > > > > > >> > sense
> > >> > > > > > > > >> > > to
> > >> > > > > > > > >> > > > > > >> support
> > >> > > > > > > > >> > > > > > >> > > > > > >>> authorizing a
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > principal for a
> topic
> > ID
> > >> > > rather
> > >> > > > > > than
> > >> > > > > > > a
> > >> > > > > > > > >> topic
> > >> > > > > > > > >> > > > name
> > >> > > > > > > > >> > > > > to
> > >> > > > > > > > >> > > > > > >> > > achieve
> > >> > > > > > > > >> > > > > > >> > > > > > >>> tighter
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > security?
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Or is the topic ID
> > >> > > > fundamentally
> > >> > > > > an
> > >> > > > > > > > >> internal
> > >> > > > > > > > >> > > > > detail
> > >> > > > > > > > >> > > > > > >> > similar
> > >> > > > > > > > >> > > > > > >> > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> epochs
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > used
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in a bunch of other
> > >> places
> > >> > in
> > >> > > > > > Kafka?
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > On Fri, Sep 11, 2020
> > at
> > >> > 4:06
> > >> > > PM
> > >> > > > > > John
> > >> > > > > > > > >> > Roesler <
> > >> > > > > > > > >> > > > > > >> > > > > > vvcep...@apache.org>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Hello Justine,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks for the
> KIP!
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > I happen to have
> > been
> > >> > > > > confronted
> > >> > > > > > > > >> recently
> > >> > > > > > > > >> > > with
> > >> > > > > > > > >> > > > > the
> > >> > > > > > > > >> > > > > > >> need
> > >> > > > > > > > >> > > > > > >> > > to
> > >> > > > > > > > >> > > > > > >> > > > > keep
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > track of
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > a
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > large number of
> > >> topics as
> > >> > > > > > compactly
> > >> > > > > > > > as
> > >> > > > > > > > >> > > > > possible. I
> > >> > > > > > > > >> > > > > > >> was
> > >> > > > > > > > >> > > > > > >> > > > going
> > >> > > > > > > > >> > > > > > >> > > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> come
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > up
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > with some way to
> > >> > dictionary
> > >> > > > > > encode
> > >> > > > > > > > the
> > >> > > > > > > > >> > topic
> > >> > > > > > > > >> > > > > names
> > >> > > > > > > > >> > > > > > >> as
> > >> > > > > > > > >> > > > > > >> > > > > integers,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> but
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > seems much better!
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Apologies if this
> > has
> > >> > been
> > >> > > > > raised
> > >> > > > > > > > >> before,
> > >> > > > > > > > >> > > but
> > >> > > > > > > > >> > > > > I’m
> > >> > > > > > > > >> > > > > > >> > > wondering
> > >> > > > > > > > >> > > > > > >> > > > > > >>> about the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > choice of UUID vs
> > >> > sequence
> > >> > > > > number
> > >> > > > > > > for
> > >> > > > > > > > >> the
> > >> > > > > > > > >> > > ids.
> > >> > > > > > > > >> > > > > > >> > Typically,
> > >> > > > > > > > >> > > > > > >> > > > > I’ve
> > >> > > > > > > > >> > > > > > >> > > > > > >>> seen
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUIDs
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > in two situations:
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 1. When processes
> > >> need to
> > >> > > > > > generate
> > >> > > > > > > > >> > > > non-colliding
> > >> > > > > > > > >> > > > > > >> > > > identifiers
> > >> > > > > > > > >> > > > > > >> > > > > > >>> without
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > coordination.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2. When the
> > identifier
> > >> > > needs
> > >> > > > to
> > >> > > > > > be
> > >> > > > > > > > >> > > > “universally
> > >> > > > > > > > >> > > > > > >> > unique”;
> > >> > > > > > > > >> > > > > > >> > > > > I.e.,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > identifier needs
> to
> > >> > > > distinguish
> > >> > > > > > the
> > >> > > > > > > > >> entity
> > >> > > > > > > > >> > > > from
> > >> > > > > > > > >> > > > > > all
> > >> > > > > > > > >> > > > > > >> > other
> > >> > > > > > > > >> > > > > > >> > > > > > >>> entities
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > that
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > could ever exist.
> > >> This is
> > >> > > > > useful
> > >> > > > > > in
> > >> > > > > > > > >> cases
> > >> > > > > > > > >> > > > where
> > >> > > > > > > > >> > > > > > >> > entities
> > >> > > > > > > > >> > > > > > >> > > > from
> > >> > > > > > > > >> > > > > > >> > > > > > all
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > kinds
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > of
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > systems get mixed
> > >> > together,
> > >> > > > > such
> > >> > > > > > as
> > >> > > > > > > > >> when
> > >> > > > > > > > >> > > > dumping
> > >> > > > > > > > >> > > > > > >> logs
> > >> > > > > > > > >> > > > > > >> > > from
> > >> > > > > > > > >> > > > > > >> > > > > all
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > processes
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a company into a
> > >> common
> > >> > > > system.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Maybe I’m being
> > >> > > > short-sighted,
> > >> > > > > > but
> > >> > > > > > > it
> > >> > > > > > > > >> > > doesn’t
> > >> > > > > > > > >> > > > > seem
> > >> > > > > > > > >> > > > > > >> like
> > >> > > > > > > > >> > > > > > >> > > > > either
> > >> > > > > > > > >> > > > > > >> > > > > > >>> really
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > applies here. It
> > seems
> > >> > like
> > >> > > > the
> > >> > > > > > > > brokers
> > >> > > > > > > > >> > > could
> > >> > > > > > > > >> > > > > and
> > >> > > > > > > > >> > > > > > >> would
> > >> > > > > > > > >> > > > > > >> > > > > achieve
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > consensus
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when creating a
> > topic
> > >> > > anyway,
> > >> > > > > > which
> > >> > > > > > > > is
> > >> > > > > > > > >> all
> > >> > > > > > > > >> > > > > that’s
> > >> > > > > > > > >> > > > > > >> > > required
> > >> > > > > > > > >> > > > > > >> > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > generate
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > non-colliding
> > sequence
> > >> > ids.
> > >> > > > For
> > >> > > > > > the
> > >> > > > > > > > >> > second,
> > >> > > > > > > > >> > > as
> > >> > > > > > > > >> > > > > you
> > >> > > > > > > > >> > > > > > >> > > mention,
> > >> > > > > > > > >> > > > > > >> > > > > we
> > >> > > > > > > > >> > > > > > >> > > > > > >>> could
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > assign
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a UUID to the
> > cluster
> > >> as
> > >> > a
> > >> > > > > whole,
> > >> > > > > > > > which
> > >> > > > > > > > >> > > would
> > >> > > > > > > > >> > > > > > render
> > >> > > > > > > > >> > > > > > >> > any
> > >> > > > > > > > >> > > > > > >> > > > > > resource
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > scoped
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the broker
> > universally
> > >> > > unique
> > >> > > > > as
> > >> > > > > > > > well.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > The reason I
> mention
> > >> this
> > >> > > is
> > >> > > > > > that,
> > >> > > > > > > > >> > although
> > >> > > > > > > > >> > > a
> > >> > > > > > > > >> > > > > UUID
> > >> > > > > > > > >> > > > > > >> is
> > >> > > > > > > > >> > > > > > >> > way
> > >> > > > > > > > >> > > > > > >> > > > > more
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > compact
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > than topic names,
> > it’s
> > >> > > still
> > >> > > > 16
> > >> > > > > > > > bytes.
> > >> > > > > > > > >> In
> > >> > > > > > > > >> > > > > > contrast,
> > >> > > > > > > > >> > > > > > >> a
> > >> > > > > > > > >> > > > > > >> > > > 4-byte
> > >> > > > > > > > >> > > > > > >> > > > > > >>> integer
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > sequence id would
> > give
> > >> > us 4
> > >> > > > > > billion
> > >> > > > > > > > >> unique
> > >> > > > > > > > >> > > > > topics
> > >> > > > > > > > >> > > > > > >> per
> > >> > > > > > > > >> > > > > > >> > > > > cluster,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> which
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > like enough ;)
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Considering the
> > >> number of
> > >> > > > > > different
> > >> > > > > > > > >> times
> > >> > > > > > > > >> > > > these
> > >> > > > > > > > >> > > > > > >> topic
> > >> > > > > > > > >> > > > > > >> > > > > > >>> identifiers are
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > sent
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > over the wire or
> > >> stored
> > >> > in
> > >> > > > > > memory,
> > >> > > > > > > it
> > >> > > > > > > > >> > seems
> > >> > > > > > > > >> > > > like
> > >> > > > > > > > >> > > > > > it
> > >> > > > > > > > >> > > > > > >> > might
> > >> > > > > > > > >> > > > > > >> > > > be
> > >> > > > > > > > >> > > > > > >> > > > > > >>> worth
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > additional 4x
> space
> > >> > > savings.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > What do you think
> > >> about
> > >> > > this?
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > John
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > On Fri, Sep 11,
> > 2020,
> > >> at
> > >> > > > 03:20,
> > >> > > > > > Tom
> > >> > > > > > > > >> > Bentley
> > >> > > > > > > > >> > > > > wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Hi Justine,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > This looks like
> a
> > >> very
> > >> > > > > welcome
> > >> > > > > > > > >> > > improvement.
> > >> > > > > > > > >> > > > > > >> Thanks!
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Maybe I missed
> it,
> > >> but
> > >> > > the
> > >> > > > > KIP
> > >> > > > > > > > >> doesn't
> > >> > > > > > > > >> > > seem
> > >> > > > > > > > >> > > > to
> > >> > > > > > > > >> > > > > > >> > mention
> > >> > > > > > > > >> > > > > > >> > > > > > changing
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > DeleteTopicsRequest
> > >> to
> > >> > > > > identify
> > >> > > > > > > the
> > >> > > > > > > > >> > topic
> > >> > > > > > > > >> > > > > using
> > >> > > > > > > > >> > > > > > an
> > >> > > > > > > > >> > > > > > >> > id.
> > >> > > > > > > > >> > > > > > >> > > > > Maybe
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > that's out
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > scope, but
> > >> > > > > DeleteTopicsRequest
> > >> > > > > > is
> > >> > > > > > > > not
> > >> > > > > > > > >> > > listed
> > >> > > > > > > > >> > > > > > among
> > >> > > > > > > > >> > > > > > >> > the
> > >> > > > > > > > >> > > > > > >> > > > > Future
> > >> > > > > > > > >> > > > > > >> > > > > > >>> Work
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > APIs
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > either.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Kind regards,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Tom
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > On Thu, Sep 10,
> > >> 2020 at
> > >> > > > 3:59
> > >> > > > > PM
> > >> > > > > > > > >> Satish
> > >> > > > > > > > >> > > > > Duggana <
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >> satish.dugg...@gmail.com
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Thanks
> > >> Lucas/Justine
> > >> > > for
> > >> > > > > the
> > >> > > > > > > nice
> > >> > > > > > > > >> KIP.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > It has several
> > >> > benefits
> > >> > > > > which
> > >> > > > > > > > also
> > >> > > > > > > > >> > > include
> > >> > > > > > > > >> > > > > > >> > > simplifying
> > >> > > > > > > > >> > > > > > >> > > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> topic
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion
> process
> > >> by
> > >> > > > > > controller
> > >> > > > > > > > and
> > >> > > > > > > > >> > logs
> > >> > > > > > > > >> > > > > > cleanup
> > >> > > > > > > > >> > > > > > >> by
> > >> > > > > > > > >> > > > > > >> > > > > brokers
> > >> > > > > > > > >> > > > > > >> > > > > > in
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > corner
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Best,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Satish.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > On Wed, Sep 9,
> > >> 2020
> > >> > at
> > >> > > > > 10:07
> > >> > > > > > PM
> > >> > > > > > > > >> > Justine
> > >> > > > > > > > >> > > > > > Olshan <
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > jols...@confluent.io
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Hello all,
> > it's
> > >> > been
> > >> > > > > > almost a
> > >> > > > > > > > >> year!
> > >> > > > > > > > >> > > I've
> > >> > > > > > > > >> > > > > > made
> > >> > > > > > > > >> > > > > > >> > some
> > >> > > > > > > > >> > > > > > >> > > > > > changes
> > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > KIP
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > and hope to
> > >> continue
> > >> > > the
> > >> > > > > > > > >> discussion.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > One of the
> > main
> > >> > > changes
> > >> > > > > > I've
> > >> > > > > > > > >> added
> > >> > > > > > > > >> > is
> > >> > > > > > > > >> > > > now
> > >> > > > > > > > >> > > > > > the
> > >> > > > > > > > >> > > > > > >> > > > metadata
> > >> > > > > > > > >> > > > > > >> > > > > > >>> response
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > include the
> > topic
> > >> ID
> > >> > > (as
> > >> > > > > > Colin
> > >> > > > > > > > >> > > suggested).
> > >> > > > > > > > >> > > > > > >> Clients
> > >> > > > > > > > >> > > > > > >> > > can
> > >> > > > > > > > >> > > > > > >> > > > > > >>> obtain the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > topicID
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of a given
> topic
> > >> > > through
> > >> > > > a
> > >> > > > > > > > >> > > > TopicDescription.
> > >> > > > > > > > >> > > > > > The
> > >> > > > > > > > >> > > > > > >> > > > topicId
> > >> > > > > > > > >> > > > > > >> > > > > > will
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > also be
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > included with
> > the
> > >> > > > > > > UpdateMetadata
> > >> > > > > > > > >> > > request.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Let me know
> > what
> > >> > you
> > >> > > > all
> > >> > > > > > > think.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Thank you,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Justine
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > On
> 2019/09/13
> > >> > > 16:38:26,
> > >> > > > > > > "Colin
> > >> > > > > > > > >> > > McCabe" <
> > >> > > > > > > > >> > > > > > >> > > > > > cmcc...@apache.org
> > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Hi Lucas,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Thanks for
> > >> > tackling
> > >> > > > > this.
> > >> > > > > > > > >> Topic
> > >> > > > > > > > >> > IDs
> > >> > > > > > > > >> > > > > are a
> > >> > > > > > > > >> > > > > > >> > great
> > >> > > > > > > > >> > > > > > >> > > > > idea,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > is
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > really good
> > >> writeup.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > For
> > >> > > > > > > /brokers/topics/[topic],
> > >> > > > > > > > >> the
> > >> > > > > > > > >> > > > schema
> > >> > > > > > > > >> > > > > > >> version
> > >> > > > > > > > >> > > > > > >> > > > > should
> > >> > > > > > > > >> > > > > > >> > > > > > be
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > bumped
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > version 3,
> > rather
> > >> > than
> > >> > > 2.
> > >> > > > > > > > KIP-455
> > >> > > > > > > > >> > > bumped
> > >> > > > > > > > >> > > > > the
> > >> > > > > > > > >> > > > > > >> > version
> > >> > > > > > > > >> > > > > > >> > > > of
> > >> > > > > > > > >> > > > > > >> > > > > > this
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > znode
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > already :)
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Given that
> > >> we're
> > >> > > > going
> > >> > > > > to
> > >> > > > > > > be
> > >> > > > > > > > >> > seeing
> > >> > > > > > > > >> > > > > these
> > >> > > > > > > > >> > > > > > >> > things
> > >> > > > > > > > >> > > > > > >> > > as
> > >> > > > > > > > >> > > > > > >> > > > > > >>> strings
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > as
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > lot
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > (in
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > logs, in
> > >> ZooKeeper,
> > >> > on
> > >> > > > the
> > >> > > > > > > > >> > command-line,
> > >> > > > > > > > >> > > > > > etc.),
> > >> > > > > > > > >> > > > > > >> > does
> > >> > > > > > > > >> > > > > > >> > > it
> > >> > > > > > > > >> > > > > > >> > > > > > make
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > sense to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > use
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > base64 when
> > >> > converting
> > >> > > > them
> > >> > > > > > to
> > >> > > > > > > > >> > strings?
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Here is an
> > >> > example
> > >> > > of
> > >> > > > > the
> > >> > > > > > > hex
> > >> > > > > > > > >> > > > > > >> representation:
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > > > > > >> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > And here
> is
> > an
> > >> > > > example
> > >> > > > > in
> > >> > > > > > > > >> base64.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > b8tRS7h4TJ2Vt43Dp85v2A
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > The base64
> > >> > version
> > >> > > > > saves
> > >> > > > > > 15
> > >> > > > > > > > >> > letters
> > >> > > > > > > > >> > > > (to
> > >> > > > > > > > >> > > > > be
> > >> > > > > > > > >> > > > > > >> > fair,
> > >> > > > > > > > >> > > > > > >> > > 4
> > >> > > > > > > > >> > > > > > >> > > > of
> > >> > > > > > > > >> > > > > > >> > > > > > >>> those
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > were
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > dashes that we
> > >> could
> > >> > > have
> > >> > > > > > > elided
> > >> > > > > > > > in
> > >> > > > > > > > >> > the
> > >> > > > > > > > >> > > > hex
> > >> > > > > > > > >> > > > > > >> > > > > > representation.)
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Another
> > thing
> > >> to
> > >> > > > > consider
> > >> > > > > > > is
> > >> > > > > > > > >> that
> > >> > > > > > > > >> > we
> > >> > > > > > > > >> > > > > > should
> > >> > > > > > > > >> > > > > > >> > > specify
> > >> > > > > > > > >> > > > > > >> > > > > > that
> > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > all-zeroes
> UUID
> > is
> > >> > not
> > >> > > a
> > >> > > > > > valid
> > >> > > > > > > > >> topic
> > >> > > > > > > > >> > > UUID.
> > >> > > > > > > > >> > > > > >  We
> > >> > > > > > > > >> > > > > > >> > can't
> > >> > > > > > > > >> > > > > > >> > > > use
> > >> > > > > > > > >> > > > > > >> > > > > > >>> null
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > for
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > this
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > because we
> can't
> > >> > pass a
> > >> > > > > null
> > >> > > > > > > UUID
> > >> > > > > > > > >> over
> > >> > > > > > > > >> > > the
> > >> > > > > > > > >> > > > > RPC
> > >> > > > > > > > >> > > > > > >> > > protocol
> > >> > > > > > > > >> > > > > > >> > > > > > >>> (there
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > is no
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > special
> pattern
> > >> for
> > >> > > null,
> > >> > > > > nor
> > >> > > > > > > do
> > >> > > > > > > > we
> > >> > > > > > > > >> > want
> > >> > > > > > > > >> > > > to
> > >> > > > > > > > >> > > > > > >> waste
> > >> > > > > > > > >> > > > > > >> > > space
> > >> > > > > > > > >> > > > > > >> > > > > > >>> reserving
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > such
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > pattern.)
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Maybe I
> > missed
> > >> > it,
> > >> > > > but
> > >> > > > > > did
> > >> > > > > > > > you
> > >> > > > > > > > >> > > > describe
> > >> > > > > > > > >> > > > > > >> > > "migration
> > >> > > > > > > > >> > > > > > >> > > > > > of...
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > existing
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > topic[s]
> without
> > >> > topic
> > >> > > > IDs"
> > >> > > > > > in
> > >> > > > > > > > >> detail
> > >> > > > > > > > >> > in
> > >> > > > > > > > >> > > > any
> > >> > > > > > > > >> > > > > > >> > section?
> > >> > > > > > > > >> > > > > > >> > > > It
> > >> > > > > > > > >> > > > > > >> > > > > > >>> seems
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > like
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the new
> > controller
> > >> > > > becomes
> > >> > > > > > > > active,
> > >> > > > > > > > >> it
> > >> > > > > > > > >> > > > should
> > >> > > > > > > > >> > > > > > >> just
> > >> > > > > > > > >> > > > > > >> > > > > generate
> > >> > > > > > > > >> > > > > > >> > > > > > >>> random
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > UUIDs for
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > these, and
> write
> > >> the
> > >> > > > random
> > >> > > > > > > UUIDs
> > >> > > > > > > > >> back
> > >> > > > > > > > >> > > to
> > >> > > > > > > > >> > > > > > >> > ZooKeeper.
> > >> > > > > > > > >> > > > > > >> > > > It
> > >> > > > > > > > >> > > > > > >> > > > > > >>> would be
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > good
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > spell that
> out.
> > >> We
> > >> > > > should
> > >> > > > > > make
> > >> > > > > > > > it
> > >> > > > > > > > >> > clear
> > >> > > > > > > > >> > > > > that
> > >> > > > > > > > >> > > > > > >> this
> > >> > > > > > > > >> > > > > > >> > > > > happens
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > regardless
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the
> inter-broker
> > >> > > protocol
> > >> > > > > > > version
> > >> > > > > > > > >> > (it's
> > >> > > > > > > > >> > > a
> > >> > > > > > > > >> > > > > > >> > compatible
> > >> > > > > > > > >> > > > > > >> > > > > > change).
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > "LeaderAndIsrRequests
> > >> > > > > > > > >> including an
> > >> > > > > > > > >> > > > > > >> > > > is_every_partition
> > >> > > > > > > > >> > > > > > >> > > > > > >>> flag"
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems a
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > bit wordy.
> Can
> > we
> > >> > just
> > >> > > > > call
> > >> > > > > > > > these
> > >> > > > > > > > >> > "full
> > >> > > > > > > > >> > > > > > >> > > > > > >>> LeaderAndIsrRequests"?
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > Then
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field
> could
> > be
> > >> > > named
> > >> > > > > > > "full".
> > >> > > > > > > > >> > Also,
> > >> > > > > > > > >> > > it
> > >> > > > > > > > >> > > > > > would
> > >> > > > > > > > >> > > > > > >> > > > probably
> > >> > > > > > > > >> > > > > > >> > > > > > be
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > better
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > for the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field to
> be
> > an
> > >> > enum
> > >> > > > of
> > >> > > > > {
> > >> > > > > > > > >> > > UNSPECIFIED,
> > >> > > > > > > > >> > > > > > >> > > INCREMENTAL,
> > >> > > > > > > > >> > > > > > >> > > > > FULL
> > >> > > > > > > > >> > > > > > >> > > > > > >>> }, so
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > that
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > we
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > can cleanly
> > handle
> > >> > old
> > >> > > > > > versions
> > >> > > > > > > > (by
> > >> > > > > > > > >> > > > treating
> > >> > > > > > > > >> > > > > > >> them
> > >> > > > > > > > >> > > > > > >> > as
> > >> > > > > > > > >> > > > > > >> > > > > > >>> UNSPECIFIED)
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > In the
> > >> > > > > > LeaderAndIsrRequest
> > >> > > > > > > > >> > section,
> > >> > > > > > > > >> > > > you
> > >> > > > > > > > >> > > > > > >> write
> > >> > > > > > > > >> > > > > > >> > "A
> > >> > > > > > > > >> > > > > > >> > > > > final
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > deletion
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > event
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will be
> > secheduled
> > >> > for
> > >> > > X
> > >> > > > ms
> > >> > > > > > > after
> > >> > > > > > > > >> the
> > >> > > > > > > > >> > > > > > >> > > > LeaderAndIsrRequest
> > >> > > > > > > > >> > > > > > >> > > > > > was
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > first
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > received..."
> I
> > >> guess
> > >> > > > the X
> > >> > > > > > > was a
> > >> > > > > > > > >> > > > > placeholder
> > >> > > > > > > > >> > > > > > >> that
> > >> > > > > > > > >> > > > > > >> > > you
> > >> > > > > > > > >> > > > > > >> > > > > > >>> intended
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > replace
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > before
> posting?
> > :)
> > >> > In
> > >> > > > any
> > >> > > > > > > case,
> > >> > > > > > > > >> this
> > >> > > > > > > > >> > > > seems
> > >> > > > > > > > >> > > > > > like
> > >> > > > > > > > >> > > > > > >> > the
> > >> > > > > > > > >> > > > > > >> > > > kind
> > >> > > > > > > > >> > > > > > >> > > > > > of
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > thing
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > we'd
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > want a
> > >> configuration
> > >> > > for.
> > >> > > > > > > Let's
> > >> > > > > > > > >> > > describe
> > >> > > > > > > > >> > > > > that
> > >> > > > > > > > >> > > > > > >> > > > > > configuration
> > >> > > > > > > > >> > > > > > >> > > > > > >>> key
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > somewhere
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in this KIP,
> > >> > including
> > >> > > > what
> > >> > > > > > its
> > >> > > > > > > > >> > default
> > >> > > > > > > > >> > > > > value
> > >> > > > > > > > >> > > > > > >> is.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > We should
> > >> > probably
> > >> > > > also
> > >> > > > > > > log a
> > >> > > > > > > > >> > bunch
> > >> > > > > > > > >> > > of
> > >> > > > > > > > >> > > > > > >> messages
> > >> > > > > > > > >> > > > > > >> > > at
> > >> > > > > > > > >> > > > > > >> > > > > WARN
> > >> > > > > > > > >> > > > > > >> > > > > > >>> level
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > when
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > something is
> > >> > scheduled
> > >> > > > for
> > >> > > > > > > > >> deletion,
> > >> > > > > > > > >> > as
> > >> > > > > > > > >> > > > > well.
> > >> > > > > > > > >> > > > > > >> > (Maybe
> > >> > > > > > > > >> > > > > > >> > > > > this
> > >> > > > > > > > >> > > > > > >> > > > > > >>> was
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > assumed, but
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > it would be
> good
> > >> to
> > >> > > > mention
> > >> > > > > > > it).
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > I feel
> like
> > >> there
> > >> > > > are a
> > >> > > > > > few
> > >> > > > > > > > >> > sections
> > >> > > > > > > > >> > > > > that
> > >> > > > > > > > >> > > > > > >> > should
> > >> > > > > > > > >> > > > > > >> > > be
> > >> > > > > > > > >> > > > > > >> > > > > > >>> moved to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "rejected
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > alternatives."
> > >> For
> > >> > > > > example,
> > >> > > > > > in
> > >> > > > > > > > the
> > >> > > > > > > > >> > > > > > DeleteTopics
> > >> > > > > > > > >> > > > > > >> > > > section,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> since
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > we're
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > not
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > going to do
> > >> option 1
> > >> > or
> > >> > > > 2,
> > >> > > > > > > these
> > >> > > > > > > > >> > should
> > >> > > > > > > > >> > > be
> > >> > > > > > > > >> > > > > > moved
> > >> > > > > > > > >> > > > > > >> > into
> > >> > > > > > > > >> > > > > > >> > > > > > >>> "rejected
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > alternatives,"
> > >> > rather
> > >> > > > than
> > >> > > > > > > > >> appearing
> > >> > > > > > > > >> > > > > inline.
> > >> > > > > > > > >> > > > > > >> > > Another
> > >> > > > > > > > >> > > > > > >> > > > > case
> > >> > > > > > > > >> > > > > > >> > > > > > >>> is
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "Should
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > we remove
> topic
> > >> name
> > >> > > from
> > >> > > > > the
> > >> > > > > > > > >> protocol
> > >> > > > > > > > >> > > > where
> > >> > > > > > > > >> > > > > > >> > > possible"
> > >> > > > > > > > >> > > > > > >> > > > > > >>> section.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > This
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > is
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > clearly
> > >> discussing a
> > >> > > > design
> > >> > > > > > > > >> > alternative
> > >> > > > > > > > >> > > > that
> > >> > > > > > > > >> > > > > > >> we're
> > >> > > > > > > > >> > > > > > >> > > not
> > >> > > > > > > > >> > > > > > >> > > > > > >>> proposing
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > implement:
> > >> removing
> > >> > the
> > >> > > > > topic
> > >> > > > > > > > name
> > >> > > > > > > > >> > from
> > >> > > > > > > > >> > > > > those
> > >> > > > > > > > >> > > > > > >> > > > protocols.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Is it
> really
> > >> > > > necessary
> > >> > > > > to
> > >> > > > > > > > have
> > >> > > > > > > > >> a
> > >> > > > > > > > >> > new
> > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> /admin/delete_topics_by_id
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > path
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in ZooKeeper?
> > It
> > >> > seems
> > >> > > > > like
> > >> > > > > > we
> > >> > > > > > > > >> don't
> > >> > > > > > > > >> > > > really
> > >> > > > > > > > >> > > > > > >> need
> > >> > > > > > > > >> > > > > > >> > > this.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> Whenever
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > there is
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > a new
> > controller,
> > >> > we'll
> > >> > > > > send
> > >> > > > > > > out
> > >> > > > > > > > >> full
> > >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequests
> > >> > > > > > > > >> > > > > > >> > > > > > >>> which
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > will
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > trigger the
> > stale
> > >> > > topics
> > >> > > > to
> > >> > > > > > be
> > >> > > > > > > > >> cleaned
> > >> > > > > > > > >> > > up.
> > >> > > > > > > > >> > > > > >  The
> > >> > > > > > > > >> > > > > > >> > > active
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > controller
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > also send the
> > full
> > >> > > > > > > > >> LeaderAndIsrRequest
> > >> > > > > > > > >> > > to
> > >> > > > > > > > >> > > > > > >> brokers
> > >> > > > > > > > >> > > > > > >> > > that
> > >> > > > > > > > >> > > > > > >> > > > > are
> > >> > > > > > > > >> > > > > > >> > > > > > >>> just
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > starting
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > up.    So we
> > don't
> > >> > > really
> > >> > > > > > need
> > >> > > > > > > > this
> > >> > > > > > > > >> > kind
> > >> > > > > > > > >> > > > of
> > >> > > > > > > > >> > > > > > >> > two-phase
> > >> > > > > > > > >> > > > > > >> > > > > > commit
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > (send
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > out
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >> StopReplicasRequest,
> > >> > > get
> > >> > > > > ACKs
> > >> > > > > > > > from
> > >> > > > > > > > >> all
> > >> > > > > > > > >> > > > > nodes,
> > >> > > > > > > > >> > > > > > >> > commit
> > >> > > > > > > > >> > > > > > >> > > by
> > >> > > > > > > > >> > > > > > >> > > > > > >>> removing
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >> /admin/delete_topics
> > >> > > > node)
> > >> > > > > > any
> > >> > > > > > > > >> more.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > You
> mention
> > >> that
> > >> > > > > > > FetchRequest
> > >> > > > > > > > >> will
> > >> > > > > > > > >> > > now
> > >> > > > > > > > >> > > > > > >> include
> > >> > > > > > > > >> > > > > > >> > > UUID
> > >> > > > > > > > >> > > > > > >> > > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> avoid
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > issues
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > where requests
> > are
> > >> > made
> > >> > > > to
> > >> > > > > > > stale
> > >> > > > > > > > >> > > > partitions.
> > >> > > > > > > > >> > > > > > >> > > However,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> adding a
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUID
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> MetadataRequest
> > is
> > >> > > listed
> > >> > > > > as
> > >> > > > > > > > future
> > >> > > > > > > > >> > > work,
> > >> > > > > > > > >> > > > > out
> > >> > > > > > > > >> > > > > > of
> > >> > > > > > > > >> > > > > > >> > > scope
> > >> > > > > > > > >> > > > > > >> > > > > for
> > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > KIP.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > How
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will the
> client
> > >> learn
> > >> > > > what
> > >> > > > > > the
> > >> > > > > > > > >> topic
> > >> > > > > > > > >> > > UUID
> > >> > > > > > > > >> > > > > is,
> > >> > > > > > > > >> > > > > > if
> > >> > > > > > > > >> > > > > > >> > the
> > >> > > > > > > > >> > > > > > >> > > > > > metadata
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > response
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > doesn't
> include
> > >> that
> > >> > > > > > > information?
> > >> > > > > > > > >> It
> > >> > > > > > > > >> > > > seems
> > >> > > > > > > > >> > > > > > like
> > >> > > > > > > > >> > > > > > >> > > adding
> > >> > > > > > > > >> > > > > > >> > > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> UUID
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> MetadataResponse
> > >> > would
> > >> > > be
> > >> > > > > an
> > >> > > > > > > > >> > improvement
> > >> > > > > > > > >> > > > > here
> > >> > > > > > > > >> > > > > > >> that
> > >> > > > > > > > >> > > > > > >> > > > might
> > >> > > > > > > > >> > > > > > >> > > > > > not
> > >> > > > > > > > >> > > > > > >> > > > > > >>> be
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > too
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > hard to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > make.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > best,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Colin
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > On Mon,
> Sep
> > 9,
> > >> > > 2019,
> > >> > > > at
> > >> > > > > > > > 17:48,
> > >> > > > > > > > >> > > Ryanne
> > >> > > > > > > > >> > > > > > Dolan
> > >> > > > > > > > >> > > > > > >> > > wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > Lucas,
> > this
> > >> > would
> > >> > > > be
> > >> > > > > > > great.
> > >> > > > > > > > >> I've
> > >> > > > > > > > >> > > run
> > >> > > > > > > > >> > > > > > into
> > >> > > > > > > > >> > > > > > >> > > issues
> > >> > > > > > > > >> > > > > > >> > > > > with
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > topics
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > being
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > resurrected
> > >> > > > > > accidentally,
> > >> > > > > > > > >> since
> > >> > > > > > > > >> > a
> > >> > > > > > > > >> > > > > client
> > >> > > > > > > > >> > > > > > >> > cannot
> > >> > > > > > > > >> > > > > > >> > > > > > easily
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > distinguish
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > between
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > a
> deleted
> > >> topic
> > >> > > > and a
> > >> > > > > > new
> > >> > > > > > > > >> topic
> > >> > > > > > > > >> > > with
> > >> > > > > > > > >> > > > > the
> > >> > > > > > > > >> > > > > > >> same
> > >> > > > > > > > >> > > > > > >> > > > name.
> > >> > > > > > > > >> > > > > > >> > > > > > I'd
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > need
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > ID
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> accessible
> > >> from
> > >> > > the
> > >> > > > > > > client
> > >> > > > > > > > to
> > >> > > > > > > > >> > > solve
> > >> > > > > > > > >> > > > > that
> > >> > > > > > > > >> > > > > > >> > issue,
> > >> > > > > > > > >> > > > > > >> > > > but
> > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > is a
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > good
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > first
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > step.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > Ryanne
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > On Wed,
> > Sep
> > >> 4,
> > >> > > 2019
> > >> > > > > at
> > >> > > > > > > 1:41
> > >> > > > > > > > >> PM
> > >> > > > > > > > >> > > Lucas
> > >> > > > > > > > >> > > > > > >> > > Bradstreet <
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> lu...@confluent.io>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Hi
> all,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > I
> would
> > >> like
> > >> > to
> > >> > > > > kick
> > >> > > > > > > off
> > >> > > > > > > > >> > > > discussion
> > >> > > > > > > > >> > > > > of
> > >> > > > > > > > >> > > > > > >> > > KIP-516,
> > >> > > > > > > > >> > > > > > >> > > > > an
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > implementation
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of topic
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > IDs
> for
> > >> > Kafka.
> > >> > > > > Topic
> > >> > > > > > > IDs
> > >> > > > > > > > >> aim
> > >> > > > > > > > >> > to
> > >> > > > > > > > >> > > > > solve
> > >> > > > > > > > >> > > > > > >> topic
> > >> > > > > > > > >> > > > > > >> > > > > > >>> uniqueness
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > problems in
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Kafka,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > where
> > >> > referring
> > >> > > > to
> > >> > > > > a
> > >> > > > > > > > topic
> > >> > > > > > > > >> by
> > >> > > > > > > > >> > > name
> > >> > > > > > > > >> > > > > > >> alone is
> > >> > > > > > > > >> > > > > > >> > > > > > >>> insufficient.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Such
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> include
> > >> when
> > >> > a
> > >> > > > > topic
> > >> > > > > > > has
> > >> > > > > > > > >> been
> > >> > > > > > > > >> > > > > deleted
> > >> > > > > > > > >> > > > > > >> and
> > >> > > > > > > > >> > > > > > >> > > > > recreated
> > >> > > > > > > > >> > > > > > >> > > > > > >>> with
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > same
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > name.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Unique
> > >> > > > identifiers
> > >> > > > > > will
> > >> > > > > > > > >> help
> > >> > > > > > > > >> > > > > simplify
> > >> > > > > > > > >> > > > > > >> and
> > >> > > > > > > > >> > > > > > >> > > > improve
> > >> > > > > > > > >> > > > > > >> > > > > > >>> Kafka's
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > topic
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> process,
> > >> as
> > >> > > well
> > >> > > > as
> > >> > > > > > > > prevent
> > >> > > > > > > > >> > > cases
> > >> > > > > > > > >> > > > > > where
> > >> > > > > > > > >> > > > > > >> > > brokers
> > >> > > > > > > > >> > > > > > >> > > > > may
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > incorrectly
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > interact
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > with
> > stale
> > >> > > > versions
> > >> > > > > > of
> > >> > > > > > > > >> topics.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > >> > > >
> > >> > > > > > > > >> > > > > > >> > >
> > >> > > > > > > > >> > > > > > >> >
> > >> > > > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > >
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> >
> > >> > > > > > > > >>
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> Looking
> > >> > forward
> > >> > > > to
> > >> > > > > > your
> > >> > > > > > > > >> > > thoughts.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Lucas
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > >> > > >
> > >> > > > > > > > >> > > > > > >> > >
> > >> > > > > > > > >> > > > > > >> >
> > >> > > > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > >
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> >
> > >> > > > > > > > >>
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> >
>

Reply via email to