> > How does the active controller know what is a valid `metadata.version` > to persist? Could the active controller learn this from the > ApiVersions response from all of the inactive controllers?
The active controller should probably validate whatever value is read from meta.properties against its own range of supported versions (statically defined in code). If the operator sets a version unsupported by the active controller, that sounds like a configuration error and we should shutdown. I'm not sure what other validation we could do here without introducing ordering dependencies (e.g., must have quorum before initializing the version) For example, let's say that we have a cluster that only has remote > controllers, what are the valid metadata.version in that case? I believe it would be the intersection of supported versions across all brokers and controllers. This does raise a concern with upgrading the metadata.version in general. Currently, the active controller only validates the target version based on the brokers' support versions. We will need to include controllers supported versions here as well (using ApiVersions, probably). On Fri, Oct 15, 2021 at 1:44 PM José Armando García Sancio <jsan...@confluent.io.invalid> wrote: > On Fri, Oct 15, 2021 at 7:24 AM David Arthur <mum...@gmail.com> wrote: > > Hmm. So I think you are proposing the following flow: > > > 1. Cluster metadata partition replicas establish a quorum using > > > ApiVersions and the KRaft protocol. > > > 2. Inactive controllers send a registration RPC to the active > controller. > > > 3. The active controller persists this information to the metadata log. > > > > > > What happens if the inactive controllers send a metadata.version range > > > that is not compatible with the metadata.version set for the cluster? > > > > > > As we discussed offline, we don't need the explicit registration step. > Once > > a controller has joined the quorum, it will learn about the finalized > > "metadata.version" level once it reads that record. > > How does the active controller know what is a valid `metadata.version` > to persist? Could the active controller learn this from the > ApiVersions response from all of the inactive controllers? For > example, let's say that we have a cluster that only has remote > controllers, what are the valid metadata.version in that case? > > > If it encounters a > > version it can't support it should probably shutdown since it might not > be > > able to process any more records. > > I think that makes sense. If a controller cannot replay the metadata > log, it might as well not be part of the quorum. If the cluster > continues in this state it won't guarantee availability based on the > replication factor. > > Thanks > -- > -Jose > -- David Arthur
