Hi folks, > If you use an unstable MV, you probably won't be able to upgrade your > software. Because whenever something changes, you'll probably get > serialization exceptions being thrown inside the controller. Fatal ones.
Thanks for this clarification. I think this concrete risk should be highlighted in the KIP and in the "unstable.metadata.versions.enable" documentation. In the test plan, should we also have one system test checking that "features with a stable MV will never have that MV changed"? On Wed, Jan 10, 2024 at 8:16 AM Colin McCabe <cmcc...@apache.org> wrote: > > On Tue, Jan 9, 2024, at 18:56, Proven Provenzano wrote: > > Hi folks, > > > > Thank you for the questions. > > > > Let me clarify about reorder first. The reorder of unstable metadata > > versions should be infrequent. > > Why does it need to be infrequent? We should be able to reorder unstable > metadata versions as often as we like. There are no guarantees about unstable > MVs. > > > The time you reorder is when a feature that > > requires a higher metadata version to enable becomes "production ready" and > > the features with unstable metadata versions less than the new stable one > > are moved to metadata versions greater than the new stable feature. When we > > reorder, we are always allocating a new MV and we are never reusing an > > existing MV even if it was also unstable. This way a developer upgrading > > their environment with a specific unstable MV might see existing > > functionality stop working but they won't see new MV dependent > > functionality magically appear. The feature set for a given unstable MV > > version can only decrease with reordering. > > If you use an unstable MV, you probably won't be able to upgrade your > software. Because whenever something changes, you'll probably get > serialization exceptions being thrown inside the controller. Fatal ones. > > Given that this is true, there's no reason to have special rules about what > we can and can't do with unstable MVs. We can do anything. > > > > > How do we define "production ready" and when should we bump > > LATEST_PRODUCTION? I would like to define it to be the point where the > > feature is code complete with tests and the KIP for it is approved. However > > even with this definition if the feature later develops a major issue it > > could still block future features until the issue is fixed which is what we > > are trying to avoid here. We could be much more formal about this and let > > the release manager for a release define what is stable for a given release > > and then do the bump just after the branch is created on the branch. When > > an RC candidate is accepted, the bump would be backported. I would like to > > hear other ideas here. > > > > Yeah, it's an interesting question. Overall, I think developers should define > when a feature is production ready. > > The question to ask is, "are you ready to take this feature to production in > your workplace?" I think most developers do have a sense of this. Obviously > bugs and mistakes can happen, but I think this standard would avoid most of > the issues that we're trying to avoid by having unstable MVs in the first > place. > > ELR is a good example. Nobody would have said that it was production ready in > 3.7 ... hence it belonged (and still belongs) in an unstable MV, until that > changes (hopefully soon :) ) > > best, > Colin > > > --Proven > > > > On Tue, Jan 9, 2024 at 3:26 PM Colin McCabe <cmcc...@apache.org> wrote: > > > >> Hi Justine, > >> > >> Yes, this is an important point to clarify. Proven can comment more, but > >> my understanding is that we can do anything to unstable metadata versions. > >> Reorder them, delete them, change them in any other way. There are no > >> stability guarantees. If the current text is unclear let's add more > >> examples of what we can do (which is anything) :) > >> > >> best, > >> Colin > >> > >> > >> On Mon, Jan 8, 2024, at 14:18, Justine Olshan wrote: > >> > Hey Colin, > >> > > >> > I had some offline discussions with Proven previously and it seems like > >> he > >> > said something different so I'm glad I brought it up here. > >> > > >> > Let's clarify if we are ok with reordering unstable metadata versions :) > >> > > >> > Justine > >> > > >> > On Mon, Jan 8, 2024 at 1:56 PM Colin McCabe <cmcc...@apache.org> wrote: > >> > > >> >> On Mon, Jan 8, 2024, at 13:19, Justine Olshan wrote: > >> >> > Hey all, > >> >> > > >> >> > I was wondering how often we plan to update LATEST_PRODUCTION metadata > >> >> > version. Is this something we should do as soon as the feature is > >> >> complete > >> >> > or something we do when we are releasing kafka. When is the time we > >> >> abandon > >> >> > a MV so that other features can be unblocked? > >> >> > >> >> Hi Justine, > >> >> > >> >> Thanks for reviewing. > >> >> > >> >> The idea is that you should bump LATEST_PRODUCTION when you want to > >> take a > >> >> feature to production. That could mean deploying it internally > >> somewhere to > >> >> production, or doing an Apache release that lets everyone deploy the > >> thing > >> >> to production. > >> >> > >> >> Not in production? No need to care about this. Make any changes you > >> like. > >> >> > >> >> As a corollary, we should keep the LATEST_PRODUCTION version as low as > >> it > >> >> can be. If you haven't tested the feature, don't freeze it in stone yet. > >> >> > >> >> > > >> >> > I am just considering a feature that may end up missing a release. It > >> >> seems > >> >> > like maybe that MV would block future metadata versions until we > >> decide > >> >> the > >> >> > feature won't make the cut. From that point, all "ready" features > >> should > >> >> be > >> >> > able to be released. > >> >> > >> >> The intention is the opposite. A feature in an unstable metadata version > >> >> doesn't block anything. You can always move a feature from one unstable > >> >> metadata version to another if the feature starts taking too long to > >> finish. > >> >> > >> >> > I'm also wondering if the KIP should include some information about > >> how a > >> >> > metadata should be abandoned. Maybe there is a specific message to > >> write > >> >> in > >> >> > the file? So folks who were maybe waiting on that version know they > >> can > >> >> > release their feature? > >> >> > > >> >> > I am also assuming that we don't shift all the waiting metadata > >> versions > >> >> > when we abandon a version, but it would be good to clarify and > >> include in > >> >> > the KIP. > >> >> > >> >> I'm not sure what you mean by abandoning a version. We never abandon a > >> >> version once it's stable. > >> >> > >> >> Unstable versions can change. I wouldn't describe this as "abandonment", > >> >> just the MV changing prior to release. > >> >> > >> >> In a similar way, the contents of the 3.7 branch will change up until > >> >> 3.7.0 is released. Once it gets released, it's never unreleased. We just > >> >> move on to 3.7.1. Same thing here. > >> >> > >> >> best, > >> >> Colin > >> >> > >> >> > > >> >> > Thanks, > >> >> > > >> >> > Justine > >> >> > > >> >> > On Mon, Jan 8, 2024 at 12:44 PM Colin McCabe <cmcc...@apache.org> > >> wrote: > >> >> > > >> >> >> Hi Proven, > >> >> >> > >> >> >> Thanks for the KIP. I think there is a need for this capability, for > >> >> those > >> >> >> of us who deploy from trunk (or branches dervied from trunk). > >> >> >> > >> >> >> With regard to "unstable.metadata.versions.enable": is this going to > >> be > >> >> a > >> >> >> documented configuration, or an internal one? I am guessing we want > >> it > >> >> to > >> >> >> be documented, so that users can use it. If we do, we should probably > >> >> also > >> >> >> very prominently warn that THIS WILL BREAK UPGRADES FOR YOUR CLUSTER. > >> >> That > >> >> >> includes logging an ERROR message on startup, etc. > >> >> >> > >> >> >> It would be good to document if a release can go out that contains > >> >> "future > >> >> >> MVs" that are unstable. Like can we make a 3.8 release that contains > >> >> >> IBP_4_0_IV0 in MetadataVersion.java, as an unstable future MV? > >> >> Personally I > >> >> >> think the answer should be "yes," but with the usual caveats. When > >> the > >> >> >> actual 4.0 comes out, the unstable 4.0 MV that shipped in 3.8 > >> probably > >> >> >> won't work, and you won't be able to upgrade. (It was unstable, we > >> told > >> >> you > >> >> >> not to use it.) > >> >> >> > >> >> >> best, > >> >> >> Colin > >> >> >> > >> >> >> > >> >> >> On Fri, Jan 5, 2024, at 07:32, Proven Provenzano wrote: > >> >> >> > Hey folks, > >> >> >> > > >> >> >> > I am starting a discussion thread for managing unstable metadata > >> >> >> > versions > >> >> >> > in Apache Kafka. > >> >> >> > > >> >> >> > >> >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1014%3A+Managing+Unstable+Metadata+Versions+in+Apache+Kafka > >> >> >> > > >> >> >> > This KIP is actually already implemented in 3.7 with PR > >> >> >> > https://github.com/apache/kafka/pull/14860. > >> >> >> > I have created this KIP to explain the motivation and how managing > >> >> >> Metadata > >> >> >> > Versions is expected to work. > >> >> >> > Comments are greatly appreciated as this process can always be > >> >> improved. > >> >> >> > > >> >> >> > -- > >> >> >> > --Proven > >> >> >> > >> >> > >>
