Hello Gergely, thank you for the thoughtful feedback. I'm likewise curious about your question about the ability to release fixes for 3.9 still, I hope some core devs may be able to respond to that. As 3.9 to 4.0 is relatively a much more complex upgrade than typical, I hope the community are willing to cater for further fixes to the migration path still.
HG1: As for the ability to reformat an existing storage, I think that's a valid question. However, for the sake of getting a fix to this bug landing in a timely manner, I would suggest to aim for the much less complex problem of just providing the baseline support first. With the --zk-rollback-compatible, a lot of situations will become solvable, as it is typically possible to provision new replacement nodes, or start over configuration from scratch for a badly formatted broker. I am also not certain how a reformatting tool would work, it seems there may be some complicated questions to answer for that: what should it do when it finds incompatible messages? Rather than attempting to answer all of that, I propose we now aim to get an initial fix out with the --zk-rollback-compatible flag, so that it can be shipped as soon as possible. BR, Anton Den fre 17 juli 2026 kl 22:53 skrev Gergely Harmadás <[email protected]>: > Hi Anton, > > Thanks for the KIP, in my opinion it is really important to cover edge > cases like this as the migration can be already tricky enough. > > HG1: Using the `--zk-rollback-compatible` flag eliminates the > compatibility issue but the operator is still at the mercy of knowing about > the flag before executing the formatting command. Chances are that the very > same issue will come up even with a kafka-storage tool updated with your > fix as the flag will be omitted for various reasons. Would it make sense to > add a reformatting ability to the kafka-storage tool to recover the > rollback scenario? Also we could include a more detailed error message when > the broker crashes in ZK mode about the reformatting possibility. > > My second question is actually aimed at the project maintainers, I hope it > is not too off-topic here. I have recently discovered that Kafka 3.9.2 was > moved to archived releases. What does this mean exactly for > contributions/KIPs like this targeting the 3.9 branch? Given enough > interest is it still possible to have a 3.9.3 release? > > > Thanks, > Gergely > > > On Thu, 9 Jul 2026 at 12:18, Anton Agestam via dev <[email protected]> > wrote: > >> Hi everyone, >> >> I would like to start the discussion for KIP-1362: Add a >> rollback-compatible storage format option for KRaft migration. >> >> KIP: >> >> https://cwiki.apache.org/confluence/spaces/KAFKA/pages/440303727/KIP-1362+Add+a+rollback-compatible+storage+format+option+for+KRaft+migration >> Jira: https://issues.apache.org/jira/browse/KAFKA-20762 >> PR/reference implementation: https://github.com/apache/kafka/pull/22751 >> >> In summary: this KIP proposes closing a current gap, where the ability to >> do rollback of an ongoing KRaft migration breaks when cluster member nodes >> are replaced. The proposed solution is to add a CLI flag to the storage >> formatting tool that causes the resulting storage to not break when rolled >> back. >> >> I'm looking forward to feedback on this proposal. >> >> Thanks, >> Anton >> >> -- >> [image: Aiven] <https://www.aiven.io/> >> *Anton Agestam* (he/him or they/them) >> Software Engineer, *Aiven* >> [email protected] | +46 704 486 289 >> aiven.io <https://www.aiven.io/> | >> <https://www.facebook.com/aivencloud> >> <https://www.linkedin.com/company/aiven/> < >> https://twitter.com/aiven_io> >> >
