[
https://issues.apache.org/jira/browse/KAFKA-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658580#comment-16658580
]
Ewen Cheslack-Postava commented on KAFKA-7481:
----------------------------------------------
[~ijuma] Arguably your suggestion isn't something we should do in a minor
version since it is a breaking semantic change, though I don't know whether
we've really committed to the reversibility of inter broker protocol version.
The upgrade notes already don't really point out that you can roll that back.
In fact, reading them now I'm not sure how many people even realize this is
possible – we never really explain anything about *why* the upgrade process is
the way it is. At a bare minimum, if we're aware of any cases where people have
rolled back without reverting the broker version entirely, I would be wary of
changing the semantics like this.
I think the tension between the two existing configs is that
inter.broker.protocol.version *today* is basically all the stateless protocol
stuff, which makes it safe to upgrade/downgrade, whereas
log.message.format.version is about persisted format, but for client-facing
data. The consumer state (and other examples from the KIP) are internal but
persistent, which makes neither a great fit.
re: being too complicated to explain, it seems like today we basically just
rely on telling them the mechanics and only for log.message.format.version do
we go into a bit more detail, and only because of the perf impact. If we just
documented what to do with the new config and it just fits in alongside one of
the other two for the common case, is it really that much more complicated?
We should probably also be clear about upgrade/downgrade scenarios. For
example, the rejected alternatives state:
{quote}
# We considered overloading `inter.broker.protocol.version` to include changes
to the persistent metadata format. The main drawback is that users cannot test
features which depend on the inter-broker protocol behavior without committing
to the upgrade. For example, KIP-320 makes changes to replication protocol
which may require additional testing.
{quote}
It took me a while to realize that "without committing to the upgrade"
specifically means the *broker version upgrade*, not the *upgrade to the
metadata format*. Even with a new config, you could roll back further use of
that format, but brokers could still process the data (as mentioned elsewhere
in the KIP wrt an in-progress rolling update to turn on the new format) as long
as it is compatible (and it seems likely we'd always have a compatibility path
since we could always have clusters with both formats).
I'm hesitant to add more configs since we already have tons, but I also
wouldn't want to conflate two things in one config unless we're really
convinced they belong together. I'd rather have the configs and then hide them
behind some aggregate config that overrides them and handles the 99% use case
as cleanly as possible than end up with fewer configs but which are harder to
reason about.
> Consider options for safer upgrade of offset commit value schema
> ----------------------------------------------------------------
>
> Key: KAFKA-7481
> URL: https://issues.apache.org/jira/browse/KAFKA-7481
> Project: Kafka
> Issue Type: Bug
> Reporter: Jason Gustafson
> Priority: Blocker
> Fix For: 2.1.0
>
>
> KIP-211 and KIP-320 add new versions of the offset commit value schema. The
> use of the new schema version is controlled by the
> `inter.broker.protocol.version` configuration. Once the new inter-broker
> version is in use, it is not possible to downgrade since the older brokers
> will not be able to parse the new schema.
> The options at the moment are the following:
> 1. Do nothing. Users can try the new version and keep
> `inter.broker.protocol.version` locked to the old release. Downgrade will
> still be possible, but users will not be able to test new capabilities which
> depend on inter-broker protocol changes.
> 2. Instead of using `inter.broker.protocol.version`, we could use
> `message.format.version`. This would basically extend the use of this config
> to apply to all persistent formats. The advantage is that it allows users to
> upgrade the broker and begin using the new inter-broker protocol while still
> allowing downgrade. But features which depend on the persistent format could
> not be tested.
> Any other options?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
