Yeah, it seems we forgot to update MetadataVersion class. All 4.1
versions are listed after this comment in trunk:
//
// NOTE: MetadataVersions after this point are unstable and may be changed.
// If users attempt to use an unstable MetadataVersion, they will
get an error.
// Please move this comment when updating the LATEST_PRODUCTION constant.
//
On Mon, Jul 14, 2025 at 10:58 AM Paolo Patierno
<paolo.patie...@gmail.com> wrote:
>
> Hi,
> I think I found something which could be a blocker for the release.
> I was adding the support for Apache Kafka 4.1.0 to Strimzi and running our
> tests but some of them are failing with the following error.
>
> java.lang.IllegalArgumentException: Version 4.1 is not a valid version. The
> minimum version is 3.3-IV3 and the maximum version is 4.2-IV1
> at
> org.apache.kafka.server.common.MetadataVersion.lambda$fromVersionString$0(MetadataVersion.java:356)
> at java.base/java.util.Optional.orElseThrow(Optional.java:403)
> at
> org.apache.kafka.server.common.MetadataVersion.fromVersionString(MetadataVersion.java:354)
>
> In these tests we use the MetadataVersion class to verify the validity of
> metadata but it fails when passing something like "4.1" (while they work
> just fine with all previous releases, i.e. "4.0", "3.9" and so on).
>
> The exception, thrown by the fromVersionString method, seems to be related
> to "4.1" missing as key entry within the IBP_VERSIONS map and such entry
> seems to be missing because, if you go through the static initialization of
> the IBP_VERSIONS map, the "4.1" entry is not added due to the
> metadataVersion.isProduction() which results false for "4.1" because the
> LATEST_PRODUCTION constant is defined as IBP_4_0_IV3 while I would assume
> should be something like IBP_4_1_IV1 instead.
>
> I think this issue wasn't caught during Kafka upstream testing because the
> MetadataVersionTest.testFromVersionString is missing the test for "4.1" (as
> we have for all other versions). So it's missing something like
>
> assertEquals(IBP_4_1_IV1, MetadataVersion.fromVersionString("4.1"));
> assertEquals(IBP_4_1_IV0,
> MetadataVersion.fromVersionString("4.1-IV0"));
> assertEquals(IBP_4_1_IV1,
> MetadataVersion.fromVersionString("4.1-IV1"));
>
> Let's add that this issue seems to break the storage tool where you can
> pass the --release-version, but if you run it by passing "4.1" you get
> exactly the same exception.
>
> > bin/kafka-storage.sh format --standalone -t kEzc4vk3TIKhCQKsh40klQ -c
> config/server.properties --release-version 4.1
> Exception in thread "main" java.lang.IllegalArgumentException: Version 4.1
> is not a valid version. The minimum version is 3.3-IV3 and the maximum
> version is 4.2-IV1
>
> Finally, I would add that the error message itself is misleading as it
> mentions 4.2-IV1 as maximum version which would not exist at all if you are
> dealing with a 4.1 release. This misleading message should be related to
> MetadataVersion.latestTesting() method which is getting last value from
> VERSIONS which is filled with all MetadataVersion enums including the 4.2
> placeholders (which AFAIU is actually a new practice). All of that has also
> an impact on the ClusterTest annotation class as well as FeatureCommandTest
> messages to be assert. So that 4.2 seems to be there on purpose but not
> sure why.
>
> I would be happy to contribute about the first issue which raises the
> exception. Not sure about the misleading error message because of the usage
> of the 4.2 placeholders within the VERSIONS collection.
>
> Thanks
> Paolo Patierno
>
> ---------- Forwarded message ---------
> From: Mickael Maison <mickael.mai...@gmail.com>
> Date: Fri, Jul 11, 2025 at 11:53 PM
> Subject: [VOTE] 4.1.0 RC0
> To: dev <d...@kafka.apache.org>, Users <users@kafka.apache.org>,
> kafka-clients <kafka-clie...@googlegroups.com>
>
>
> Hello Kafka users, developers and client-developers,
>
> This is the first candidate for release of Apache Kafka 4.1.0.
>
> The release brings many new features and improvements:
> - Mechanism for plugin to register metrics
> - Allow running multiple versions of connector plugins
> - Queue are now in preview access
> - New Streams rebalance protocol in early access
> - Support for OAuth jwt-bearer grant type
> - Improved metadata replication
>
> Release notes for the 4.1.0 release:
> https://dist.apache.org/repos/dist/dev/kafka/4.1.0-rc0/RELEASE_NOTES.html
>
> *** Please download, test and vote by Monday July 21
>
> Kafka's KEYS file containing PGP keys we use to sign the release:
> https://kafka.apache.org/KEYS
>
> * Release artifacts to be voted upon (source and binary):
> https://dist.apache.org/repos/dist/dev/kafka/4.1.0-rc0/
>
> * Docker release artifacts to be voted upon:
> apache/kafka:4.1.0-rc0
> apache/kafka-native:4.1.0-rc0
>
> * Maven artifacts to be voted upon:
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>
> * Javadoc:
> https://dist.apache.org/repos/dist/dev/kafka/4.1.0-rc0/javadoc/
>
> * Tag to be voted upon (off 4.1 branch) is the 4.1.0 tag:
> https://github.com/apache/kafka/releases/tag/4.1.0-rc0
>
> * Documentation:
> https://kafka.apache.org/41/documentation.html
> PR (https://github.com/apache/kafka-site/pull/702) is not merged yet
>
> * Protocol:
> https://kafka.apache.org/41/protocol.html
> PR (https://github.com/apache/kafka-site/pull/702) is not merged yet
>
> * Successful CI builds for the 4.1 branch:
> Unit/integration tests:
> https://github.com/apache/kafka/actions/runs/16198079614
> * System tests results:
> - core:
> https://drive.google.com/file/d/1JLpFBFbPV1N6cdh9EA9TGw9S5SZwkIpv/view?usp=drive_link
> - non-core:
> https://drive.google.com/file/d/15jCnTS-dssSHauW7uWxroiupkWaeUK7A/view?usp=drive_link
> I got 2 failures in the system tests:
> -
> 'tests/kafkatest/tests/client/consumer_test.py::OffsetValidationTest.test_broker_rolling_bounce@
> {"metadata_quorum":"ISOLATED_KRAFT","group_protocol":"classic"}'
> -
> 'tests/kafkatest/tests/core/network_degrade_test.py::NetworkDegradeTest.test_rate@
> {"task_name":"rate-1000-latency-50","device_name":"eth0","latency_ms":50,"rate_limit_kbit":1000000,"metadata_quorum":"COMBINED_KRAFT"}'
>
> * Successful Docker Image Github Actions Pipeline for 4.1 branch:
> Docker Build Test Pipeline (JVM):
> https://github.com/apache/kafka/actions/runs/16172687407/
> Docker Build Test Pipeline (Native):
> https://github.com/apache/kafka/actions/runs/16220242161
>
> Note:
> As described in the release thread, the native image configuration
> files had to be updated. I'm not able to run the full system test
> suite with the GraalVM agent to generate the new configuration files.
> Luke has kindly run a few commands to produce a fix to the config
> files. I validated that the kafka-native:4.1.0-rc0 image passes the
> quickstart (clients, connect and streams), but as that image is only
> for development, I've not tested it any further.
> Also all the other release artifacts were generated before the fix to
> the native image configuration files was merged. As these files are
> not part of any of the other release artifacts and as this is rc0, I
> deemed that acceptable.
>
> Thanks,
> Mickael
>
> >
>
> --
> Paolo Patierno
>
> *Senior Principal Software Engineer @ Red Hat**CNCF Ambassador*
> *Microsoft MVP on **Azure*
>
> Twitter : @ppatierno <http://twitter.com/ppatierno>
> Linkedin : paolopatierno <http://it.linkedin.com/in/paolopatierno>
> GitHub : ppatierno <https://github.com/ppatierno>
> Paolo Patierno
>
> *Senior Principal Software Engineer @ Red Hat**CNCF Ambassador*
> *Microsoft MVP on **Azure*
>
> Twitter : @ppatierno <http://twitter.com/ppatierno>
> Linkedin : paolopatierno <http://it.linkedin.com/in/paolopatierno>
> GitHub : ppatierno <https://github.com/ppatierno>
