Hi,
I think I found something which could be a blocker for the release.
I was adding the support for Apache Kafka 4.1.0 to Strimzi and running our
tests but some of them are failing with the following error.
java.lang.IllegalArgumentException: Version 4.1 is not a valid version. The
minimum version is 3.3-IV3 and the maximum version is 4.2-IV1
at
org.apache.kafka.server.common.MetadataVersion.lambda$fromVersionString$0(MetadataVersion.java:356)
at java.base/java.util.Optional.orElseThrow(Optional.java:403)
at
org.apache.kafka.server.common.MetadataVersion.fromVersionString(MetadataVersion.java:354)
In these tests we use the MetadataVersion class to verify the validity of
metadata but it fails when passing something like "4.1" (while they work
just fine with all previous releases, i.e. "4.0", "3.9" and so on).
The exception, thrown by the fromVersionString method, seems to be related
to "4.1" missing as key entry within the IBP_VERSIONS map and such entry
seems to be missing because, if you go through the static initialization of
the IBP_VERSIONS map, the "4.1" entry is not added due to the
metadataVersion.isProduction() which results false for "4.1" because the
LATEST_PRODUCTION constant is defined as IBP_4_0_IV3 while I would assume
should be something like IBP_4_1_IV1 instead.
I think this issue wasn't caught during Kafka upstream testing because the
MetadataVersionTest.testFromVersionString is missing the test for "4.1" (as
we have for all other versions). So it's missing something like
assertEquals(IBP_4_1_IV1, MetadataVersion.fromVersionString("4.1"));
assertEquals(IBP_4_1_IV0,
MetadataVersion.fromVersionString("4.1-IV0"));
assertEquals(IBP_4_1_IV1,
MetadataVersion.fromVersionString("4.1-IV1"));
Let's add that this issue seems to break the storage tool where you can
pass the --release-version, but if you run it by passing "4.1" you get
exactly the same exception.
> bin/kafka-storage.sh format --standalone -t kEzc4vk3TIKhCQKsh40klQ -c
config/server.properties --release-version 4.1
Exception in thread "main" java.lang.IllegalArgumentException: Version 4.1
is not a valid version. The minimum version is 3.3-IV3 and the maximum
version is 4.2-IV1
Finally, I would add that the error message itself is misleading as it
mentions 4.2-IV1 as maximum version which would not exist at all if you are
dealing with a 4.1 release. This misleading message should be related to
MetadataVersion.latestTesting() method which is getting last value from
VERSIONS which is filled with all MetadataVersion enums including the 4.2
placeholders (which AFAIU is actually a new practice). All of that has also
an impact on the ClusterTest annotation class as well as FeatureCommandTest
messages to be assert. So that 4.2 seems to be there on purpose but not
sure why.
I would be happy to contribute about the first issue which raises the
exception. Not sure about the misleading error message because of the usage
of the 4.2 placeholders within the VERSIONS collection.
Thanks
Paolo Patierno
---------- Forwarded message ---------
From: Mickael Maison <mickael.mai...@gmail.com>
Date: Fri, Jul 11, 2025 at 11:53 PM
Subject: [VOTE] 4.1.0 RC0
To: dev <d...@kafka.apache.org>, Users <users@kafka.apache.org>,
kafka-clients <kafka-clie...@googlegroups.com>
Hello Kafka users, developers and client-developers,
This is the first candidate for release of Apache Kafka 4.1.0.
The release brings many new features and improvements:
- Mechanism for plugin to register metrics
- Allow running multiple versions of connector plugins
- Queue are now in preview access
- New Streams rebalance protocol in early access
- Support for OAuth jwt-bearer grant type
- Improved metadata replication
Release notes for the 4.1.0 release:
https://dist.apache.org/repos/dist/dev/kafka/4.1.0-rc0/RELEASE_NOTES.html
*** Please download, test and vote by Monday July 21
Kafka's KEYS file containing PGP keys we use to sign the release:
https://kafka.apache.org/KEYS
* Release artifacts to be voted upon (source and binary):
https://dist.apache.org/repos/dist/dev/kafka/4.1.0-rc0/
* Docker release artifacts to be voted upon:
apache/kafka:4.1.0-rc0
apache/kafka-native:4.1.0-rc0
* Maven artifacts to be voted upon:
https://repository.apache.org/content/groups/staging/org/apache/kafka/
* Javadoc:
https://dist.apache.org/repos/dist/dev/kafka/4.1.0-rc0/javadoc/
* Tag to be voted upon (off 4.1 branch) is the 4.1.0 tag:
https://github.com/apache/kafka/releases/tag/4.1.0-rc0
* Documentation:
https://kafka.apache.org/41/documentation.html
PR (https://github.com/apache/kafka-site/pull/702) is not merged yet
* Protocol:
https://kafka.apache.org/41/protocol.html
PR (https://github.com/apache/kafka-site/pull/702) is not merged yet
* Successful CI builds for the 4.1 branch:
Unit/integration tests:
https://github.com/apache/kafka/actions/runs/16198079614
* System tests results:
- core:
https://drive.google.com/file/d/1JLpFBFbPV1N6cdh9EA9TGw9S5SZwkIpv/view?usp=drive_link
- non-core:
https://drive.google.com/file/d/15jCnTS-dssSHauW7uWxroiupkWaeUK7A/view?usp=drive_link
I got 2 failures in the system tests:
-
'tests/kafkatest/tests/client/consumer_test.py::OffsetValidationTest.test_broker_rolling_bounce@
{"metadata_quorum":"ISOLATED_KRAFT","group_protocol":"classic"}'
-
'tests/kafkatest/tests/core/network_degrade_test.py::NetworkDegradeTest.test_rate@
{"task_name":"rate-1000-latency-50","device_name":"eth0","latency_ms":50,"rate_limit_kbit":1000000,"metadata_quorum":"COMBINED_KRAFT"}'
* Successful Docker Image Github Actions Pipeline for 4.1 branch:
Docker Build Test Pipeline (JVM):
https://github.com/apache/kafka/actions/runs/16172687407/
Docker Build Test Pipeline (Native):
https://github.com/apache/kafka/actions/runs/16220242161
Note:
As described in the release thread, the native image configuration
files had to be updated. I'm not able to run the full system test
suite with the GraalVM agent to generate the new configuration files.
Luke has kindly run a few commands to produce a fix to the config
files. I validated that the kafka-native:4.1.0-rc0 image passes the
quickstart (clients, connect and streams), but as that image is only
for development, I've not tested it any further.
Also all the other release artifacts were generated before the fix to
the native image configuration files was merged. As these files are
not part of any of the other release artifacts and as this is rc0, I
deemed that acceptable.
Thanks,
Mickael
>
--
Paolo Patierno
*Senior Principal Software Engineer @ Red Hat**CNCF Ambassador*
*Microsoft MVP on **Azure*
Twitter : @ppatierno <http://twitter.com/ppatierno>
Linkedin : paolopatierno <http://it.linkedin.com/in/paolopatierno>
GitHub : ppatierno <https://github.com/ppatierno>
Paolo Patierno
*Senior Principal Software Engineer @ Red Hat**CNCF Ambassador*
*Microsoft MVP on **Azure*
Twitter : @ppatierno <http://twitter.com/ppatierno>
Linkedin : paolopatierno <http://it.linkedin.com/in/paolopatierno>
GitHub : ppatierno <https://github.com/ppatierno>
