Sophie Blee-Goldman created KAFKA-9323:
------------------------------------------
Summary: Refactor Streams' upgrade system tests
Key: KAFKA-9323
URL: https://issues.apache.org/jira/browse/KAFKA-9323
Project: Kafka
Issue Type: Improvement
Components: streams
Reporter: Sophie Blee-Goldman
With the introduction of version probing in 2.0 and cooperative rebalancing in
2.4, the specific upgrade path depends heavily on the to & from version. This
can be a complex operation, and we should make sure to test a realistic upgrade
scenario across all possible combinations. The current system tests have gaps
however, which have allowed bugs in the upgrade path to slip by unnoticed for
several versions.
Our current system tests include a metadata upgrade test, a version probing
test, and a cooperative upgrade test. This has a few drawbacks:
a) only the version probing test tests "forwards compatibility" (upgrade from
latest to future version)
b) nothing tests version probing "backwards compatibility" (upgrade from older
version to latest), except:
c) the cooperative rebalancing test actually happens to involve a version
probing step, and so coincidentally DOES test VP (but only starting with 2.4)
d) each test itself tries to test the upgrade across different versions,
meaning there may be overlap and/or unnecessary tests
e) as new versions are released, it is unclear to those not directly involved
in these tests and/or projects whether and what needs to be updated (eg should
this new version be added to the cooperative test? should the old version be
aded to the metadata test?)
We should definitely fill in the testing gap here, but how to do so is of
course up for discussion.
I would propose to refactor the upgrade tests, and rather than maintain
different lists of versions to pass as input to each different test, we should
have a single matrix that we update with each new version that specifies which
upgrade path that version combination actually requires. We can then loop
through each version combination and test only the actual upgrade path that
users would actually need to follow. This way we can be sure we are not missing
anything, as each and every possible upgrade would be tested.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
