[ https://issues.apache.org/jira/browse/KAFKA-9323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nikolay Izhikov reassigned KAFKA-9323: -------------------------------------- Assignee: Nikolay Izhikov > Refactor Streams' upgrade system tests > --------------------------------------- > > Key: KAFKA-9323 > URL: https://issues.apache.org/jira/browse/KAFKA-9323 > Project: Kafka > Issue Type: Improvement > Components: streams > Reporter: Sophie Blee-Goldman > Assignee: Nikolay Izhikov > Priority: Major > > With the introduction of version probing in 2.0 and cooperative rebalancing > in 2.4, the specific upgrade path depends heavily on the to & from version. > This can be a complex operation, and we should make sure to test a realistic > upgrade scenario across all possible combinations. The current system tests > have gaps however, which have allowed bugs in the upgrade path to slip by > unnoticed for several versions. > Our current system tests include a metadata upgrade test, a version probing > test, and a cooperative upgrade test. This has a few drawbacks: > a) only the version probing test tests "forwards compatibility" (upgrade from > latest to future version) > b) nothing tests version probing "backwards compatibility" (upgrade from > older version to latest), except: > c) the cooperative rebalancing test actually happens to involve a version > probing step, and so coincidentally DOES test VP (but only starting with 2.4) > d) each test itself tries to test the upgrade across different versions, > meaning there may be overlap and/or unnecessary tests > e) as new versions are released, it is unclear to those not directly involved > in these tests and/or projects whether and what needs to be updated (eg > should this new version be added to the cooperative test? should the old > version be aded to the metadata test?) > We should definitely fill in the testing gap here, but how to do so is of > course up for discussion. > I would propose to refactor the upgrade tests, and rather than maintain > different lists of versions to pass as input to each different test, we > should have a single matrix that we update with each new version that > specifies which upgrade path that version combination actually requires. We > can then loop through each version combination and test only the actual > upgrade path that users would actually need to follow. This way we can be > sure we are not missing anything, as each and every possible upgrade would be > tested. -- This message was sent by Atlassian Jira (v8.3.4#803005)