Our current upgrade criterion says: "For each one of the release-blocking package sets, it must be possible to successfully complete an upgrade from a fully updated installation of the previous stable Fedora release with that package set installed." https://fedoraproject.org/wiki/Fedora_24_Beta_Release_Criteria#Upgrade_requirements
Currently we have no criterion that would cover upgrading across 2 releases, e.g. from F21 to F23 (directly, not one by one). But in the real world this very often happens. It's even one of the reasons we support our releases until N+2 release is available + 1 month (i.e. F21 is supported until F23 is out + 1 month). The often cited reason is for people to be upgrading just once per year (and have one month to do that). And of course many (probably most) of them don't upgrade one by one, but skip a release. I feel that for something as important as system upgrade, we should provide a better level of quality and assurance for upgrading across 2 releases. Currently we have no criterion and testing it is just an afterthought, not even tracked anywhere. I'd like to amend the existing criterion to include N-2 release as well, i.e.: "For each one of the release-blocking package sets, it must be possible to successfully complete an upgrade from a fully updated installation of any of the two previous stable Fedora releases with that package set installed." (language corrections very welcome) We can discuss whether N+2 upgrading should be a separate Final criterion, not joined with the Beta one. I don't feel strongly either way. I'd also set up a new test case in our installation matrix in the upgrade section: https://fedoraproject.org/wiki/Template:Installation_test_matrix#Upgrade Something like QA:Testcase_upgrade_dnf_skip_release. The question is whether to have just a single test case and let people choose which package set they test, or whether to pick some particular package set. We probably don't want to test all combinations, at least not manually. Just a single "please test something" test case would be satisfactory here, I think. Something will get tested, and we will block on important bugs we discover, that's the important change. If we decide to not go this route for some reason, I think we should adjust our tools (system-upgrade) and documentation (wiki, fedora docs) and provide very clear and visible warning that the only officially supported means of upgrading is to go up releases one by one. And that skipping releases might be dangerous (considerably more than doing it the recommended way). Because I feel we would be doing our users a disservice if we neither tested skipping releases nor warned them against doing that. Thoughts? Kamil -- test mailing list test@lists.fedoraproject.org To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org
