Adding infra mailing lists to the loop. Il 08/09/2014 11:34, Sandro Bonazzola ha scritto: > Il 20/08/2014 10:05, Sandro Bonazzola ha scritto: >> Hi, >> while we're near to release 3.5.0 I would like to start the discussion on >> the release process for 3.5.z and 3.6.0. >> The reference page for the release process has been last updated 2 years ago >> and is not reflecting the way we're managing the releases anymore. > > Following up to this email with proposal for missing bits. Comments are > welcome. > > > >> In order to start the discussion here's what we are currently doing: >> >> = oVirt maintenance releases = >> >> * After a new 3.y.0 release, maintenance releases are scheduled every month >> until a new 3.y+1.0 release is published. >> * After first 3.y.0 release candidate is published a new release management >> wiki page is created (see >> http://www.ovirt.org/OVirt_3.4.z_release-management) >> * For each 3.y.z release, the release management page is updated >> ** Defining the schedule for RC and GA date >> ** Pointing to the new release note page to be created (see >> http://www.ovirt.org/OVirt_3.4.4_Release_Notes) >> ** Pointing to a testing page to be created (see >> http://www.ovirt.org/Testing/oVirt_3.4.4_Testing) >> ** Pointing to a blocker bug to be created (see >> http://bugzilla.redhat.com/1118689) >> * Weekly status email are sent to users and devel lists >> * QA page is updated with pointers to the new release (see >> http://www.ovirt.org/OVirt_Quality_Assurance) >> * Release manager update the release notes pages starting from RC >> >> Usually RC date is scheduled one week before GA. >> RC releases are created by taking the latest nightly snapshot available >> after verifying that the build passes basic sanity test and repository >> closure. >> A new stabilization branch is created with the same git hash used for >> building the snapshot. >> Between RC and GA release criteria are tested on the RC and if the release >> met the criteria a GA release will be built just updating the versioning >> code to drop master, git hash and timestamp suffix from tarballs and rpms. >> A new RC will be created and GA postponed by one week if release criteria >> are not met. >> >> >> >> = oVirt Release Process = >> >> * After a new 3.y.0 release, a new 3.y+1.0 release is tentatively scheduled >> after 6 months. >> * A new release management page is created (see >> http://www.ovirt.org/OVirt_3.5_release-management) >> * A new tracker bug is created (see http://bugzilla.redhat.com/1073943) >> * A discussion is started on devel and users mailing list gathering idea for >> next release features >> * Teams prepare a list of accepted features collecting / creating bug >> tracker, devel owner, qa owner, feature page for each of them >> * Several presentations are scheduled by the teams presenting the accepted >> features >> * An alpha release is scheduled 4 months before GA >> * Feature freeze is scheduled 3 months before GA >> * A beta release is scheduled 2 months before GA, git tree is branched for >> stabilization >> * A release candidate is scheduled 1 month before GA >> * Weekly status email are sent starting 3 weeks before alpha release. >> * Test days are scheduled after every milestone release >> * Release manager update the release notes pages starting from Alpha >> >> All milestones releases are created by taking the latest nightly snapshot >> available after verifying that the build passes basic sanity test and >> repository closure. >> >> RC will be postponed until all known blockers are fixed >> >> Between RC and GA release criteria are tested on the RC and if the release >> met the criteria a GA release will be built just updating the versioning >> code to drop master, git hash and timestamp suffix from tarballs and rpms. >> A new RC will be created and GA postponed by one week if release criteria >> are not met. >> >> [[Category:Release management]] >> >> >> What I think we're missing / doing wrong: >> * A discussion about release criteria *after* accepted features list is ready > > 1) I think we should require that each feature must come with a set of test > cases covering the functionality provided by the new feature. > The test cases becomes part of the release criteria as > " * All test cases defined for each accepted feature must be verified and > pass the test for the build to be released" > > This means that the whole test suite must be verified on each release > candidate for deciding if it's ready to be released. > Test suite should cover whatever is in the documentation of the feature. > This means that the feature must contain enough documentation to be > considered "a contract" between the feature owner and the feature tester / > user. > > > 2) I think that the current criteria: "MUST: No blockers on the lower level > components - libvirt, lvm,device-mapper,qemu-kvm, Jboss, postgres, > iscsi-initiator" should be dropped. > We're not acting as QA for lower level components. Instead I think we should > change above statement with: > "MUST: all lower level components must be available on supported > distributions at required version" > > This means we're not hosting any package we don't develop inside oVirt > repository just because it's not yet released officially. > > I think we can allow to require third party repositories like JPackage, > Gluster, virt-preview, epel-testing, fedora-updates-testing. > We shouldn't allow to add to oVirt repositories packages which belongs to > third party repositories. > Example of those are libguestfs, python-cpopen, slf4j, kexec-tools and so on. > > > 3) MUST: No known regressions from previous releases. > this means that bugs with Regression keyword will become automatically > blockers. > > 4) MUST: Have Release Notes with feature specific information > I think this should be a requirement for entering beta phase: all features > page should contains enough information for allowing features maintainers > to add some information to the release notes page. > > 5) MUST: Have release announcement for front page of ovirt.org and for > mailing lists > Well, this is something usually we do after we released the packages, so it > can't be a MUST. > > 6) MUST: All sources must be available on ovirt.org > > I think all remaining existing requirements should be on a per subproject > base. > > 7) MUST each subproject must allow upgrade from a previous version either > providing documented manual or automated upgrade procedures. > > 8) MUST each subproject must pass it's own regression test matrix. > This means that each subproject maintainer should create one and ensure it's > tested. > This can be rephrased as "All features working on previous release must still > work in the new release if not dropped as deprecated" > While verifying the features, regression bugs can be found. We just can't > re-verify all fixed bugs, can we? > > 9) MUST each subproject should list packages expected to be in the release. > This just for ensuring we can verify the final repository for completeness. > > > Points 1 and 8 ensures also we have a test day page ready for the first test > day, just listing test cases to be covered. > >> * A clear schedule like http://fedoraproject.org/wiki/Releases/21/Schedule > > Will follow up on the schedule once intermediate steps are more defined. > > >> * A clear feature freeze policy like >> https://fedoraproject.org/wiki/Changes_Freeze_Policy > > Ignoring the Feature Freeze > Introducing significant changes after Feature Freeze can result in your patch > to be reverted. > > Allowed after Feature Freeze: > - Adding code meant for automated testing of the feature > - Adding bug fixes > - Adding localization > - Cleaning or optimizing the code without introducing new features > > Not allowed after Feature Freeze: > - Continue to add new enhancements (note that supporting a new distribution > is an enhancement) > - Try to cover new enhancements under bug fixes > - Make changes that require other subprojects to make changes as well (API, > ABI and configuration files included) if not absolutely required for > fixing a bug > - Make changes that require packages not yet available on supported > distributions. > > Exceptions > About this, I think we shouldn't allow exceptions. > But if needed, I think that a public vote on the exception should be required. > Exceptions should be asked at least one week before feature freeze and vote > on them should be closed on feature freeze. > > This means that you're not allowed to open new features but you can complete > those that can't be completed for feature freeze if the vote allow it. > > > Incomplete features > I think we should require for each feature to define a contingency plan, > allowing either to disable the incomplete feature or ensuring that the > feature code is isolated in a directory or package that can be easily dropped. > Those that can't be done that way may live in their own branch until they're > finished and merged in the main tree just when completed. > > >> * A clear policy about changes we must allow and not allow between RC and GA > > I think that only patches fixing blockers bug should be allowed between RC > and GA. > This means that ystream branches must fork on RC release git hash and release > maintainers must ensure nothing will be merged there if not related to a > blocker bug. > >> * Alpha release should come after feature freeze > > Nothing more to say about it > >> * A string freeze schedule, allowing translators to have enough time to >> translate new strings > > I think that we should freeze string between beta and 1 week before RC. > Considering we have 1 month between beta and RC this means that string freeze > should go no later than 2 weeks before RC and translators have 2 weeks > for finalizing last translation bits and verify them. > So, I think we should have a beta release requirement: > - Supported localizations must be at least at 70% of completeness at beta > stage for being included in the release. > Without such requirement translators won't make it on time for GA. > > >> * Maintainers should pay more attention to release notes and test day pages > > This is enforced by rules on features acceptance requirements. > >> >> > >
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel