On Sun, Dec 29, 2013 at 9:00 PM, Abhinandan Prateek <abhinandan.prat...@citrix.com> wrote: > Every time a RC is made it is flagged off due to some bug or the other at > the last moment. > > I was wondering if before a RC is cut we take a pre-vote making sure that > everyone has got sufficient time to test the branch for the features they > are looking for ? > > Another suggestion is that since 4.3 is now close, why put our energies in > 4.2.1 and instead focus on 4.3. > > -abhi >
So I have several comments here. Basically it boils down to this: 1. Most people won't test until there is a real RC. It's a catch-22 - manual testing is time intensive, people don't want to waste their time if things are constantly changing; so they will only expend the effort on testing when something is a release candidate. If we want more confidence that a branch is in a decent state, that requires more automation and testing. (see more on this below) 2. IMO we can't abandon 4.2.1 - we set the expectation that it would be delivered and need to follow on with that expectation. 3. Patch releases (or feature releases for that matter) shouldn't result in core functionality being worse than the previous release. So some quick comments on testing - our testing should have caught CLOUDSTACK-5393 (actually it did, it just caught it for 4.3 instead; and manual testing by Nux and Andrei found that it applied to 4.2.1.) This suggests that we either didn't run the test suite against RC3 or that we did and didn't catch/report the bug. Reviewing jenkins.buildacloud.org doesn't show any suggestion that the 4.2 branch is having anything past unit tests run against it, so that leaves us really with manual reviewing, which as you noted, ends up with last minute bugs. We also don't have a good way to compare a 4.2.1 release candidate against what was released for 4.2. (I know, plenty of complaints, not a lot of solutions.) --David
