We just have to do it. We just freeze master at some point, do all of the release bugfixes, and when it is solid enough to pass a release vote we branch a release from it, and then only allow merges to master that have been tested in a merge branch, or something along those lines. Things will slip through, because our testing isn't full coverage, but we can begin to add to it.
I've said it before, but I think we're also a bit stingy regarding bugfix releases. Unless we cause a regression, there should be no need for bugfix releases to go through multiple RCs. We get caught on bugs that are already in the shipping version and make them blockers for the other bug fixes, or a pet patch needs to slip in (which also only matters because bugfix releases are so few and hard to release). On Wed, Apr 22, 2015 at 12:32 PM, Remi Bergsma <r...@remi.nl> wrote: > Hi, > > The '2 week cycle' was intended as something to work towards, like a > mission. The nice thing is that it immediately shows that we cannot achieve > such a thing if we keep our current method of (semi)manual testing, as you > said. Totally agree. > > That's why we need to improve on our automated functional testing. And that > will of course take time and effort. As I don't currently have a clear > overview of what is already available, I'll try to get that first and work > from there. I spoke to several people recently and most seem to do testing > on their own way (with sometimes cool automatic stuff going on!). It'd be > interesting to bring that together and share. > I think improving this is important, so I'll try to spend as much time on > this as possible. > > However, the tread started with comments on 4.5. Let's try to get it stable > and deliver 4.5.1. What is still to be done here? Can we share the load > somehow to fasten it? > > Regards, > Remi > > 2015-04-22 20:13 GMT+02:00 Paul Angus <paul.an...@shapeblue.com>: > >> I fully support the idea of a stable master with an automated CD process >> to protect against regressions. >> >> ....However, we obviously don't currently have fantastic integration >> testing otherwise we wouldn't have relied on 'people' to pick up the >> release blockers recently. A 2 week release cycle IMHO is way too frequent >> to expect 'people' to be carrying out integration testing. >> >> Maybe 1 month is a workable compromise until the integration testing is of >> a coverage and standard that can give real confidence. >> >> >> >> I'm also going to compile a list of people who vote "+1 - it works on my >> laptop" and devise a Guinness related punishment for any of them that show >> up at the CloudStack day in Dublin. >> >> Regards >> >> Paul Angus >> Cloud Architect >> S: +44 20 3603 0540 | M: +447711418784 | T: CloudyAngus >> paul.an...@shapeblue.com >> >> -----Original Message----- >> From: Remi Bergsma [mailto:r...@remi.nl] >> Sent: 22 April 2015 10:25 >> To: dev@cloudstack.apache.org >> Subject: Re: Next ACS release? >> >> I'd be happy to help here as well. Last week in Austin, we also discussed >> this topic a couple of times. I do agree shorter release cycles are better. >> >> In my opinion, the first thing to improve is the minor releases in both the >> 4.4 and 4.5 branches. If we speed those up to let's say once every 2-weeks >> we will be able to do the next minor release with less effort and users can >> choose to either wait to start using 4.5 or start now and upgrade when the >> next minor release is available. This would have helped in getting 4.5 out >> on time and quickly fixing issues after the initial release. Also, it will >> save time which we can invest in getting the next release out on time, etc. >> >> The common thing here is we need more automated testing, preferably >> functional testing in addition to unit testing. There might also be other >> steps that we do manually now that can be automated. I'll try to understand >> the current process first and then come up with a proposal to improve which >> we can discuss. >> >> Regards, >> Remi >> >> 2015-04-22 10:56 GMT+02:00 Erik Weber <terbol...@gmail.com>: >> >> > On Wed, Apr 22, 2015 at 10:46 AM, Daan Hoogland >> > <daan.hoogl...@gmail.com> >> > wrote: >> > >> > > On Wed, Apr 22, 2015 at 10:29 AM, Erik Weber <terbol...@gmail.com> >> > wrote: >> > > > On Wed, Apr 22, 2015 at 9:49 AM, Stephen Turner < >> > > stephen.tur...@citrix.com> >> > > > wrote: >> > > > >> > > >> I have to admit I'm a bit sceptical because when we did have a >> > > four-month >> > > >> release cycle, we never seemed to manage to meet it. Personally I >> > think >> > > >> six-monthly might be easier. >> > > >> >> > > >> Having said that, part of the problem was the long close-down >> > > >> period >> > > where >> > > >> we kept finding critical bugs, so more automated testing might >> > > >> help to shorten the cycle. >> > > >> >> > > >> >> > > > >> > > > - Shorter cycle means that it's less of a problem to miss a >> > > > deadline, meaning you can spend more time on QA. >> > > > - Longer cycle means that it could be critical to meet the >> > > > deadline, meaning you might end up doing less QA while stressing >> > > > to get your >> > > feature >> > > > in. >> > > > >> > > > My $.002 >> > > >> > > >> > Yes, Eric and the amount of qa needed is less as well. Bare with me >> > > for a little rant here. >> > > >> > > When less new features are introduced less new interdependencies are >> > > introduced. I could try and expand on this but people make phd on >> > > the subject so that would be presumptuous of me. >> > > >> > > un-educated guess is (m + n)! - m! >> > > where m is old features and n is new features >> > > >> > > example: >> > > 1 old feature + 1 new feature mean 1 check to see if they (still) >> > > work >> > > 1 old feature + 2 new features means 3 checks to see if they all >> > > work >> > > 2 + 2 = 6 of which only 1 is old and should be fine >> > > 2 + 3 = 10, see the n! progressing ;) >> > > >> > > >> > > this is an over simplification as the complexity of features comes >> > > in play as well. I make them out for being function points here. >> > > those are fables of course. >> > > >> > > thanks for baring that. >> > > >> > > >> > I'm all with you on this Daan. >> > Just for the record, my notion of QA was meant at the feature >> > developer, ie. that they could use more time on test coverage etc. >> > without having to worry that much about feature freeze dates. >> > >> > -- >> > Erik >> > >> Find out more about ShapeBlue and our range of CloudStack related services >> >> IaaS Cloud Design & Build< >> http://shapeblue.com/iaas-cloud-design-and-build//> >> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/> >> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> >> CloudStack Software Engineering< >> http://shapeblue.com/cloudstack-software-engineering/> >> CloudStack Infrastructure Support< >> http://shapeblue.com/cloudstack-infrastructure-support/> >> CloudStack Bootcamp Training Courses< >> http://shapeblue.com/cloudstack-training/> >> >> This email and any attachments to it may be confidential and are intended >> solely for the use of the individual to whom it is addressed. Any views or >> opinions expressed are solely those of the author and do not necessarily >> represent those of Shape Blue Ltd or related companies. If you are not the >> intended recipient of this email, you must neither take any action based >> upon its contents, nor copy or show it to anyone. Please contact the sender >> if you believe you have received this email in error. Shape Blue Ltd is a >> company incorporated in England & Wales. ShapeBlue Services India LLP is a >> company incorporated in India and is operated under license from Shape Blue >> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil >> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is >> a company registered by The Republic of South Africa and is traded under >> license from Shape Blue Ltd. ShapeBlue is a registered trademark. >>
