Again, I am not disputing that more features will make it in given Dave's argument. In fact, I'm not really that keen on using the fixed cost of release mgmt. for the reason of the move as well. Heck, I'm not even sure a 6 month cycle would fix any of the issues that have already been outlined but it'd be nice to try. However, I do know a couple of things:
1. We all want CS releases to be of certain quality. It needs to work and upgrades need to work. 2. ACS still relies too heavily on manual testing and most of it unfortunately comes from 1 company. We cannot be dependent on another company's schedule to ensure ACS has gone through enough proper testing to achieve (1). 3. Most importantly, the extra 2 months will give QA more time, give features more time to settle in, and more automated tests to be written for existing features as well. I agree with Dave that more feature will come in. So what? If they are not of good enough quality, we don't release it as part of the ACS release. However, that extra 2 month will give earlier written features more time to be potentially tested and used by people so that we fix the most egregious bugs before we ship it. It will also better accommodate people's schedule so they can fix bugs for their features. Personally, so far, I feel 4 months seems a bit rushed. Will > -----Original Message----- > From: David Nalley [mailto:da...@gnsa.us] > Sent: Monday, April 22, 2013 7:52 PM > To: dev@cloudstack.apache.org > Cc: cloudstack-...@incubator.apache.org > Subject: Re: [DISCUSS] ACS Release 4 month v/s 6 month > > On Mon, Apr 22, 2013 at 5:19 PM, Animesh Chaturvedi > <animesh.chaturv...@citrix.com> wrote: > > Folks > > > > We started discussing 4 month v/s 6 month release cycle in a another > thread [1]. Since the subject of that thread was different, community may > not have participated in this important discussion fully. I am are bringing > this discussion to its own thread. Here is the summary so far please refer to > [1] for more details. > > > > Summary of discussion: > > - Animesh pointed out the technical debt that we have accumulated so > > far needs extra time to resolve > > - David, Chip favor shorter release cycle of 4 month and keeping > > master always stable and in good quality and enhancing automation as a > > solution to reduce QA manual effort. A focused defect fixing activity > > may be needed to reduce technical debt > > - Will brought up several points in the discussion: He called out heavy > dependence on manual QA for a release and pointed out that manual QA > may not be always available to match up ACS release schedule. Release > overhead for 4 month release is still high and suggest that moving to 6 > month will save on release overhead and that time can be used for > strengthening automation. > > - Joe agrees partly in release overhead being significant for major > > release > > > > If I missed out any important point please feel free to bring into the > thread. > > > > There were some other discussion in [1] on release planning conference > and chip's clarification on time based v/s feature based releases but we will > not discuss those in this thread. Community has agreed to time-based > release already. > > > > [1] http://markmail.org/thread/6suq2fhltdvgvcxd > > > > > Hi Animesh: > > I thought I would add a few comments. To the folks reading - I apologize for > the length. If you haven't started yet, you may want to get some coffee/tea. > You've been warned. :) > > I think there are two concerns people think about when talking about > changing the length of the release cycle. > > The first concern is workload. Generating a release has a certain fixed > amount of overhead. Writing release notes, running votes, and a key part of > the discussion currently is the QA cycle. Currently a large portion of our QA > cycle is manual testing, which means we need lots of fingers on keyboards > to get it done. > > The other concern is quality. We always want to try and put our best foot > forward, and minimize bugs, yet the very act of developing software, and > especially developing new features, introduces bugs. > > Before I start telling you my reasoning, I want to lay out something that > only hit me tonight. We've talked about how extending the release cycle > only extends the length of development. We've talked about keeping the > length of the development period (e.g. pre-code freeze) the same, and > essentially only extending the amount of time for QA. > That works for the first release after a move from 4 to 6 months. It falls > apart there after though. The problem is that at code freeze we branch. > Master immediately becomes open for future feature development, and > you've just extended it by an additional 2 months, because your cycle is > that much longer. > > I personally don't think that either concern is addressed well by lengthening > the cycle. Let me explain why > > On the quality front - the immediate threat to quality is instability inserted > by additional development/new features/improvements. By increasing the > length of time for that disruption to occur, we get more of it, as people will > continue adding things that might have been deferred to a later release. We > have code merged that, realistically, we won't know if it works properly > until QA has gone in and verified it, perhaps months later. The more paths > changed in a given release, the greater the propensity for failure. > > On the overhead front - the overhead of writing release notes and > generating releases goes down. In addition the 'basic' testing that QA does > now is additional overhead in each cycle as well. What isn't overhead > though is testing for new features. Because effectively longer cycles mean > greater amount of codebase change, more features, etc. The actual testing > in a given cycle must also increase. > > If we were really focused on absolute best quality we could deliver with our > current tooling, I believe the easiest would be to make our release cycle the > length of a single feature development period + the QA cycle (e.g. > shortening the release cycle.) > > Is there a right or wrong answer? There shouldn't be. Our software should > be deliverable at any point in the development cycle we choose. > 4 months or 6 months would not be a quality, or QA workload issue. We > would have automated test suites written for 80+% of our testing and we > would run comprehensive tests against propose merges to guarantee that > an incoming features both works as intended, and doesn't harm the rest of > the platform in the process. We'd make release cycle decisions based upon > the same kind of preferences that make us choose a monkey mascot over a > polar bear mascot. > > That sadly isn't where we are today, but I do think it illuminates a clear > path > of where we need to go. (more automated testing) > > So where do I fall on 4 months versus 6 months? > > I think we should stay the 4 month course. Heres why: > * We set the 4-month release cycle expectation already. In theory we've > been through 2 releases, but really only 4.1 - 4.0 was dominated by getting > into the incubator, getting releases okayed legally, etc. > It took us 7 months to release. I am loathe to change something we only > recently set, and have only been through once. I am also worried about the > signal that such rapid contrary decision changes would send to our users. > > * I am worried about the impact to quality of our releases. Greater > introduction of features means an increased threat to stability. It may not > impact us in the 4.2 cycle, but without massive automated QA it will impact > us going forward in the 4.3 and later time frames. > > > If you've made it thus far, I hope your tea/coffee was enjoyable. > Thanks for reading. :) > > --David
