The more I think about this, though, we can't really reasonably create an RC at Feature Freeze time.
It's not generally until the week or two before Feature Freeze that features start coming into the master branch. We would need at least a couple weeks after Feature Freeze to perform our own integration tests to be confident that our new feature(s) play well with other recently introduced new features. On Thu, Mar 13, 2014 at 5:53 PM, Mike Tutkowski < mike.tutkow...@solidfire.com> wrote: > Exactly...it is not practical for 4.4 to spin up a RC so soon. > > Perhaps with more clearly defined criteria for 4.5, this may be possible. > > > On Thu, Mar 13, 2014 at 5:48 PM, Sudha Ponnaganti < > sudha.ponnaga...@citrix.com> wrote: > >> Feature freeze and Merge criteria is not defined to change the milestone >> suddenly. We can create build and call it RC but that would take another >> 2+ months to harden to build next RC. There are only 50 defects logged for >> 4.4 so far. We are away from another 600+ defects to get acceptable RC >> candidate ( based on 4.3 comparison) >> >> Merge might have some sanity tests submitted but I don't think any >> confidence tests are done on the features. There hasn't been big discussion >> around integration tests on 4.4 threads except find bugs and some basic >> integration or sanity tests. >> >> -----Original Message----- >> From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] >> Sent: Thursday, March 13, 2014 4:34 PM >> To: dev >> Subject: Re: Release cadence >> >> I agree that we can't move to our end goal in on go. But I disagree that >> we should go on with business as usual right now. baby steps but never stop >> taking steps. >> >> On Fri, Mar 14, 2014 at 12:20 AM, Mike Tutkowski < >> mike.tutkow...@solidfire.com> wrote: >> > My reasoning here is that otherwise we will just be futilely creating >> > RCs for 4.4 since this expectation was not clearly defined ahead of the >> release. >> > >> > If we set expectations appropriately for 4.5, then we should expect we >> > can begin RC building right after Feature Freeze for that release. >> > >> > >> > On Thu, Mar 13, 2014 at 5:18 PM, Mike Tutkowski < >> > mike.tutkow...@solidfire.com> wrote: >> > >> >> I think we should set that as a goal for 4.5. We should treat 4.4 as >> >> business as usual at this point and give "fair warning" for the next >> >> release. >> >> >> >> We should formally define what "tested" means for 4.5 and then take >> >> the appropriate course of action from a RC point of view. >> >> >> >> >> >> On Thu, Mar 13, 2014 at 5:00 PM, Daan Hoogland < >> daan.hoogl...@gmail.com>wrote: >> >> >> >>> That's how i like to see it and why I asked. Is there a reason >> >>> people merge and then commit their features instead of rebasing and >> >>> running a standard set of integration tests to validate before >> >>> merging. I am not better then average on this myself but I think >> >>> here is where we have room to improve if anywhere. >> >>> >> >>> So do we create 4.4 RC1 next Monday? >> >>> >> >>> >> >>> >> >>> On Thu, Mar 13, 2014 at 11:19 PM, Mike Tutkowski >> >>> <mike.tutkow...@solidfire.com> wrote: >> >>> > I think many people (myself included) are used to performing >> >>> > rigorous, >> >>> but >> >>> > focused feature-specific testing before feature freeze, but are >> >>> > under >> >>> the >> >>> > impression that once feature freeze arrives that we are in >> >>> > integration-testing mode (where our feature is tested in >> >>> > combination >> >>> with >> >>> > other features...not so isolated anymore). At this point, we tend >> >>> > to >> >>> find >> >>> > bugs that were not hit pre feature freeze because that mode of >> >>> > testing >> >>> was >> >>> > more confined. >> >>> > >> >>> > Perhaps we simply need to decide on how tested a feature should be >> >>> > for feature freeze. Does it need to be fully tested from an >> >>> > integration with other features standpoint or not? If yes, then we >> are basically "done" >> >>> with >> >>> > the release at feature freeze time and can begin the >> >>> > release-candidate process. >> >>> > >> >>> > >> >>> > On Thu, Mar 13, 2014 at 4:11 PM, David Nalley <da...@gnsa.us> >> wrote: >> >>> > >> >>> >> Thats a very good point - we are effectively saying we know the >> >>> >> features we merged in have potentially months worth of bugs. >> >>> >> Though really, our hiccups don't seem to generally be in new >> >>> >> features, it's old features. >> >>> >> >> >>> >> On Thu, Mar 13, 2014 at 3:44 PM, Marcus <shadow...@gmail.com> >> wrote: >> >>> >> > Its a good point. I had thought about. Essentially we are >> >>> >> > saying >> >>> that we >> >>> >> > know the features we just merged need another few months of work. >> >>> >> > On Mar 13, 2014 1:01 PM, "Daan Hoogland" >> >>> >> > <daan.hoogl...@gmail.com> >> >>> >> wrote: >> >>> >> > >> >>> >> >> Just a thought, >> >>> >> >> >> >>> >> >> Why isn't the freshly cut branch the first RC from the get go? >> >>> >> >> It is quite sure not to pass but it should cantain what we ant >> >>> >> >> to ship feature wise. >> >>> >> >> >> >>> >> >> On Thu, Mar 13, 2014 at 6:35 PM, Mike Tutkowski >> >>> >> >> <mike.tutkow...@solidfire.com> wrote: >> >>> >> >> > OK, so it sounds like a 3-month dev cycle for a four-month >> >>> >> >> > release >> >>> >> was on >> >>> >> >> > purpose. >> >>> >> >> > >> >>> >> >> > Just curious...thanks :) >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > On Thu, Mar 13, 2014 at 11:31 AM, David Nalley >> >>> >> >> > <da...@gnsa.us> >> >>> wrote: >> >>> >> >> > >> >>> >> >> >> This was (IIRC) part of the explicit decision in how to do >> >>> things. >> >>> >> The >> >>> >> >> >> thought being that if you are restricting what people can >> >>> >> >> >> do >> >>> with a >> >>> >> >> >> release branch, people still need to be able to have a >> >>> >> >> >> place to >> >>> base >> >>> >> >> >> their ongoing work; and master should be that place. Some >> >>> features >> >>> >> >> >> will take more than a cycle to get integrated. >> >>> >> >> >> >> >>> >> >> >> --David >> >>> >> >> >> >> >>> >> >> >> On Thu, Mar 13, 2014 at 1:11 PM, Mike Tutkowski >> >>> >> >> >> <mike.tutkow...@solidfire.com> wrote: >> >>> >> >> >> > Yeah, if you "abandon" the "old" release as soon as a >> >>> >> >> >> > release >> >>> >> branch >> >>> >> >> is >> >>> >> >> >> cut >> >>> >> >> >> > for it, then you essentially have three months on the new >> >>> release >> >>> >> >> before >> >>> >> >> >> > its release branch is cut and you move on to the newer >> >>> release. I'm >> >>> >> >> not >> >>> >> >> >> > sure that was the intent when such a schedule was >> >>> >> >> >> > created. It >> >>> means >> >>> >> >> we're >> >>> >> >> >> > releasing every four months, but developing for only three. >> >>> >> >> >> > >> >>> >> >> >> > >> >>> >> >> >> > On Thu, Mar 13, 2014 at 11:03 AM, Marcus >> >>> >> >> >> > <shadow...@gmail.com> >> >>> >> wrote: >> >>> >> >> >> > >> >>> >> >> >> >> The overlap is simply a byproduct of cutting the branch, >> >>> >> >> >> >> I'm >> >>> not >> >>> >> sure >> >>> >> >> >> >> there's a way around it. It's a good point though, that >> >>> >> essentially >> >>> >> >> >> >> the window is 1 month shorter than I think was intended. >> >>> Better >> >>> >> >> >> >> testing will help that, however, with the point being >> >>> >> >> >> >> that we shouldn't be doing a ton of work to make the >> >>> >> >> >> >> release branch >> >>> >> stable. >> >>> >> >> It >> >>> >> >> >> >> should push the majority of the work back into the >> >>> >> >> >> >> pre-branch >> >>> >> stage. >> >>> >> >> >> >> >> >>> >> >> >> >> On Thu, Mar 13, 2014 at 10:50 AM, Mike Tutkowski >> >>> >> >> >> >> <mike.tutkow...@solidfire.com> wrote: >> >>> >> >> >> >> > I wanted to add a little comment/question in general >> >>> >> >> >> >> > about >> >>> our >> >>> >> >> release >> >>> >> >> >> >> > process: >> >>> >> >> >> >> > >> >>> >> >> >> >> > Right now we typically have a one-month overlap >> >>> >> >> >> >> > between >> >>> >> releases. >> >>> >> >> That >> >>> >> >> >> >> > being the case, if you are focusing on the current >> >>> >> >> >> >> > release >> >>> until >> >>> >> >> it is >> >>> >> >> >> >> out >> >>> >> >> >> >> > the door, you effectively lose a month of development >> >>> >> >> >> >> > for >> >>> the >> >>> >> >> future >> >>> >> >> >> >> > release. It might be tempting during this one-month >> >>> >> >> >> >> > time >> >>> period >> >>> >> to >> >>> >> >> >> focus >> >>> >> >> >> >> > instead on the future release and leave the current >> >>> >> >> >> >> > release >> >>> >> alone. >> >>> >> >> >> >> > >> >>> >> >> >> >> > Would it make sense to keep a four-month release >> >>> >> >> >> >> > cycle, but >> >>> not >> >>> >> >> have >> >>> >> >> >> an >> >>> >> >> >> >> > overlapping month of two releases? >> >>> >> >> >> >> > >> >>> >> >> >> >> > Just a thought >> >>> >> >> >> >> > >> >>> >> >> >> >> > >> >>> >> >> >> >> > On Thu, Mar 13, 2014 at 10:42 AM, David Nalley < >> >>> da...@gnsa.us> >> >>> >> >> wrote: >> >>> >> >> >> >> > >> >>> >> >> >> >> >> The RC7 vote thread contained a lot of discussion >> >>> >> >> >> >> >> around >> >>> >> release >> >>> >> >> >> >> >> cadence, and I figured I'd move that to a thread that >> >>> >> >> >> >> >> has a >> >>> >> better >> >>> >> >> >> >> >> subject so there is better visibility to list >> >>> >> >> >> >> >> participants >> >>> who >> >>> >> >> don't >> >>> >> >> >> >> >> read every thread. >> >>> >> >> >> >> >> >> >>> >> >> >> >> >> When I look at things schedule wise, I see our aims >> >>> >> >> >> >> >> and our >> >>> >> >> reality. >> >>> >> >> >> >> >> We have a relatively short development window (in the >> >>> schedule) >> >>> >> >> and >> >>> >> >> >> we >> >>> >> >> >> >> >> have almost 50% of our time in the schedule allocated >> >>> >> >> >> >> >> to >> >>> >> testing. >> >>> >> >> >> >> >> (over two months). However, it seems that a lot of >> >>> >> >> >> >> >> testing >> >>> - >> >>> >> or at >> >>> >> >> >> >> >> least a lot of testing for what became blockers to >> >>> >> >> >> >> >> the >> >>> release >> >>> >> >> >> didn't >> >>> >> >> >> >> >> appear to happen until RCs were kicked out - and >> >>> >> >> >> >> >> that's >> >>> where >> >>> >> our >> >>> >> >> >> >> >> schedule has fallen apart for multiple releases. The >> >>> automated >> >>> >> >> tests >> >>> >> >> >> >> >> we have were clean when we issued RCs, so we clearly >> >>> >> >> >> >> >> don't >> >>> have >> >>> >> >> the >> >>> >> >> >> >> >> depth needed from an automated standpoint. >> >>> >> >> >> >> >> >> >>> >> >> >> >> >> Two problems, one cultural and one technical. The >> >>> >> >> >> >> >> technical >> >>> >> >> problem >> >>> >> >> >> is >> >>> >> >> >> >> >> that our automated test suite isn't deep enough to >> >>> >> >> >> >> >> give us >> >>> a >> >>> >> high >> >>> >> >> >> >> >> level of confidence that we should release. The >> >>> >> >> >> >> >> cultural >> >>> >> problem >> >>> >> >> is >> >>> >> >> >> >> >> that many of us wait until the release period of the >> >>> schedule >> >>> >> to >> >>> >> >> >> test. >> >>> >> >> >> >> >> >> >>> >> >> >> >> >> What does that have to do with release cadence? Well >> >>> inherently >> >>> >> >> not >> >>> >> >> >> >> >> much; but let me describe my concerns. As a project; >> >>> >> >> >> >> >> the >> >>> >> schedule >> >>> >> >> is >> >>> >> >> >> >> >> meaningless if we don't follow it; and effectively >> >>> >> >> >> >> >> the >> >>> release >> >>> >> >> date >> >>> >> >> >> is >> >>> >> >> >> >> >> held hostage. Personally, I do want as few bugs as >> >>> possible, >> >>> >> but >> >>> >> >> it's >> >>> >> >> >> >> >> a balancing act where people doubt our ability if we >> >>> >> >> >> >> >> aren't >> >>> >> able >> >>> >> >> to >> >>> >> >> >> >> >> ship. I don't think it matters if we move to 6 month >> >>> cycles, if >> >>> >> >> this >> >>> >> >> >> >> >> behavior continues, we'd miss the 6 month date as >> >>> >> >> >> >> >> well and >> >>> push >> >>> >> >> to 8 >> >>> >> >> >> >> >> or 9 months. See my radical proposition at the bottom >> >>> >> >> >> >> >> for >> >>> an >> >>> >> idea >> >>> >> >> on >> >>> >> >> >> >> >> dealing with this. >> >>> >> >> >> >> >> >> >>> >> >> >> >> >> I also find myself agreeing with Daan on the >> >>> >> >> >> >> >> additional >> >>> >> >> complexity. >> >>> >> >> >> >> >> Increasing the window for release inherently >> >>> >> >> >> >> >> increases the >> >>> >> window >> >>> >> >> for >> >>> >> >> >> >> >> feature development. As soon as we branch a release, >> >>> master is >> >>> >> >> open >> >>> >> >> >> >> >> for feature development again. This means a potential >> >>> >> >> >> >> >> for >> >>> >> greater >> >>> >> >> >> >> >> change at each release. Change is a risk to quality; >> >>> >> >> >> >> >> or at >> >>> >> least >> >>> >> >> an >> >>> >> >> >> >> >> unknown that we again have to test. The greater that >> >>> quantity >> >>> >> of >> >>> >> >> >> >> >> change, the greater the potential threat to quality. >> >>> >> >> >> >> >> >> >>> >> >> >> >> >> Radical proposition: >> >>> >> >> >> >> >> >> >>> >> >> >> >> >> Because we have two problems, of different nature, we >> >>> >> >> >> >> >> are >> >>> in a >> >>> >> >> >> >> >> difficult situation. This is a possible solution, and >> >>> >> >> >> >> >> I'd >> >>> >> >> appreciate >> >>> >> >> >> >> >> you reading and considering it. Feedback is welcome. >> >>> >> >> >> >> >> I >> >>> propose >> >>> >> >> that >> >>> >> >> >> >> >> after we enter the RC stage that we not entertain any >> >>> >> >> >> >> >> bugs >> >>> as >> >>> >> >> >> blockers >> >>> >> >> >> >> >> that don't have automated test cases associated with >> them. >> >>> This >> >>> >> >> means >> >>> >> >> >> >> >> that you are still welcome to do manual testing of >> >>> >> >> >> >> >> your pet >> >>> >> >> feature >> >>> >> >> >> >> >> and the things that are important to you; during the >> >>> testing >> >>> >> >> window >> >>> >> >> >> >> >> (or anytime really). However, if the automation suite >> >>> >> >> >> >> >> isn't >> >>> >> also >> >>> >> >> >> >> >> failing then we consider the release as high enough >> >>> quality to >> >>> >> >> ship. >> >>> >> >> >> >> >> This isn't something we can codify, but the PMC can >> >>> certainly >> >>> >> >> adopt >> >>> >> >> >> >> >> this attitude as a group when voting. Which also >> >>> >> >> >> >> >> means >> >>> that we >> >>> >> can >> >>> >> >> >> >> >> deviate from it. If you brought up a blocker for >> >>> >> >> >> >> >> release - >> >>> we >> >>> >> >> should >> >>> >> >> >> >> >> be immediately looking at how we can write a test for >> >>> >> >> >> >> >> that >> >>> >> >> behavior. >> >>> >> >> >> >> >> This should also mean several other behaviors need to >> >>> become a >> >>> >> >> valid >> >>> >> >> >> >> >> part of our process. We need to ensure that things >> >>> >> >> >> >> >> are well >> >>> >> tested >> >>> >> >> >> >> >> before allowing a merge. This means we need a known >> >>> >> >> >> >> >> state >> >>> of >> >>> >> >> master, >> >>> >> >> >> >> >> and we need to perform testing that allows us to >> >>> >> >> >> >> >> confirm >> >>> that a >> >>> >> >> patch >> >>> >> >> >> >> >> does no harm. We also need to insist on >> >>> >> >> >> >> >> implementation of comprehensive tests for every >> inbound feature. >> >>> >> >> >> >> >> >> >>> >> >> >> >> >> Thoughts, comments, flames, death threats? :) >> >>> >> >> >> >> >> >> >>> >> >> >> >> >> --David >> >>> >> >> >> >> >> >> >>> >> >> >> >> > >> >>> >> >> >> >> > >> >>> >> >> >> >> > >> >>> >> >> >> >> > -- >> >>> >> >> >> >> > *Mike Tutkowski* >> >>> >> >> >> >> > *Senior CloudStack Developer, SolidFire Inc.* >> >>> >> >> >> >> > e: mike.tutkow...@solidfire.com >> >>> >> >> >> >> > o: 303.746.7302 >> >>> >> >> >> >> > Advancing the way the world uses the >> >>> >> >> >> >> > cloud<http://solidfire.com/solution/overview/?video=pl >> >>> >> >> >> >> > ay> >> >>> >> >> >> >> > *(tm)* >> >>> >> >> >> >> >> >>> >> >> >> > >> >>> >> >> >> > >> >>> >> >> >> > >> >>> >> >> >> > -- >> >>> >> >> >> > *Mike Tutkowski* >> >>> >> >> >> > *Senior CloudStack Developer, SolidFire Inc.* >> >>> >> >> >> > e: mike.tutkow...@solidfire.com >> >>> >> >> >> > o: 303.746.7302 >> >>> >> >> >> > Advancing the way the world uses the >> >>> >> >> >> > cloud<http://solidfire.com/solution/overview/?video=play> >> >>> >> >> >> > *(tm)* >> >>> >> >> >> >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > -- >> >>> >> >> > *Mike Tutkowski* >> >>> >> >> > *Senior CloudStack Developer, SolidFire Inc.* >> >>> >> >> > e: mike.tutkow...@solidfire.com >> >>> >> >> > o: 303.746.7302 >> >>> >> >> > Advancing the way the world uses the >> >>> >> >> > cloud<http://solidfire.com/solution/overview/?video=play> >> >>> >> >> > *(tm)* >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> -- >> >>> >> >> Daan >> >>> >> >> >> >>> >> >> >>> > >> >>> > >> >>> > >> >>> > -- >> >>> > *Mike Tutkowski* >> >>> > *Senior CloudStack Developer, SolidFire Inc.* >> >>> > e: mike.tutkow...@solidfire.com >> >>> > o: 303.746.7302 >> >>> > Advancing the way the world uses the >> >>> > cloud<http://solidfire.com/solution/overview/?video=play> >> >>> > *(tm)* >> >>> >> >>> >> >>> >> >>> -- >> >>> Daan >> >>> >> >> >> >> >> >> >> >> -- >> >> *Mike Tutkowski* >> >> *Senior CloudStack Developer, SolidFire Inc.* >> >> e: mike.tutkow...@solidfire.com >> >> o: 303.746.7302 >> >> Advancing the way the world uses the >> >> cloud<http://solidfire.com/solution/overview/?video=play> >> >> *(tm)* >> >> >> > >> > >> > >> > -- >> > *Mike Tutkowski* >> > *Senior CloudStack Developer, SolidFire Inc.* >> > e: mike.tutkow...@solidfire.com >> > o: 303.746.7302 >> > Advancing the way the world uses the >> > cloud<http://solidfire.com/solution/overview/?video=play> >> > *(tm)* >> >> >> >> -- >> Daan >> > > > > -- > *Mike Tutkowski* > *Senior CloudStack Developer, SolidFire Inc.* > e: mike.tutkow...@solidfire.com > o: 303.746.7302 > Advancing the way the world uses the > cloud<http://solidfire.com/solution/overview/?video=play> > *(tm)* > -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play> *(tm)*
