Exactly...it is not practical for 4.4 to spin up a RC so soon. Perhaps with more clearly defined criteria for 4.5, this may be possible.
On Thu, Mar 13, 2014 at 5:48 PM, Sudha Ponnaganti < sudha.ponnaga...@citrix.com> wrote: > Feature freeze and Merge criteria is not defined to change the milestone > suddenly. We can create build and call it RC but that would take another > 2+ months to harden to build next RC. There are only 50 defects logged for > 4.4 so far. We are away from another 600+ defects to get acceptable RC > candidate ( based on 4.3 comparison) > > Merge might have some sanity tests submitted but I don't think any > confidence tests are done on the features. There hasn't been big discussion > around integration tests on 4.4 threads except find bugs and some basic > integration or sanity tests. > > -----Original Message----- > From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] > Sent: Thursday, March 13, 2014 4:34 PM > To: dev > Subject: Re: Release cadence > > I agree that we can't move to our end goal in on go. But I disagree that > we should go on with business as usual right now. baby steps but never stop > taking steps. > > On Fri, Mar 14, 2014 at 12:20 AM, Mike Tutkowski < > mike.tutkow...@solidfire.com> wrote: > > My reasoning here is that otherwise we will just be futilely creating > > RCs for 4.4 since this expectation was not clearly defined ahead of the > release. > > > > If we set expectations appropriately for 4.5, then we should expect we > > can begin RC building right after Feature Freeze for that release. > > > > > > On Thu, Mar 13, 2014 at 5:18 PM, Mike Tutkowski < > > mike.tutkow...@solidfire.com> wrote: > > > >> I think we should set that as a goal for 4.5. We should treat 4.4 as > >> business as usual at this point and give "fair warning" for the next > >> release. > >> > >> We should formally define what "tested" means for 4.5 and then take > >> the appropriate course of action from a RC point of view. > >> > >> > >> On Thu, Mar 13, 2014 at 5:00 PM, Daan Hoogland <daan.hoogl...@gmail.com > >wrote: > >> > >>> That's how i like to see it and why I asked. Is there a reason > >>> people merge and then commit their features instead of rebasing and > >>> running a standard set of integration tests to validate before > >>> merging. I am not better then average on this myself but I think > >>> here is where we have room to improve if anywhere. > >>> > >>> So do we create 4.4 RC1 next Monday? > >>> > >>> > >>> > >>> On Thu, Mar 13, 2014 at 11:19 PM, Mike Tutkowski > >>> <mike.tutkow...@solidfire.com> wrote: > >>> > I think many people (myself included) are used to performing > >>> > rigorous, > >>> but > >>> > focused feature-specific testing before feature freeze, but are > >>> > under > >>> the > >>> > impression that once feature freeze arrives that we are in > >>> > integration-testing mode (where our feature is tested in > >>> > combination > >>> with > >>> > other features...not so isolated anymore). At this point, we tend > >>> > to > >>> find > >>> > bugs that were not hit pre feature freeze because that mode of > >>> > testing > >>> was > >>> > more confined. > >>> > > >>> > Perhaps we simply need to decide on how tested a feature should be > >>> > for feature freeze. Does it need to be fully tested from an > >>> > integration with other features standpoint or not? If yes, then we > are basically "done" > >>> with > >>> > the release at feature freeze time and can begin the > >>> > release-candidate process. > >>> > > >>> > > >>> > On Thu, Mar 13, 2014 at 4:11 PM, David Nalley <da...@gnsa.us> wrote: > >>> > > >>> >> Thats a very good point - we are effectively saying we know the > >>> >> features we merged in have potentially months worth of bugs. > >>> >> Though really, our hiccups don't seem to generally be in new > >>> >> features, it's old features. > >>> >> > >>> >> On Thu, Mar 13, 2014 at 3:44 PM, Marcus <shadow...@gmail.com> > wrote: > >>> >> > Its a good point. I had thought about. Essentially we are > >>> >> > saying > >>> that we > >>> >> > know the features we just merged need another few months of work. > >>> >> > On Mar 13, 2014 1:01 PM, "Daan Hoogland" > >>> >> > <daan.hoogl...@gmail.com> > >>> >> wrote: > >>> >> > > >>> >> >> Just a thought, > >>> >> >> > >>> >> >> Why isn't the freshly cut branch the first RC from the get go? > >>> >> >> It is quite sure not to pass but it should cantain what we ant > >>> >> >> to ship feature wise. > >>> >> >> > >>> >> >> On Thu, Mar 13, 2014 at 6:35 PM, Mike Tutkowski > >>> >> >> <mike.tutkow...@solidfire.com> wrote: > >>> >> >> > OK, so it sounds like a 3-month dev cycle for a four-month > >>> >> >> > release > >>> >> was on > >>> >> >> > purpose. > >>> >> >> > > >>> >> >> > Just curious...thanks :) > >>> >> >> > > >>> >> >> > > >>> >> >> > On Thu, Mar 13, 2014 at 11:31 AM, David Nalley > >>> >> >> > <da...@gnsa.us> > >>> wrote: > >>> >> >> > > >>> >> >> >> This was (IIRC) part of the explicit decision in how to do > >>> things. > >>> >> The > >>> >> >> >> thought being that if you are restricting what people can > >>> >> >> >> do > >>> with a > >>> >> >> >> release branch, people still need to be able to have a > >>> >> >> >> place to > >>> base > >>> >> >> >> their ongoing work; and master should be that place. Some > >>> features > >>> >> >> >> will take more than a cycle to get integrated. > >>> >> >> >> > >>> >> >> >> --David > >>> >> >> >> > >>> >> >> >> On Thu, Mar 13, 2014 at 1:11 PM, Mike Tutkowski > >>> >> >> >> <mike.tutkow...@solidfire.com> wrote: > >>> >> >> >> > Yeah, if you "abandon" the "old" release as soon as a > >>> >> >> >> > release > >>> >> branch > >>> >> >> is > >>> >> >> >> cut > >>> >> >> >> > for it, then you essentially have three months on the new > >>> release > >>> >> >> before > >>> >> >> >> > its release branch is cut and you move on to the newer > >>> release. I'm > >>> >> >> not > >>> >> >> >> > sure that was the intent when such a schedule was > >>> >> >> >> > created. It > >>> means > >>> >> >> we're > >>> >> >> >> > releasing every four months, but developing for only three. > >>> >> >> >> > > >>> >> >> >> > > >>> >> >> >> > On Thu, Mar 13, 2014 at 11:03 AM, Marcus > >>> >> >> >> > <shadow...@gmail.com> > >>> >> wrote: > >>> >> >> >> > > >>> >> >> >> >> The overlap is simply a byproduct of cutting the branch, > >>> >> >> >> >> I'm > >>> not > >>> >> sure > >>> >> >> >> >> there's a way around it. It's a good point though, that > >>> >> essentially > >>> >> >> >> >> the window is 1 month shorter than I think was intended. > >>> Better > >>> >> >> >> >> testing will help that, however, with the point being > >>> >> >> >> >> that we shouldn't be doing a ton of work to make the > >>> >> >> >> >> release branch > >>> >> stable. > >>> >> >> It > >>> >> >> >> >> should push the majority of the work back into the > >>> >> >> >> >> pre-branch > >>> >> stage. > >>> >> >> >> >> > >>> >> >> >> >> On Thu, Mar 13, 2014 at 10:50 AM, Mike Tutkowski > >>> >> >> >> >> <mike.tutkow...@solidfire.com> wrote: > >>> >> >> >> >> > I wanted to add a little comment/question in general > >>> >> >> >> >> > about > >>> our > >>> >> >> release > >>> >> >> >> >> > process: > >>> >> >> >> >> > > >>> >> >> >> >> > Right now we typically have a one-month overlap > >>> >> >> >> >> > between > >>> >> releases. > >>> >> >> That > >>> >> >> >> >> > being the case, if you are focusing on the current > >>> >> >> >> >> > release > >>> until > >>> >> >> it is > >>> >> >> >> >> out > >>> >> >> >> >> > the door, you effectively lose a month of development > >>> >> >> >> >> > for > >>> the > >>> >> >> future > >>> >> >> >> >> > release. It might be tempting during this one-month > >>> >> >> >> >> > time > >>> period > >>> >> to > >>> >> >> >> focus > >>> >> >> >> >> > instead on the future release and leave the current > >>> >> >> >> >> > release > >>> >> alone. > >>> >> >> >> >> > > >>> >> >> >> >> > Would it make sense to keep a four-month release > >>> >> >> >> >> > cycle, but > >>> not > >>> >> >> have > >>> >> >> >> an > >>> >> >> >> >> > overlapping month of two releases? > >>> >> >> >> >> > > >>> >> >> >> >> > Just a thought > >>> >> >> >> >> > > >>> >> >> >> >> > > >>> >> >> >> >> > On Thu, Mar 13, 2014 at 10:42 AM, David Nalley < > >>> da...@gnsa.us> > >>> >> >> wrote: > >>> >> >> >> >> > > >>> >> >> >> >> >> The RC7 vote thread contained a lot of discussion > >>> >> >> >> >> >> around > >>> >> release > >>> >> >> >> >> >> cadence, and I figured I'd move that to a thread that > >>> >> >> >> >> >> has a > >>> >> better > >>> >> >> >> >> >> subject so there is better visibility to list > >>> >> >> >> >> >> participants > >>> who > >>> >> >> don't > >>> >> >> >> >> >> read every thread. > >>> >> >> >> >> >> > >>> >> >> >> >> >> When I look at things schedule wise, I see our aims > >>> >> >> >> >> >> and our > >>> >> >> reality. > >>> >> >> >> >> >> We have a relatively short development window (in the > >>> schedule) > >>> >> >> and > >>> >> >> >> we > >>> >> >> >> >> >> have almost 50% of our time in the schedule allocated > >>> >> >> >> >> >> to > >>> >> testing. > >>> >> >> >> >> >> (over two months). However, it seems that a lot of > >>> >> >> >> >> >> testing > >>> - > >>> >> or at > >>> >> >> >> >> >> least a lot of testing for what became blockers to > >>> >> >> >> >> >> the > >>> release > >>> >> >> >> didn't > >>> >> >> >> >> >> appear to happen until RCs were kicked out - and > >>> >> >> >> >> >> that's > >>> where > >>> >> our > >>> >> >> >> >> >> schedule has fallen apart for multiple releases. The > >>> automated > >>> >> >> tests > >>> >> >> >> >> >> we have were clean when we issued RCs, so we clearly > >>> >> >> >> >> >> don't > >>> have > >>> >> >> the > >>> >> >> >> >> >> depth needed from an automated standpoint. > >>> >> >> >> >> >> > >>> >> >> >> >> >> Two problems, one cultural and one technical. The > >>> >> >> >> >> >> technical > >>> >> >> problem > >>> >> >> >> is > >>> >> >> >> >> >> that our automated test suite isn't deep enough to > >>> >> >> >> >> >> give us > >>> a > >>> >> high > >>> >> >> >> >> >> level of confidence that we should release. The > >>> >> >> >> >> >> cultural > >>> >> problem > >>> >> >> is > >>> >> >> >> >> >> that many of us wait until the release period of the > >>> schedule > >>> >> to > >>> >> >> >> test. > >>> >> >> >> >> >> > >>> >> >> >> >> >> What does that have to do with release cadence? Well > >>> inherently > >>> >> >> not > >>> >> >> >> >> >> much; but let me describe my concerns. As a project; > >>> >> >> >> >> >> the > >>> >> schedule > >>> >> >> is > >>> >> >> >> >> >> meaningless if we don't follow it; and effectively > >>> >> >> >> >> >> the > >>> release > >>> >> >> date > >>> >> >> >> is > >>> >> >> >> >> >> held hostage. Personally, I do want as few bugs as > >>> possible, > >>> >> but > >>> >> >> it's > >>> >> >> >> >> >> a balancing act where people doubt our ability if we > >>> >> >> >> >> >> aren't > >>> >> able > >>> >> >> to > >>> >> >> >> >> >> ship. I don't think it matters if we move to 6 month > >>> cycles, if > >>> >> >> this > >>> >> >> >> >> >> behavior continues, we'd miss the 6 month date as > >>> >> >> >> >> >> well and > >>> push > >>> >> >> to 8 > >>> >> >> >> >> >> or 9 months. See my radical proposition at the bottom > >>> >> >> >> >> >> for > >>> an > >>> >> idea > >>> >> >> on > >>> >> >> >> >> >> dealing with this. > >>> >> >> >> >> >> > >>> >> >> >> >> >> I also find myself agreeing with Daan on the > >>> >> >> >> >> >> additional > >>> >> >> complexity. > >>> >> >> >> >> >> Increasing the window for release inherently > >>> >> >> >> >> >> increases the > >>> >> window > >>> >> >> for > >>> >> >> >> >> >> feature development. As soon as we branch a release, > >>> master is > >>> >> >> open > >>> >> >> >> >> >> for feature development again. This means a potential > >>> >> >> >> >> >> for > >>> >> greater > >>> >> >> >> >> >> change at each release. Change is a risk to quality; > >>> >> >> >> >> >> or at > >>> >> least > >>> >> >> an > >>> >> >> >> >> >> unknown that we again have to test. The greater that > >>> quantity > >>> >> of > >>> >> >> >> >> >> change, the greater the potential threat to quality. > >>> >> >> >> >> >> > >>> >> >> >> >> >> Radical proposition: > >>> >> >> >> >> >> > >>> >> >> >> >> >> Because we have two problems, of different nature, we > >>> >> >> >> >> >> are > >>> in a > >>> >> >> >> >> >> difficult situation. This is a possible solution, and > >>> >> >> >> >> >> I'd > >>> >> >> appreciate > >>> >> >> >> >> >> you reading and considering it. Feedback is welcome. > >>> >> >> >> >> >> I > >>> propose > >>> >> >> that > >>> >> >> >> >> >> after we enter the RC stage that we not entertain any > >>> >> >> >> >> >> bugs > >>> as > >>> >> >> >> blockers > >>> >> >> >> >> >> that don't have automated test cases associated with > them. > >>> This > >>> >> >> means > >>> >> >> >> >> >> that you are still welcome to do manual testing of > >>> >> >> >> >> >> your pet > >>> >> >> feature > >>> >> >> >> >> >> and the things that are important to you; during the > >>> testing > >>> >> >> window > >>> >> >> >> >> >> (or anytime really). However, if the automation suite > >>> >> >> >> >> >> isn't > >>> >> also > >>> >> >> >> >> >> failing then we consider the release as high enough > >>> quality to > >>> >> >> ship. > >>> >> >> >> >> >> This isn't something we can codify, but the PMC can > >>> certainly > >>> >> >> adopt > >>> >> >> >> >> >> this attitude as a group when voting. Which also > >>> >> >> >> >> >> means > >>> that we > >>> >> can > >>> >> >> >> >> >> deviate from it. If you brought up a blocker for > >>> >> >> >> >> >> release - > >>> we > >>> >> >> should > >>> >> >> >> >> >> be immediately looking at how we can write a test for > >>> >> >> >> >> >> that > >>> >> >> behavior. > >>> >> >> >> >> >> This should also mean several other behaviors need to > >>> become a > >>> >> >> valid > >>> >> >> >> >> >> part of our process. We need to ensure that things > >>> >> >> >> >> >> are well > >>> >> tested > >>> >> >> >> >> >> before allowing a merge. This means we need a known > >>> >> >> >> >> >> state > >>> of > >>> >> >> master, > >>> >> >> >> >> >> and we need to perform testing that allows us to > >>> >> >> >> >> >> confirm > >>> that a > >>> >> >> patch > >>> >> >> >> >> >> does no harm. We also need to insist on > >>> >> >> >> >> >> implementation of comprehensive tests for every inbound > feature. > >>> >> >> >> >> >> > >>> >> >> >> >> >> Thoughts, comments, flames, death threats? :) > >>> >> >> >> >> >> > >>> >> >> >> >> >> --David > >>> >> >> >> >> >> > >>> >> >> >> >> > > >>> >> >> >> >> > > >>> >> >> >> >> > > >>> >> >> >> >> > -- > >>> >> >> >> >> > *Mike Tutkowski* > >>> >> >> >> >> > *Senior CloudStack Developer, SolidFire Inc.* > >>> >> >> >> >> > e: mike.tutkow...@solidfire.com > >>> >> >> >> >> > o: 303.746.7302 > >>> >> >> >> >> > Advancing the way the world uses the > >>> >> >> >> >> > cloud<http://solidfire.com/solution/overview/?video=pl > >>> >> >> >> >> > ay> > >>> >> >> >> >> > *(tm)* > >>> >> >> >> >> > >>> >> >> >> > > >>> >> >> >> > > >>> >> >> >> > > >>> >> >> >> > -- > >>> >> >> >> > *Mike Tutkowski* > >>> >> >> >> > *Senior CloudStack Developer, SolidFire Inc.* > >>> >> >> >> > e: mike.tutkow...@solidfire.com > >>> >> >> >> > o: 303.746.7302 > >>> >> >> >> > Advancing the way the world uses the > >>> >> >> >> > cloud<http://solidfire.com/solution/overview/?video=play> > >>> >> >> >> > *(tm)* > >>> >> >> >> > >>> >> >> > > >>> >> >> > > >>> >> >> > > >>> >> >> > -- > >>> >> >> > *Mike Tutkowski* > >>> >> >> > *Senior CloudStack Developer, SolidFire Inc.* > >>> >> >> > e: mike.tutkow...@solidfire.com > >>> >> >> > o: 303.746.7302 > >>> >> >> > Advancing the way the world uses the > >>> >> >> > cloud<http://solidfire.com/solution/overview/?video=play> > >>> >> >> > *(tm)* > >>> >> >> > >>> >> >> > >>> >> >> > >>> >> >> -- > >>> >> >> Daan > >>> >> >> > >>> >> > >>> > > >>> > > >>> > > >>> > -- > >>> > *Mike Tutkowski* > >>> > *Senior CloudStack Developer, SolidFire Inc.* > >>> > e: mike.tutkow...@solidfire.com > >>> > o: 303.746.7302 > >>> > Advancing the way the world uses the > >>> > cloud<http://solidfire.com/solution/overview/?video=play> > >>> > *(tm)* > >>> > >>> > >>> > >>> -- > >>> Daan > >>> > >> > >> > >> > >> -- > >> *Mike Tutkowski* > >> *Senior CloudStack Developer, SolidFire Inc.* > >> e: mike.tutkow...@solidfire.com > >> o: 303.746.7302 > >> Advancing the way the world uses the > >> cloud<http://solidfire.com/solution/overview/?video=play> > >> *(tm)* > >> > > > > > > > > -- > > *Mike Tutkowski* > > *Senior CloudStack Developer, SolidFire Inc.* > > e: mike.tutkow...@solidfire.com > > o: 303.746.7302 > > Advancing the way the world uses the > > cloud<http://solidfire.com/solution/overview/?video=play> > > *(tm)* > > > > -- > Daan > -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play> *(tm)*
