For milestone build, I think BVT + fidelity automation(load/save some samples) running is ok. we needn't have extra test plan on it, how do you think about it? -Xue Fei
On Wed, Sep 26, 2012 at 6:50 PM, Shenfeng Liu <[email protected]> wrote: > 2012/9/26 Ji Yan <[email protected]> > > > Simon, > > > > IMO, milestone test plan should base on milestone schedule and feature > > plan, what feature goes in and which defects are fixed in this milestone. > > Based on this scope we can define our testing plan. Otherwise we are > > running into target unclear testing, defects will be missed. > > > > > Ji, > Thanks for your complete thought! > While IMO, the milestone build is still a development snapshot. We don't > need to ensure a kind of GA quality on it. And we just need to ensure this > build: > > 1. Will not cause data lost (e.g. damage the file in editing). > 2. Will not lead to operation system crash. > 3. No severe defects that blocks user to try out the new features in this > build. > > Of course we need to test the new features, but I think it should fall to > another category of our planned testing. And for milestone build testing, I > think an acceptance test should be able to cover above goals, and we can > record the tested scenarios/platforms/languages on a wiki page. > We don't need to define a perfect test plan from beginning. Let's just > practise and refine. > > - Simon > > > > > 2012/9/26 Shenfeng Liu <[email protected]> > > > > > 2012/9/26 Kay Schenk <[email protected]> > > > > > > > On Tue, Sep 25, 2012 at 7:51 AM, Shenfeng Liu <[email protected]> > > > wrote: > > > > > > > > > 2012/9/24 Keith N. McKenna <[email protected]> > > > > > > > > > > > Rob Weir wrote: > > > > > > > > > > > >> On Mon, Sep 24, 2012 at 8:59 AM, Keith N. McKenna > > > > > >> <[email protected]> wrote: > > > > > >> > > > > > >>> Rob Weir wrote: > > > > > >>> > > > > > >>>> > > > > > >>>> On Sun, Sep 23, 2012 at 10:13 AM, Shenfeng Liu < > > > [email protected]> > > > > > >>>> wrote: > > > > > >>>> > > > > > >>>>> > > > > > >>>>> Hi, all, > > > > > >>>>> After 3.4.1, we are focusing on preparation of the > > community > > > > > >>>>> graduation. > > > > > >>>>> But I still want to remind us to take some time to think > about > > > our > > > > > >>>>> future > > > > > >>>>> releases. > > > > > >>>>> > > > > > >>>>> We have the discussion early about what 3.5 and 4.0 > should > > > look > > > > > >>>>> like. > > > > > >>>>> If > > > > > >>>>> I remember correctly: > > > > > >>>>> (1) 3.5 should be more about fidelity, reliability, > performance > > > and > > > > > >>>>> translation, new platform support... > > > > > >>>>> (2) While 4.0, in addition to the same focuses as 3.5, should > > > also > > > > > add > > > > > >>>>> significant UX enhancements (e.g. sidebar, modern UI) and new > > > > values > > > > > >>>>> (e.g. > > > > > >>>>> Accessibility, social integration capability, enhanced > > installer, > > > > new > > > > > >>>>> features...). If we make good progress on those items at the > > same > > > > > time, > > > > > >>>>> we > > > > > >>>>> may consider to skip 3.5. > > > > > >>>>> (3) There are also more requirements (e.g. fixpack mechanism, > > > > > >>>>> simplifying > > > > > >>>>> the build structure, OOMXL export, smartArt...) we need to > put > > > > into > > > > > >>>>> our > > > > > >>>>> backlog and consider their priority. > > > > > >>>>> > > > > > >>>>> Even we don't need to discuss the solid plan now, but > there > > > are > > > > > >>>>> already a > > > > > >>>>> lot of development activities on the trunk. So I think we > need > > to > > > > > keep > > > > > >>>>> certain track on it. Though it may be too early to set a > target > > > > date > > > > > >>>>> for > > > > > >>>>> the next release, but it is important for us to tell more > about > > > > what > > > > > we > > > > > >>>>> think the next release should contain. > > > > > >>>>> > > > > > >>>>> So I'm suggesting the following: > > > > > >>>>> > > > > > >>>>> 1. Keep updating the current release planning wiki: > > > > > >>>>> - > > > > > >>>>> > > > > > >>>>> https://cwiki.apache.org/**confluence/display/OOOUSERS/** > > > > > >>>>> AOO+3.5+Release+Planning< > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/OOOUSERS/AOO+3.5+Release+Planning > > > > > > > > > > > >>>>> - > > > > > >>>>> > > > > > >>>>> https://cwiki.apache.org/**confluence/display/OOOUSERS/** > > > > > >>>>> AOO+4.0+Release+Planning< > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/OOOUSERS/AOO+4.0+Release+Planning > > > > > > > > > > > >>>>> I know it is a little confusing for 2 places to input. > But > > > > think > > > > > >>>>> about > > > > > >>>>> the scope we agreed above. You can input to the wiki that you > > > think > > > > > >>>>> your > > > > > >>>>> work belong to. I personally will monitor both wiki pages. > > > > > >>>>> > > > > > >>>>> 2. Figure out a better way to manage our release backlog. > e.g. > > > set > > > > > >>>>> Target > > > > > >>>>> Milestone to 3.5 or 4.0 in Bugzilla for what we recommended. > > > > > >>>>> > > > > > >>>>> 3. Deliver milestone builds to harvest our development > fruits. > > A > > > > > >>>>> milestone > > > > > >>>>> build is: > > > > > >>>>> (a) a development snapshot that contains the > > > > > >>>>> features/enhancements > > > > > >>>>> that implemented till now; > > > > > >>>>> (b) passed regression test to ensure no severe > defects; > > > > > >>>>> (c) announced on a development wiki; > > > > > >>>>> (d) with documents on the wiki for the list of > features > > > and > > > > > bug > > > > > >>>>> fixes > > > > > >>>>> in this milestone build (like a release notes). > > > > > >>>>> Since whatever 3.5 or 4.0 sounds to me like some thing > in > > > next > > > > > >>>>> year > > > > > >>>>> or > > > > > >>>>> at least close to the end of this year, milestone builds can > be > > > > light > > > > > >>>>> weigh > > > > > >>>>> on process to show our development progress, and give people > a > > > more > > > > > >>>>> clear > > > > > >>>>> view on how far are we to the next release. > > > > > >>>>> > > > > > >>>>> Looking forward every one's comments! > > > > > >>>>> > > > > > >>>>> > > > > > >>>> Maybe also start a "release notes" page on the wiki. > Whenever a > > > new > > > > > >>>> feature or important bug fix is added to the trunk also add > > > > something > > > > > >>>> to the release notes. If something can be show with a > "before > > > and > > > > > >>>> after" screen shot, include that. This might be easier than > > > waiting > > > > > >>>> until the end to prepare the release notes. > > > > > >>>> > > > > > >>>> -Rob > > > > > >>>> > > > > > >>>> > > > > > >>>>> - Simon > > > > > >>>>> > > > > > >>>> > > > > > >>>> > > > > > >>>> Rob; > > > > > >>> > > > > > >>> A Release Notes page already exists or 3.5 and one or 4.0 can > be > > > > easily > > > > > >>> added. The complication I see here is since we have not decided > > > > whether > > > > > >>> the > > > > > >>> next release will be 3.5 or 4.0 that would require adding it in > > two > > > > > >>> places. > > > > > >>> I see that as a lot of overhead at this point. > > > > > >>> > > > > > >>> > > > > > >> IMHO, the name is not so important. Everything in the trunk > goes > > > into > > > > > >> the next release. Nothing not in the trunk goes into the next > > > > > >> release. So if we want a wiki page that is called "Release > notes > > > for > > > > > >> AOO Target January 2013" then it would be sufficient. Just > > describe > > > > > >> significant changes there made in the trunk. Maybe in the end > we > > > call > > > > > >> it "Apache OpenOffice 2013", or "Apache OpenOffice Adventitious > > > > > >> Armadillo" or something like that. That decision can come > later. > > > > > >> > > > > > >> -Rob > > > > > >> > > > > > >> > > > > > > In that case lets use the existing 3.5 Release Notes as Armin has > > > > already > > > > > > put a number of entries in there the "name can be change to > protect > > > the > > > > > > innocent later". > > > > > > > > > > > > > > > > +1 to use the existing 3.5 Release Notes wiki. > > > > > > > > > > And I just made a query in BZ, for defects fixed after 3.4 (May 8), > > and > > > > > excluded (1) some Products as qa, www, (2) those Target Milestone > set > > > to > > > > > 3.4.1, and (3) Issue Type not in (DEFECT, FEATURE, ENHANCEMENT). > And > > I > > > > got > > > > > about 500 results. I picked some of them in the list and believe > > there > > > > are > > > > > still many items need to be taken out, e.g. those fixed 1 year ago, > > but > > > > > just validated recently. So I think I can quickly go through them, > > and > > > > for > > > > > those who are really fixed/implemented in trunk after 3.4 and not > in > > > > 3.4.1, > > > > > I will set the Target Milestone to AOO 3.5.0. And this list can be > a > > > base > > > > > for our release notes. How do you think? > > > > > > > > > > Another thing is that we need to define a test plan for the > milestone > > > > > build, which can be a lightweight regression test suite. The plan > can > > > be > > > > > published on a wiki, and executed against very milestone build. > > > > > I agree with Juergen that we should start as early as possible. > > While I > > > > > still hope to get the confirmation from our QE team, since IMO they > > are > > > > the > > > > > key to this plan. :) > > > > > > > > > > - Simon > > > > > > > > > > > > > Simon -- > > > > > > > > Great that you started this. Will all new features ( these seem to be > > all > > > > in Calc by the current 3.5 Release Planning doc), be given an issue > > > > assignment at some point to correspond to your BZ search filter? The > > one > > > > listed as i110133 doesn't seem to show up in either your "features" > or > > > > "all" filters you posted> I think it needs a target change (which I > > > didn't > > > > make). > > > > > > > > All this will be very very helpful for planning our next release. So, > > > thank > > > > you. > > > > > > > > > > > Kay, > > > The list on the wiki is out of date. So I update it and marked it > out. > > I > > > think i110133 is a defect that proposed to fix in 3.5, but further on > > there > > > is no update for this defect... > > > Per our lesson&learned from 3.4.1, a better way is to leverage > Bugzilla > > > query. So I created one: TargetTo350AllFix. Also, I put the link on 3.5 > > > wiki. > > > > > > - Simon > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Regards > > > > > > Keith > > > > > > > > > > > > > > > > > > > > > > > >> I could create a separate page as a sandbox where what you > > suggest > > > > > could > > > > > >>> be > > > > > >>> input, then when the release comes it is just a matter of > moving > > > the > > > > > data > > > > > >>> from the sandbox into the formal Release Notes page. > > > > > >>> > > > > > >>> Regards > > > > > >>> Keith > > > > > >>> > > > > > >>> > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > ---------------------------------------------------------------------------------------- > > > > MzK > > > > > > > > "Just 'cause you got the monkey off your back > > > > doesn't mean the circus has left town." > > > > -- George Carlin > > > > > > > > > > > > > > > -- > > > > > > Thanks & Best Regards, Yan Ji > > >
