Re: Desktop updates to consider for beta1?
On 09/01/2012 05:50 AM, Sebastien Bacher wrote: > Hey release team, > > Here is a status update from desktop: > > * we finally landed the unity stack, sorry for the delay and thanks > for accepting it, it looks like the stack can be copied to "quantal" > (from proposed) at this point (I prefer to let the release team to do > it so I'm not sure what the copy rules are during the freeze) > > * we would appreciate if those upates could be considered as well for > beta1: > - indicator-session: it has some UI tweaks and we would like feedback > early on that (bugs linked in the changelog are UIFe ones) > - indicator-messages: it should be a no-op compared to what we have > for most user but it adds some extra api that are needed for the > webapp integration work going on, as well as documentation which > should help the porting of the client apps > - system-coonfig-printer: it includes some fixes > > * the libreoffice update from yesterday failed to build on amd64, we > uploaded a fixed version, Bjoern described the issue we had with > testing in the changelog (basically we need to mangle the build on > ppas due to disk use limitation in place on those builders), the new > version got tested locally by Bjoern and we are confident it fixes the > problem from the previous version > The new libreoffice failed on i386. -- Ubuntu-release mailing list Ubuntu-release@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-release
Re: Release Process concerns (QA) and suggestions
Hi Gema, Thanks for starting this thread off. :) On Fri, 2012-08-31 at 11:38 +0100, Gema Gomez wrote: > On 30/08/12 19:53, Stéphane Graber wrote: > > The release team is in charge of releasing a pre-defined set of > > images, for a given list of media at a given date. That's how > > things are. > > > > When we unfortunately hit a bug at the last minute, like happened > > last week, the release team needs to check how critical it's. If > > it's considered as a show-stopper, like was the case here, the > > only action to take is to fix it as soon as possible, re-test and > > then release. > > > > If we know it's technically impossible to get it re-tested in > > time, then we need to release a day later, but that's a very last > > resort as releasing on a Friday brings its own set of problems. > > Which kind of problems do you face when releasing on Friday? I think > it'd be good for us to know the consequences as well. The problem with releasing on Friday is that we don't have as good coverage available to react to problems if they occur. This is standard policy for stable release updates as well as releases, and has become so as a result lessons learned the hard way. Exceptions do occur, but they are very special cases, and contingency/monitoring plans need to be figured out in advance. > > > In the case of 12.04.1, we noticed on release day that an image > > didn't actually fit on its target media and apparently no tester > > bothered to actually burn it to a standard CD... > > You could use du next time right after the image is built to satisfy > yourself that the size is good, it could be a standard check that you > guys do. Certain mandatory manual tests can only be run if a CD is burned, specifically the AMD64+MAC based systems don't work with USB. > We have added some static validation tests to jenkins and are > in the process of publishing them. The information was already published on http://cdimage.ubuntu.com/precise/daily-live/current/ (and related pages). Its standard practice that an oversize indicator in bold red is published on the page, when an image is over a predetermined size as specified by the development teams. It was a failure of both the QA and release teams that no one looked at the page before Thursday. Release team had been looking at them pretty heavily the prior week, and thought we had all the issues solved. Based on discussions with Stéphane there are now plans to be adding an indicator to the ISO tracker to make oversize issues more visible in the future, as that is where some folks are now focusing, rather than the original publishing pages. > I don't think we need to burn a CD > to know if the image is going to fit or not. But if you want us to > validate things manually, adding a test case to the current set in the > iso tracker will help track that someone has bothered. Unfortunately I > don't feel confident enough yet with the admin mode of the iso tracker > to change anything, so your expertise there would be appreciated. There is the implication that a CD is burned in some of the test cases already, so I'm not sure that another test case need to be added, but rather an existing one be split to make it explicit a CD or when appropriate a DVD be burned as part of the test. If you have specific questions on admin'ing the iso tracker, please feel free to join us in #ubuntu-iso-tracker. There are multiple folk available (me, Jean-Baptiste, Nick), that can help as well. > Anyway, looking forward rather than backwards, for Quantal the size is > 800MB so, what media do you suggest we test on for size next week? > > https://blueprints.launchpad.net/ubuntu/+spec/desktop-q-one-iso-for-q > For Desktop, please test on both USB "and" DVD. For Ubuntu Server, please test on both USB "and" CD. We want to make sure both paths work since they are likely to be common based on what hardware folks have access to, and we'll be manufacturing CD's for Server, and DVD's for desktop, so making sure there are no significant problems is important. > > We found an obvious way of fixing it (removing a langpack) within > > just a couple of hours, got the change reviewed, tested, the image > > rebuilt, the content checked and then fully re-tested by 3 testers > > in less than 3 hours. Leaving us a good 10-12h before we actually > > released the set. > > In my opinion, it is not possible for 3 people to do 10 installs + 3 > upgrades each to a good level of details in less than 3 hours. Yes, > you can rush through things or split the test cases between the three > of you, or consider some tests done because one test case is sort of a > subset of another, and do some risk based testing, but the level of > risk we are accepting is not clear nor understood by all the parties. Its a case of testing smart, which we should all be aiming for. I had good confidence after Stéphane, Jean-Baptiste, and Nick had exercised the tests, after discussing their metho
Re: Release Process concerns (QA) and suggestions
Hi Gema, Thanks for starting this thread off. :) On Fri, 2012-08-31 at 11:38 +0100, Gema Gomez wrote: > On 30/08/12 19:53, Stéphane Graber wrote: > > The release team is in charge of releasing a pre-defined set of > > images, for a given list of media at a given date. That's how > > things are. > > > > When we unfortunately hit a bug at the last minute, like happened > > last week, the release team needs to check how critical it's. If > > it's considered as a show-stopper, like was the case here, the > > only action to take is to fix it as soon as possible, re-test and > > then release. > > > > If we know it's technically impossible to get it re-tested in > > time, then we need to release a day later, but that's a very last > > resort as releasing on a Friday brings its own set of problems. > > Which kind of problems do you face when releasing on Friday? I think > it'd be good for us to know the consequences as well. The problem with releasing on Friday is that we don't have as good coverage available to react to problems if they occur. This is standard policy for stable release updates as well as releases, and has become so as a result lessons learned the hard way. Exceptions do occur, but they are very special cases, and contingency/monitoring plans need to be figured out in advance. > > > In the case of 12.04.1, we noticed on release day that an image > > didn't actually fit on its target media and apparently no tester > > bothered to actually burn it to a standard CD... > > You could use du next time right after the image is built to satisfy > yourself that the size is good, it could be a standard check that you > guys do. Certain mandatory manual tests can only be run if a CD is burned, specifically the AMD64+MAC based systems don't work with USB. > We have added some static validation tests to jenkins and are > in the process of publishing them. The information was already published on http://cdimage.ubuntu.com/precise/daily-live/current/ (and related pages). Its standard practice that an oversize indicator in bold red is published on the page, when an image is over a predetermined size as specified by the development teams. It was a failure of both the QA and release teams that no one looked at the page before Thursday. Release team had been looking at them pretty heavily the prior week, and thought we had all the issues solved. Based on discussions with Stéphane there are now plans to be adding an indicator to the ISO tracker to make oversize issues more visible in the future, as that is where some folks are now focusing, rather than the original publishing pages. > I don't think we need to burn a CD > to know if the image is going to fit or not. But if you want us to > validate things manually, adding a test case to the current set in the > iso tracker will help track that someone has bothered. Unfortunately I > don't feel confident enough yet with the admin mode of the iso tracker > to change anything, so your expertise there would be appreciated. There is the implication that a CD is burned in some of the test cases already, so I'm not sure that another test case need to be added, but rather an existing one be split to make it explicit a CD or when appropriate a DVD be burned as part of the test. If you have specific questions on admin'ing the iso tracker, please feel free to join us in #ubuntu-iso-tracker. There are multiple folk available (me, Jean-Baptiste, Nick), that can help as well. > Anyway, looking forward rather than backwards, for Quantal the size is > 800MB so, what media do you suggest we test on for size next week? > > https://blueprints.launchpad.net/ubuntu/+spec/desktop-q-one-iso-for-q > For Desktop, please test on both USB "and" DVD. For Ubuntu Server, please test on both USB "and" CD. We want to make sure both paths work since they are likely to be common based on what hardware folks have access to, and we'll be manufacturing CD's for Server, and DVD's for desktop, so making sure there are no significant problems is important. > > We found an obvious way of fixing it (removing a langpack) within > > just a couple of hours, got the change reviewed, tested, the image > > rebuilt, the content checked and then fully re-tested by 3 testers > > in less than 3 hours. Leaving us a good 10-12h before we actually > > released the set. > > In my opinion, it is not possible for 3 people to do 10 installs + 3 > upgrades each to a good level of details in less than 3 hours. Yes, > you can rush through things or split the test cases between the three > of you, or consider some tests done because one test case is sort of a > subset of another, and do some risk based testing, but the level of > risk we are accepting is not clear nor understood by all the parties. Its a case of testing smart, which we should all be aiming for. I had good confidence after Stéphane, Jean-Baptiste, and Nick had exercised the tests, after discussing their method
Desktop updates to consider for beta1?
Hey release team, Here is a status update from desktop: * we finally landed the unity stack, sorry for the delay and thanks for accepting it, it looks like the stack can be copied to "quantal" (from proposed) at this point (I prefer to let the release team to do it so I'm not sure what the copy rules are during the freeze) * we would appreciate if those upates could be considered as well for beta1: - indicator-session: it has some UI tweaks and we would like feedback early on that (bugs linked in the changelog are UIFe ones) - indicator-messages: it should be a no-op compared to what we have for most user but it adds some extra api that are needed for the webapp integration work going on, as well as documentation which should help the porting of the client apps - system-coonfig-printer: it includes some fixes * the libreoffice update from yesterday failed to build on amd64, we uploaded a fixed version, Bjoern described the issue we had with testing in the changelog (basically we need to mangle the build on ppas due to disk use limitation in place on those builders), the new version got tested locally by Bjoern and we are confident it fixes the problem from the previous version Thanks for considering those, Sebastien Bacher -- Ubuntu-release mailing list Ubuntu-release@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-release
