[Libreoffice-qa] Looking for a new approach in QA workflow.
Hi all, Reading comments out there, like Ask, Bugzilla and other sites, seems some people have the impression that LibreOffice is released with bugs, what leaves secondly the great improvements, what do not make me feel too well. Well IMHO, we need to look for a new approach to change this appreciation, and if was possible at the same time improve the project. IMHO we have not enough verification for Patch and Enhancements (P&E) previously to their incorporation in the RC and final releases. E.g. we have not a place where to find how a P&E must work and when is in a build?. Invest the time before release, to reduce multiplied the spent time after release, I think could be the most effective way to get results. How many hours spent on repair can save every hour invest in a previous QA to avoid a bug?, this is a true saving for users and their companies, and up to footprint on the environment. We are green :). My proposal: 1) Have a place where the patch and enhancements (P&E) are published at the same time of the build where they are pushed, is made available, with the detailed information needed to make possible their verification, with the link to the build. Some indication about priority, specially the urgent patches also would be of interest. 2) The quality team, maybe developers, and who can help, then can make a systematic and orderly checking thereof. 3) After test, report the positive result, and the negative with the number of the report in bugzilla with cc to the author for found bugs. 4) Restrict to only patch and enhancements with positives test, without any negative, can reach RCs or final releases. 5) Special cases can be discussed ESC call. IMO, the point, is review as soon as possible, making easier solve the issues while the P&E are fresh in mind, reducing the options for new cross issues, decreasing at the same time, I think significantly, the need for future bibisect. The public explanation, I am sure will make developers think in a wider perspective, getting a good feedback quickly and making more visible the quality of their work. In this way, one can know what are the new P&E for test in the build, and know much better what kind of test to do. Now, it is hard to know what is new in every build and what is there for review. I think it would be an invaluable reference place for everybody. To know about what can be expected from the P&E and how it must work. I am thinking specially in documentation people and who could help devs on the help update. A place where to see quickly what is going on. Where to find when the P&E have been pushed. I know carry on the place is not easy, but maybe all things are there in different places, in any case, I think the benefits could be beyond on what we can think now. I am sorry for mistakes and if find the proposal has no interest or maybe a bit revolutionary, but I hope at least help to point in the right direction. Regards. Miguel Ángel. * Inglés - detectado * Inglés * Español * Gallego * Italiano * Inglés * Español * Gallego * Italiano <#> ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] After EOL has been reached...
Hi Robinson, On Fri, Apr 26, 2013 at 12:54:05PM -0400, Robinson Tryon wrote: > I don't know if an easier-to-read interface would increase our ability > to attract users to join us and help us retain people as members of > the QA Team, but it is one of the things I thought about when making a > mock-up. We definitely need to grow our ranks to be able to address > the influx of bug reports. Yes, but just a warning to spare you from disappointment: Putting a lot of work into mockups, when there is no bugzilla-hacker around that runs around crying "Im really bored, I have nothing to do and have no idea on my own how to improve bugzilla" is quite futile as implementing it is the tricky part. Once you have that guy, you can start discussing the details. If I had a dollar for every mockup showing how a completely new LibreOffice UI should look like, I would be a very rich man. ;) Best, Bjoern ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] After EOL has been reached...
On Fri, Apr 26, 2013 at 12:36 PM, Bjoern Michaelsen wrote: > > So, as a developer, I would think comments (and the platform field) would be > sufficient for these rare cornercases. > > For fixed and reappearing, I think its > not really an issue: > - if the bug is really fixed, it should have a dev having insight in the some > issue already (and reading a few comments is not so much work compared to > the > total effort in these cornercases) > - if the bug was not fixed but just "disappeared" -- that is, stopped being > there without a dev claiming to intentionally having fixed it, it is likely > a > Heisenburg anyway and the version info to be used carefully (or even > ignored) > anyway. If the developers think that the current system provides enough information for them to fix the bugs, then perhaps we don't need to make any updates to the system for the devs, which is good to know. My thought was that a cleaner, more visual display of the information could be helpful for people doing QA (both on the team or just regular users) to understand the helpfulness of narrowing-down the precise point at which a bug was introduced into the code. The FDO interface can be very cryptic! I don't know if an easier-to-read interface would increase our ability to attract users to join us and help us retain people as members of the QA Team, but it is one of the things I thought about when making a mock-up. We definitely need to grow our ranks to be able to address the influx of bug reports. Cheers, --R ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] After EOL has been reached...
Hi, On Fri, Apr 26, 2013 at 11:26:33AM -0400, Robinson Tryon wrote: > On Fri, Apr 26, 2013 at 11:08 AM, Bjoern Michaelsen > wrote: > > On Fri, Apr 26, 2013 at 10:56:21AM -0400, Robinson Tryon wrote: > >> This kind of scenario is one of the reasons I was interested in > >> implementing my "Repro Table". The current version drop-down is both > >> too long and not granular enough to visually indicate the NOREPRO in > >> 3.6.2 rc1 and the REPRO in 3.6.2 release. The Repro Table could > >> display that information very clearly, but I am digressing a bit :-) > > > > But we dont really need a table, but only the first version the bug > > reproduces > > (and possibly the last version that does not reproduce it), right? > > For most cases, yes, we would just need 2 versions (and notes on each > OS used for repro), but... there are special cases :-) > > 1. Platform-specific bugs > > If a bug is platform-specific, then the table can help to show that. > It could also show the case (which many agree would be very rare) of a > bug that might affect just Mac/Linux and not Windows. > > 2. Fixed and then reappearing > > Joel made a note that some bugs might be fixed and then reappear in a > later build. He was specifically talking about issues when > bibisecting, but it could just as easily appear in testing with other > builds. So, as a developer, I would think comments (and the platform field) would be sufficient for these rare cornercases. For fixed and reappearing, I think its not really an issue: - if the bug is really fixed, it should have a dev having insight in the some issue already (and reading a few comments is not so much work compared to the total effort in these cornercases) - if the bug was not fixed but just "disappeared" -- that is, stopped being there without a dev claiming to intentionally having fixed it, it is likely a Heisenburg anyway and the version info to be used carefully (or even ignored) anyway. Best, Bjoern ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] After EOL has been reached...
On Fri, Apr 26, 2013 at 11:08 AM, Bjoern Michaelsen wrote: > On Fri, Apr 26, 2013 at 10:56:21AM -0400, Robinson Tryon wrote: >> This kind of scenario is one of the reasons I was interested in >> implementing my "Repro Table". The current version drop-down is both >> too long and not granular enough to visually indicate the NOREPRO in >> 3.6.2 rc1 and the REPRO in 3.6.2 release. The Repro Table could >> display that information very clearly, but I am digressing a bit :-) > > But we dont really need a table, but only the first version the bug reproduces > (and possibly the last version that does not reproduce it), right? For most cases, yes, we would just need 2 versions (and notes on each OS used for repro), but... there are special cases :-) 1. Platform-specific bugs If a bug is platform-specific, then the table can help to show that. It could also show the case (which many agree would be very rare) of a bug that might affect just Mac/Linux and not Windows. 2. Fixed and then reappearing Joel made a note that some bugs might be fixed and then reappear in a later build. He was specifically talking about issues when bibisecting, but it could just as easily appear in testing with other builds. --R ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] After EOL has been reached...
On Fri, Apr 26, 2013 at 10:56:21AM -0400, Robinson Tryon wrote: > This kind of scenario is one of the reasons I was interested in > implementing my "Repro Table". The current version drop-down is both > too long and not granular enough to visually indicate the NOREPRO in > 3.6.2 rc1 and the REPRO in 3.6.2 release. The Repro Table could > display that information very clearly, but I am digressing a bit :-) But we dont really need a table, but only the first version the bug reproduces (and possibly the last version that does not reproduce it), right? Best, Bjoern ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] After EOL has been reached...
On Fri, Apr 26, 2013 at 5:24 AM, Bjoern Michaelsen wrote: > On Fri, Apr 26, 2013 at 09:47:28AM +0200, Jean-Baptiste Faure wrote: >> Le 19/04/2013 23:21, Robinson Tryon a écrit : >> > AGREED: We de-list versions in Bugzilla 6 months after the release has >> > been EOLed >> >> I do not see any rationale for that. >> What I suggest is to remove all RC and beta version from BugZilla as >> soon as the corresponding final version has been published. > > Hmm, sounds like a good idea too. +1 I think we need to be even more aggressive in de-listing versions, but if this is a cut we can all agree on, let's start there :-) > but If someone triages it even further (e.g. a > bug was not there in 3.6.2 rc1 but is in the final version), a comment in the > bug might suffice. This kind of scenario is one of the reasons I was interested in implementing my "Repro Table". The current version drop-down is both too long and not granular enough to visually indicate the NOREPRO in 3.6.2 rc1 and the REPRO in 3.6.2 release. The Repro Table could display that information very clearly, but I am digressing a bit :-) > Note that the reduction of the numbers of versions to select from is valuable > in itself. I find myself being annoyed by scrolling through that long list in > a > small box, and imagine it is even more irritating to e.g. a first time bug > reporter. > Here are some numbers for reference. I might add that since the beginning of this thread, (I believe) several old versions have already been de-listed from Bugzilla, so I feel like we're making progress already! The version drop-down in Bugzilla currently contains some 86 different entries (see Appendix #1). Here's a breakdown by minor version: Minor Version - Total, non-Release - 3.3 - 6, 1 3.4 - 22, 15 3.5 - 23, 15 3.6 - 19, 12 4.0 - 12, 9 4.1 - 1, 1 Other - 3, 1 If we were to de-list the non-release stuff from 3.3 - 3.5, that would shorten things by a good 31 entries, but the list would still be pretty long. If we were to remove all non-Release alpha/beta/RC items from every version (even open ones) , that would help a bit and pare the list down to 32 items, but remember that we'd still be growing the list by 7-8 new entries for each new minor version. That means that just by the end of 2013, we'd add another 10 entries to be at 42. One of the reasons I like the idea of de-listing all of the versions for a minor release and replacing them with just an X.x (e.g. "3.5") is that it would slash the number of versions that are in EOL. If we did that for the table above, we'd be down to 38. If we then removed out-of-date non-Releases for 3.6. and 4.0, we'd cut the list to a svelte 17: --- (See in Summary) 3.3 3.4 3.5 3.6.0.4 release 3.6.1.2 release 3.6.2.2 release 3.6.3.2 release 3.6.4.3 release 3.6.5.2 release 3.6.6.2 release 4.0.0.3 release 4.0.1.2 release 4.0.2.2 release 4.0.3.1 rc 4.1.0.0.alpha0+ Master unspecified --- I think the shorter list is *much* more manageable for our users in a drop-down. I do agree that it shortens the list that can be selected when hunting-down the particular version that introduced the bug, but I think that the version field isn't ultimately the best tool for those purposes -- something like the Repro Table is a much better bet. Cheers, --R Appendix 1: --- (See in Summary) 3.3.0 Beta2 3.3.0 release 3.3.1 release 3.3.2 release 3.3.3 release 3.3.4 release 3.4.0 Beta1 3.4.0 Beta2 3.4.0 Beta3 3.4.0 Beta4 3.4.0 Beta5 3.4.0 RC1 3.4.0 release 3.4.1 RC1 3.4.1 RC2 3.4.1 release 3.4.2 RC1 3.4.2 RC2 3.4.2 release 3.4.3 RC1 3.4.3 release 3.4.4 RC1 3.4.4 release 3.4.5 RC1 3.4.5 release 3.4.6 RC1 3.4.6 release 3.4 Daily 3.5.0 Beta0 3.5.0 Beta1 3.5.0 Beta2 3.5.0 Beta3 3.5.0 RC1 3.5.0 RC2 3.5.0 release 3.5.1 RC1 3.5.1 release 3.5.2 RC1 3.5.2 release 3.5.3 RC1 3.5.3 release 3.5.4 RC1 3.5.4 release 3.5.5.1 rc 3.5.5.2 rc 3.5.5.3 release 3.5.6.1 rc 3.5.6.2 release 3.5.7.1 rc 3.5.7.2 release 3.5 Daily 3.6.0.0.alpha1 3.6.0.0.beta1 3.6.0.0.beta2 3.6.0.0.beta3 3.6.0.1 rc 3.6.0.2 rc 3.6.0.3 rc 3.6.0.4 release 3.6.1.1 rc 3.6.1.2 release 3.6.2.1 rc 3.6.2.2 release 3.6.3.1 rc 3.6.3.2 release 3.6.4.1 rc 3.6.4.3 release 3.6.5.2 release 3.6.6.1 rc 3.6.6.2 release 4.0.0.0.alpha0+ Master 4.0.0.0.alpha1 4.0.0.0.beta1 4.0.0.0.beta2 4.0.0.1 rc 4.0.0.2 rc 4.0.0.3 release 4.0.1.1 rc 4.0.1.2 release 4.0.2.1 rc 4.0.2.2 release 4.0.3.1 rc 4.1.0.0.alpha0+ Master Master old -3.6 unspecified ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] Role of the QA calls
Rainer Bielefeld píše v Čt 25. 04. 2013 v 19:40 +0200: > Hi, > > in discussions I sometimes read statements like „...formal vote next > call”. I think here some clarification is required. I agree that the wording is not ideal and we should improve it. The processes are often just suggested and you could do it your way if it does not break others and if it does not create mess in bugzilla. I hope most of the proposed changes are not in conflict with your processes. In each case, many of the QA call decisions are about tasks for the participants and they do not affect others that much. For example, the contest preparation, talkyoo problems, attachments to FDO. Some others are about processes that were discussed on the mailing list where it was hard to reach a conclusion. IMHO, it is worth to get into a conclusion and the call is the most effective solution. But it does not mean that it is the final solution. Everything can be improved and changed. Finally, we have started to discusse things about the formal structure of the team recently. It is just proposal and I hope that we will not end there. IMHO, open source projects can't be successful with any deep and complex structure of decision making. And you are right that this project is meriocratic. Well, we need a group of people that will drive forward some long pending activities, who would make an agreement when there is long discussions on the mailing list, many different ideas and no conclusion. We also need someone who would actually do the action or document it. I think that the QA call is the best place. We need to have the most active people on the call. I am very sad that you could not make it. > The working sphere in the LibO project caused considerable discomfort > for me, and because I doubt that this incompatibility can be solved, I > decided to leave the LibO project. In future I will contribute to Open > Source PLC Programming Libraries. I am really sad to read this. I hope that we could change this and make you happy again. Best Regards, Petr ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] After EOL has been reached...
Hi, On Fri, Apr 26, 2013 at 09:47:28AM +0200, Jean-Baptiste Faure wrote: > Le 19/04/2013 23:21, Robinson Tryon a écrit : > > Per discussion at the meeting today, we generally agreed to the following: > > > > AGREED: For users inquiring about tech support after the EOL date of a > > release, we politely indicate that the release is EOL and ask them to > > upgrade to a new version (or go talk to their vendor/distro/paid tech > > support) > > What do you mean by "users inquiring about tech support" ? > > Who is "we" here ? I think that subscribers of users mailing lists do > what they want and answer the questions they want. The user list does not provide tech support, though ;). This is about people complaining about bugs not getting a bugfix in an ancient and unsupported version of LibreOffice. If someone is using LibreOffice 3.5.x and reports a bug, it will not get fixed on 3.5 unless he has a support contractor of some kind that is able to provide him with a custom build. Note that this is invincible in principle as TDF will not release any further versions of 3.5.x. I think its good to make reporters aware of that. > > AGREED: We de-list versions in Bugzilla 6 months after the release has > > been EOLed > > I do not see any rationale for that. > What I suggest is to remove all RC and beta version from BugZilla as > soon as the corresponding final version has been published. Hmm, sounds like a good idea too. If someone triages it even further (e.g. a bug was not there in 3.6.2 rc1 but is in the final version), a comment in the bug might suffice. While the info is highly valuable to a developer trying to fix the bug (in which case a reading of comments happens anyway), it is not something we can query bugzilla for esp. since the version we set in bugzilla is the earliest _known_ version to have the bug. So a bug reported against 3.6.2 rc1 can just as well have already been there in 3.6.1 final. That being said, I expect most regressions in a minor update (4.0.2->4.0.3) to be in the RC1 already, and most regressions in a major update (4.0.x->4.1.x) to be in the beta already anyway. > You know, we have users who work with LO 3.3.x and others who plan to > upgrade to LO 3.5.7 at the end of this year ... They don't decide for > themselves. Well, they are running unsupported versions of LibreOffice. For these reports to be interesting for the LibreOffice project at least the following has to be true: The bug has to be reproducable in a supported version of LibreOffice. If you have a bug in 3.5.x and its fixed in 3.6.x everything is fine from the perspective of the LibreOffice project. If you want a fix for 3.5 you need a custom build from a support partner (or do the fix yourself: this is open source). We could consider leaving in the latest version of each major series (e.g. 3.5.7, 3.4.6), even if unsupported to keep it easy to mark a bug a being around for longer. Note however that even this actually _lowers_ the priority of a bug anyway: If something is a recent regression (e.g. broken in 4.0, worked in 3.6) it has priority over something that is broken since the dawn of time. Note that the reduction of the numbers of versions to select from is valuable in itself. I find myself being annoyed by scrolling through that long list in a small box, and imagine it is even more irritating to e.g. a first time bug reporter. Best, Bjoern ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] After EOL has been reached...
Hi, Le 19/04/2013 23:21, Robinson Tryon a écrit : > Per discussion at the meeting today, we generally agreed to the following: > > AGREED: For users inquiring about tech support after the EOL date of a > release, we politely indicate that the release is EOL and ask them to > upgrade to a new version (or go talk to their vendor/distro/paid tech > support) What do you mean by "users inquiring about tech support" ? Who is "we" here ? I think that subscribers of users mailing lists do what they want and answer the questions they want. > > AGREED: We de-list versions in Bugzilla 6 months after the release has > been EOLed I do not see any rationale for that. What I suggest is to remove all RC and beta version from BugZilla as soon as the corresponding final version has been published. You know, we have users who work with LO 3.3.x and others who plan to upgrade to LO 3.5.7 at the end of this year ... They don't decide for themselves. I think that removing old versions from bugzilla will give a bad picture how the technical contributors consider the users community. Best regards. JBF -- Seuls des formats ouverts peuvent assurer la pérennité de vos documents. ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/