Re: Triage for AOO 4.0.0 Release Blockers
>IMHO there are no bad ideas here. They are just mistimed ;-) It would be mad to change the bug policy now, but for the future it might have an impact. 2013/6/21 Rob Weir > On Fri, Jun 21, 2013 at 11:09 AM, Sub Phil wrote: > > Bug policy debate and so on.. > > > > Well, just sharing experience from as a past developper and notorious bug > > fighter. > > > > There is no other point in creating a database of bugs and not treating > > them, than to have an ultra-complicated list of known issues (so complex > > that users report are duplicated) and frustrated reporters (and testers). > > > > "It is no a display of disrespect when a certain bug is not fixed." > > Yes until you have a list of open bugs, which generated itself a list of > > duplicates... > > What matters is the evolution ratio closed / confirmed by release (e.g. > > 3.4.1) and cumulated version (3.4.x). > > > > Metrics like this could be useful if we're measuring defects not > defect reports. Unfortunately the duplicates, plus the "how do I?" > and other non-defects that users enter , make this difficult to > estimate. My impression is that close to half of the Bugzilla reports > are invalid. We'd need to do a major clean up of Bugzilla before we > even knew how many actual defects are there. > > > "I would like to add another one. I am a developer. When I spend time > on > > fixing bugs then I sometimes wish there where a big list of all open bugs > > that are sorted according to importance. I would go down the list and > grab > > a bug from the top that lies in the area of my expertise (Impress, UI, > > Slideshow, Sidebar). Without such a list I have to prioritize bugs by > > myself. And if I do that then I use my own judgement of which bug is > > important and which is not." > > > > To the extent that rated priority is a motivation to volunteers, this > is true. But we have all sorts of volunteers. I think some of them > have enough stress in their real lives that they don't want to take > "high priority" issues and world rather work on lower priority ones, > where there is less urgency. > > Also, much of this depends on where we are in the release. Once we > have completed regression testing we want to avoid introducing > additional risk. So it is no longer "open season" for fixing just any > defect. We need to weigh the risks involved. In many cases, for less > serious bugs, we fix them in a later release, so they can undergo a > full test pass. > > > This further illustrate the fact that in order to handle these list, one > > has introduced techniques such as priority and vote for bug. > > > > My own experience shows, that all these are not good policy for bug > > treatment, it's a dispair. > > > > Prioritization is not despair. It is just acknowledging that changing > the code after testing has completed is risky, and that we should only > make changes now where absolutely necessary. Despair is what users > feel when we don't show such precautions and see a last minute fix > introduce a serious bug that is not caught before release. > > > Bug should be assigned in groups regarding a feature with final objective > > in mind. > > What do I mean by feature. > > Take for instance in Calc, the pivot table function. > > All bug regarding pivot table should be treated together, or most of > them. > > Why concentrating? > > Because: > > 1/ It takes time to fix a bug, because one needs to understand or > > re-understand the implementation. > > "- Fixing a bug usually takes a lot more time and effort than reporting > one. > > Also, often because test case scenario are not narrow enough - see my > other > > post. > > > > 2/ By taking all bugs related to a piece of source code, each time you > > become more familliar and so more efficient. In a sense you proof reading > > it several times, you get leverage on bug handling. > > > > 3/ By reviewing several related bugs, you reduce the risk or regression, > as > > you've been through it several times, each time with a deeper > understanding. > > > > This is a good practice to follow, before the main test pass > completes. After than we intentionally aim to minimize the changes to > the code. Otherwise we don't have a good sense of what the quality is > we're release. > > Bugs are bad, but with testing we can at least have a degree of > confidence that there are no defects above a certain severity level. > If we make unrestrained changes after the test pass th
Re: Triage for AOO 4.0.0 Release Blockers
Bug policy debate and so on.. Well, just sharing experience from as a past developper and notorious bug fighter. There is no other point in creating a database of bugs and not treating them, than to have an ultra-complicated list of known issues (so complex that users report are duplicated) and frustrated reporters (and testers). "It is no a display of disrespect when a certain bug is not fixed." Yes until you have a list of open bugs, which generated itself a list of duplicates... What matters is the evolution ratio closed / confirmed by release (e.g. 3.4.1) and cumulated version (3.4.x). "I would like to add another one. I am a developer. When I spend time on fixing bugs then I sometimes wish there where a big list of all open bugs that are sorted according to importance. I would go down the list and grab a bug from the top that lies in the area of my expertise (Impress, UI, Slideshow, Sidebar). Without such a list I have to prioritize bugs by myself. And if I do that then I use my own judgement of which bug is important and which is not." This further illustrate the fact that in order to handle these list, one has introduced techniques such as priority and vote for bug. My own experience shows, that all these are not good policy for bug treatment, it's a dispair. Bug should be assigned in groups regarding a feature with final objective in mind. What do I mean by feature. Take for instance in Calc, the pivot table function. All bug regarding pivot table should be treated together, or most of them. Why concentrating? Because: 1/ It takes time to fix a bug, because one needs to understand or re-understand the implementation. "- Fixing a bug usually takes a lot more time and effort than reporting one. Also, often because test case scenario are not narrow enough - see my other post. 2/ By taking all bugs related to a piece of source code, each time you become more familliar and so more efficient. In a sense you proof reading it several times, you get leverage on bug handling. 3/ By reviewing several related bugs, you reduce the risk or regression, as you've been through it several times, each time with a deeper understanding. "The other thing to note is that bug fixes are not risk-free. Studies have shown that 25% of defects in code are side effects of changes". FFmpeg is really a good example for regression, to a point I stopped reporting (first you have to convince it's a bug, then it lays pending to a wish to fix it and then you find older release which works...) 4/ Having fixed a bulk of related feature bugs, report the fix is implemented to all different reporters. They will(hopefully if done in a reasonable time, not like me who got a mail for a bug I posted 2 years ago...) test "their" bug and by doing so improve the total quality control and reduce regression risk. What do I mean by assigning by objective. Well, here is for me where the votes should take place, grouped by user, tester and developer (might have different interests). Suppose, pivot table code is considered as a mess by developpers and ought to be re-written by next release. There is no point in fixing pivot table bug now. Suppose users consider that priority should be assigned in this order 1. fix doc format issues 2. fix grammar check issues It has to be respected and if there is a "critical" bug in another field, well if it will wait. One should not vote for a bug, but rather for an expected behaviour. Phil 2013/6/21 Andre Fischer > On 21.06.2013 13:42, Edwin Sharp wrote: > >> Dear Andre >> This is exactly the problem! >> When you grab a bug from the top and critical bugs continuously enter the >> list - >> The minor bugs at the bottom will never get attention. >> > You make that sound like it where a bad thing. > A bug is critical because it affects man users and/or causes data loss > (and one or the other additional property). Therefore it should be fixed > before minor bugs are fixed. > This is how it should work. If it does not then I see two reasons for it: > > - There is not one single sorted list of bugs. Everybody has his or her > own idea of which bug is important and which isn't. > > - The way in which bugs are ordered is not good enough. This ordering is > certainly not easy to define and it will be impossible to define it in a > way that makes everybody happy. But if there where one central list of > ordered bugs then we could at least try. And everybody could help to > improve it. With such a list you have to rely on my judgement or that of > other developers. > > -Andre > > >> On Fri, Jun 21, 2013, at 14:24, Andre Fischer wrote: >> >>> >>> All good ideas. >>> >>> I would like to add another one. I am a developer. When I spend time >>> on fixing bugs then I sometimes wish there where a big list of all open >>> bugs that are sorted according to importance. I would go down the list >>> and grab a bug from the top that lies in the area of my expertise >>> (Impress, UI, Slideshow, Sideba
Re: Triage for AOO 4.0.0 Release Blockers
Regarding "classification" of bugs (critical, etc.) and bug fixing. The core question is how to increase the throughput of bug fixing, i.e. increase the number of fixes? If one analyse bug reporting process & fixing, it is usually done as 1/ User report bug 2/ Tester try to reproduce it, then 2.1/ confirm or not 2.2/ sets a priority 3/ Some developper will fix it when has time and very othen if feels like it, instead of developping some new features (s)he likes. This is a very bad way to handle bug and discourage reporter, tester and possibly developper. Indeed, often bug are fixed weeks later, not months if ever and priority are very subjective. How to improve that? 1/ First by reducing time between bug reporting and fixing and only delaying fixing when it's too ressource consumming. Work on it as it is hot reported and user expect to verify the fix. 2/ By notifing bug reporter of current ressources (for instance, red flag lack ressouce so handles only critical bugs) How to improve the time between bug reporting and fixing? By cracking down the problem to its core root. Irrelevant of the bug seriousness, there are user familliar with bug reporting and some not. So the first role of the bug tester is to validate it and have a test scenario. When there is a test scenario, then 6 things are often short-circuited, but would accelerate the bug fixing process. 0/ When lacking time, well how frequent in its USE does the bug appear, i.e. is it rather of general use or rather specific to some expert user? 1/ Simplify the reproducable bug test case to its core form. For instance, if it's a spreadsheet. Are all columns usefull? No, delete the rest. Are all lines usefull? No, delete the rest. Is the formating relevant? No, remove it. Is it file format specific? Is it language specific? Etc. Snif the trail of the bug to find its root. 2/ Narrow down the process involved to generate the bug, in order to identify the precise component For example, In the process to reproduce the bug, are they steps one can omit, what about changing the order sequence. 3/ If applicable, make a run-time debug run to narrow down which component of the code has the problem. See my post on run-time debug & tracing tools. Try to report something as Crash when executing line 1234 of file blabla.c 4/ Set a priority according the IMPACT it has one the confidence one has in the product. Examples. * Imagine Calc can't handle CSV files, with semi-colon as delimiter. Well, annoying, but there are temporary work around. * Now suppose Calc, take https://issues.apache.org/ooo/show_bug.cgi?id=119836 Won't you fear to use pivot? 5/ Ideally, make a macro to run the test case. Yours, Philippe 2013/6/21 Oliver-Rainer Wittmann > Hi, > > On 21.06.2013 08:57, Jürgen Schmidt wrote: > >> On 6/20/13 8:01 PM, Edwin Sharp wrote: >> >>> Yes. please allow to share the following objections: 1. in general, >>> bugs shouldn't be rated >>> >> >> I disagree here and you always rate issues starting with setting a >> priority. And as always you can't fix them all with a fix number of >> developers. It simply doesn't work. That means you have to >> prioritize and that is quite normal. >> >> 2. most are crashes - "release blocker" can be other than crash - >>> i.e. copy chart from Calc into Writer results in chart area without >>> curve. this is a huge bug that doesn't involve a crash >>> >> exactly and we have defined some criteria for issues, crashes, data >> loss and security issues have always high priority. Regressions are >> also very important especially when often used functionality is >> broken. >> >> It is documented in the wiki but I haven't the reference in place >> yet, have to look for it. >> >> 3. with all the respect to Norwegian users, print dialog too wide >>> is a negligible problem >>> >> not for this specific user group and the fix is already available. >> >> 4. there should be a documented evidence to the bug evaluation >>> process, based on approved SOP >>> >> I am not sure if I understand you here >> >> 5. the general public has no declared release date - no need to >>> rush as there is no commitment to fulfill >>> >> no but the project have committed to a release date and I believe >> the majority of the community support the new release. You have to >> set a date and have define what is important for this date. You can >> always do more and can fix more issues but that won't work very >> well. >> >> 6. "minor" bugs should also get attention and be targeted >>> >> definitely but not short in front of a release. And of course nobody >> prevent anybody from fixing such issues. We have a lot of them, many >> are probably not longer valid and can be closed, other are not easy >> to fix and so on. A good start would be of course to simply check >> older issues if they are still valid or not. >> >> 7. the judgment of introducing the sidebar with so many pending >>> bugs should be justified to demonstrate openness >>> >> as far a
TOOLS for (runtime) tracing, debbuging and performance measurements
Not exhausitive list FYI: * function call tree with gcc http://ndevilla.free.fr/etrace/ Remark : Building cumulative time lapse statistics of functions calls is a great way to pinpoint the area of crucial improvements. Improving a function runtime by 0.01 sec might seem ridiculous, but if this function is called 1 million times, then it comes very relevant. * http://stackoverflow.com/questions/659780/what-is-your-favorite-open-source-debugging-tool * http://en.wikipedia.org/wiki/Comparison_of_debuggers * http://www.testingtv.com/tag/open-source-tools/
