On Sun, Jan 13, 2013 at 2:50 PM, Martin Gräßlin <mgraess...@kde.org> wrote: > Hi all, > > *warning* long mail! Please take time reading it. Given how long I am now > writing on it: schedule half an hour or so ;-) > > I will try to formulate some thoughts on what I think might help to improve > the Quality Management in Plasma. Some ideas are based on what works well in > KWin. > > First of all I want to say that I find it awesome how many mentioned Quality > in Aaron's 4.10 reflection thread. It means that we identified an area for > improvement. I think that's the most important part to actually get something > changed. And considering where we have been just a few years ago, I find it > even more awesome that we consider Quality as the issue of the release cycle. > We are complaining here on a very high scale. Yes we had regressions, but we > also fixed them in time for the release. > > I think the problem with our QM process is, that we don't have a tool to > support it. Our bugtracker is (in it's current state) totally useless. Let me > just show a few stats for Plasma: > * 858 bugs (including wishlist) created since 4.9.0 tagging > ** 343 are marked as resolved > *** 43 (!) are fixed - that are 5 % of all reported bugs > *** 29 are invalid > *** 222 are duplicates - a quote of 26 % > ** 384 are still unconfirmed (45 %) > *** 96 of those are crash reports > ** 130 of the not resolved bugs are still in component general (25 % of the > not resolved bugs) > > now for comparison I show the stats of KWin: > * 432 bugs (including wishlist) created since 4.9.0 tagging > ** 329 are marked as resolved > *** 87 are fixed - that are 20 % of all reported bugs > *** 28 are invalid > *** 149 are duplicates - a quote of 34 % > ** 42 are still unconfirmed (9.7 %) > *** 4 of those are crash reports > ** 17 of the not resolved (or needsinfo) bugs are still in component general > > Two people work on the KWin bugs. Plasma gets twice the amount of bug reports > than KWin, so it would need 4 people to keep the bug tracker in a managable > state. Looking at the stats it would need 4 people triaging one bug per day to > keep it clean. That is doable! > > I have two requirements for bug reports: > 1. There should not be any unconfirmed bugs > 2. There should not be any bugs in component general > > What is an unconfirmed bug? It's a bug which has not been reproduced/verified > that it is valid! A bug report should not stay in this status for a long time. > Our users spend time to report it, so we should process them. If the user > provided the correct input, it should be possible to get the bug to status > "CONFIRMED". If the user cannot provide the necessary data to get the bug into > status CONFIRMED, it has to be set to RESOLVED WAITINGFORINFO. A bug should > never be longer than one week in status UNCONFIRMED without any action. If the > user provided input which is not sufficient: ask for more. If he cannot > provide it -> WAITINGFORINFO. Yes it sounds harsh, but you want to keep the > tracker clean. > > What is a bug in component general? A bug that applies to everything of > Plasma? Impossible! There is no general. A bug in general means that it is > unknown where it belongs to. It should be moved into the best fitting > component. If it doesn't exist create a new component! But don't keep it in > general - nobody is responsible for those bugs. > This morning I spent an hour going through the list of "general", I found a few problems.
There were several bugs I looked at where the relevant component didn't exist (webslice plasmoid, timer plasmoid). All plasmoids should have a component, it's much easier to manage lots of small lists than one big one, especially for delegating. I created components for the items I found (hope that's OK), and will continue to do so. If anyone else finds one of these, ping someone with bugzilla superpowers (like me) There are a lot of crashes filed against general, looking at the backtrace it's impossible to see who's at fault. The backtrace only shows some timer firing, or a python binding call or something generic - it's impossible to tell what's at fault. I have no idea how we can fix this. Some bugs weren't even against plasma (as devs know it). As we use the phrase "plasma workspaces" I guess it's natural for bugs to end up here which are totally generic. I found bugs on kwin, dolphin, and other KDE things. I don't think there's really a fix for this. There's a lot of wishlists. I found these very hard to triage because I can't say on someone else's project saying "we're never doing this, wontfix", which I would do on my projects (maybe slightly more politely). I imagine many others are in the same position, how can we manage this? > Now the action items I would suggest to do: > > *Re-evaluate all components for Plasma* > Let's check whether the components we have are still valid and fitting. Let's > rename what can be improved and create what makes sense to create. Maybe even > rethink whether it makes sense to split the product "plasma" into multiple > products. > > *Find a maintainer for each component* > The current maintainer model of Aaron and Marco for everything doesn't scale. > We have so many people who could take over maintainership for a small > component. Whoever is maintainer of a component, should become the default > assignee for bugs in that component. No longer a one address for all assignee. > Let's make the people responsible for their components. > > *Assign some "super" maintainers* > Having a maintainer for each component carries the risk of people dropping out > without being noticed and that would render the component unmaintained > (example: all the kde-workspace components currently maintained by Lubos). We > need some people who regularly check whether there is activity and poke the > right people when lots of bugs get unmanaged (hey Jimmy are you still working > on those? No? Should we get a new maintainer?). In Bugzilla talk this is > called a QA contact IIRC. Random thought: give those super maintainers the > power to pull the plug on a component if the quality decreases. > > *Restrict the available version numbers* > At KWin we only allow bugs to be reported for: > * the latest beta/RC > * the last two minor versions of the current stable > * the last two minor version of the last stable > That is currently: 4.8.4, 4.8.5, 4.9.4, 4.9.5, 4.10 RC2 > > We won't do a release for 4.8 or 4.9 any more, so we hardly care about these > versions. There are just too many changes to properly support those versions. > I would restrict even further, but there are users out there and it's not nice > telling them that the distro they installed two months ago is already so > outdated that we don't support it. The restriction on the latest two minor > versions is to catch all the .1 bug reports we get through distro install > media, but are fixed in .2 or .3. It helps the user to tell them: look there > are bug fix releases, you might want to update. > > *Close all existing bug reports* > We are currently in the process of developing Plasma 2 and porting everything > to QML. Do we really care about a bug reported for the C++ version? Do we > really care about a crash report which is for an old version and the backtrace > doesn't match the code any more? Do we really want to spend person years of > going through the bugs to confirm them? Now with the step towards Qt 5/KF > 5/libplasma2/QML we have the chance to do the clear cut. We did it before > (KDesktop/Kicker). Let's do it again. I'm pretty sure our users prefer the > honesty that we won't work on it then having the bug open for so long. > I just checked and there are 811 reports open which haven't changed over the > last 365 days (and all of them are wishlist). And there are an additional 561 > bugs which have not changed over the last half year (no comment, nothing). > > That's all for the bugzilla part, now let's do some more general ideas I have > :-) > > *Every commit should be referenced to a bug* > What is the motivation for a commit? It's either a bug fix or it is a new > feature/improvement. If it's a bug it's clear that there has to be a bug > report for it (out of experience: there is always a bug, if not: create it). > If it's a feature, it should also have a feature request in the tracker. > Create it yourself if you need to. Sounds beaurocratic, but it comes quite > handy as it allows to generate changelogs from it, allows people to easily > test new features. > > *Bug fixes should come with a unit test* > Plasma so far does not have any unit tests. Why? Let's not fool ourself with > "it's UI, that's difficult to test". No it's not, because Plasma has a great > separation between UI and logic. It would not be a problem to create a unit > test for the data engines. If there is a bug do yourself the favor of creating > the test. It helps you understanding the problem and fixing it. Also remember: > we have now a Jenkins installation running the tests after each commit, so we > see when they break (currently 3 tests in kde-workspace are failing). > > *New features should come with unit tests* > If you work on something new it's easy to restructure the code in a way that > it gets testable. New logic should be covered by tests. GUI is difficult, > though we could spend a GSoC on getting a test framework. > > So that's it. Looking forwards to your comments on it :-) > > Cheers > Martin > > _______________________________________________ > Plasma-devel mailing list > Plasma-devel@kde.org > https://mail.kde.org/mailman/listinfo/plasma-devel > _______________________________________________ Plasma-devel mailing list Plasma-devel@kde.org https://mail.kde.org/mailman/listinfo/plasma-devel