On Tue, May 29, 2012 at 06:58:53PM +0200, Bertrik Sikken wrote: > With respect to test strategy: In my opinion, we can start with > a set of rather basic tests to verify high-level behaviour, rather > than to go for test completeness. A test suite taking about (say) > one hour should keep the barrier low for people who want to > participate in testing.
I agree in general, but I think one hour for the basic set of tests is too much. I stronly believe we need to have a *very* resstricted set of basic tests that only tests basic playback, including things like playing a file from the file browser or database, pause, skip, volume change, and not much more. After that, we can (should) have more advanced tests, again divided into small sets, that ideally cover all functionality. The reasin I think this is better than larger chunks (at least for the basics) is that I think we mainly need *wide* testing. We apparently have targets (and I'd guess these days this goes for at least half our stable targets) that see no testing at all during a release cycle, and on some of those targets we then only get two or three comments after the release about major issues (such as sound not working at all). We *need* someone to test these targets, and if there are only (say) five people who might be tempted to do that, fifteen minutes (for starters, I hope they'll continue after the basic bit) is easier to sell than an hour. Frank > Kind regards, > Bertrik -- "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." - Brian W. Kernighan
