On Fri, Sep 23, 2011 at 1:51 PM, Sami Liedes <slie...@cc.hut.fi> wrote:
> > > For a decision of the form: > > > > if( A && B )... > > > > The test suite for SQLite tries at least three cases: > > > > (1) A false > > (2) A true and B false > > (3) A true and B true > > > > You seem to be saying that this is only MC/DC if we split case (1) into > two > > subcases: > > > > (1a) A false and B false > > (1b) A false and B true > > Hmm, no, not quite, although you do have a point here that in > shortcutting languages it's different. I would concede that in a > shortcutting language you do not need to test for (1a) and (1b). > > I think it means that for the decision of the form if( A && B ), you > additionally need to show that A and B both can independently affect > the outcome; For case (1), since B is uncomputable, we can deem it to be true. Then since case (3) has a different outcome from cases (1) and (2), we do show that each term is independent of the other. The fact that B is deemed true in case (1) might raise eyebrows, but in a shortcutting language, that's the best you can do, I think. > that is, > > For A: Either (2) or (3) results in different branch being taken from > (1) (which in this case would be true for (3), but the analysis is not > always so straightforward) > > For B: The branches taken in (2) and (3) are different (which is the > case here). > > So in case of if( A && B ), I think you are correct that these three > test cases suffice for MC/DC. But you *have* to consider the outcome > too for it to be MC/DC. > > Taking a look at the Wikipedia page[1] for MC/DC, I believe what you > describe is plain Condition/Decision Coverage, not MC/DC: > > ------------------------------------------------------------ > * Condition/Decision Coverage > > Every point of entry and exit in the program has been invoked > at least once, every condition in a decision in the program > has taken all possible outcomes at least once, and every > decision in the program has taken all possible outcomes at > least once. > > * Modified Condition/Decision Coverage > > Every point of entry and exit in the program has been invoked > at least once, every condition in a decision in the program > has taken on all possible outcomes at least once, and each > condition has been shown to affect that decision outcome > independently. A condition is shown to affect a decision’s > outcome independently by varying just that condition while > holding fixed all other possible conditions. The > condition/decision criterion does not guarantee the coverage > of all conditions in the module because in many test cases, > some conditions of a decision are masked by the other > conditions. Using the modified condition/decision criterion, > each condition must be shown to be able to act on the decision > outcome by itself, everything else being held fixed. The MC/DC > criterion is thus much stronger than the condition/decision > coverage. > ------------------------------------------------------------ > > > Your argument is that there might be redundancy; that if B is always > false > > when A is false, then it would be possible to simplify the decision to > just: > > > > if( B )... > > > > True enough. But there are countless ways to refactor a boolean > > expression. I don't think that MC/DC has any requirement that the number > of > > conditions be minimized, does it? > > I think that's one of the central ideas of MC/DC. Well, I think it > doesn't require *minimizing* per se, only that each condition > independently is enough to flip the outcome. (I now think I was wrong > to talk about subexpressions in my original post; The "conditions" are > the atomic conditions, not subexpressions.) I certainly cannot imagine > of another plausible reading of the fourth condition. Perhaps you have > a different plausible interpretation? > > In fact nearly all definitions of MC/DC I can find contain this > paragraph, which explains it quite well: > > "A condition is shown to affect a decision's outcome independently > by varying just that decision while holding fixed all other > possible conditions." > > > One of the key benefits we derive from testing SQLite to 100% branch > > coverage is that we end up testing the object code, not the source code. > So > > even if the compiler makes a mistake, we will still likely catch it. I > > don't think your more rigorous definition of MC/DC is necessary to > achieve > > that goal, is it? > > I certainly think that even 100% branch coverage (also called Decision > Coverage) is quite impressive in itself without MC/DC, and if what the > SQLite test harness does is plain Condition/Decision Coverage, I'm > sure that already places SQLite within the best tested 1% of software. > I'm happy and impressed with how tested it is, I just don't think it's > MC/DC :) > > As far as I can tell 100% MC/DC coverage is only required in highly > safety critical systems, most notably the DO-178B standard for > avionics for the most critical (Class A) software only, where failure > would be "catastrophic", i.e. result in planes falling from the sky. I > don't know how much it would make sense to strive for such a standard > in SQLite, unless someone really wants to use it in such a highly > critical system. MC/DC testing is often criticized for its high > cost, after all[2]. > > [1] http://en.wikipedia.org/wiki/Modified_Condition/Decision_Coverage > [2] http://sunnyday.mit.edu/papers/dupuy.pdf > > Sami > _______________________________________________ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > -- D. Richard Hipp d...@sqlite.org _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users