On 7/9/2004 4:57 PM, Paul Johnson wrote:
> On Fri, Jul 09, 2004 at 12:10:52PM -0500, Pete Krawczyk wrote:
>
>> Consider [code with unreachable path] Devel::Cover will always see that as
>> a partial test, and never a full test: Is that a bug, then?
That's for you to decide. The lack of coverage serves to bring it to your
attention so you can ask (and answer) that very question. As others have said,
coverage is just a tool. The amount of coverage you get does not (by itself) say
anything at all about the correctness of your code.
> There is some initial code in place in Devel::Cover to handle this situation,
> [...] Michael Carman is looking at making this more usable.
I am, but I hit a roadblock with compound conditionals.
A little background for those unfamiliar with the guts of Devel::Cover: The
.uncoverable file captures analysis data at the same level of granularity as
Devel::Cover uses internally -- each individual operation.
e.g. the expression '$a && $b || $c' is actually seen as two completely separate
conditions:
L op R
-------------------------
1) $a && $b
2) $a && $b || $c
The truth table you see when you look at the condition coverage report (or
generate a "unified" report) is rather crudely and painfully calculated from
these primitives.
The problem is that for compound expressions, the unreachable paths are really
part of the composite expression and are best analyzed there. e.g. you can't
drive the "1 0 0" combination.
The "simple" solution is probably to add a new type of entry to .uncoverable for
composite expressions that tells us which row of a truth table can't be reached.
It occurred to me that if I'm missing a path in a composite expression I should
be missing coverage for one (or more) of the simple expressions that feed into
it. If I were really clever I could use that to propagate coverage analysis data
when building a truth table. (The reverse mapping -- from a truth table to the
primitive ops that build it -- is much harder.)
This got me thinking about something that had tickled my thoughts before. I'm
afraid that I have uncovered a serious flaw/limitation in Devel::Cover. Take the
expression '$a && $b || $c'
Its truth table is:
a b c | Z
---------
0) 0 X 0 | 0
1) 0 X 1 | 1
2) 1 0 0 | 0
3) 1 0 1 | 1
4) 1 1 X | 1
Suppose for the moment that all rows except row 2 (1 0 0) can be driven. The
following (contrived) example does this:
#!/usr/bin/perl
use strict;
use warnings;
for my $a (0 .. 1) {
for my $b (0 .. 1) {
for my $c (0 .. 1) {
foo($a, $b, $c);
}
}
}
sub foo {
my ($a, $b, $c) = @_;
$a = 0 unless ($b || $c); # Make '1 0 0' unreachable
print "$a $b $c ";
if ($a && $b || $c) {
print "T\n";
}
else {
print "F\n";
}
}
__END__
Now run that with -MDevel::Cover and generate a report. Shows full coverage,
doesn't it? The reason is that to Devel::Cover, the "if ($a && $b || $c)" line
has two *independent* conditions:
a b | Y Y c | Z
------- -------
0) 0 X | 0 0) 0 0 | 0
1) 1 0 | 0 1) 0 1 | 1
2) 1 1 | 1 2) 1 X | 1
Independently, all rows of both tables can be covered. What can't happen is
driving row 1 of table 1 at the same time as row 0 of table 2 (these are the
states that correspond to the "1 0 0" state in the composite table).
The truth table showing "1 0 0" as covered is my fault. I wrote the portion of
the reporting back end that builds the table from the simple operations. I
thought that I could calculate the coverage for a composite row from the
coverage of the simple rows that feed into it. I can see now that that
assumption was wrong.
Unfortunately, the information necessary to correct this isn't available.
Devel::Cover would have to either track condition coverage at an expression
level (instead of an operation level) or keep track of which combinations of
simple coverage were witnessed.
-mjc
