On Friday, February 11, 2011 14:13:29 Steve Schveighoffer wrote: > The limit should be 100% coverage. If std.datetime passes that, it should > be trimmed.
The problem with that is that all code coverage indicates is whether each line of code is run. In some cases, that may be enough, but as soon as anything like math is involved, it certainly isn't. Just because you covered a particular code path doesn't mean that that code path works correctly. What if a particular value ends up on a particular code path when it should be on another, or if it _does_ belong on that code path, but the result is incorrect for that particular value even though it works with other values? Also, does code coverage take into account take into account the various specializations of a template? And even if it does, it could be that the template is correctly written for a particular type but not another, and yet both instantiate with it, generating the same code. You'd only catch that if you tested both of those types with the template. Code coverage is a useful metric, but I don't think that it grasps the whole picture at all. I think that a module which has 100% code coverage can still be buggy, because its tests didn't cover enough cases. There is a difference between testing every code path and testing that the function works correctly for all inputs. Obviously, you're not going to test for _all_ possible inputs - it's completely impractical to do so in most cases - but often, I don't think that even the number of inputs required to get 100% code coverage is sufficient. By the way, with how -cov is currently set up, getting 100% test coverage is often impossible anyway, simply because you end up with code paths with assert(0) in them which should _never_ happen and so that code path _never_ gets hit. A prime example would be a function which is nothrow but calls functions which can technically throw even though you know that they never will with the given input. You're pretty much forced to use a catch block with assert(0) in it. Even perfectly tested code will never hit that code path, because it should never actually happen if the code is logically correct. - Jonathan M Davis _______________________________________________ phobos mailing list [email protected] http://lists.puremagic.com/mailman/listinfo/phobos
