On 11/19/2015 11:44 AM, Bernd Schmidt wrote:
On 11/19/2015 07:08 PM, David Malcolm wrote:
gcc_assert terminates the process and no further testing is done,
whereas the approach the kit tries to run as much of the testsuite as
possible, and then fail if any errors occurred.
Yeah, but let's say someone is working on bitmaps and one of the bitmap
tests fail, it's somewhat unlikely that cfg will also fail (or if it
does, it'll be a consequence of the earlier failure). You debug the
issue, fix it, and run cc1 -fself-test again to see if that sorted it out.
As I said, it's a matter of taste and style and I won't claim that my
way is necessarily the right one, but I do want to see if others feel
the same.
I was originally going to say that immediate abort would be the
preferred method of operation, but as I thought more about it....
I think this really is a question of how the tests are likely used. I
kind of expect that most of the time they'll be used as part of an early
sanity test.
So to continue with the bitmap botch causing a CFG failure, presumably
the developer was mucking around in the bitmap code already and when
they see the CFG test failure, they're going to suspect they've mucked
up the bitmap code in some way.
The first question should then be did the bitmap tests pass or fail and
if they passed, then those tests clearly need extending :-)
The patch kit does use a lot of "magic" via macros and C++.
Taking registration/discovery/running in completely the other direction,
another approach could be a completely manual approach, with something
like this in toplev.c:
bitmap_selftest ();
et_forest_selftest ();
/* etc */
vec_selftest ();
This has the advantage of being explicit, and the disadvantage of
requiring a bit more typing.
The one advantage of explicit registration I see is the ability to order
the tests so that the lowest level data structures are tested first,
moving to increasingly more complex stuff.
But if we're in a mode of run everything, then ordering won't be that
important.
In the end I think I lean towards run everything with automatic
registration/discovery. But I still have state worries. Or to put it
another way, given a test of tests, we should be able to run them in an
arbitrary order with no changes in the expected output or pass/fail results.
jeff
