On 01/30/15 01:19, Jakub Jelinek wrote:
The biggest problem is that what fails and what does not varries between
targets and between optimization levels. Right now we have no way to xfail
test XYZ for -Os on x86_64-linux and for -O2 and -O3 on i686-linux ia32, and
the lists would become very large. Some tests in guality are xfaileded just
in case, even when they actually XPASS on many targets.
I thought we added that kind of capability a while back. There's still
significant potential for them to get unwieldy. The hope would be that
we'd have a set for x86, x86_64, aarch64, etc, but not have to do
anything special for the OS.
The way to look for regressions in the guality area, at least as I do it
regularly, is just compare test_summary results.
If we'd disable this by default, I'm sure our debug quality would sink very
quickly.
Yup. But it'd still be nicer if our test runs were cleaner.
jeff
