[Bug testsuite/25352] xfail within dg-do command has no effect
--- Comment #10 from janis at gcc dot gnu dot org 2007-11-01 16:22 --- Subject: Bug 25352 Author: janis Date: Thu Nov 1 16:22:36 2007 New Revision: 129823 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=129823 Log: gcc/ PR testsuite/25352 * doc/sourcebuild.texi (Test Directives): Document that xfail has no effect for dg-do except when dg-do-what is "run". libstdc++-v3/ PR testsuite/25352 * testsuite/26_numerics/headers/cmath/c99_classification_macros_c.cc: Remove xfail from dg-do and add dg-skip-if. gcc/testsuite/ PR testsuite/25352 * gcc.c-torture/compile/2804-1.c: Remove xfail from dg-do and add dg-skip-if. * gcc.c-torture/compile/20001205-1.c: Ditto. * gcc.c-torture/compile/20001226-1.c: Ditto. * gcc.c-torture/compile/20020312-1.c: Ditto. * gcc.c-torture/compile/20020604-1.c: Ditto. * gcc.c-torture/compile/920501-12.c: Ditto. * gcc.c-torture/compile/920501-4.c: Ditto. * gcc.c-torture/compile/920520-1.c: Ditto. * gcc.c-torture/compile/980506-1.c: Ditto. * gcc.c-torture/compile/990617-1.c: Ditto. * gcc.dg/compare6.c: Ditto. * g++.old-deja/g++.brendan/crash43.C: Ditto. * g++.old-deja/g++.brendan/synth1.C: Ditto. * g++.old-deja/g++.oliva/dwarf2.C: Ditto. * g++.old-deja/g++.oliva/dwarf3.C: Ditto. * g++.old-deja/g++.pt/static6.C: Ditto. * g++.old-deja/g++.pt/crash16.C: Ditto. * g++.old-deja/g++.mike/p10416.C: Ditto. * g++.old-deja/g++.mike/ns15.C: Ditto. * g++.old-deja/g++.mike/eh46.C: Ditto. Modified: trunk/gcc/ChangeLog trunk/gcc/doc/sourcebuild.texi trunk/gcc/testsuite/ChangeLog trunk/gcc/testsuite/g++.old-deja/g++.brendan/crash43.C trunk/gcc/testsuite/g++.old-deja/g++.brendan/synth1.C trunk/gcc/testsuite/g++.old-deja/g++.mike/eh46.C trunk/gcc/testsuite/g++.old-deja/g++.mike/ns15.C trunk/gcc/testsuite/g++.old-deja/g++.mike/p10416.C trunk/gcc/testsuite/g++.old-deja/g++.oliva/dwarf2.C trunk/gcc/testsuite/g++.old-deja/g++.oliva/dwarf3.C trunk/gcc/testsuite/g++.old-deja/g++.pt/crash16.C trunk/gcc/testsuite/g++.old-deja/g++.pt/static6.C trunk/gcc/testsuite/gcc.c-torture/compile/2804-1.c trunk/gcc/testsuite/gcc.c-torture/compile/20001205-1.c trunk/gcc/testsuite/gcc.c-torture/compile/20001226-1.c trunk/gcc/testsuite/gcc.c-torture/compile/20020312-1.c trunk/gcc/testsuite/gcc.c-torture/compile/20020604-1.c trunk/gcc/testsuite/gcc.c-torture/compile/920501-12.c trunk/gcc/testsuite/gcc.c-torture/compile/920501-4.c trunk/gcc/testsuite/gcc.c-torture/compile/920520-1.c trunk/gcc/testsuite/gcc.c-torture/compile/980506-1.c trunk/gcc/testsuite/gcc.c-torture/compile/990617-1.c trunk/gcc/testsuite/gcc.dg/compare6.c trunk/libstdc++-v3/ChangeLog trunk/libstdc++-v3/testsuite/26_numerics/headers/cmath/c99_classification_macros_c.cc -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25352
[Bug testsuite/25352] xfail within dg-do command has no effect
--- Comment #11 from janis at gcc dot gnu dot org 2008-05-13 22:19 --- Fixed, via documentation and workarounds. -- janis at gcc dot gnu dot org changed: What|Removed |Added Status|ASSIGNED|RESOLVED Resolution||FIXED http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25352
[Bug testsuite/25352] xfail within dg-do command has no effect
--- Comment #7 from ghazi at gcc dot gnu dot org 2007-05-10 17:44 --- Ben, Janis, Would you please offer an update on the status of this PR? I'm running into another situation where I need to xfail a dg-do link, and due to the issues discussed in this PR it doesn't work. Has dejagnu cvs sources been fixed to handle this case? Has 1.4.5 been released? (I don't see it on ftp.gnu.org). Thanks, --Kaveh -- ghazi at gcc dot gnu dot org changed: What|Removed |Added CC||bje at gcc dot gnu dot org http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25352
[Bug testsuite/25352] xfail within dg-do command has no effect
--- Comment #8 from janis at gcc dot gnu dot org 2007-05-11 00:51 --- I had forgotten all about this. Today I checked to see how older versions of DejaGnu and GCC tests handled xfail for dg-do keywords other than "run". I only went back as far as GCC 3.0.4 and DejaGnu 1.4, but they also ignored xfail in dg-do for preprocess/compile/assemble/link. There are currently 27 tests that use this construct: three in libstdc++-v3 and 24 in gcc. My recommendations are to document, in the description of test directives in the GCC Internals manual, that xfail for dg-do is only honored for dg-do run and ignored for other keywords, recommending the use of dg-xfail-if instead; and to remove the xfail section from the 27 tests that use it when it's ignored. I'll do these if it sounds reasonable. I don't think this is worth working around in the GCC testsuite support or fixing in DejaGnu. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25352
[Bug testsuite/25352] xfail within dg-do command has no effect
--- Comment #9 from ghazi at gcc dot gnu dot org 2007-05-15 17:47 --- (In reply to comment #8) > My recommendations are to document, in the description of test directives in > the GCC Internals manual, that xfail for dg-do is only honored for dg-do run > and ignored for other keywords, recommending the use of dg-xfail-if instead; > and to remove the xfail section from the 27 tests that use it when it's > ignored. > I'll do these if it sounds reasonable. I don't think this is worth working > around in the GCC testsuite support or fixing in DejaGnu. Sounds fine. I just want to clear up the confusion. There may be some similarly vague stuff in the Wiki (on the topic of testcases) that should be clarified as well IMHO. Thanks! -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25352
[Bug testsuite/25352] xfail within dg-do command has no effect
--- Comment #1 from ghazi at gcc dot gnu dot org 2005-12-11 15:14 --- We probably should have some sort of dg conformance test directory with simple tests ensuring all the directives work as advertised. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25352
[Bug testsuite/25352] xfail within dg-do command has no effect
--- Comment #2 from janis at gcc dot gnu dot org 2005-12-12 17:57 --- This is odd, if the test case has a syntax error the xfail works as expected, but with #error it doesn't. I'll look into it. Take a look at gcc/testsuite/gcc.test-framework, which tests lots of combinations of directives as well as specific cases. I run it as part of my nightly build of trunk. It's meant to be run separately from the rest of the testsuite since the outcome of a test is not the results that the test itself reports, but whether the result is the expected one. -- janis at gcc dot gnu dot org changed: What|Removed |Added AssignedTo|unassigned at gcc dot gnu |janis at gcc dot gnu dot org |dot org | Status|UNCONFIRMED |ASSIGNED Ever Confirmed|0 |1 Last reconfirmed|-00-00 00:00:00 |2005-12-12 17:57:54 date|| http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25352
[Bug testsuite/25352] xfail within dg-do command has no effect
--- Comment #3 from janis at gcc dot gnu dot org 2005-12-12 22:14 --- How embarrassing, I tried these things by adding new tests to gcc.test-framework and the awk script that checks the results was ignoring some of the results because of magic needed for the generated tests. Now I'm able to properly reproduce failures. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25352
[Bug testsuite/25352] xfail within dg-do command has no effect
--- Comment #4 from janis at gcc dot gnu dot org 2005-12-13 19:08 --- xfail within a dg-do command other than run has no effect, at least not with DejaGnu 1.4.4 used with GCC at least as far back as 3.0. The DejaGnu documentation doesn't mention the dg-* directives. The comments in dejagnu/dg.exp file implies that xfail should be recognized for all dg-do options. There are 50+ tests in gcc/testsuite that use xfail with dg-do assemble, compile, and link, so perhaps it worked at one time. The generated tests in gcc/testsuite/gcc.test-framework only test xfail with "dg-do run", and there are no permanent tests for xfail with dg-do and other actions. There's undoubtedly a way for the GCC dg scripts to capture the information that the test's dg-do had an xfail, although it will probably be ugly. I'll keep investigating to find out just how ugly it will need to be. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25352
[Bug testsuite/25352] xfail within dg-do command has no effect
--- Comment #5 from janis at gcc dot gnu dot org 2005-12-14 18:17 --- I've got an ugly fix that reaches up a couple of levels to get the variable from the DejaGnu proc that records that the test should be xfailed. Ben Elliston, who is a DejaGnu maintainer, says that 1.4.5 will be released in a month or so and it could be fixed there as well. Having GCC depend on a fix in DejaGnu would not be popular unless there were other compelling reasons to require people to upgrade. Thoughts? -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25352
[Bug testsuite/25352] xfail within dg-do command has no effect
--- Comment #6 from ghazi at gcc dot gnu dot org 2005-12-14 18:36 --- (In reply to comment #5) > Having GCC depend on a fix in > DejaGnu would not be popular unless there were other compelling reasons to > require people to upgrade. Thoughts? If the breakage is in dejagnu, certainly a fix should go there. However I don't feel strongly about whether to fix it in GCC additionally and separately. If forced to pick, I'd lean towards dejagnu only since I'll upgrade once the new version is out. If we don't fix it in GCC, then we should remove the documentation that says it works. Perhaps we could recommend dg-xfail-if instead? Either way we should probably continue to remove the existing dg-do xfails since we'll start getting a bunch of spurious XPASSes once people start upgrading to the new dejagnu. I'll try and handle the ones that trigger on my platforms but some I can't test to know whether simply to remove the xfail or change it to dg-xfail-if. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25352
