[google/4.6] Add xfails for arm-grtev2-linux-gnueabi (issue5794063)
Hi Diego This patch adds arm-grtev2-linux-gnueabi.xfail for our 4.6 branch so that we can track regressions. This just established the test baseline. The failures need to be investigated. -Doug 2012-03-12 Doug Kwan dougk...@google.com * contrib/testsuite-management/arm-grtev2-linux-gnueabi.xfail: New file. Index: contrib/testsuite-management/arm-grtev2-linux-gnueabi.xfail === --- contrib/testsuite-management/arm-grtev2-linux-gnueabi.xfail (revision 0) +++ contrib/testsuite-management/arm-grtev2-linux-gnueabi.xfail (revision 0) @@ -0,0 +1,74 @@ +# Failures in ./gcc/testsuite/gcc/gcc.sum: +# *** gcc: +FAIL: gcc.dg/automversn_1.c (test for excess errors) +UNRESOLVED: gcc.dg/automversn_1.c compilation failed to produce executable +UNRESOLVED: gcc.dg/automversn_1.c scan-tree-dump auto_clone foo.autoclone.original +UNRESOLVED: gcc.dg/automversn_1.c scan-tree-dump auto_clone foo.autoclone.0 +FAIL: gcc.dg/builtin-apply2.c execution test +FAIL: gcc.dg/tls/pr42894.c (test for excess errors) +FAIL: gcc.dg/torture/stackalign/builtin-apply-2.c -O0 execution test +FAIL: gcc.dg/torture/stackalign/builtin-apply-2.c -O0 execution test +FAIL: gcc.dg/torture/stackalign/builtin-apply-2.c -O1 execution test +FAIL: gcc.dg/torture/stackalign/builtin-apply-2.c -Os execution test +FAIL: gcc.dg/torture/tls/tls-test.c -O0 execution test +FAIL: gcc.dg/torture/tls/tls-test.c -O1 execution test +FAIL: gcc.dg/torture/tls/tls-test.c -O2 execution test +FAIL: gcc.dg/torture/tls/tls-test.c -O3 -fomit-frame-pointer execution test +FAIL: gcc.dg/torture/tls/tls-test.c -O3 -g execution test +FAIL: gcc.dg/torture/tls/tls-test.c -Os execution test +FAIL: gcc.dg/torture/tls/tls-test.c -O2 -flto -flto-partition=none execution test +FAIL: gcc.dg/torture/tls/tls-test.c -O2 -flto execution test +FAIL: gcc.dg/tree-prof/switch-case-1.c scan-rtl-dump-times expand Succ edge[^\n]*count:2000 1 +FAIL: gcc.dg/vect/vect-multitypes-11.c scan-tree-dump-times vect vectorized 1 loops 1 +FAIL: gcc.dg/vect/vect-multitypes-12.c scan-tree-dump-times vect vectorized 1 loops 1 +FAIL: gcc.dg/vect/vect-reduc-dot-s16b.c scan-tree-dump-times vect vectorized 1 loops 0 +FAIL: gcc.dg/vect/vect-reduc-pattern-1a.c scan-tree-dump-times vect vectorized 1 loops 0 +FAIL: gcc.dg/vect/vect-reduc-pattern-1b.c scan-tree-dump-times vect vectorized 1 loops 0 +FAIL: gcc.dg/vect/vect-reduc-pattern-1c.c scan-tree-dump-times vect vectorized 1 loops 0 +FAIL: gcc.dg/vect/vect-reduc-pattern-2a.c scan-tree-dump-times vect vectorized 1 loops 0 +FAIL: gcc.dg/vect/vect-reduc-pattern-2b.c scan-tree-dump-times vect vectorized 1 loops 0 +XPASS: gcc.dg/vect/slp-reduc-3.c scan-tree-dump-times vect vectorizing stmts using SLP 1 +FAIL: gcc.dg/vect/wrapv-vect-reduc-pattern-2c.c scan-tree-dump-times vect vectorized 1 loops 0 +XPASS: gcc.dg/vect/no-scevccp-outer-16.c scan-tree-dump-times vect OUTER LOOP VECTORIZED. 1 +XPASS: gcc.dg/vect/no-scevccp-outer-17.c scan-tree-dump-times vect OUTER LOOP VECTORIZED. 1 +XPASS: gcc.dg/vect/no-scevccp-outer-19.c scan-tree-dump-times vect OUTER LOOP VECTORIZED. 1 +XPASS: gcc.dg/vect/no-scevccp-outer-21.c scan-tree-dump-times vect OUTER LOOP VECTORIZED. 1 +FAIL: gcc.target/arm/pr42575.c scan-assembler-not mov +FAIL: gcc.target/arm/thumb-ltu.c (test for excess errors) +UNRESOLVED: gcc.target/arm/thumb-ltu.c scan-assembler-not uxtb + +# Failures in ./gcc/testsuite/gfortran/gfortran.sum: +# *** gfortran: +FAIL: gfortran.dg/pr45636.f90 -O scan-tree-dump-times forwprop2 memset 0 +FAIL: gfortran.dg/vect/fast-math-pr38968.f90 execution test +FAIL: gfortran.dg/vect/fast-math-pr38968.f90 scan-tree-dump vect vectorized 1 loops + +# Failures in ./gcc/testsuite/g++/g++.sum: +# *** g++: +FAIL: g++.dg/abi/forced.C execution test +FAIL: g++.dg/abi/local1.C execution test +FAIL: g++.dg/thread-ann/thread_annot_lock-82.C (test for warnings, line 47) +XPASS: g++.dg/uninit-pred-3_b.C (test for excess errors) +FAIL: g++.dg/tree-prof/mversn13.C execution,-g -fprofile-generate +UNRESOLVED: g++.dg/tree-prof/mversn13.C compilation, -g -fprofile-use +UNRESOLVED: g++.dg/tree-prof/mversn13.C execution,-g -fprofile-use +FAIL: g++.dg/tree-prof/mversn13.C execution,-O0 -fprofile-generate +UNRESOLVED: g++.dg/tree-prof/mversn13.C compilation, -O0 -fprofile-use +UNRESOLVED: g++.dg/tree-prof/mversn13.C execution,-O0 -fprofile-use +FAIL: g++.dg/tree-prof/mversn13.C execution,-O1 -fprofile-generate +UNRESOLVED: g++.dg/tree-prof/mversn13.C compilation, -O1 -fprofile-use +UNRESOLVED: g++.dg/tree-prof/mversn13.C execution,-O1 -fprofile-use +FAIL: g++.dg/tree-prof/mversn13.C execution,-O2 -fprofile-generate +UNRESOLVED: g++.dg/tree-prof/mversn13.C compilation, -O2 -fprofile-use +UNRESOLVED: g++.dg/tree-prof/mversn13.C execution,-O2 -fprofile-use +FAIL: g++.dg/tree-prof/mversn13.C execution,-O3
Re: [google/4.6] Add xfails for arm-grtev2-linux-gnueabi (issue5794063)
On 12/03/12 14:58 , Doug Kwan wrote: 2012-03-12 Doug Kwandougk...@google.com * contrib/testsuite-management/arm-grtev2-linux-gnueabi.xfail: New file. OK. Diego.
Re: [google/4.6] Add xfails for arm-grtev2-linux-gnueabi (issue5794063)
On Mar 12, 2012, at 11:59 AM, Diego Novillo wrote: On 12/03/12 14:58 , Doug Kwan wrote: 2012-03-12 Doug Kwandougk...@google.com * contrib/testsuite-management/arm-grtev2-linux-gnueabi.xfail: New file. OK. Hum, kinda makes be wish we had all the xfail files for the release branches and trunk for all primary and secondary platforms and that make check did something more intelligent. :-) One advantage, people doing day to day work, would rarely have to do two make check runs, as they could do just one and do the analysis against the checked in file. They would only have to do two, if the file isn't up-to-date. So, I guess the question is, is there a down side to doing that? I can imagine a if [ -r $srcdir/contrib/testsuite-management/$target.xfail ]; then $srcdir/contrib/testsuite-management/validate_failures.py bla bla fi added into make check somewhere.
Re: [google/4.6] Add xfails for arm-grtev2-linux-gnueabi (issue5794063)
On 12/03/12 15:27 , Mike Stump wrote: So, I guess the question is, is there a down side to doing that? I can imagine a if [ -r $srcdir/contrib/testsuite-management/$target.xfail ]; then $srcdir/contrib/testsuite-management/validate_failures.py bla bla fi Yeah. I had something like this in mind originally, but never followed through (we use the validator from within our own build harness). Ideally, though, we would not even need this hack. 'make check' should return 0 when every test is nominal. Period. Our own guidelines are the culprit here: '... , this means comparing post-patch test results to pre-patch results by testing twice or comparing with recent posts to the gcc-testresults list.' (http://gcc.gnu.org/contribute.html#testing). I have argued before that we should change this, but I am yet to do anything concrete about it. Diego.
Re: [google/4.6] Add xfails for arm-grtev2-linux-gnueabi (issue5794063)
On Mar 12, 2012, at 12:37 PM, Diego Novillo wrote: Ideally, though, we would not even need this hack. 'make check' should return 0 when every test is nominal. Period. Yeah, that pig has yet to achieve lift-off. :-)
Re: [google/4.6] Add xfails for arm-grtev2-linux-gnueabi (issue5794063)
On 12 March 2012 18:58, Doug Kwan dougk...@google.com wrote: Hi Diego This patch adds arm-grtev2-linux-gnueabi.xfail for our 4.6 branch so that we can track regressions. This just established the test baseline. The failures need to be investigated. Just out of curiosity, were these when you ran them cross on qemu or when you ran these native. It's probably worth noting that as well. There are times when you'll see differences in test results especially on recent trunk ( atomic tests depend on the version of gdb installed , cross testing on qemu pretty much means threaded tests are well let's say flaky) . This is probably something that ought to be recorded along with the environment in which the tests were run to ease comparison. This failure here suggests that you are runing on qemu. +FAIL: gfortran.dg/vect/fast-math-pr38968.f90 execution test I've noticed that this is something that times out depending on the orientation of the sun , moon and earth and the performance of qemu on your machine but on a native device runs just fine. regards, Ramana
Re: [google/4.6] Add xfails for arm-grtev2-linux-gnueabi (issue5794063)
Hi Ramana, We know the limit of the QEMU and already noticed failure due to the simulator. Like I said, this is used as the baseline. We are going to look at the failures carefully to categorize them. We noticed that some tests fail randomly on QEMU, these are marked as flaky and the validator script ignores those. Thanks -Doug On Mon, Mar 12, 2012 at 3:57 PM, Ramana Radhakrishnan ramana.radhakrish...@linaro.org wrote: On 12 March 2012 18:58, Doug Kwan dougk...@google.com wrote: Hi Diego This patch adds arm-grtev2-linux-gnueabi.xfail for our 4.6 branch so that we can track regressions. This just established the test baseline. The failures need to be investigated. Just out of curiosity, were these when you ran them cross on qemu or when you ran these native. It's probably worth noting that as well. There are times when you'll see differences in test results especially on recent trunk ( atomic tests depend on the version of gdb installed , cross testing on qemu pretty much means threaded tests are well let's say flaky) . This is probably something that ought to be recorded along with the environment in which the tests were run to ease comparison. This failure here suggests that you are runing on qemu. +FAIL: gfortran.dg/vect/fast-math-pr38968.f90 execution test I've noticed that this is something that times out depending on the orientation of the sun , moon and earth and the performance of qemu on your machine but on a native device runs just fine. regards, Ramana
