On Tue, Jan 20, 2026 at 01:04:15PM +0100, Georg-Johann Lay via Gcc wrote: > Since quite some time now I am observing differences in testsuite > results, even in the case when the compiler is build on the same > host from the same sources, and configured the same way. > > What I did: > > Host is cfarm421: x86_64 GNU/Linux, Debian 6.12.57-1 > > 1) Checked out same GCC sources (usually trunk) to > > $HOME/gnu/source/gcc-master > $HOME/gnu/source/gcc-x > > 2) In $HOME/gnu/build/gcc-host configure + build + test: > > $ ../../source/gcc-master/configure --enable-languages=c,c++ > --disable-bootstrap > $ make -j44 > $ (cd gcc; make -j44 -k check-gcc) > > 3) In $HOME/gnu/build/gcc-x-host configure + build + test: > > $ ../../source/gcc-x/configure --enable-languages=c,c++ --disable-bootstrap > $ make -j44 > $ (cd gcc; make -j44 -k check-gcc) > > 4) Then compare the test outcomes: > > $ $HOME/gnu/source/gcc-master/contrib/compare_tests > ~/gnu/build/gcc-host/gcc/testsuite/ ~/gnu/build/gcc-x-host/gcc/testsuite/ > > compare_tests's output reads: > > # Comparing directories > ## Dir1=/home/gjl/gnu/build/gcc-host/gcc/testsuite/: 1 sum files > ## Dir2=/home/gjl/gnu//build/gcc-x-host/gcc/testsuite/: 1 sum files > > # Comparing 1 common sum files > ## /bin/sh /home/gjl/gnu/source/gcc-master/contrib/compare_tests > /tmp/gxx-sum1.4194012 /tmp/gxx-sum2.4194012 > Non-unique test names: (Eeek!) > 2 PASS: gcc: c-c++-common/diagnostic-format-sarif-file-Wbidi-chars.c > -Wc++-compat scan-sarif-file "text": "unpaired UTF-8 bidirectional control > characters detected" > 2 PASS: gcc: > c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-1.c -Wc++-compat > scan-sarif-file "text": "invalid UTF-8 character '<80>'" > 2 PASS: gcc: > c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c -Wc++-compat > scan-sarif-file "text": "invalid UTF-8 character '<80>'" > 6 PASS: gcc: > c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c -Wc++-compat > scan-sarif-file "text": "invalid UTF-8 character '<bf>'" > 2 PASS: gcc: > c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c -Wc++-compat > scan-sarif-file "text": "invalid UTF-8 character '<c0>'" > 2 PASS: gcc: > c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c -Wc++-compat > scan-sarif-file "text": "invalid UTF-8 character '<c1>'" > 2 PASS: gcc: > c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c -Wc++-compat > scan-sarif-file "text": "invalid UTF-8 character '<c2>'" > 2 PASS: gcc: > c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c -Wc++-compat > scan-sarif-file "text": "invalid UTF-8 character '<e0>'" > 2 PASS: gcc: > c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c -Wc++-compat > scan-sarif-file "text": "invalid UTF-8 character '<e0><80><bf>'" > 2 PASS: gcc: > c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c -Wc++-compat > scan-sarif-file "text": "invalid UTF-8 character '<e0><9f><80>'" > 2 PASS: gcc: > c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c -Wc++-compat > scan-sarif-file "text": "invalid UTF-8 character '<e0><bf>'" > 2 PASS: gcc: > c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c -Wc++-compat > scan-sarif-file "text": "invalid UTF-8 character '<ec><80>'" > 2 PASS: gcc: > c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c -Wc++-compat > scan-sarif-file "text": "invalid UTF-8 character '<ed><a0><80>'" > 2 PASS: gcc: > c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c -Wc++-compat > scan-sarif-file "text": "invalid UTF-8 character '<f0><80><80><80>'" > 2 PASS: gcc: > c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c -Wc++-compat > scan-sarif-file "text": "invalid UTF-8 character '<f0><8f><bf><bf>'" > 2 PASS: gcc: > c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c -Wc++-compat > scan-sarif-file "text": "invalid UTF-8 character '<f4><90><80><80>'" > 2 PASS: gcc: > c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c -Wc++-compat > scan-sarif-file "text": "invalid UTF-8 character '<f5>'" > 2 PASS: gcc: > c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c -Wc++-compat > scan-sarif-file "text": "invalid UTF-8 character '<fd><bf><bf><bf>'" > 2 PASS: gcc: > c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c -Wc++-compat > scan-sarif-file "text": "invalid UTF-8 character '<ff>'" > 3 PASS: gcc: c-c++-common/goacc/combined-reduction.c > scan-tree-dump-times gimple "acc loop reduction.+:v1. private.i." 1 > 3 PASS: gcc: c-c++-common/gomp/append-args-6.c scan-tree-dump-times > gimple "D.[0-9]+ = __builtin_omp_get_mapped_ptr \\(cp2, 4\\);" 1 > 2 PASS: gcc: c-c++-common/gomp/requires-5.c scan-tree-dump original > "#pragma omp atomic release" > 3 PASS: gcc: c-c++-common/pr57371-1.c -Wc++-compat scan-tree-dump > original "return (<retval> = )?x != 0" > 2 PASS: gcc: gcc.dg/attr-noipa.c scan-tree-dump optimized "fn20 \\(\\)" > 2 PASS: gcc: gcc.dg/format/diagnostic-ranges.c > diagnostic-ranges-html.py::test_annotation > 2 PASS: gcc: gcc.dg/format/diagnostic-ranges.c > diagnostic-ranges-html.py::test_messag > 3 PASS: gcc: gcc.dg/lto/20111207-1 c_lto_20111207-1_0.o assemble, > -fcommon > 3 PASS: gcc: gcc.dg/lto/20111207-1 > c_lto_20111207-1_0.o-c_lto_20111207-1_3.o execute -fcommon > 3 PASS: gcc: gcc.dg/lto/20111207-1 > c_lto_20111207-1_0.o-c_lto_20111207-1_3.o link, -fcommon > 3 PASS: gcc: gcc.dg/lto/20111207-1 c_lto_20111207-1_1.o assemble, > -fcommon > 3 PASS: gcc: gcc.dg/lto/20111207-1 c_lto_20111207-1_2.o assemble, > -fcommon > 3 PASS: gcc: gcc.dg/lto/20111207-1 c_lto_20111207-1_3.o assemble, > -fcommon > 2 PASS: gcc: gcc.dg/sarif-output/multiple-outputs.c (test .sarif output > for UTF-8-encoded parseable JSON) > 2 UNSUPPORTED: gcc: gcc.dg/sarif-output/multiple-outputs.c > verify-sarif-file: check-jsonschema is missing > 2 PASS: gcc: gcc.dg/vect/vect-fmin-1.c -flto -ffat-lto-objects > scan-tree-dump vect "Detected reduction" > 2 PASS: gcc: gcc.dg/vect/vect-fmin-1.c scan-tree-dump vect "Detected > reduction" > 2 PASS: gcc: gcc.dg/vect/vect-reduc-dot-s16a.c -flto -ffat-lto-objects > scan-tree-dump-times vect "vectorized 1 loops" 1 > 2 PASS: gcc: gcc.dg/vect/vect-reduc-dot-s16a.c scan-tree-dump-times > vect "vectorized 1 loops" 1 > > ## Differences found > # 1 differences in 1 common sum files found > > > I also see similar diffs for other targets. > > Can someone give a hint what I am doing wrong and how to fix that?
You're not doing anything wrong, and there are no real differences. It's just that the test script is misleading. All of the issues reported by the script are relating to duplicate test names, which can be a problem because it leads to ambiguity about which test sum file line refers to. I recall that Richard Earnshaw eliminated a lot of duplicate names last year, but obviously some still remain. The line "1 differences in 1 common sum files found" doesn't actually count differences. The second number is the number of sum files that are present in both sets of test results, while the first number counts how many of these files the script detected an issue in. As you have encountered here, not all such issues are actual "differences". Alice > > Johann > >
