https://gcc.gnu.org/bugzilla/show_bug.cgi?id=116260
--- Comment #2 from Sam James <sjames at gcc dot gnu.org> ---
(In reply to Thiago Jung Bauermann from comment #1)
> Hello Sam,
>
> I mostly work with the GDB testsuite, so I'm not very familiar with
> GCC-specific details of DejaGNU and the sum files, so advance warning:
> naive questions ahead.
No problem -- I'm really not an expert on DejaGNU yet :)
>
> I added Christophe Lyon to the Cc list. He's much more familiar with the
> GCC testsuite and how we use it.
>
> (In reply to Sam James from comment #0)
> > We've started using validate_failures.py in our packaging (and I'm using it
> > in my own workflow too) and it's great, thanks for it.
>
> Nice!
>
> > I tried wiring up multilib tests next in our packaging and noticed that the
> > manifest ended up mixing results between ABIs, like:
>
> I'm not sure I understand the scenario. Are you doing two testsuite runs,
> one for each ABI, and then using the one's manifest to compare with the
> other?
>
So, the scenario right now is, say, on x86_64-pc-linux-gnu:
$ ./configure --enable-multilib --with-multilib-list=m32,m64
$ make
$ make check RUNTESTFLAGS="--target_board=unix\{,-m32\}"
dejagnu will then run the whole testsuite parametrised into "" and "-m32".
This is AFAIK the primary way to test GCC w/ multilib, but people also use
this method to do stuff like an extra testrun with -O3 or whatever.
I think what makes sense there is to accept a parameter indicating which
target (in the dejagnu sense, see below)/variation we're wanting to produce
or compare a manifest for.
That way, we can run validate_failures.py repeatedly per-variant to both
generate and then compare the manifests? I think it makes more sense that
way around than it trying to do it for us (and spitting out a suffix to the
file
we chose), but not sure.
> > dejagnu emits lines indicating which ABI tests were run for,
>
> Do you mean these lines?
>
> ```
> Native configuration is armv8l-unknown-linux-gnueabihf
>
> === gcc tests ===
>
> Schedule of variations:
> tcwg-local
>
> Running target tcwg-local
> ```
The lines I'm thinking of are these:
```
=== libgomp tests ===
Schedule of variations:
unix
unix/-m32
```
It'll execute these in-order, looking like...
```
=== libgomp tests ===
Schedule of variations:
unix
unix/-m32
Running target unix
[...]
Executing on host: ...
[...]
=== libgomp Summary for unix ===
# of expected passes 15407
# of unexpected failures 1281
# of unexpected successes 190
# of expected failures 75
# of unresolved testcases 50
# of unsupported tests 619
Running target unix/-m32
[...]
Executing on host: ...
[...]
```
