Re: RISC-V Test Errors and Failures
On 5/25/23 13:29, Thomas Schwinge wrote: Hi! On 2023-05-17T09:52:13+0200, Andreas Schwab via Gcc-patches wrote: On Mai 16 2023, Vineet Gupta wrote: Yes I was seeing similar tcl errors and such - and in my case an even higher count. They are coming from commit d6654a4be3b. I call FUD. Until you prove otherwise, of coures. Grüße Thomas Just as a data point, with those patches reverted, I don't see the tcl errors. 2023-05-25 7a3c9f8e8362 Revert "Let each 'lto_init' determine the default 'LTO_OPTIONS', and 'torture-init' the 'LTO_TORTURE_OPTIONS'" 2023-05-25 22206cb760ee Revert "Testsuite: Add missing 'torture-init'/'torture-finish' around 'LTO_TORTURE_OPTIONS' usage" 2023-05-25 db46b946dd6d Revert "Testsuite: Add 'torture-init-done', and use it to conditionalize implicit 'torture-init'" 2023-05-25 bd412162fd0d Revert "xxx vineet fixup" 2023-05-22 97a5e2241d33 xxx vineet fixup 2023-05-24 ec2e86274427 Fortran: reject bad DIM argument of SIZE intrinsic in simplification [PR104350] ... = Summary of gcc testsuite = | # of unexpected case / # of unique unexpected case | gcc | g++ | gfortran | rv64imafdc/ lp64d/ medlow | 25 / 4 | 1 / 1 | 72 / 12 | rv32imafdc/ ilp32d/ medlow | 26 / 5 | 3 / 2 | 72 / 12 | rv32imac/ ilp32/ medlow | 25 / 4 | 3 / 2 | 109 / 19 | rv64imac/ lp64/ medlow | 26 / 5 | 1 / 1 | 109 / 19 |
Re: RISC-V Test Errors and Failures
Hi! On 2023-05-17T09:52:13+0200, Andreas Schwab via Gcc-patches wrote: > On Mai 16 2023, Vineet Gupta wrote: > >> Yes I was seeing similar tcl errors and such - and in my case an even >> higher count. > > They are coming from commit d6654a4be3b. I call FUD. Until you prove otherwise, of coures. Grüße Thomas
Re: RISC-V Test Errors and Failures
On 5/17/23 00:52, Andreas Schwab wrote: On Mai 16 2023, Vineet Gupta wrote: Yes I was seeing similar tcl errors and such - and in my case an even higher count. They are coming from commit d6654a4be3b. As of a726d007f197 today I get a gazzilion splat for riscv multilib dejagnu runs and over 5k fails ERROR: torture-init: torture_without_loops is not empty as expected ERROR: tcl error code NONE ... ... = Summary of gcc testsuite = | # of unexpected case / # of unique unexpected case | gcc | g++ | gfortran | rv64imafdc/ lp64d/ medlow | 5033 / 4 | 1 / 1 | 72 / 12 | rv32imafdc/ ilp32d/ medlow | 5032 / 3 | 3 / 2 | 72 / 12 | rv32imac/ ilp32/ medlow | 1 / 1 | 3 / 2 | 109 / 19 | rv64imac/ lp64/ medlow | 5034 / 5 | 1 / 1 | 109 / 19 | For a non multilib run things are sane: = Summary of gcc testsuite = | # of unexpected case / # of unique unexpected case | gcc | g++ | gfortran | rv64imafdc/ lp64d/ medlow | 11 / 4 | 1 / 1 | 72 / 12 | It is really hard to test anything on upstream ATM. -Vineet
Re: RISC-V Test Errors and Failures
On Mai 16 2023, Vineet Gupta wrote: > Yes I was seeing similar tcl errors and such - and in my case an even > higher count. They are coming from commit d6654a4be3b. -- Andreas Schwab, SUSE Labs, sch...@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different."
Re: RISC-V Test Errors and Failures
On 5/16/23 20:39, Palmer Dabbelt wrote: By "chroot environment" you mean something like a debootstrap-into-chroot with qemu-user/binfmt-misc? I don't have that setup right now, but it wouldn't be a big lift. Essentially, yes. I actually have a home built ones for the various targets. There was a time when they needed to be <100M so they could be stored on github. Going forward I just want them to be a docker container for Fedora or Ubuntu. With binfmt you can run non-native containers trivially. jeff
Re: RISC-V Test Errors and Failures
On 5/16/23 21:33, Kito Cheng via Gcc-patches wrote:
diff --git a/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
b/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
index 94d6ec5..efc3a80 100755
--- a/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
+++ b/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
@@ -12,4 +12,4 @@ done
xlen="$(readelf -h $1 | grep 'Class' | cut -d: -f 2 | xargs echo | sed
's/^ELF//')"
-qemu-riscv$xlen -r 5.10 "${qemu_args[@]}" -L ${RISC_V_SYSROOT} -cpu
rv$xlen,zba=on,zbb=on,zbc=on,zbs=on "$@"
+qemu-riscv$xlen -r 5.10 "${qemu_args[@]}" -L ${RISC_V_SYSROOT} -cpu
rv$xlen,zba=on,zbb=on,zbc=on,zbs=on,v=on "$@"
This not work when you testing some combination e.g. Z*inx and zve*,
but anyway I guess those configurations are not matter for you guys :P
What you could do is install a suitable binfmt handler, then you don't
need the wrappers at all. That's how we're handling this stuff in
Ventana. It also means you don't need magic dejagnu baseboard files or
anything like that. In fact from dejagnu's standpoint it looks native.
jeff
Re: RISC-V Test Errors and Failures
> diff --git a/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
> b/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
> index 94d6ec5..efc3a80 100755
> --- a/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
> +++ b/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
> @@ -12,4 +12,4 @@ done
>
> xlen="$(readelf -h $1 | grep 'Class' | cut -d: -f 2 | xargs echo | sed
> 's/^ELF//')"
>
> -qemu-riscv$xlen -r 5.10 "${qemu_args[@]}" -L ${RISC_V_SYSROOT} -cpu
> rv$xlen,zba=on,zbb=on,zbc=on,zbs=on "$@"
> +qemu-riscv$xlen -r 5.10 "${qemu_args[@]}" -L ${RISC_V_SYSROOT} -cpu
> rv$xlen,zba=on,zbb=on,zbc=on,zbs=on,v=on "$@"
This not work when you testing some combination e.g. Z*inx and zve*,
but anyway I guess those configurations are not matter for you guys :P
>
> for now. I'm going to throw together hwprobe for qemu-user, from looking at
> the AVX stuff it should be pretty easy to plumb that into DG and then get the
> detection going.
Re: RISC-V Test Errors and Failures
On Tue, 16 May 2023 20:08:26 PDT (-0700), Vineet Gupta wrote:
On 5/16/23 19:53, Palmer Dabbelt wrote:
Probably, I'll go try and bump stuff and see if it works...
Word of caution: Best to not disturb your existing setup, a try a fresh
checkout first
Even easier, I think I can get away with just
diff --git a/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
b/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
index 94d6ec5..efc3a80 100755
--- a/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
+++ b/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
@@ -12,4 +12,4 @@ done
xlen="$(readelf -h $1 | grep 'Class' | cut -d: -f 2 | xargs echo | sed
's/^ELF//')"
-qemu-riscv$xlen -r 5.10 "${qemu_args[@]}" -L ${RISC_V_SYSROOT} -cpu
rv$xlen,zba=on,zbb=on,zbc=on,zbs=on "$@"
+qemu-riscv$xlen -r 5.10 "${qemu_args[@]}" -L ${RISC_V_SYSROOT} -cpu
rv$xlen,zba=on,zbb=on,zbc=on,zbs=on,v=on "$@"
for now. I'm going to throw together hwprobe for qemu-user, from looking at
the AVX stuff it should be pretty easy to plumb that into DG and then get the
detection going.
Re: RISC-V Test Errors and Failures
On 5/16/23 19:53, Palmer Dabbelt wrote: Probably, I'll go try and bump stuff and see if it works... Word of caution: Best to not disturb your existing setup, a try a fresh checkout first
Re: RISC-V Test Errors and Failures
On Tue, 16 May 2023 19:51:48 PDT (-0700), Patrick O'Neill wrote: On 5/16/23 19:47, Palmer Dabbelt wrote: On Tue, 16 May 2023 19:46:28 PDT (-0700), Vineet Gupta wrote: On 5/16/23 19:21, Kito Cheng wrote: Palmer: For short-term, this should help your internal test: https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1233 That only helps if using bleeding edge toolchain scripts (which I regularly do and so did Patrick). Palmer has a fork of toolchain scripts and I'm assuming he hasn't caught up to that point ;-) I'm fine dropping the fork if the bugs have been fixed. IIRC last week we were still waiting for them to merge something? The testsuite was broken last week, but was fixed by https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1247 which was merged last Friday. That might be the thing you were thinking about? Probably, I'll go try and bump stuff and see if it works... Thanks!
Re: RISC-V Test Errors and Failures
On 5/16/23 19:47, Palmer Dabbelt wrote: On Tue, 16 May 2023 19:46:28 PDT (-0700), Vineet Gupta wrote: On 5/16/23 19:21, Kito Cheng wrote: Palmer: For short-term, this should help your internal test: https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1233 That only helps if using bleeding edge toolchain scripts (which I regularly do and so did Patrick). Palmer has a fork of toolchain scripts and I'm assuming he hasn't caught up to that point ;-) I'm fine dropping the fork if the bugs have been fixed. IIRC last week we were still waiting for them to merge something? The testsuite was broken last week, but was fixed by https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1247 which was merged last Friday. That might be the thing you were thinking about?
Re: RISC-V Test Errors and Failures
On Tue, 16 May 2023 19:46:28 PDT (-0700), Vineet Gupta wrote: On 5/16/23 19:21, Kito Cheng wrote: Palmer: For short-term, this should help your internal test: https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1233 That only helps if using bleeding edge toolchain scripts (which I regularly do and so did Patrick). Palmer has a fork of toolchain scripts and I'm assuming he hasn't caught up to that point ;-) I'm fine dropping the fork if the bugs have been fixed. IIRC last week we were still waiting for them to merge something? -Vineet
Re: RISC-V Test Errors and Failures
On 5/16/23 19:21, Kito Cheng wrote: Palmer: For short-term, this should help your internal test: https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1233 That only helps if using bleeding edge toolchain scripts (which I regularly do and so did Patrick). Palmer has a fork of toolchain scripts and I'm assuming he hasn't caught up to that point ;-) -Vineet
Re: RISC-V Test Errors and Failures
On Tue, 16 May 2023 19:32:21 PDT (-0700), jeffreya...@gmail.com wrote: On 5/16/23 20:05, Palmer Dabbelt wrote: On Tue, 16 May 2023 19:00:12 PDT (-0700), Jeff Law wrote: On 5/16/23 19:29, Palmer Dabbelt wrote: I think the most pressing need is bleeding edge gcc regression tracking. @Jeff is anything setup on sourceware and/or usable ? I thought they do have existing bots for some arches to spin up build / run - perhaps runs are native and not qemu. IIRC Jeff said his builders were hanging right now. Correct. More precisely, the riscv64 builds hang. Not sure if it's stage2 or stage3 of the bootstrap. Been happening for the last couple weeks. I suspect some codegen bug in the riscv port. I'll have to bisect it which will be quite painful. Can anyone else do it? If the only blocker for having an upstream regression CI thing is just sorting out why it broke over the last few weeks then I'm happy to try and trick someone around here into doing some work... Probably easiest for me unless someone else has a chroot environment handy. It's not hard to do the bisection, it just involves a lot of waiting. By "chroot environment" you mean something like a debootstrap-into-chroot with qemu-user/binfmt-misc? I don't have that setup right now, but it wouldn't be a big lift. I've just about got the my problem from earlier today under control, then I can probably start bisection. That's fine with me, I have plenty of other stuff to do ;)
Re: RISC-V Test Errors and Failures
On 5/16/23 20:05, Palmer Dabbelt wrote: On Tue, 16 May 2023 19:00:12 PDT (-0700), Jeff Law wrote: On 5/16/23 19:29, Palmer Dabbelt wrote: I think the most pressing need is bleeding edge gcc regression tracking. @Jeff is anything setup on sourceware and/or usable ? I thought they do have existing bots for some arches to spin up build / run - perhaps runs are native and not qemu. IIRC Jeff said his builders were hanging right now. Correct. More precisely, the riscv64 builds hang. Not sure if it's stage2 or stage3 of the bootstrap. Been happening for the last couple weeks. I suspect some codegen bug in the riscv port. I'll have to bisect it which will be quite painful. Can anyone else do it? If the only blocker for having an upstream regression CI thing is just sorting out why it broke over the last few weeks then I'm happy to try and trick someone around here into doing some work... Probably easiest for me unless someone else has a chroot environment handy. It's not hard to do the bisection, it just involves a lot of waiting. I've just about got the my problem from earlier today under control, then I can probably start bisection. Jeff
Re: Re: RISC-V Test Errors and Failures
Palmer:
For short-term, this should help your internal test:
https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1233
On Wed, May 17, 2023 at 10:20 AM Kito Cheng wrote:
>
> Currently we are highly rely on simulator can setup correctly by ELF
> attribute or -march setting, but seems not true for everyone, for
> longer term we need something like
> check_effective_target_aarch64_sve_hw, but as Palmer point out, we
> might need...bunch of that for different extensions
>
> On Wed, May 17, 2023 at 10:13 AM Palmer Dabbelt wrote:
> >
> > On Tue, 16 May 2023 19:07:01 PDT (-0700), juzhe.zh...@rivai.ai wrote:
> > > Oh, I see. Kito has add /* { dg-do run { target { riscv_vector } } } */
> > > But not all RVV tests has use this and I not sure whether it can work.
> > > I think Kito can answer it.
> > > If yes, I think we should add all of them.
> >
> > Unless I'm missing something, it looks like that only checks if GCC is
> > compiling for V. Nothing appears to be checking if the system the tests
> > are running on supports V.
> >
> > # Return 1 if the target has RISC-V vector extension, 0 otherwise.
> > # Cache the result.
> >
> > proc check_effective_target_riscv_vector { } {
> > # Check that we are compiling for v by checking the __riscv_v marco.
> > return [check_no_compiler_messages riscv_vector assembly {
> >#if !defined(__riscv_v)
> >#error "__riscv_v not defined!"
> >#endif
> > }]
> > }
> >
> > Those are really just two different things.
> >
> > It seems pretty reasonably to me to just avoid running the tests when
> > the DUT lacks V, but I'm never great with DG. We should probably add
> > similar checks for the other ISA extensions, there's going to be a bunch
> > of this.
> >
> > >
> > > Thanks.
> > >
> > >
> > > juzhe.zh...@rivai.ai
> > >
> > > From: Andrew Pinski
> > > Date: 2023-05-17 10:02
> > > To: juzhe.zh...@rivai.ai
> > > CC: gcc-patches; palmer; Kito.cheng
> > > Subject: Re: RISC-V Test Errors and Failures
> > > On Tue, May 16, 2023 at 6:58 PM juzhe.zh...@rivai.ai
> > > wrote:
> > >>
> > >> Hi, Palmer.
> > >> I saw your patch showed there are a lot of run time fail (execution
> > >> fail) of C++.
> > >> bug-*.C
> > >>
> > >> These tests are RVV api intrinsics tests coming from Kito's that I have
> > >> already fixed all of them.
> > >> I just double checked again they all passed.
> > >> I think it may be your regression environment does not set up simulator
> > >> (QEMU or SPIKE or GEM5) correctly.
> > >> For example, did not enable vector extension in simulator, I don't you
> > >> may try.
> > >
> > > So on x86_64, we test to see if you have the right vector unit before
> > > running those tests? The same thing was true on powerpc (and I think
> > > aarch64 does the same for SVE now too). The reason why I am asking is
> > > that I would need to run the testsuite using the simulator as setup
> > > for the RISCV ISA I am using rather than the one with everything on.
> > > So does the RVV runtime testsuite tests to see if you can run RVV
> > > before running them (or running them and return they passed)?
> > >
> > > Thanks,
> > > Andrew Pinski
> > >
> > >>
> > >> Thanks.
> > >>
> > >>
> > >> juzhe.zh...@rivai.ai
> > >
Re: Re: RISC-V Test Errors and Failures
Currently we are highly rely on simulator can setup correctly by ELF
attribute or -march setting, but seems not true for everyone, for
longer term we need something like
check_effective_target_aarch64_sve_hw, but as Palmer point out, we
might need...bunch of that for different extensions
On Wed, May 17, 2023 at 10:13 AM Palmer Dabbelt wrote:
>
> On Tue, 16 May 2023 19:07:01 PDT (-0700), juzhe.zh...@rivai.ai wrote:
> > Oh, I see. Kito has add /* { dg-do run { target { riscv_vector } } } */
> > But not all RVV tests has use this and I not sure whether it can work.
> > I think Kito can answer it.
> > If yes, I think we should add all of them.
>
> Unless I'm missing something, it looks like that only checks if GCC is
> compiling for V. Nothing appears to be checking if the system the tests
> are running on supports V.
>
> # Return 1 if the target has RISC-V vector extension, 0 otherwise.
> # Cache the result.
>
> proc check_effective_target_riscv_vector { } {
> # Check that we are compiling for v by checking the __riscv_v marco.
> return [check_no_compiler_messages riscv_vector assembly {
>#if !defined(__riscv_v)
>#error "__riscv_v not defined!"
>#endif
> }]
> }
>
> Those are really just two different things.
>
> It seems pretty reasonably to me to just avoid running the tests when
> the DUT lacks V, but I'm never great with DG. We should probably add
> similar checks for the other ISA extensions, there's going to be a bunch
> of this.
>
> >
> > Thanks.
> >
> >
> > juzhe.zh...@rivai.ai
> >
> > From: Andrew Pinski
> > Date: 2023-05-17 10:02
> > To: juzhe.zh...@rivai.ai
> > CC: gcc-patches; palmer; Kito.cheng
> > Subject: Re: RISC-V Test Errors and Failures
> > On Tue, May 16, 2023 at 6:58 PM juzhe.zh...@rivai.ai
> > wrote:
> >>
> >> Hi, Palmer.
> >> I saw your patch showed there are a lot of run time fail (execution fail)
> >> of C++.
> >> bug-*.C
> >>
> >> These tests are RVV api intrinsics tests coming from Kito's that I have
> >> already fixed all of them.
> >> I just double checked again they all passed.
> >> I think it may be your regression environment does not set up simulator
> >> (QEMU or SPIKE or GEM5) correctly.
> >> For example, did not enable vector extension in simulator, I don't you may
> >> try.
> >
> > So on x86_64, we test to see if you have the right vector unit before
> > running those tests? The same thing was true on powerpc (and I think
> > aarch64 does the same for SVE now too). The reason why I am asking is
> > that I would need to run the testsuite using the simulator as setup
> > for the RISCV ISA I am using rather than the one with everything on.
> > So does the RVV runtime testsuite tests to see if you can run RVV
> > before running them (or running them and return they passed)?
> >
> > Thanks,
> > Andrew Pinski
> >
> >>
> >> Thanks.
> >>
> >>
> >> juzhe.zh...@rivai.ai
> >
Re: Re: RISC-V Test Errors and Failures
On Tue, 16 May 2023 19:07:01 PDT (-0700), juzhe.zh...@rivai.ai wrote:
Oh, I see. Kito has add /* { dg-do run { target { riscv_vector } } } */
But not all RVV tests has use this and I not sure whether it can work.
I think Kito can answer it.
If yes, I think we should add all of them.
Unless I'm missing something, it looks like that only checks if GCC is
compiling for V. Nothing appears to be checking if the system the tests
are running on supports V.
# Return 1 if the target has RISC-V vector extension, 0 otherwise.
# Cache the result.
proc check_effective_target_riscv_vector { } {
# Check that we are compiling for v by checking the __riscv_v marco.
return [check_no_compiler_messages riscv_vector assembly {
#if !defined(__riscv_v)
#error "__riscv_v not defined!"
#endif
}]
}
Those are really just two different things.
It seems pretty reasonably to me to just avoid running the tests when
the DUT lacks V, but I'm never great with DG. We should probably add
similar checks for the other ISA extensions, there's going to be a bunch
of this.
Thanks.
juzhe.zh...@rivai.ai
From: Andrew Pinski
Date: 2023-05-17 10:02
To: juzhe.zh...@rivai.ai
CC: gcc-patches; palmer; Kito.cheng
Subject: Re: RISC-V Test Errors and Failures
On Tue, May 16, 2023 at 6:58 PM juzhe.zh...@rivai.ai
wrote:
Hi, Palmer.
I saw your patch showed there are a lot of run time fail (execution fail) of
C++.
bug-*.C
These tests are RVV api intrinsics tests coming from Kito's that I have already
fixed all of them.
I just double checked again they all passed.
I think it may be your regression environment does not set up simulator (QEMU
or SPIKE or GEM5) correctly.
For example, did not enable vector extension in simulator, I don't you may try.
So on x86_64, we test to see if you have the right vector unit before
running those tests? The same thing was true on powerpc (and I think
aarch64 does the same for SVE now too). The reason why I am asking is
that I would need to run the testsuite using the simulator as setup
for the RISCV ISA I am using rather than the one with everything on.
So does the RVV runtime testsuite tests to see if you can run RVV
before running them (or running them and return they passed)?
Thanks,
Andrew Pinski
Thanks.
juzhe.zh...@rivai.ai
Re: RISC-V Test Errors and Failures
On 5/16/23 20:08, Vineet Gupta wrote: I think the most pressing need is bleeding edge gcc regression tracking. @Jeff is anything setup on sourceware and/or usable ? I thought they do have existing bots for some arches to spin up build / run - perhaps runs are native and not qemu. IIRC Jeff said his builders were hanging right now. Jeff it seems has his own test infra. I was ask Mine is the closest we've got for project-wide testing. Various orgs have their own servers/bots. jeff
Re: RISC-V Test Errors and Failures
On 5/16/23 18:29, Palmer Dabbelt wrote: On Tue, 16 May 2023 18:04:37 PDT (-0700), Vineet Gupta wrote: + Christoph, Jiawei On 5/16/23 17:20, Palmer Dabbelt wrote: We really need to add some CI around RV toolchains to trip on these sooner ! Sounds like you're volunteering to set one up? Patrick's github CI patch seems to be a great start. Lets wait for it to get merged, that will at least catch rv toolchain snafus: although the granularity of testing is not ideal (tc changes are not so frequent) You mean riscv-gnu-toolchain changes? That's not super useful for GCC development, they're on a fork. Well they are still useful to catch various snafus in toolchain plumbing itself - I've run into 2 of those and Patcrick 2 himself, when trying to use latest upstream toolchain scripts. But sure they are not testing bleeding edge gcc. I think the most pressing need is bleeding edge gcc regression tracking. @Jeff is anything setup on sourceware and/or usable ? I thought they do have existing bots for some arches to spin up build / run - perhaps runs are native and not qemu. IIRC Jeff said his builders were hanging right now. Jeff it seems has his own test infra. I was asking if sourceware (or whatever the custodian of gcc project has). I'd be really surprised if primary arches such as x86/aarch64 don't have any test bots there ? FWIW rivos gitlab CI (not public) has capability to track upstream gcc (Kevin almost has it working), but there is no easy way to publish it for rest of the world and I'd rather that be done in a public infra. +Kevin At least having the failure lists public would be a must-have, and I think that's tricky to do with gitlab. Yep. Bjorn and Conor have something glued to the kernel patchwork that uploads test results to github as snippits, but IIRC we're trying to replace it with something more directly visible. Didn't ISCAS/PLCT have such infra - sorry Kito asked the same question this morning, but I was not fully awoke so don't remember what Jiawei replied. I didn't even remember he asked ;)
Re: Re: RISC-V Test Errors and Failures
Oh, I see. Kito has add /* { dg-do run { target { riscv_vector } } } */
But not all RVV tests has use this and I not sure whether it can work.
I think Kito can answer it.
If yes, I think we should add all of them.
Thanks.
juzhe.zh...@rivai.ai
From: Andrew Pinski
Date: 2023-05-17 10:02
To: juzhe.zh...@rivai.ai
CC: gcc-patches; palmer; Kito.cheng
Subject: Re: RISC-V Test Errors and Failures
On Tue, May 16, 2023 at 6:58 PM juzhe.zh...@rivai.ai
wrote:
>
> Hi, Palmer.
> I saw your patch showed there are a lot of run time fail (execution fail) of
> C++.
> bug-*.C
>
> These tests are RVV api intrinsics tests coming from Kito's that I have
> already fixed all of them.
> I just double checked again they all passed.
> I think it may be your regression environment does not set up simulator (QEMU
> or SPIKE or GEM5) correctly.
> For example, did not enable vector extension in simulator, I don't you may
> try.
So on x86_64, we test to see if you have the right vector unit before
running those tests? The same thing was true on powerpc (and I think
aarch64 does the same for SVE now too). The reason why I am asking is
that I would need to run the testsuite using the simulator as setup
for the RISCV ISA I am using rather than the one with everything on.
So does the RVV runtime testsuite tests to see if you can run RVV
before running them (or running them and return they passed)?
Thanks,
Andrew Pinski
>
> Thanks.
>
>
> juzhe.zh...@rivai.ai
Re: RISC-V Test Errors and Failures
On Tue, 16 May 2023 19:00:12 PDT (-0700), Jeff Law wrote: On 5/16/23 19:29, Palmer Dabbelt wrote: I think the most pressing need is bleeding edge gcc regression tracking. @Jeff is anything setup on sourceware and/or usable ? I thought they do have existing bots for some arches to spin up build / run - perhaps runs are native and not qemu. IIRC Jeff said his builders were hanging right now. Correct. More precisely, the riscv64 builds hang. Not sure if it's stage2 or stage3 of the bootstrap. Been happening for the last couple weeks. I suspect some codegen bug in the riscv port. I'll have to bisect it which will be quite painful. Can anyone else do it? If the only blocker for having an upstream regression CI thing is just sorting out why it broke over the last few weeks then I'm happy to try and trick someone around here into doing some work...
Re: RISC-V Test Errors and Failures
On Tue, May 16, 2023 at 6:58 PM juzhe.zh...@rivai.ai wrote: > > Hi, Palmer. > I saw your patch showed there are a lot of run time fail (execution fail) of > C++. > bug-*.C > > These tests are RVV api intrinsics tests coming from Kito's that I have > already fixed all of them. > I just double checked again they all passed. > I think it may be your regression environment does not set up simulator (QEMU > or SPIKE or GEM5) correctly. > For example, did not enable vector extension in simulator, I don't you may > try. So on x86_64, we test to see if you have the right vector unit before running those tests? The same thing was true on powerpc (and I think aarch64 does the same for SVE now too). The reason why I am asking is that I would need to run the testsuite using the simulator as setup for the RISCV ISA I am using rather than the one with everything on. So does the RVV runtime testsuite tests to see if you can run RVV before running them (or running them and return they passed)? Thanks, Andrew Pinski > > Thanks. > > > juzhe.zh...@rivai.ai
Re: RISC-V Test Errors and Failures
On 5/16/23 19:29, Palmer Dabbelt wrote: I think the most pressing need is bleeding edge gcc regression tracking. @Jeff is anything setup on sourceware and/or usable ? I thought they do have existing bots for some arches to spin up build / run - perhaps runs are native and not qemu. IIRC Jeff said his builders were hanging right now. Correct. More precisely, the riscv64 builds hang. Not sure if it's stage2 or stage3 of the bootstrap. Been happening for the last couple weeks. I suspect some codegen bug in the riscv port. I'll have to bisect it which will be quite painful. jeff
Re: RISC-V Test Errors and Failures
On Tue, 16 May 2023 18:04:37 PDT (-0700), Vineet Gupta wrote: + Christoph, Jiawei On 5/16/23 17:20, Palmer Dabbelt wrote: We really need to add some CI around RV toolchains to trip on these sooner ! Sounds like you're volunteering to set one up? Patrick's github CI patch seems to be a great start. Lets wait for it to get merged, that will at least catch rv toolchain snafus: although the granularity of testing is not ideal (tc changes are not so frequent) You mean riscv-gnu-toolchain changes? That's not super useful for GCC development, they're on a fork. I think the most pressing need is bleeding edge gcc regression tracking. @Jeff is anything setup on sourceware and/or usable ? I thought they do have existing bots for some arches to spin up build / run - perhaps runs are native and not qemu. IIRC Jeff said his builders were hanging right now. FWIW rivos gitlab CI (not public) has capability to track upstream gcc (Kevin almost has it working), but there is no easy way to publish it for rest of the world and I'd rather that be done in a public infra. +Kevin At least having the failure lists public would be a must-have, and I think that's tricky to do with gitlab. Bjorn and Conor have something glued to the kernel patchwork that uploads test results to github as snippits, but IIRC we're trying to replace it with something more directly visible. Didn't ISCAS/PLCT have such infra - sorry Kito asked the same question this morning, but I was not fully awoke so don't remember what Jiawei replied. I didn't even remember he asked ;)
Re: RISC-V Test Errors and Failures
+ Christoph, Jiawei On 5/16/23 17:20, Palmer Dabbelt wrote: We really need to add some CI around RV toolchains to trip on these sooner ! Sounds like you're volunteering to set one up? Patrick's github CI patch seems to be a great start. Lets wait for it to get merged, that will at least catch rv toolchain snafus: although the granularity of testing is not ideal (tc changes are not so frequent) I think the most pressing need is bleeding edge gcc regression tracking. @Jeff is anything setup on sourceware and/or usable ? I thought they do have existing bots for some arches to spin up build / run - perhaps runs are native and not qemu. FWIW rivos gitlab CI (not public) has capability to track upstream gcc (Kevin almost has it working), but there is no easy way to publish it for rest of the world and I'd rather that be done in a public infra. Didn't ISCAS/PLCT have such infra - sorry Kito asked the same question this morning, but I was not fully awoke so don't remember what Jiawei replied. Thx, -Vineet
Re: RISC-V Test Errors and Failures
On Tue, 16 May 2023 17:16:11 PDT (-0700), Vineet Gupta wrote: On 5/16/23 16:06, Palmer Dabbelt wrote: A few of us were talking about test-related issues in the patchwork meeting this morning. I bumped to trunk and did a full rebuild, I'm getting the following (it's in riscv-systems-ci/riscv-gnu-toolchain). This is about what I remember seeing last time I ran the tests, which was a week or so ago. I figured it'd be best to just blast the lists, as Jeff said his test running had been hanging so there might be some issue preventing folks from seeing the failures. I guess I didn't get time to look last time and I doubt things are looking any better right now. I'll try and take a look at some point, but any help would of course be appreciated. Yes I was seeing similar tcl errors and such - and in my case an even higher count. Also for posterity, what was your configure cmdline ? multilibs or no If only I'd saved those in the build somewhere... :) It's all in github.com/palmer-dabbelt/riscv-systems-ci, which points to riscv-gnu-toolchain. I've always got uncommitted diff in my various local checkous, but I think this would only be toolchain: toolchain/install.stamp toolchain/install.stamp: toolchain/Makefile $(MAKE) -C $(dir $<) date > $@ toolchain/Makefile: riscv-gnu-toolchain/configure mkdir -p $(dir $@) env -C $(dir $@) $(abspath $<) --prefix="$(abspath $(dir $@)/install)" --enable-linux --enable-multilib --enable-gcc-checking=yes toolchain/check.log: toolchain/install.stamp $(MAKE) -C $(dir $<) check \ GLIBC_TARGET_BOARDS_EXTRA="riscv-sim/-march=rv64gczba_zbb_zbc_zbs/-mabi=lp64d riscv-sim/-march=rv64imafdcv/-mabi=lp64d riscv-sim/-march=rv32imafdcv/-mabi=ilp32d" |& tee $@ touch -c $@ toolchain/report: toolchain/check.log $(MAKE) -C $(dir $<) report \ GLIBC_TARGET_BOARDS_EXTRA="riscv-sim/-march=rv64gczba_zbb_zbc_zbs/-mabi=lp64d riscv-sim/-march=rv64imafdcv/-mabi=lp64d riscv-sim/-march=rv32imafdcv/-mabi=ilp32d" |& tee $@ touch -c $@ We really need to add some CI around RV toolchains to trip on these sooner ! Sounds like you're volunteering to set one up? $ cat toolchain/report make[1]: Entering directory '/scratch/merges/rgt-gcc-trunk/toolchain' /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/scripts/testsuite-filter gcc glibc /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/test/allowlist `find build-gcc-linux-stage2/gcc/testsuite/ -name *.sum |paste -sd "," -` === g++: Unexpected fails for rv64imac lp64 medlow === FAIL: g++.dg/contracts/contracts-tmpl-spec2.C output pattern test === g++: Unexpected fails for rv32imac ilp32 medlow === FAIL: g++.dg/contracts/contracts-tmpl-spec2.C output pattern test FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2a (test for excess errors) FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2b (test for excess errors) === g++: Unexpected fails for rv64imafdc lp64d medlow === FAIL: g++.dg/contracts/contracts-tmpl-spec2.C output pattern test === g++: Unexpected fails for rv32imafdc ilp32d medlow === FAIL: g++.dg/contracts/contracts-tmpl-spec2.C output pattern test FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2a (test for excess errors) FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2b (test for excess errors) === g++: Unexpected fails for rv64imafdcv lp64d === FAIL: g++.dg/contracts/contracts-tmpl-spec2.C output pattern test FAIL: g++.target/riscv/rvv/base/bug-10.C execution test FAIL: g++.target/riscv/rvv/base/bug-11.C execution test FAIL: g++.target/riscv/rvv/base/bug-12.C execution test FAIL: g++.target/riscv/rvv/base/bug-13.C execution test FAIL: g++.target/riscv/rvv/base/bug-14.C execution test FAIL: g++.target/riscv/rvv/base/bug-15.C execution test FAIL: g++.target/riscv/rvv/base/bug-16.C execution test FAIL: g++.target/riscv/rvv/base/bug-17.C execution test FAIL: g++.target/riscv/rvv/base/bug-2.C execution test FAIL: g++.target/riscv/rvv/base/bug-23.C execution test FAIL: g++.target/riscv/rvv/base/bug-3.C execution test FAIL: g++.target/riscv/rvv/base/bug-4.C execution test FAIL: g++.target/riscv/rvv/base/bug-5.C execution test FAIL: g++.target/riscv/rvv/base/bug-6.C execution test FAIL: g++.target/riscv/rvv/base/bug-7.C execution test FAIL: g++.target/riscv/rvv/base/bug-8.C execution test FAIL: g++.target/riscv/rvv/base/bug-9.C execution test === g++: Unexpected fails for rv32imafdcv ilp32d === FAIL: g++.dg/contracts/contracts-tmpl-spec2.C output pattern test FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2a (test for excess errors) FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2b (test for excess errors) FAIL: g++.target/riscv/rvv/base/bug-10.C execution test FAIL: g++.target/riscv/rvv/base/bug-11.C execution test FAIL: g++.target/riscv/rvv/base/bug-12.C execution test FAIL: g++.target/riscv/rvv/base/bug-13.
Re: RISC-V Test Errors and Failures
On 5/16/23 16:06, Palmer Dabbelt wrote: A few of us were talking about test-related issues in the patchwork meeting this morning. I bumped to trunk and did a full rebuild, I'm getting the following (it's in riscv-systems-ci/riscv-gnu-toolchain). This is about what I remember seeing last time I ran the tests, which was a week or so ago. I figured it'd be best to just blast the lists, as Jeff said his test running had been hanging so there might be some issue preventing folks from seeing the failures. I guess I didn't get time to look last time and I doubt things are looking any better right now. I'll try and take a look at some point, but any help would of course be appreciated. Yes I was seeing similar tcl errors and such - and in my case an even higher count. Also for posterity, what was your configure cmdline ? multilibs or no We really need to add some CI around RV toolchains to trip on these sooner ! $ cat toolchain/report make[1]: Entering directory '/scratch/merges/rgt-gcc-trunk/toolchain' /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/scripts/testsuite-filter gcc glibc /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/test/allowlist `find build-gcc-linux-stage2/gcc/testsuite/ -name *.sum |paste -sd "," -` === g++: Unexpected fails for rv64imac lp64 medlow === FAIL: g++.dg/contracts/contracts-tmpl-spec2.C output pattern test === g++: Unexpected fails for rv32imac ilp32 medlow === FAIL: g++.dg/contracts/contracts-tmpl-spec2.C output pattern test FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2a (test for excess errors) FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2b (test for excess errors) === g++: Unexpected fails for rv64imafdc lp64d medlow === FAIL: g++.dg/contracts/contracts-tmpl-spec2.C output pattern test === g++: Unexpected fails for rv32imafdc ilp32d medlow === FAIL: g++.dg/contracts/contracts-tmpl-spec2.C output pattern test FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2a (test for excess errors) FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2b (test for excess errors) === g++: Unexpected fails for rv64imafdcv lp64d === FAIL: g++.dg/contracts/contracts-tmpl-spec2.C output pattern test FAIL: g++.target/riscv/rvv/base/bug-10.C execution test FAIL: g++.target/riscv/rvv/base/bug-11.C execution test FAIL: g++.target/riscv/rvv/base/bug-12.C execution test FAIL: g++.target/riscv/rvv/base/bug-13.C execution test FAIL: g++.target/riscv/rvv/base/bug-14.C execution test FAIL: g++.target/riscv/rvv/base/bug-15.C execution test FAIL: g++.target/riscv/rvv/base/bug-16.C execution test FAIL: g++.target/riscv/rvv/base/bug-17.C execution test FAIL: g++.target/riscv/rvv/base/bug-2.C execution test FAIL: g++.target/riscv/rvv/base/bug-23.C execution test FAIL: g++.target/riscv/rvv/base/bug-3.C execution test FAIL: g++.target/riscv/rvv/base/bug-4.C execution test FAIL: g++.target/riscv/rvv/base/bug-5.C execution test FAIL: g++.target/riscv/rvv/base/bug-6.C execution test FAIL: g++.target/riscv/rvv/base/bug-7.C execution test FAIL: g++.target/riscv/rvv/base/bug-8.C execution test FAIL: g++.target/riscv/rvv/base/bug-9.C execution test === g++: Unexpected fails for rv32imafdcv ilp32d === FAIL: g++.dg/contracts/contracts-tmpl-spec2.C output pattern test FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2a (test for excess errors) FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2b (test for excess errors) FAIL: g++.target/riscv/rvv/base/bug-10.C execution test FAIL: g++.target/riscv/rvv/base/bug-11.C execution test FAIL: g++.target/riscv/rvv/base/bug-12.C execution test FAIL: g++.target/riscv/rvv/base/bug-13.C execution test FAIL: g++.target/riscv/rvv/base/bug-14.C (test for excess errors) FAIL: g++.target/riscv/rvv/base/bug-15.C execution test FAIL: g++.target/riscv/rvv/base/bug-16.C execution test FAIL: g++.target/riscv/rvv/base/bug-17.C execution test FAIL: g++.target/riscv/rvv/base/bug-18.C (test for excess errors) FAIL: g++.target/riscv/rvv/base/bug-19.C (test for excess errors) FAIL: g++.target/riscv/rvv/base/bug-2.C execution test FAIL: g++.target/riscv/rvv/base/bug-20.C (test for excess errors) FAIL: g++.target/riscv/rvv/base/bug-21.C (test for excess errors) FAIL: g++.target/riscv/rvv/base/bug-22.C (test for excess errors) FAIL: g++.target/riscv/rvv/base/bug-23.C execution test FAIL: g++.target/riscv/rvv/base/bug-3.C execution test FAIL: g++.target/riscv/rvv/base/bug-4.C execution test FAIL: g++.target/riscv/rvv/base/bug-5.C execution test FAIL: g++.target/riscv/rvv/base/bug-6.C execution test FAIL: g++.target/riscv/rvv/base/bug-7.C execution test FAIL: g++.target/riscv/rvv/base/bug-8.C execution test FAIL: g++.target/riscv/rvv/base/bug-9.C (test for excess errors) === g++: Unexpected fails for rv64gczba_zbb_zbc_zbs lp64d === FAIL: g++.dg/contracts/contracts-tmpl-spec2.C output pattern test === gcc: Unexpected fails for rv64imac lp64 medlow === ERROR: tcl error sourcing /scratch/
