Yes, some sort of fixed environment would solve the problem I think. Really for reproducibility in general dynamic binaries should not resolve libraries on the host, they should have their own set of libraries as if they were cross compiled. Updating the host (like I did) should not change the behavior of experiments, especially since that's often outside of people's control and may even happen silently in the background without their knowledge. This is particularly important for tests though, since those are more or less handed down as a standard by gem5 itself, unlike how experiments are designed which I think is more of an active choice.
The stdout matching test probably failed because the underlying test failed, and didn't produce the expected "Hello world" (exact text may differ). For such a simple test program, 99% of the work happens initializing the standard library, so most problems would crop up before main, aka any visible output. Gabe On Mon, May 24, 2021 at 1:12 PM Jason Lowe-Power via gem5-dev < gem5-dev@gem5.org> wrote: > We shouldn't remove that test! It's a important feature that people rely > on. > > The solution would be to run the test within a "known" environment. The > docker containers we provide should work perfectly for that. > > Cheers, > Jason > > On Mon, May 24, 2021 at 11:54 AM Bobby Bruce via gem5-dev < > gem5-dev@gem5.org> wrote: > >> Hey Gabe, >> >> When you say the 'hello' binary is dynamically linked, we're talking >> about the 'hello64-dynamic' binary? I agree these tests should probably be >> removed. (Does anyone have any objection to this?) >> >> This only appears to be half your problem though, as you've got >> "MatchStdoutNoPerf" tests failing also. Did you ever figure out why these >> tests are failing for you? These tests fail when the STDOUT of the test has >> changed (each test has a "ref" file somewhere recording what the output >> should be. It's more or less a simple diff). >> >> Bobby >> -- >> Dr. Bobby R. Bruce >> Room 3050, >> Kemper Hall, UC Davis >> Davis, >> CA, 95616 >> >> web: https://www.bobbybruce.net >> >> >> On Sat, May 22, 2021 at 4:17 AM Gabe Black via gem5-dev < >> gem5-dev@gem5.org> wrote: >> > >> > Well, I think I figured out the problem. The GCN3 hello world >> executable is dynamically linked, and I'm assuming it's pulling in >> libraries from my system. That seems to use a system call that's not >> implemented, see below. These tests should *not* be dependent on the host >> system to work correctly! I will not be able to run regressions on my >> machine until this is fixed... >> > >> > $ cat >> ./testing-results/SuiteUID:test-hello64-dynamic-DerivO3CPU-GCN3_X86-x86_64-opt/TestUID:test-hello64-dynami >> > c-DerivO3CPU-GCN3_X86-x86_64-opt/simerr >> > warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports` >> > warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports` >> > warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports` >> > warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports` >> > warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports` >> > warn: membus.master is deprecated. `master` is now called >> `mem_side_ports` >> > warn: membus.master is deprecated. `master` is now called >> `mem_side_ports` >> > warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports` >> > build/GCN3_X86/mem/mem_interface.cc:785: warn: DRAM device capacity >> (8192 Mbytes) does not match the address range assigned (512 Mb >> > ytes) >> > 0: system.remote_gdb: listening for remote gdb on port 7001 >> > build/GCN3_X86/sim/simulate.cc:104: info: Entering event queue @ 0. >> Starting simulation... >> > build/GCN3_X86/arch/x86/cpuid.cc:177: warn: x86 cpuid family 0x0000: >> unimplemented function 13 >> > build/GCN3_X86/sim/syscall_emul.cc:70: warn: ignoring syscall >> access(...) >> > build/GCN3_X86/sim/mem_state.cc:440: info: Increasing stack size by one >> page. >> > build/GCN3_X86/sim/syscall_emul.cc:63: fatal: syscall newfstatat (#262) >> unimplemented. >> > Memory Usage: 653788 KBytes >> > >> > On Sat, May 22, 2021 at 3:35 AM Gabe Black <gabe.bl...@gmail.com> >> wrote: >> >> >> >> Even that failed... Maybe something ended up damaged in one of the >> docker images? I don't think those are used, but I'm not sure what else it >> could be... I'll have docker clear all those and try again. >> >> >> >> Gabe >> >> >> >> On Sat, May 22, 2021 at 3:17 AM Gabe Black <gabe.bl...@gmail.com> >> wrote: >> >>> >> >>> Hmm, I thought this might be that something was stale, but I did a >> fresh checkout and the same tests fail at ToT there too. I'm trying on a >> much older checkout since hopefully I'll be able to get back to a point >> where the tests pass... If not, then there's something screwy about my >> setup, although I'm at a loss to say what. >> >>> >> >>> Gabe >> >>> >> >>> On Sat, May 22, 2021 at 12:51 AM Gabe Black <gabe.bl...@gmail.com> >> wrote: >> >>>> >> >>>> This isn't a build issue, this is failing tests (see below). It's >> hard to extract more information than that from the test runs, so I can't >> say much more. It could be something got messed up in my local environment >> somehow? I'm still trying to find a revision that works, but there may be >> something stale in my build directory which is causing problems which I'm >> trying to clear out. >> >>>> >> >>>> Gabe >> >>>> >> >>>> Test: test-hello64-static-TimingSimpleCPU-GCN3_X86-x86_64-opt Passed >> >>>> Test: >> test-hello64-static-TimingSimpleCPU-GCN3_X86-x86_64-opt-MatchStdoutNoPerf >> Failed >> >>>> Redirecting stdout to /tmp/gem5outijxfv1tq/simout >> >>>> Redirecting stderr to /tmp/gem5outijxfv1tq/simerr >> >>>> Test: test-hello64-static-AtomicSimpleCPU-GCN3_X86-x86_64-opt Passed >> >>>> Test: >> test-hello64-static-AtomicSimpleCPU-GCN3_X86-x86_64-opt-MatchStdoutNoPerf >> Failed >> >>>> Redirecting stdout to /tmp/gem5outeg1h1b3e/simout >> >>>> Redirecting stderr to /tmp/gem5outeg1h1b3e/simerr >> >>>> Test: test-hello64-static-DerivO3CPU-GCN3_X86-x86_64-opt Passed >> >>>> Test: >> test-hello64-static-DerivO3CPU-GCN3_X86-x86_64-opt-MatchStdoutNoPerf Failed >> >>>> Redirecting stdout to /tmp/gem5outfcqm4wyl/simout >> >>>> Redirecting stderr to /tmp/gem5outfcqm4wyl/simerr >> >>>> Test: test-hello32-static-TimingSimpleCPU-GCN3_X86-x86_64-opt Passed >> >>>> Test: >> test-hello32-static-TimingSimpleCPU-GCN3_X86-x86_64-opt-MatchStdoutNoPerf >> Failed >> >>>> Redirecting stdout to /tmp/gem5out9p26oogk/simout >> >>>> Redirecting stderr to /tmp/gem5out9p26oogk/simerr >> >>>> Test: test-hello32-static-AtomicSimpleCPU-GCN3_X86-x86_64-opt Passed >> >>>> Test: >> test-hello32-static-AtomicSimpleCPU-GCN3_X86-x86_64-opt-MatchStdoutNoPerf >> Failed >> >>>> Redirecting stdout to /tmp/gem5outyn_zm9ks/simout >> >>>> Redirecting stderr to /tmp/gem5outyn_zm9ks/simerr >> >>>> Test: test-hello32-static-DerivO3CPU-GCN3_X86-x86_64-opt Passed >> >>>> Test: >> test-hello32-static-DerivO3CPU-GCN3_X86-x86_64-opt-MatchStdoutNoPerf Failed >> >>>> Redirecting stdout to /tmp/gem5outxx9s4yse/simout >> >>>> Redirecting stderr to /tmp/gem5outxx9s4yse/simerr >> >>>> Test: test-hello64-static-TimingSimpleCPU-ARM-x86_64-opt Passed >> >>>> Test: >> test-hello64-static-TimingSimpleCPU-ARM-x86_64-opt-MatchStdoutNoPerf Failed >> >>>> Redirecting stdout to /tmp/gem5out91qvmwnt/simout >> >>>> Redirecting stderr to /tmp/gem5out91qvmwnt/simerr >> >>>> Test: test-hello64-static-AtomicSimpleCPU-ARM-x86_64-opt Passed >> >>>> Test: >> test-hello64-static-AtomicSimpleCPU-ARM-x86_64-opt-MatchStdoutNoPerf Failed >> >>>> Redirecting stdout to /tmp/gem5out6y5lsj7b/simout >> >>>> Redirecting stderr to /tmp/gem5out6y5lsj7b/simerr >> >>>> Test: test-hello64-static-DerivO3CPU-ARM-x86_64-opt Passed >> >>>> Test: >> test-hello64-static-DerivO3CPU-ARM-x86_64-opt-MatchStdoutNoPerf Failed >> >>>> Redirecting stdout to /tmp/gem5out0mupa7_1/simout >> >>>> Redirecting stderr to /tmp/gem5out0mupa7_1/simerr >> >>>> Test: test-hello32-static-TimingSimpleCPU-ARM-x86_64-opt Passed >> >>>> Test: >> test-hello32-static-TimingSimpleCPU-ARM-x86_64-opt-MatchStdoutNoPerf Failed >> >>>> Redirecting stdout to /tmp/gem5outel9p_9hi/simout >> >>>> Redirecting stderr to /tmp/gem5outel9p_9hi/simerr >> >>>> Test: test-hello32-static-AtomicSimpleCPU-ARM-x86_64-opt Passed >> >>>> Test: >> test-hello32-static-AtomicSimpleCPU-ARM-x86_64-opt-MatchStdoutNoPerf Failed >> >>>> Redirecting stdout to /tmp/gem5outxybwrfn2/simout >> >>>> Redirecting stderr to /tmp/gem5outxybwrfn2/simerr >> >>>> Test: test-hello32-static-DerivO3CPU-ARM-x86_64-opt Passed >> >>>> Test: >> test-hello32-static-DerivO3CPU-ARM-x86_64-opt-MatchStdoutNoPerf Failed >> >>>> Redirecting stdout to /tmp/gem5outlhyny0vi/simout >> >>>> Redirecting stderr to /tmp/gem5outlhyny0vi/simerr >> >>>> Test: test-hello-TimingSimpleCPU-RISCV-x86_64-opt Passed >> >>>> Test: test-hello-TimingSimpleCPU-RISCV-x86_64-opt-MatchStdoutNoPerf >> Failed >> >>>> Redirecting stdout to /tmp/gem5outvwqx5_8_/simout >> >>>> Redirecting stderr to /tmp/gem5outvwqx5_8_/simerr >> >>>> Test: test-hello-AtomicSimpleCPU-RISCV-x86_64-opt Passed >> >>>> Test: test-hello-AtomicSimpleCPU-RISCV-x86_64-opt-MatchStdoutNoPerf >> Failed >> >>>> Redirecting stdout to /tmp/gem5out3dt2vzne/simout >> >>>> Redirecting stderr to /tmp/gem5out3dt2vzne/simerr >> >>>> Test: test-hello-DerivO3CPU-RISCV-x86_64-opt Passed >> >>>> Test: test-hello-DerivO3CPU-RISCV-x86_64-opt-MatchStdoutNoPerf Failed >> >>>> Redirecting stdout to /tmp/gem5outi3op2hir/simout >> >>>> Redirecting stderr to /tmp/gem5outi3op2hir/simerr >> >>>> Test: test-hello-MinorCPU-RISCV-x86_64-opt Passed >> >>>> Test: test-hello-MinorCPU-RISCV-x86_64-opt-MatchStdoutNoPerf Failed >> >>>> Redirecting stdout to /tmp/gem5outyfy82zlg/simout >> >>>> Redirecting stderr to /tmp/gem5outyfy82zlg/simerr >> >>>> Test: test-hello64-dynamic-TimingSimpleCPU-GCN3_X86-x86_64-opt Failed >> >>>> Test: >> test-hello64-dynamic-TimingSimpleCPU-GCN3_X86-x86_64-opt-MatchStdoutNoPerf >> Failed >> >>>> Redirecting stdout to /tmp/gem5outz38uz_tb/simout >> >>>> Redirecting stderr to /tmp/gem5outz38uz_tb/simerr >> >>>> Test: test-hello64-dynamic-AtomicSimpleCPU-GCN3_X86-x86_64-opt Failed >> >>>> Test: >> test-hello64-dynamic-AtomicSimpleCPU-GCN3_X86-x86_64-opt-MatchStdoutNoPerf >> Failed >> >>>> >> >>>> >> >>>> On Fri, May 21, 2021 at 10:10 PM Matt Sinclair <sincl...@cs.wisc.edu> >> wrote: >> >>>>> >> >>>>> Hi Gabe, >> >>>>> >> >>>>> I'm not sure which error you are seeing, but develop is currently >> broken for GCN3_X86 yes. The hope is the patches you pushed yesterday (to >> fix the gcc < 8 problems) will resolve the problem. Is that not what >> you're seeing? >> >>>>> >> >>>>> Previous to those patches, Bobby and Kyle did the bisecting and >> found the patch Bobby and I commented on yesterday was the one that seemed >> to be causing the problems (which prompted your new patches). >> >>>>> >> >>>>> Thanks, >> >>>>> Matt >> >>>>> >> >>>>> On Fri, May 21, 2021 at 9:45 PM Gabe Black via gem5-dev < >> gem5-dev@gem5.org> wrote: >> >>>>>> >> >>>>>> Hey folks, I'm trying to test a change of mine, and it looks like >> the tests are broken on ToT for GCN3? Not sure if it's just something >> that's out of whack on my system, or if incompatible patches passed each >> other on gerrit. >> >>>>>> >> >>>>>> I'm attempting to bisect where things went wrong, but I wouldn't >> complain if anyone else wants to try to figure it out too. >> >>>>>> >> >>>>>> Gabe >> >>>>>> _______________________________________________ >> >>>>>> gem5-dev mailing list -- gem5-dev@gem5.org >> >>>>>> To unsubscribe send an email to gem5-dev-le...@gem5.org >> >>>>>> %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s >> > >> > _______________________________________________ >> > gem5-dev mailing list -- gem5-dev@gem5.org >> > To unsubscribe send an email to gem5-dev-le...@gem5.org >> > %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s >> _______________________________________________ >> gem5-dev mailing list -- gem5-dev@gem5.org >> To unsubscribe send an email to gem5-dev-le...@gem5.org >> %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s > > _______________________________________________ > gem5-dev mailing list -- gem5-dev@gem5.org > To unsubscribe send an email to gem5-dev-le...@gem5.org > %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
_______________________________________________ gem5-dev mailing list -- gem5-dev@gem5.org To unsubscribe send an email to gem5-dev-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
