[Bug 1872945] Re: risc-v doubles getting clobbered somehow
This bug was fixed in the package qemu - 1:4.2-3ubuntu6 --- qemu (1:4.2-3ubuntu6) focal; urgency=medium [ Christian Ehrhardt ] * enable riscv build (LP: #1872931) [ changes picked from Debian ] - enable support for riscv64 hosts - only enable librbd on architectures where it is built - ceph: do not list librados-dev as we only use librbd-dev and the latter depends on the former - seccomp grew up, no need in versioned build-dep - enable seccomp only on architectures where it can be built * d/p/ubuntu/lp-1872931-*: fix build on non KVM platforms * d/p/ubuntu/lp-1872945-*: fix riscv emulation errors that e.g. hung ssh and clobbered doubles (LP: #1872945) [ William Grant ] * d/control-in: disable rbd support unavailable on riscv (LP: 1872931) -- Christian Ehrhardt Wed, 15 Apr 2020 14:27:15 +0200 ** Changed in: qemu (Ubuntu) Status: Triaged => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1872945 Title: risc-v doubles getting clobbered somehow To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1872945/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1872945] Re: risc-v doubles getting clobbered somehow
Sorry, I said 4.2-3ubuntu6 but meant 1:4.2-3ubuntu6~ppa5 from the PPA (which works). Sounds like Colin got 1:4.2-3ubuntu6 from the PPA; I will test 1:4.2-3ubuntu6 from -proposed to be sure. $ apt-cache policy qemu-system-misc qemu-system-misc: Installed: 1:4.2-3ubuntu6~ppa5 Candidate: 1:4.2-3ubuntu6~ppa5 Version table: *** 1:4.2-3ubuntu6~ppa5 500 500 http://ppa.launchpad.net/ci-train-ppa-service/4015/ubuntu focal/main amd64 Packages 100 /var/lib/dpkg/status 1:4.2-3ubuntu5 500 500 http://us.archive.ubuntu.com/ubuntu focal/universe amd64 Packages -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1872945 Title: risc-v doubles getting clobbered somehow To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1872945/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1872945] Re: risc-v doubles getting clobbered somehow
Yep, 1:4.2-3ubuntu6 from focal-proposed test case works for me. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1872945 Title: risc-v doubles getting clobbered somehow To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1872945/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1872945] Re: risc-v doubles getting clobbered somehow
I think you got the wrong version. PPA version 1:4.2-3ubuntu6~ppa5 has the fix, archive version 1:4.2-3ubuntu6 has the fix, but PPA version 1:4.2-3ubuntu6 (which was to be archive -3ubuntu6, but while it was waiting in the queue we found this fix) doesn't have the fix. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1872945 Title: risc-v doubles getting clobbered somehow To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1872945/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1872945] Re: risc-v doubles getting clobbered somehow
Updated with the new version in the PPA: dpkg -S qemu-system-riscv64 qemu-system-misc: /usr/share/man/man1/qemu-system-riscv64.1.gz qemu-system-misc: /usr/bin/qemu-system-riscv64 cking@skylake:~/risc-v-ubuntu$ dpkg -l | grep qemu-system-misc ii qemu-system-misc 1:4.2-3ubuntu6 amd64QEMU full system emulation binaries (miscellaneous) and tested the reproducer: [00:02:05.328] cking@ubuntu:~$ ./a.out [00:04:29.189] 1587408297.940333 0.00 -1587408297.940333 [00:04:29.190] START : 1587408297.940333 1587408297 940333 (5e9deda9 e592d) errno=0 [00:04:29.191] NOW BAD: 0.00 1587408440 627126 (5e9dee38 991b6) errno=0 [00:04:29.192] NOW OK: 1587408440.633365 1587408440 633365 (5e9dee38 9aa15) errno=0 so still an issue for me. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1872945 Title: risc-v doubles getting clobbered somehow To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1872945/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1872945] Re: risc-v doubles getting clobbered somehow
Confirmed qemu 4.2-3ubuntu6 fixes the OpenSSH sshd hang on riscv64 guest. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1872945 Title: risc-v doubles getting clobbered somehow To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1872945/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1872945] Re: risc-v doubles getting clobbered somehow
As coordinated with Wgrant who is working on riscv in general I uploaded 4.2-3ubuntu6 to focal-unapproved including this fix. Pre-build uploaded to: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/4015/+packages General pre-builds (before this fix) are also in https://launchpad.net/~wgrant/+archive/ubuntu/nonvirt/+packages -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1872945 Title: risc-v doubles getting clobbered somehow To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1872945/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1872945] Re: risc-v doubles getting clobbered somehow
Trying Ckings testcase on a 4.2 build with those five patches on top. I can reproduce it on 4.2 - 1st Try ~ 3 seconds - 2st Try ~ 50 seconds On 4.2 plus the fixes it works for ~30 minutes now I'll run a few more to be sure but it looks good. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1872945 Title: risc-v doubles getting clobbered somehow To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1872945/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1872945] Re: risc-v doubles getting clobbered somehow
Started 4 of ckings programs. - Broken qemu ~1 hit per 15 seconds - Fixed qemu good for ~30 minutes I think we can add those patches ... -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1872945 Title: risc-v doubles getting clobbered somehow To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1872945/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1872945] Re: risc-v doubles getting clobbered somehow
** Changed in: qemu (Ubuntu) Status: New => Triaged -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1872945 Title: risc-v doubles getting clobbered somehow To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1872945/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1872945] Re: risc-v doubles getting clobbered somehow
FYI v5.0.0-rc3 - working as well Bisect log (looking for the fix): git bisect start # old: [b0ca999a43a22b38158a33d3f5881648bb4f] Update version for v4.2.0 release git bisect old b0ca999a43a22b38158a33d3f5881648bb4f # new: [20038cd7a8412feeb49c01f6ede89e36c8995472] Update version for v5.0.0-rc3 release git bisect new 20038cd7a8412feeb49c01f6ede89e36c8995472 # new: [971b2a1e5b1a8cc8f597ac5d7016908f9fa880de] Merge remote-tracking branch 'remotes/palmer/tags/riscv-for-master-5.0-sf2' into staging git bisect new 971b2a1e5b1a8cc8f597ac5d7016908f9fa880de # old: [bd5de61e7b926b5bb9dae254630bf3c8be575bc0] migration: add savevm_state_handler_remove() git bisect old bd5de61e7b926b5bb9dae254630bf3c8be575bc0 # new: [cd45c506c8ec37c05fdfe06441ad350ab8e19138] hw/s390x/ipl: replace deprecated qdev_reset_all registration git bisect new cd45c506c8ec37c05fdfe06441ad350ab8e19138 # old: [cdc497c6925be745bc895355bd4674a17a4b2a8b] virtiofsd: prevent FUSE_INIT/FUSE_DESTROY races git bisect old cdc497c6925be745bc895355bd4674a17a4b2a8b # new: [760df0d121a836dcbf3726b80b820115aef21b30] Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging git bisect new 760df0d121a836dcbf3726b80b820115aef21b30 # old: [4f67d30b5e74e060b8dbe10528829b47345cd6e8] qdev: set properties with device_class_set_props() git bisect old 4f67d30b5e74e060b8dbe10528829b47345cd6e8 # old: [c0248b36d8d190933a43919b9f71013a255e866c] Merge remote-tracking branch 'remotes/kraxel/tags/ui-20200123-pull-request' into staging git bisect old c0248b36d8d190933a43919b9f71013a255e866c # old: [a4abb6f2061f3a40f01cfe4c79c1527db655ac89] hw/misc/stm32f4xx_syscfg: Fix copy/paste error git bisect old a4abb6f2061f3a40f01cfe4c79c1527db655ac89 # new: [ba2ed84fe6a78f64b2da441750fc6e925d94106a] Merge remote-tracking branch 'remotes/palmer/tags/riscv-for-master-5.0-sf1' into staging git bisect new ba2ed84fe6a78f64b2da441750fc6e925d94106a # old: [1d59b1b210d7c3b0bdf4b10ebe0bb1fccfcb8b95] virtiofsd: add some options to the help message git bisect old 1d59b1b210d7c3b0bdf4b10ebe0bb1fccfcb8b95 # old: [613fa160e19abe8e1fe44423fcfa8ec73d3d48e5] target/riscv: Fix tb->flags FS status git bisect old 613fa160e19abe8e1fe44423fcfa8ec73d3d48e5 # old: [a37f21c27d3e2342c2080aafd4cfe7e949612428] riscv: Set xPIE to 1 after xRET git bisect old a37f21c27d3e2342c2080aafd4cfe7e949612428 # new: [82f014671cf057de51c4a577c9e2ad637dcec6f9] target/riscv: update mstatus.SD when FS is set dirty git bisect new 82f014671cf057de51c4a577c9e2ad637dcec6f9 # old: [a59796eb6d59bbd74ce28ddbddb1b83e60674e96] target/riscv: fsd/fsw doesn't dirty FP state git bisect old a59796eb6d59bbd74ce28ddbddb1b83e60674e96 # first new commit: [82f014671cf057de51c4a577c9e2ad637dcec6f9] target/riscv: update mstatus.SD when FS is set dirty $ git bisect old 82f014671cf057de51c4a577c9e2ad637dcec6f9 is the first new commit commit 82f014671cf057de51c4a577c9e2ad637dcec6f9 Author: ShihPo Hung Date: Tue Jan 14 22:17:33 2020 -0800 target/riscv: update mstatus.SD when FS is set dirty remove the check becuase SD bit should summarize FS and XS fields unconditionally. Signed-off-by: ShihPo Hung Reviewed-by: Richard Henderson Reviewed-by: Alistair Francis Signed-off-by: Palmer Dabbelt target/riscv/csr.c | 3 +-- target/riscv/translate.c | 2 +- 2 files changed, 2 insertions(+), 3 deletions(-) I double checked - 82f014671cf057de51c4a577c9e2ad637dcec6f9 - works - a59796eb6d59bbd74ce28ddbddb1b83e60674e96 - fails Also [10:49] Both 613fa16 and 82f0146 revert parts of b345b48 [10:51] Confirmed that those two on top of v4.2.0 gets sshd good But we might need/want more of that series ?! 97a254b3f0 target/openrisc: Fix FPCSR mask to allow setting DZF 82f014671c target/riscv: update mstatus.SD when FS is set dirty a59796eb6d target/riscv: fsd/fsw doesn't dirty FP state 613fa160e1 target/riscv: Fix tb->flags FS status a37f21c27d riscv: Set xPIE to 1 after xRET bb8136df69 riscv/sifive_u: fix a memory leak in soc_realize() -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1872945 Title: risc-v doubles getting clobbered somehow To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1872945/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1872945] Re: risc-v doubles getting clobbered somehow
mistyped before: - broken 4.1 -> 4.2 - some fix in 4-2 -> master -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1872945 Title: risc-v doubles getting clobbered somehow To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1872945/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1872945] Re: risc-v doubles getting clobbered somehow
Test via https://people.ubuntu.com/~wgrant/riscv64/ Talking with Wgrant there also was an atomic fix that he has on his build. The upstream version of it is [1] and some other bits in that series. That is in since 4.0. Another testcase (unsure if it is the same root cause, but easier to test for now). Start via: $ qemu-system-riscv64 -machine virt -m 8192 -smp 4 -kernel bbl-vmlinuz-5.4.0-23-generic -initrd initrd.img-5.4.0-23-generic -append "console=ttyS0 rw noquiet root=/dev/vda1" -device virtio-blk-device,drive=vda -drive file=focal-riscv64-20200411.qcow2,id=vda -device virtio-net-device,netdev=eth0 -netdev user,id=eth0,hostfwd=tcp::2-:22 Check SSH which will hang: $ ssh -v root@localhost -p 2 ... debug1: SSH2_MSG_SERVICE_ACCEPT received Wgrant reports that this also worked with his patched 3.1 but fails with 4.2 as in Ubuntu. I tried that with: - qemu 4.2 Ubuntu archive - failing - git master (5.0-rc) - working - git 4.0 - working - git 4.1 - working - git 4.2 - failing So it is not in the Ubuntu Delta but between 5.1 and 4.2. I think I can bisect from here - once we know what it is we can cross check if it is the same root case that also caused ckings failure. [1]: https://git.qemu.org/?p=qemu.git;a=commit;h=c13b169f1a3dd158d6c75727cdc388f95988db39 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1872945 Title: risc-v doubles getting clobbered somehow To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1872945/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1872945] Re: risc-v doubles getting clobbered somehow
Past 22 hours now and still no issues, even though I've intermittently been saturating all cores with builds. So yeah, the kernel is probably fine. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1872945 Title: risc-v doubles getting clobbered somehow To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1872945/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1872945] Re: risc-v doubles getting clobbered somehow
Thanks William for verifying this. At least I can stop worrying about registers getting trashed by context switching by the kernel. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1872945 Title: risc-v doubles getting clobbered somehow To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1872945/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1872945] Re: risc-v doubles getting clobbered somehow
I've been running your test code on a HiFive Unleashed board with our 5.4.0-24 for a couple of hours so far, and no issues yet. I saw widespread crashes with qemu 4.2 that went away when downgrading (though I went straight back to 3.1), so I'm pretty confident there is a qemu bug, but I haven't had time to dig into it. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1872945 Title: risc-v doubles getting clobbered somehow To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1872945/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1872945] Re: risc-v doubles getting clobbered somehow
A bit more debugging with the same issue on stress-ng and it seems to occur during logging when using the vsnprintf family of va_arg printing. My current hypothesis is that the va_args handling with formatting doubles to a string is randomly clobbering doubles somehow. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1872945 Title: risc-v doubles getting clobbered somehow To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1872945/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1872945] Re: risc-v doubles getting clobbered somehow
This does not happen when executing the *same* binary with qemu- riscv64-static -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1872945 Title: risc-v doubles getting clobbered somehow To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1872945/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1872945] Re: risc-v doubles getting clobbered somehow
OK, not a kernel bug. I'm getting random clobbering when printing via the printf() family of calls when passing a double. Sometimes it's corrupting passed variables, it looks like this only occurs when using va_args and doubles. ** No longer affects: linux (Ubuntu) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1872945 Title: risc-v doubles getting clobbered somehow To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1872945/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1872945] Re: risc-v doubles getting clobbered somehow
Does this occur on real H/W? Can somebody with risc-v H/W test this out? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1872945 Title: risc-v doubles getting clobbered somehow To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1872945/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
