[Bug 2069674] Re: Stress Tests failed during Server Certification test
** Package changed: ubuntu => stress-ng -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2069674 Title: Stress Tests failed during Server Certification test To manage notifications about this bug go to: https://bugs.launchpad.net/stress-ng/+bug/2069674/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2069674] Re: Stress Tests failed during Server Certification test
** Attachment added: "CPUStress-MultipleStressor-Fail.zip" https://bugs.launchpad.net/ubuntu/+bug/2069674/+attachment/5800374/+files/CPUStress-MultipleStressor-Fail.zip -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2069674 Title: Stress Tests failed during Server Certification test To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+bug/2069674/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2069674] Re: Stress Tests failed during Server Certification test
update the test status and need help to move forward: For CPU Stress, with CPU 960 Cores, af-alg stressors fails when multiple stressors execute as below such error, see attachment "CPUStress-MultipleStressor-Fail.zip" for reference. stress-ng: fail: [256886] af-alg: xts(aes): read failed: errno=22 (Invalid argument) stress-ng: debug: [256886] af-alg: [256886] exited (instance 32 on CPU 263) stress-ng: info: [256780] for a 9620.77s run time: stress-ng: info: [256780] 9235939.57s available CPU time stress-ng: info: [256780] 5689357.95s user time ( 61.60%) stress-ng: info: [256780] 3386150.07s system time ( 36.66%) stress-ng: info: [256780] 9075508.02s total time ( 98.26%) stress-ng: info: [256780] load average: 227.75 897.78 3301.64 stress-ng: info: [256780] skipped: 0 stress-ng: info: [256780] passed: 13994: af-alg (554) bsearch (960) context (960) cpu (960) crypt (960) hsearch (960) longjmp (960) lsearch (960) matrix (960) qsort (960) str (960) stream (960) tsearch (960) vecmath (960) wcs (960) stress-ng: info: [256780] failed: 405: af-alg (405) stress-ng: info: [256780] metrics untrustworthy: 0 stress-ng: info: [256780] unsuccessful run completed in 2 hours, 40 mins, 20.77 secs -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2069674 Title: Stress Tests failed during Server Certification test To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+bug/2069674/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2069674] Re: Stress Tests failed during Server Certification test
Update the latest status for CPU Stress: 1. with reduced CPU 480, (from 60 Core to 30Core for each), the CPU Stress can achieve Pass result. 2. with CPU 960 Core, Extend the execution time, fail on both 4 hours and 6 hours. 3. with CPU 960 Cores, execute the test with command stress-ng and figure the --af-alg stressor failed. sudo stress-ng --aggressive --verify --timeout 7200 --metrics-brief --tz --times --verbose --af-alg 960 --bsearch 960 --context 960 --cpu 960 --crypt 960 --hsearch 960 --longjmp 960 --lsearch 960 --matrix 960 --qsort 960 --str 960 --stream 960 --tsearch 960 --vecmath 960 --wcs 960 >StresssngTest.log 4. with CPU 960 Cores, investigate the individual stressor --af-alg result. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2069674 Title: Stress Tests failed during Server Certification test To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+bug/2069674/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2069674] Re: Stress Tests failed during Server Certification test
** Attachment added: "Stress-SR950V3-0722.zip" https://bugs.launchpad.net/ubuntu/+bug/2069674/+attachment/5799236/+files/Stress-SR950V3-0722.zip -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2069674 Title: Stress Tests failed during Server Certification test To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+bug/2069674/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2069674] Re: Stress Tests failed during Server Certification test
Hi Michael, with Swap file >2 times of MEm size, the stress test on CPU/MEM still fail as time out. see attachment "Stress-SR950V3-0722.zip" for detailed info reference. besides, I do the other investigation, separate the CPU and MEM stress, till now, CPU Stress still failed as below, MEM stress is still under ongoing, assume it root caused the high cores, will reduce the core count to see the result once after MEM stress test completed tomorrow: ubuntu@SR950V3:~$ sudo /usr/lib/checkbox-provider-base/bin/stress_ng_test.py cpu -b 7200 Estimated total run time is 120 minutes 22 Jul 03:52: Running multiple stress-ng stressors in parallel for 7200 seconds... ** stress-ng timed out and was forcefully terminated retval is 1 ** ** stress-ng test failed! ** ubuntu@SR950V3:~$ sudo /usr/lib/checkbox-provider-base/bin/stress_ng_test.py memory -b 300 -t 10 -s 0 -k Minimum swap space is set to 0 GiB Total memory is 2015.1 GiB Constant run time is 300 seconds per stressor Variable run time is 20451 seconds per stressor Number of NUMA nodes is 8 Estimated total run time is 1473 minutes 22 Jul 07:13: Running stress-ng bsearch stressor for 300 seconds... stress-ng: info: [1451389] setting to a 5 mins run per stressor stress-ng: info: [1451389] dispatching hogs: 960 bsearch stress-ng: warn: [1451389] WARNING! using HPET clocksource (refer to /sys/devices/system/clocksource/clocksource0), this may impact benchmarking performance stress-ng: info: [1451389] skipped: 0 stress-ng: info: [1451389] passed: 959: bsearch (959) stress-ng: info: [1451389] failed: 0 stress-ng: info: [1451389] metrics untrustworthy: 0 stress-ng: info: [1451389] successful run completed in 5 mins -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2069674 Title: Stress Tests failed during Server Certification test To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+bug/2069674/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2069674] Re: Stress Tests failed during Server Certification test
Hi Michael, The test case "test-stress" run stress-ng and keeps running about 20 hours, CPU usage is about 100% and there is CPU Hard Lockup under dmesg. ubuntu@SR950V3:/var/log$ cat dmesg | grep -i lockup [ 17.361583] kernel: watchdog: Watchdog detected hard LOCKUP on cpu 817 [ 17.361583] kernel: ? watchdog_hardlockup_check+0x1cb/0x3b0 due to the system operation feedback very slow, i will collect the log once it is better response status. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2069674 Title: Stress Tests failed during Server Certification test To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+bug/2069674/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2069674] Re: Stress Tests failed during Server Certification test
Hi Michael, For Disk Stress, the current latest Kernel can fix the issue and achieve pass result, see attachment "Disk-Test_2024-07-18T05.30.24.485316.html". For CPU Stress and Mem Stress, we expend the swap size and ongoing with the test execution, due to Physical CPU Core=60*8=480 and Physical MEM=256GB*8=2TB, the stress takes more time than ever, 1-2 days, will do the update once we get any result later then. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2069674 Title: Stress Tests failed during Server Certification test To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+bug/2069674/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2069674] Re: Stress Tests failed during Server Certification test
** Attachment added: "Disk-Test_2024-07-18T05.30.24.485316.html" https://bugs.launchpad.net/ubuntu/+bug/2069674/+attachment/5798297/+files/Disk-Test_2024-07-18T05.30.24.485316.html -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2069674 Title: Stress Tests failed during Server Certification test To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+bug/2069674/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2069674] Re: Stress Tests failed during Server Certification test
Thank you Amy for the stress-ng version information. One thing we can try is increasing the swap space to 2X (RAM on the system) There are occasions where this may help. Please run test-stress to run memory and cpu stres-ng after increasing the swap space. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2069674 Title: Stress Tests failed during Server Certification test To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+bug/2069674/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2069674] Re: Stress Tests failed during Server Certification test
hi Michael, for Stress-ng version, the System is ruined for the other OS test, here is the version from the log for your reference: name: stress-ng version: 0.17.08-0~202405302230~ubuntu24.04.1 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2069674 Title: Stress Tests failed during Server Certification test To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+bug/2069674/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2069674] Re: Stress Tests failed during Server Certification test
What version of stress-ng are you running? Install devscripts sudo apt install devscripts Find the version rmadison stress-ng | grep noble -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2069674 Title: Stress Tests failed during Server Certification test To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+bug/2069674/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2069674] Re: Stress Tests failed during Server Certification test
hi Michael, for Stress_ng CPU,it fails as timeout. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2069674 Title: Stress Tests failed during Server Certification test To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+bug/2069674/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2069674] Re: Stress Tests failed during Server Certification test
The stress_ng cpu tests appears to have timed out? Or did you terminate the test manually? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2069674 Title: Stress Tests failed during Server Certification test To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+bug/2069674/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2069674] Re: Stress Tests failed during Server Certification test
It appears that disk stress also failed and the system crashed? This has been fixed in a later kernel in version 6.8.0-35.35. The version that is deployed in these results is 6.8.0-31.31. The fix is described in this bug https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2058557.Once the kernel is updated run test-storage to see if disk stress-ng passes. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2069674 Title: Stress Tests failed during Server Certification test To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+bug/2069674/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2069674] Re: Stress Tests failed during Server Certification test
Hi Ubuntu team, is there any update for the defect? Many thanks. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2069674 Title: Stress Tests failed during Server Certification test To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+bug/2069674/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2069674] Re: Stress Tests failed during Server Certification test
** Project changed: launchpad => ubuntu -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2069674 Title: Stress Tests failed during Server Certification test To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+bug/2069674/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
