** Description changed: [impact] - these tests fail intermittently, due to read failures when the test - expects to read data, but the umockdev 'replay' script device doesn't - provide the data to read. + these tests fail intermittently, due to various timing issues, as well + as an actual code bug in how data fuzz is calculated. - this appears to just be a timing issue, as the test expects the umockdev device to provide specific data replayed at specific times, and those times are very short (ms). Looking at the autopkgtest results: - http://autopkgtest.ubuntu.com/packages/umockdev - - it looks like the failures are mostly on armhf. + it looks like the failures are mostly on armhf, but do happen less often + on other archs. [test case] check the previous autopkgtest logs, e.g. https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-cosmic/cosmic/armhf/u/umockdev/20190601_015323_8f795@/log.gz [regression potential] - the tests on armhf have been flaky ever since trusty, and people just - retry and/or ignore them; any regression would likely result in (still) - flaky tests and/or false positive or negative test results on armhf. + any regression would likely result in incorrectly failing/passing autopkgtests, or in a incorrect pass or incorrect fail of the ScriptRunner + to verify the data's level of fuzz. [scope] as the tests are flaky on armhf all the way back to trusty, this is needed for all releases. [other info] - the armhf testbeds are unfortunately "different" from all the other archs: - 1) they don't run directly in a virtual instance, they run in an armhf lxd container inside an arm64 virtual instance. - 2) all other archs get are run inside a small instance, with limited memory and 1 cpu, but the armhf tests are run in what appears to be a large instance, and the armhf container inherits the 4 cpus that the arm64 instance has. + Fixed upstream in PR https://github.com/martinpitt/umockdev/pull/103 + + Most of the changes are test case fixes, but there is one fix to source + code to fix the ScriptRunning function that validates the level of fuzz + in data, so this is not *only* a testcase fix.
-- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to umockdev in Ubuntu. https://bugs.launchpad.net/bugs/1831467 Title: test-umockdev tests flaky on armhf (and sometimes other archs) Status in umockdev package in Ubuntu: In Progress Status in umockdev source package in Xenial: In Progress Status in umockdev source package in Bionic: In Progress Status in umockdev source package in Focal: In Progress Status in umockdev source package in Groovy: In Progress Bug description: [impact] these tests fail intermittently, due to various timing issues, as well as an actual code bug in how data fuzz is calculated. it looks like the failures are mostly on armhf, but do happen less often on other archs. [test case] check the previous autopkgtest logs, e.g. https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-cosmic/cosmic/armhf/u/umockdev/20190601_015323_8f795@/log.gz [regression potential] any regression would likely result in incorrectly failing/passing autopkgtests, or in a incorrect pass or incorrect fail of the ScriptRunner to verify the data's level of fuzz. [scope] as the tests are flaky on armhf all the way back to trusty, this is needed for all releases. [other info] Fixed upstream in PR https://github.com/martinpitt/umockdev/pull/103 Most of the changes are test case fixes, but there is one fix to source code to fix the ScriptRunning function that validates the level of fuzz in data, so this is not *only* a testcase fix. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/umockdev/+bug/1831467/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
