On Sun, 5 Sept 2021 at 09:44, D. Hugh Redelmeier <[email protected]> wrote:
> I ran the tests suite last night on two machines. It used to be that one > would expect different results from two runs "due to non-determanism", > even on the same machine. > > I got the same results on two machines. That's either good news or luck. > This is probably the result of hard work: thanks! > It's a bit of both, for instance, these two runs had 36 failures: https://testing.libreswan.org/?run=v4.5-36-gcb013f4154-main&run=v4.5-35-g6a4754e1fd-main But yes, things are definitely better. > > kvmrunner 6:22:43.06: stats/skiptest/ignored/status!=good: 9 > kvmrunner 6:22:43.06: stats/wip/ignored/status!=good: 79 > kvmrunner 6:22:43.06: tests/good/errors/EXPECTATION: 3 > kvmrunner 6:22:43.06: tests/good/errors/output-different: 43 > kvmrunner 6:22:43.06: tests/good/results/failed: 46 > kvmrunner 6:22:43.06: tests/good/results/passed: 832 > kvmrunner 6:22:43.06: tests/skiptest/results/untested: 9 > kvmrunner 6:22:43.06: tests/wip/results/untested: 79 > kvmrunner 6:22:43.06: total: 966 > kvmrunner 6:22:43.06: total/failed: 46 > kvmrunner 6:22:43.06: total/ignored: 88 > kvmrunner 6:22:43.06: total/passed: 832 > > The machines are essentially identical except that one has an 8G of RAM > and an SSD and the other has 16G of RAM and a HDD. The different disk > speeds might tend to amplify non-determinism. The run times were 5:34 and > 6:23. > _______________________________________________ > Swan-dev mailing list > [email protected] > https://lists.libreswan.org/mailman/listinfo/swan-dev >
_______________________________________________ Swan-dev mailing list [email protected] https://lists.libreswan.org/mailman/listinfo/swan-dev
