On Thu, Apr 05, 2018 at 08:05:39AM +0200, Gábor Boskovits wrote: > Actually running tests test the behaviour of a software. Unfortunately > reproducible build does not guarantee reproducible behaviour. > Furthermore there are still cases, where the environment is > not the same around these running software, like hardware or > kernel configuration settings leaking into the environment. > These can be spotted by running tests. Nondeterministic > failures can also be spotted more easily. There are a lot of > packages where pulling tests can be done, I guess, but probably not > for all of them. WDYT?
Hi Gabor, If that were a real problem we should not be providing substitutes - same problem. With substitutes we also provide software with tests that have been run once (at least). We should not forbid people to run tests. But I don't think it should be the default once tests have been run in a configuation. Think of it as functional programming. In my opinion rerunning tests can be cached. My point is that we should not overestimate/overdo the idea of leakage. Save the planet. We have responsibility. Pj.
