On Tue, Aug 26, 2025 at 06:55:28AM -0700, '[email protected]' via sage-devel wrote: > The biggest issue with the reliability of the CI is a deep design > decision in the way the tests are setup. Many doctests have an > inherent random element, and this is mostly on purpose to increase > the surface of code paths that are tested and thereby discover new > bugs. The disadvantage is that unfortunately some test runs will > produce failures that are not connected to the changes of the PR. > ...
I know nothing about the internal implementation here, but just this description suggests a change in practice: when a test on a PR fails, rerun that test on the existing branch (without the pull request) with the same random seed to see if it also fails there. If it does, then automatically file a separate issue. (I see potential issues with different numbers of random numbers being used pre- and post-PR, but there are also ways to mitigate that, eg maintaining a separate random seed used only for generating test cases.) --Dylan -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/sage-devel/aK3Kvnxq0sDn7603%40whitehail.bostoncoop.net.
