nlopes added a comment. In D98179#2639491 <https://reviews.llvm.org/D98179#2639491>, @davezarzycki wrote:
> In D98179#2639476 <https://reviews.llvm.org/D98179#2639476>, @nlopes wrote: > >> Why are timeouts important? Our use case is running Alive2 with the test >> suite. Alive2 is heavy machinery and runs into timeouts. Running the tests >> in roughly the same order every time is important to avoid timeout tests >> flipping to failed or vice-versa. Plus slow tests = tests that consume a lot >> of memory (in our scenario), so we can't bundle slow tests together. >> Adding a `--ignore-timing-data` would be great, yes! But I still feel that >> sorting the list of failed tests is important for user experience. I diff >> these all the time. > > That still sounds incredibly brittle. If there is any variety in test machine > performance, then any and all attempts at sorting should be futile because > the underlying hardware will perturb different timeouts. Is this not your > experience? How do you reconcile hardware performance and configuration > details (like SMT) with timeout settings? Of course it's brittle :) Changing from a time-based setting to a ticks-based system is ongoing work, such that resource exhaustion becomes deterministic. Nevertheless, on a same machine, we don't see many test flips. It's quite stable most of the times (just one test flip once in a while). Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D98179/new/ https://reviews.llvm.org/D98179 _______________________________________________ lldb-commits mailing list lldb-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
