AaronBallman wrote: > > tests will fail locally but pass everywhere else (for example, people > > working on slightly out-of-norm configurations will sometimes have > > persistent local failures that are unrelated to changes in the patch). > > It never happened to me, in a way that can't be root caused and fixed (that > is for example cases like that would be the name of my directory interfere > with a CHECK somehow), what do you have in mind?
I do most of my work on Windows which is not the best-tested platform we have. I've frequently had failures locally of the variety "someone committed code that causes an MSVC STL debug iterator assertion", "mingw behaves differently than cygwin which behaves differently than Windows", git's autocrlf was not set properly, AV software impacting test behavior, tests hitting timeouts, etc. @Endilll has one locally where libclang symbols test fails for him in inexplicable ways that don't reproduce on any bots. That sort of thing. > > When precommit CI comes back green or with only false positives > > The point of my message was that the CI wasn't a false positive: the CI > didn't run. A false positive would be seeing a "Passed" for the test that you > see failing locally. Except CI *did* run and we got failure reports that were believed to be false positives in clang-tools-extra. The only way we could notice that *Clang* tests did not run was to see: ``` Total Discovered Tests: 2777 Unsupported : 7 (0.25%) Passed : 2765 (99.57%) Expectedly Failed: 2 (0.07%) Failed : 3 (0.11%) ``` and realize that's a very small number of tests. That's not a particularly obvious signal for code reviewers and it's even less obvious to newer contributors. > > speculative commits to see whether an issue "is real" or not do happen on > > occasion and are appropriate > > The cases I have seen for "speculative commits" are the opposite of what > you're describing: iterating on issues that only happens on a bot and not > reproducible locally. That is you **have** to push a commit because this is > the only way to iterate on debugging an issue. Pushing a commit when you have > a local failure seems backward to me, I don't quite follow actually. That is definitely the more common scenario, to be sure. But local failures that are not reproduced on bots are a reality unfortunately. Obviously, the goal is for tests to pass everywhere before committing. https://github.com/llvm/llvm-project/pull/71322 _______________________________________________ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
