Well, let’s take a look at the commits that have failed CI builds lately. Of course CI was green before, MutableThreadContextMapFilterTest was only introduced recently. Since then, the following completely unrelated commits have CI build failures:
* https://github.com/apache/logging-log4j2/commit/22382a42bb888cdfb371553ec74c1da1f343ea02 <https://github.com/apache/logging-log4j2/commit/22382a42bb888cdfb371553ec74c1da1f343ea02> (unrelated dependency change) * https://github.com/apache/logging-log4j2/commit/a4d562b0363d7f4ceef22aa7ab3d0ff11c2d6376 <https://github.com/apache/logging-log4j2/commit/a4d562b0363d7f4ceef22aa7ab3d0ff11c2d6376> (change in manual) * https://github.com/apache/logging-log4j2/commit/26efbc801b8e87088c0924abde17001cff34a88e <https://github.com/apache/logging-log4j2/commit/26efbc801b8e87088c0924abde17001cff34a88e> (unrelated test updates for arbiters) * https://github.com/apache/logging-log4j2/commit/ecd2d7783090683d3b7ba0ceb26e5c953f5c5a44 <https://github.com/apache/logging-log4j2/commit/ecd2d7783090683d3b7ba0ceb26e5c953f5c5a44> (modifying the Dependabot config) * several Dependabot PRs because fuck it at that point So a test was added that flakes on Windows CI which causes churn in trying to merge PRs. The fact that this test was left enabled in production like this for as long as it was is surprising. The GitHub action build failure spam sent directly to committers doesn’t help the case, either. Once I start getting those after I manually verify my commits before pushing them, unless the test failures are related to changes I just made (which does happen once in a while due to platform-specific differences or forgetting to commit a file), these emails are simply just spam. Thus, I go and fix the source of spam. If you prefer the spam, then change the action config to email yourself instead for all build failures and re-enable all the flaky tests. — Matt Sicker > On May 28, 2022, at 14:26, Ralph Goers <[email protected]> wrote: > > There is only one flaw with that argument. When I did release 2.17.2 we > weren’t having random test failures like this. So something changed. > > Ralph > >> On May 28, 2022, at 11:34 AM, Matt Sicker <[email protected]> wrote: >> >> Oh, and if these tests used to work, they wouldn’t be flaking from unrelated >> changes such as when someone updates a README file or other non-code area >> that somehow results in a failed CI run. >> — >> Matt Sicker >> >>> On May 28, 2022, at 10:29, Matt Sicker <[email protected]> wrote: >>> >>> I sent a separate email complaining about how Dependabot does this. >>> Anyways, the disabled tests are flakes. As I’ve said before, I run the full >>> build and suite of tests locally before pushing commits, but I’ve been >>> getting tons of build failure emails regardless. So instead of ignoring CI >>> failures as seems to be standard right now, I disabled the flaky tests >>> where applicable until someone cares enough to fix them. I filed Jira >>> issues so we don’t forget, either. It was also the only real feasible way >>> to get through dependency upgrade PRs without them randomly failing due to >>> unrelated flaky tests. >>> >>> — >>> Matt Sicker >>> >>>> On May 28, 2022, at 03:46, Piotr P. Karwasz <[email protected]> >>>> wrote: >>>> >>>> Hi Ralph, >>>> >>>> On Sat, 28 May 2022 at 10:17, Ralph Goers <[email protected]> >>>> wrote: >>>> >>>>> What I don’t understand is why several of the Jira issues seemingly have >>>>> 100 commits on various branches and flooded my inbox with email. >>>>> >>>>> This is Dependabot-related: every time a commit with "LOG4J2" appears in >>>> *any* branch, JIRA sends an e-mail. Rebasing many Dependabot branches >>>> caused a storm of e-mails (I had some 150 e-mails this morning). >>>> >>>> Piotr >> >
