Re: [log4j] Unstable tests on Windows

There’s a JUnit annotation `@Issue` or something like that (Spock has a similar annotation) which you can use for linking to an issue. Otherwise, adding `@Tag` annotations allows for arbitrary tags (of which we use several already such as “functional” and “sleepy”). > On Nov 28, 2023, at 5:55 

Re: [log4j] Unstable tests on Windows

I think JUnit can group tests in categories with annotations (AFK). Gary On Tue, Nov 28, 2023, 12:01 AM Ralph Goers wrote: > I would be -1 if the issues are going to be ignored or not tracked in any > way. I don’t know if GitHub has something like a Jira Epic or if they can > be tagged in some

Re: [log4j] Unstable tests on Windows

Hi Ralph, On Tue, 28 Nov 2023 at 06:01, Ralph Goers wrote: > > I would be -1 if the issues are going to be ignored or not tracked in any > way. I don’t know if GitHub has something like a Jira Epic or if they can be > tagged in some way so that they can be easily located but something like

Re: [log4j] Unstable tests on Windows

Yesterday I disabled one and created #2011 (labeled with `tests`) for it. You can navigate all `tests` labeled issues or simply search for `@DisabledOnOs` in the code base – there are not

Re: [log4j] Unstable tests on Windows

I would be -1 if the issues are going to be ignored or not tracked in any way. I don’t know if GitHub has something like a Jira Epic or if they can be tagged in some way so that they can be easily located but something like that would be fine. Even tracking them in Confluence would be fine.

Re: [log4j] Unstable tests on Windows

Ralph did not agree, but did not strongly object either. Ralph, are you -1 on disabling tests only Windows that are failing frequently on Windows and capturing them in tickets to be addressed? On Thu, Nov 23, 2023 at 12:23 AM Christian Grobmeier wrote: > Ralph said, nobody would ever fix these

Re: [log4j] Unstable tests on Windows

Ralph said, nobody would ever fix these tests if you do it like this. I think you should create the ticket but keep the tests until we find the issue. Otherwise there issues will rot On Wed, Nov 22, 2023, at 09:13, Volkan Yazıcı wrote: > AFAIC, nobody[1] shows a strong opposition against the

Re: [log4j] Unstable tests on Windows

AFAIC, nobody[1] shows a strong opposition against the idea of disabling frequently failing Windows tests only on Windows and creating a ticket for each one. I will proceed with that. [1] Except Piotr, whom I discussed the issue with in Slack and he agreed with the above shared approach. On Mon,

Re: [log4j] Unstable tests on Windows

The GH CI build also fails randomly compared to the green builds that I push in the first place. Occasionally, this might be a platform-specific issue (which is what I’d ideally be discovering from CI), but the failing tests are generally fragile ones in the first place. While I have ideas on

Re: [log4j] Unstable tests on Windows

The GH CI builds on every push (as opposed to commi) IIRC. Gary On Mon, Nov 20, 2023, 9:34 AM Ralph Goers wrote: > Gary uses Windows as his development OS. He is probably the only one of us > who does. So he inevitably finds these issues before the rest of us. > > I don’t know if the CI has a

Re: [log4j] Unstable tests on Windows

Gary uses Windows as his development OS. He is probably the only one of us who does. So he inevitably finds these issues before the rest of us. I don’t know if the CI has a build that runs on Windows for every commit. Ralph > On Nov 20, 2023, at 6:12 AM, Robert Middleton wrote: > > Are the

Re: [log4j] Unstable tests on Windows

Are the tests run on Windows through Github workflows? It doesn't look like it to me. If you need access to a Windows machine, you can download a development VM straight from Microsoft: https://developer.microsoft.com/en-us/windows/downloads/virtual-machines/ -Robert Middleton On Mon, Nov 20,

Re: [log4j] Unstable tests on Windows

In my experience they never get fixed. To be honest, when I was doing the releases I would have these failures investigated to determine if it was a trait problem vs a problem in the code being released. If it was the latter I would cancel the vote. The only time tests should be disabled is if

Re: [log4j] Unstable tests on Windows

I am not asking to disable Windows tests. I am asking to disable tests and only those tests that have a failure rate on Windows higher than, say, 30%. To be precise, I think there are 2-3 of them dealing with network sockets and rolling file appenders. I am not talking about dozens or such. After

Re: [log4j] Unstable tests on Windows

Hi Volkan, On Mon, 20 Nov 2023 at 09:36, Volkan Yazıcı wrote: > > As Gary (the only Windows user among the active Log4j maintainers, > AFAIK) has noticed several times, Log4j tests on Windows are pretty > unstable. It not only fails on Gary's laptop, but Piotr and I need to > give Windows tests

[log4j] Unstable tests on Windows

As Gary (the only Windows user among the active Log4j maintainers, AFAIK) has noticed several times, Log4j tests on Windows are pretty unstable. It not only fails on Gary's laptop, but Piotr and I need to give Windows tests in CI a kick on a regular basis. Approximately one out of three CI runs