On Thu, Oct 23, 2025 at 1:23 PM Thomas Wolf <[email protected]> wrote: > > On 23.10.25 15:00, Gary Gregory wrote: > > I tested the src zip file: > > - ASC OK > > - SHA512 OK > > - mvn clean verify > > > > This causes errors in > > org.apache.sshd.common.forward.ApacheServerJSchClientTest as "flakes" > > warnings?. What does it mean? > > > > Please see > > https://gist.github.com/garydgregory/622094942ed1d1a34e22e97c46a63c9a > > TL;DR: yes, some tests are flaky. Ignore that as long as the full build > reports success.
+1. TY for the details. Gary > > Long version: > > Flaky tests are a perennial pain point. I spent quite some time a while > back (I think in 2022) to get stable builds on my machine and in CI, but > that doesn't mean that all our tests are fully stable. > > Also: re-trying surefire tests multiple times actually hides the > flakiness in CI (unless you'd go inspect every build output for flakes). > > From time to time some tests still are flaky. I don't know what exactly > went wrong in your run; if the same test is frequently flaky, you'd > have to debug failing runs live to have a chance to figure out where > the problem is. > > From my past experience, problems that cause flaky tests can be > > - bugs in Apache MINA SSHD, typically a race condition somewhere. > - problems with sockets; sometimes in tests re-using sockets too > quickly, but sometimes also for unspecific reasons in the > environment or OS. > - bugs in Apache MINA SSHD test code, typically expecting some > assertion to be true synchronously when in fact the condition > is fulfilled only asynchronously. > - some tests are only flaky on some I/O transports. A test might > work perfectly on NIO2 or MINA, but be flaky on Netty. Or be > fine on NIO2 and Netty but be flaky on MINA. Or some other > combination. > - bugs and race conditions in third-party libraries used in tests. > (In particular JSch.) > > (The list is not exhaustive!) > > It's also possible that the refactorings done in 3.0.0 so far have > introduced a new problem that might make some tests flaky now, but > locally I have not observed anything like that. (Nor in CI). > > However: I use a nearly 10 year old Intel Mac as development machine, > which is not the fastest. CI build in Github are also not exactly > fast. It's possible that running the tests on your much more powerful > ARM Mac uncovers new flakiness. You might hit race conditions that > just never occur on slower machines. > > Cheers, > > Thomas > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
