[ https://issues.apache.org/jira/browse/SUREFIRE-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17499188#comment-17499188 ]
Olivier Lamy commented on SUREFIRE-2027: ---------------------------------------- It's exactly the problem. It's performance test which measure numbers on a machine. And they are machine dependant. To run performance test you must ensure the node running those can handle it. In case of a poor performance machine or overloaded node, the test will fail whereas it's part of the default build. Such test cannot be part of the default build. > Flaky test AsyncSocketTest > -------------------------- > > Key: SUREFIRE-2027 > URL: https://issues.apache.org/jira/browse/SUREFIRE-2027 > Project: Maven Surefire > Issue Type: Bug > Reporter: Olivier Lamy > Priority: Critical > > This looks to be a flaky test > {quote}{\{Error: Tests run: 193, Failures: 0, Errors: 1, Skipped: 0, Time > elapsed: 16.435 s <<< FAILURE! - in org.apache.maven.JUnit4SuiteTest }} > {\{Error: org.apache.maven.surefire.api.util.internal.AsyncSocketTest.test > Time elapsed: 5.311 s <<< ERROR! }} > {{[694|https://github.com/apache/maven-surefire/runs/5349295808?check_suite_focus=true#step:7:694]java.lang.AssertionError: > }} > {{[695|https://github.com/apache/maven-surefire/runs/5349295808?check_suite_focus=true#step:7:695]}} > {{[696|https://github.com/apache/maven-surefire/runs/5349295808?check_suite_focus=true#step:7:696]Expecting > actual: }} > {{[697|https://github.com/apache/maven-surefire/runs/5349295808?check_suite_focus=true#step:7:697] > 1237L }} > {{[698|https://github.com/apache/maven-surefire/runs/5349295808?check_suite_focus=true#step:7:698]to > be less than: }} > {{[699|https://github.com/apache/maven-surefire/runs/5349295808?check_suite_focus=true#step:7:699] > 1000L }} > {{[700|https://github.com/apache/maven-surefire/runs/5349295808?check_suite_focus=true#step:7:700] > at > org.apache.maven.surefire.api.util.internal.AsyncSocketTest.test(AsyncSocketTest.java:185)}} > {quote} > > The code here > [https://github.com/apache/maven-surefire/blob/3050d2e32b957bbc98112979c27e977a39933fa6/surefire-api/src/test/java/org/apache/maven/surefire/api/util/internal/AsyncSocketTest.java#L182] > is making some assumptions which really depends on the network of the node > where tests are running. Time to transfer data on a node cannot be evaluate > with such strict figures it depends how busy is the operating system. -- This message was sent by Atlassian Jira (v8.20.1#820001)