winglam edited a comment on pull request #348: URL: https://github.com/apache/maven-surefire/pull/348#issuecomment-1077771434
@aslakhellesoy Thanks for the clarification. I was aware from the https://issues.apache.org/jira/browse/SUREFIRE-2041 issue that you wanted a global test method ordering feature that supports the interleaving of test methods across test classes and I agree that it may not be realistic to attempt. More importantly, I'm not sure how one would implement such a feature. Namely, how should one handle class setup and teardown methods (e.g., @BeforeClass from JUnit) for an order such as testing.Y#a,testing.X#c,testing.Y#b? Should one run _testing.Y#beforeClass_,testing.Y#a,_testing.X#beforeClass_,testing.X#c,_testing.Y#beforeClass_,testing.Y#b? What if running testing.Y#beforeClass twice without running testing.Y#afterClass causes a failure? We could then run _testing.Y#afterClass_ after testing.Y#a and before _testing.X#beforeClass_ but then that can add substantial time cost to running tests and still cause unexpected failures if these class setup and teardown methods are not expected to run twice in one JVM. Not supporting a global test method ordering feature is a purposeful design feature when I did my implementation of test method ordering. For the same reason we may not want to support the feature of interleaving test classes across different modules (e.g., module1.testing.Y,module2.testing.X,module1.testing.Z), I'm not sure whether a global test method ordering feature should be supported or not based on the way modules and test classes are defined and commonly used. I believe there is still much value to support test method ordering within classes (and also the ordering of test classes) and the changes I have in this PR accomplishes both test class ordering and test method ordering, just not the interleaving of test methods across classes and in the way that Tibor suggests us to do so for this feature. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
