I mean changing parameter http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#runOrder which is fileSystem by default.
It is not about hiding. If a problematic test affects other test is will continue to affect. The main point here is only about the test, which will be affected. With unpredictable order, testA may break testB, testC, testD. But for predictive & fixed order test affected by a failure of testA will be always testC, and B&D will not be considered flaky because of the randomized nature of execution. It will help for building at least good statistics in the TC Bot. ср, 6 февр. 2019 г. в 17:37, Eduard Shangareev <eduard.shangar...@gmail.com >: > Dmitriy, > > Please, clarify the idea. > What do we want to achieve by this? Making tests more stable by hiding some > issues in our tests/codebase? > > > On Tue, Feb 5, 2019 at 6:57 PM Павлухин Иван <vololo...@gmail.com> wrote: > > > Dmitriy, > > > > Sounds like a good idea to me. Problems related to tests isolation are > > very common in practice. And things can be complicated when order > > varies. > > > > But by the way does the order of Ignite tests vary today? Junit 4 > > javadocs claims something about "default deterministic order" [1]. > > > > [1] > https://junit.org/junit4/javadoc/latest/org/junit/FixMethodOrder.html > > > > вт, 5 февр. 2019 г. в 17:40, Dmitriy Pavlov <dpav...@apache.org>: > > > > > > Dear Ignite Developers, > > > > > > The original idea came from our recent habr.ru post related to Apache > > > Ignite TeamCity Bot (for Russian native speakers, you can read an > > original > > > https://habr.com/ru/company/sberbank/blog/436070/#comment_19616976 ) > > > > > > It is a known phenomenon when tests have an influence on each other. > The > > > simplest case when Ignite Native persistence is used, and not properly > > > cleared after a test run. This can make some test failed afterward. > > > > > > So, what if we will set predictable, for example, alphabetical tests > > > execution order (maven-surefire-plugin/runOrder/alphabetical). This may > > > have the following effect: the set of tests failed because of being > > > affected by the previous run will be constant, will be exactly the same > > > each run. > > > > > > At some point, when we stabilize flaky tests enough, we may select > random > > > order, but for now, this solution seems valid to me. > > > > > > What do you think? > > > > > > Sincerely, > > > Dmitriy Pavlov > > > > > > > > -- > > Best regards, > > Ivan Pavlukhin > > >
