Dmitriy, Please, clarify the idea. What do we want to achieve by this? Making tests more stable by hiding some issues in our tests/codebase?
On Tue, Feb 5, 2019 at 6:57 PM Павлухин Иван <vololo...@gmail.com> wrote: > Dmitriy, > > Sounds like a good idea to me. Problems related to tests isolation are > very common in practice. And things can be complicated when order > varies. > > But by the way does the order of Ignite tests vary today? Junit 4 > javadocs claims something about "default deterministic order" [1]. > > [1] https://junit.org/junit4/javadoc/latest/org/junit/FixMethodOrder.html > > вт, 5 февр. 2019 г. в 17:40, Dmitriy Pavlov <dpav...@apache.org>: > > > > Dear Ignite Developers, > > > > The original idea came from our recent habr.ru post related to Apache > > Ignite TeamCity Bot (for Russian native speakers, you can read an > original > > https://habr.com/ru/company/sberbank/blog/436070/#comment_19616976 ) > > > > It is a known phenomenon when tests have an influence on each other. The > > simplest case when Ignite Native persistence is used, and not properly > > cleared after a test run. This can make some test failed afterward. > > > > So, what if we will set predictable, for example, alphabetical tests > > execution order (maven-surefire-plugin/runOrder/alphabetical). This may > > have the following effect: the set of tests failed because of being > > affected by the previous run will be constant, will be exactly the same > > each run. > > > > At some point, when we stabilize flaky tests enough, we may select random > > order, but for now, this solution seems valid to me. > > > > What do you think? > > > > Sincerely, > > Dmitriy Pavlov > > > > -- > Best regards, > Ivan Pavlukhin >
