Hi, Igniters! During work on iep-30[1] I discovered that IgniteConfigVariationsAbstractTest subclasses - it is about 15_000 tests[2] - do not work. You can check it just run one of the tests with log output, for example ConfigVariationsTestSuiteBuilderTest#LegacyLifecycleTest#test1 [3]. There is no warning notification in the console. The same situation with other IgniteConfigVariationsAbstractTest subclasses - tests run, but they simply represent empty code.
So, I created a ticket on such issue [4] and it turned out that the problem is with ruleChain in IgniteConfigVariationsAbstractTest [5]. The rule that is responsible for running a test statement does not start indeed [6] under ruleChain runRule. I suggested a solution - move testsCfg initialization to IgniteConfigVariationsAbstractTest#beforeTestsStarted method. After such changes ruleChain becomes not necessary. But I faced another problem - multiple failures on TeamCity [7]. From logs, it seems that failures are related to what tests check, but not JUnit error. I can not track TeamCity history on that fact were tests failed or not on the previous JUnit version - the oldest log is dated the start of the March when JUnit4 already was implemented (for example, this [8] test). Moreover, there are not so much failed tests, but because of running with multiple configurations (InterceptorCacheConfigVariationsFullApiTestSuite_0 ..._95) it turns out about 400 failed tests. TeamCity results also confirm that tests do not work in the master branch - duration time is less than 1ms. Now all tests are green and that is not surprising - under @Test annotation, nothing happens. Could some of us confirm or disprove my guess that tests are red because of its functionality, but not JUnit implementation? And if it is true, how should I take such fact into account in this ticket? [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-30%3A+Migration+to+JUnit+5 [2] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testsuites/InterceptorCacheConfigVariationsFullApiTestSuite.java [3] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/test/ConfigVariationsTestSuiteBuilderTest.java#L434 [4] https://issues.apache.org/jira/browse/IGNITE-11708 [5] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteConfigVariationsAbstractTest.java#L62 [6] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/GridAbstractTest.java#L181 [7] https://mtcga.gridgain.com/pr.html?serverId=apache&suiteId=IgniteTests24Java8_RunAll&branchForTc=pull/6434/head&action=Latest [8] https://ci.ignite.apache.org/project.html?tab=testDetails&projectId=IgniteTests24Java8&testNameId=-9037806478172035481&page=8 -- Ivan Fedotov. ivanan...@gmail.com
