Hi Dmitry, Maybe if it will turn out that all tests fail because of functionality we should ignore or mute these tests in the context of the ticket IGNITE-11708?
ср, 17 апр. 2019 г. в 23:20, Dmitriy Pavlov <dpav...@apache.org>: > Hi Ivan F. > > Thank you for finding it and bringing it here. > > Please feel free to fix test to run (and fail) if the code it tests is > faulty now. If the community knows the moment when tests run will be > enabled, it is absolutely not an issue, that TeamCity Bot will complain > about new failures. I strongly believe that we should see a true picture of > tests output rather than having these tests not running. > > Sincerely, > Dmitriy Pavlov > > ср, 17 апр. 2019 г. в 22:05, Ivan Rakov <ivan.glu...@gmail.com>: > > > Hi Ivan, > > > > I've checked your branch. Seems like these tests fail due to real issue > > in functionality. > > I'll take a look. > > > > Best Regards, > > Ivan Rakov > > > > On 17.04.2019 13:54, Ivan Fedotov wrote: > > > Hi, Igniters! > > > > > > During work on iep-30[1] I discovered that > > > IgniteConfigVariationsAbstractTest subclasses - it is about 15_000 > > tests[2] > > > - do not work. > > > You can check it just run one of the tests with log output, for example > > > ConfigVariationsTestSuiteBuilderTest#LegacyLifecycleTest#test1 [3]. > > > There is no warning notification in the console. The same situation > with > > > other IgniteConfigVariationsAbstractTest subclasses - tests run, but > they > > > simply represent empty code. > > > > > > So, I created a ticket on such issue [4] and it turned out that the > > problem > > > is with ruleChain in IgniteConfigVariationsAbstractTest [5]. > > > The rule that is responsible for running a test statement does not > start > > > indeed [6] under ruleChain runRule. I suggested a solution - move > > testsCfg > > > initialization to > > > IgniteConfigVariationsAbstractTest#beforeTestsStarted method. After > such > > > changes ruleChain becomes not necessary. > > > > > > But I faced another problem - multiple failures on TeamCity [7]. From > > logs, > > > it seems that failures are related to what tests check, but not JUnit > > error. > > > I can not track TeamCity history on that fact were tests failed or not > on > > > the previous JUnit version - the oldest log is dated the start of the > > March > > > when JUnit4 > > > already was implemented (for example, this [8] test). > > > Moreover, there are not so much failed tests, but because of running > with > > > multiple configurations > > (InterceptorCacheConfigVariationsFullApiTestSuite_0 > > > ..._95) it turns out about 400 failed tests. TeamCity results also > > confirm > > > that tests do not work in the master branch - duration time is less > than > > > 1ms. Now all tests are green and that is not surprising - under @Test > > > annotation, nothing happens. > > > > > > Could some of us confirm or disprove my guess that tests are red > because > > of > > > its functionality, but not JUnit implementation? > > > And if it is true, how should I take such fact into account in this > > ticket? > > > > > > [1] > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-30%3A+Migration+to+JUnit+5 > > > [2] > > > > > > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testsuites/InterceptorCacheConfigVariationsFullApiTestSuite.java > > > [3] > > > > > > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/test/ConfigVariationsTestSuiteBuilderTest.java#L434 > > > [4] https://issues.apache.org/jira/browse/IGNITE-11708 > > > [5] > > > > > > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteConfigVariationsAbstractTest.java#L62 > > > [6] > > > > > > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/GridAbstractTest.java#L181 > > > [7] > > > > > > https://mtcga.gridgain.com/pr.html?serverId=apache&suiteId=IgniteTests24Java8_RunAll&branchForTc=pull/6434/head&action=Latest > > > [8] > > > > > > https://ci.ignite.apache.org/project.html?tab=testDetails&projectId=IgniteTests24Java8&testNameId=-9037806478172035481&page=8 > > > > > > -- Ivan Fedotov. ivanan...@gmail.com
