Igniters, I created corresponding tickets [1][2][3][4] in Jira and outlined description of problems in a nutshell. In the context of the current ticket (IGNITE-11708), I ignored them.
If some of us have a comprehension of the problem why tests are failed, please let know here or in the tickets. [1] https://issues.apache.org/jira/browse/IGNITE-11883 [2] https://issues.apache.org/jira/browse/IGNITE-11884 [3] https://issues.apache.org/jira/browse/IGNITE-11885 [4] https://issues.apache.org/jira/browse/IGNITE-11886 чт, 30 мая 2019 г. в 14:55, Ivan Fedotov <[email protected]>: > Hi Igniters! > > I found the solution for how to run IgniteConfigVariationsAbstractTest. I > removed garbage injection [1] and static variable [2]. > Now assigning of VariationsTestsConfig proceeds in constructor class which > is created dynamically [3]. > > But I faced with the problem that a certain amount of tests are failed. It > seems that they failed because of the real reasons, not because > of the test workflow. Despite those fact that a big amount of tests on TC > became red, really failed tests are not so much. Derivatives tests appear > from > different configs. > > Could some of us take a look on failed tests? I am going to ignore them in > the context of the current ticket [5] and create separate > issues. > > [1] > https://github.com/apache/ignite/pull/6434/files#diff-cd3a07ce10f21d396c1eac0c328850e0L102 > [2] > https://github.com/apache/ignite/pull/6434/files#diff-cd3a07ce10f21d396c1eac0c328850e0L84 > [3] > https://github.com/apache/ignite/pull/6434/files#diff-3da5dbb6f22e5bf3bf18734933f1bfc5R434 > [4] > https://ci.ignite.apache.org/viewLog.html?buildId=3978807&buildTypeId=IgniteTests24Java8_RunAllNightly > [5]https://issues.apache.org/jira/browse/IGNITE-11708 > > вт, 14 мая 2019 г. в 16:31, Ivan Pavlukhina <[email protected]>: > >> Ivan F., >> >> Agree with referring tickets in @Ignore annotation. >> >> Currently I have no access to a computer. Will be able to look at updated >> PR next week. >> >> Sent from my iPhone >> >> > On 14 May 2019, at 10:55, Ivan Fedotov <[email protected]> wrote: >> > >> > Ivan P., >> > I managed with IgniteConfigVariationsAbstractTest - now subclasses >> enable >> > [1]. >> > I ignored all the failed tests that were mentioned in our conversation. >> If >> > you don't mind, I can create appropriate tickets and mention them in >> Ignore >> > annotation. >> > >> > [1] https://github.com/apache/ignite/pull/6434/files >> > >> > ср, 1 мая 2019 г. в 15:29, Павлухин Иван <[email protected]>: >> > >> >> Ivan F., >> >> >> >> I think that it is better to enable IgniteConfigVariationsAbstractTest >> >> subclasses first, so no new broken tests will appear. After that we >> >> can fix IgniteConfigVariationsAbstractTest subclasses one by one in >> >> separate ticket(s). >> >> >> >> вт, 30 апр. 2019 г. в 13:23, Ivan Fedotov <[email protected]>: >> >>> >> >>> Ivan R., Ivan P., thank you for the suggestions, I took a look at >> them. >> >>> >> >>> According to checking CacheAtomicityMode on null in >> >>> IgniteCacheConfigVariationsAbstractTest#atomicityMode - are you sure >> >>> that checking should proceed on test level? May be it will be better >> to >> >>> make default cache value in case if cc.getAtomicityMode() == null >> >>> in CacheConfiguration constructor [1]? >> >>> >> >>> Also let me suggest one minor change according cache specification in >> >>> IgniteCacheReadThroughEvictionSelfTest. The same logic is used in >> >>> GridCacheAbstractSelfTest#cacheConfiguration [2]. >> >>> I think we can excract this code block in a separate static methods >> >>> (initCacheConfig, for example) in >> IgniteCacheReadThroughEvictionSelfTest >> >> and >> >>> invoke it in IgniteCacheReadThroughEvictionSelfTest#variationConfig >> and >> >>> GridCacheAbstractSelfTest#cacheConfiguration methods. >> >>> >> >>> In addition, I want to draw your attention to >> >>> IgniteContinuousQueryConfigVariationsSuite test runs [3]. >> >>> CacheContinuousQueryVariationsTest are generated dynamically. >> >>> The first batch (12 tests) works fine, but on the next runs we got >> NPE in >> >>> IgniteCacheConfigVariationsAbstractTest#afterTest - default cache does >> >> not >> >>> exist and we can not >> >>> invoke unwrap() on this [4]. >> >>> Seems that the problem is in >> >>> >> >> >> IgniteCacheConfigVariationsAbstractTest#beforeTestsStarted/afterTestsStopped >> >>> methods, cache is not properly created (or already existed cache is >> >>> destroyed). >> >>> >> >>> By the way, should I resolve these issues in the context of >> IGNITE-11708 >> >> or >> >>> create a separate ticket(s)? >> >>> >> >>> [1] >> >>> >> >> >> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/configuration/CacheConfiguration.java#L434 >> >>> [2] >> >>> >> >> >> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/GridCacheAbstractSelfTest.java#L235 >> >>> [3] >> >>> >> >> >> https://ci.ignite.apache.org/viewLog.html?buildId=3701865&buildTypeId=IgniteTests24Java8_RunAllNightly >> >>> [4] >> >>> >> >> >> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteCacheConfigVariationsAbstractTest.java#L212 >> >>> >> >>> пт, 26 апр. 2019 г. в 18:01, Ivan Rakov <[email protected]>: >> >>> >> >>>> Ivan P., >> >>>> >> >>>> Good catch, thanks. >> >>>> >> >>>> I was wrong, test scenario is correct. The problem was in >> >>>> atomicityMode() method - it could have returned null (which was okay >> >> for >> >>>> config generation, but wasn't expected in the test code). >> >>>> Please take a look at tx_out_test_fixed.patch (attached to >> >>>> IGNITE-11708). To sum it up, both issues should be fixed now. >> >>>> >> >>>> Best Regards, >> >>>> Ivan Rakov >> >>>> >> >>>>> On 26.04.2019 14:40, Павлухин Иван wrote: >> >>>>> Ivan R., >> >>>>> >> >>>>> As I can see IgniteCacheConfigVariationsFullApiTest#testGetOutTx >> does >> >>>>> not expect lock/unlock events due to line: >> >>>>> if (atomicityMode() == ATOMIC) >> >>>>> return lockEvtCnt.get() == 0; >> >>>>> >> >>>>> Could you please elaborate? >> >>>>> >> >>>>> пт, 26 апр. 2019 г. в 13:32, Ivan Rakov <[email protected]>: >> >>>>>> Ivan, >> >>>>>> >> >>>>>> Seems like IgniteCacheReadThroughEvictionSelfTest is broken. Test >> >>>>>> scenario assumes that even after expiration entry will be present >> in >> >>>>>> IgniteCache as per it will be loaded from CacheStore. However, >> >>>>>> CacheStore is not specified in node config. I've added patch that >> >>>>>> enables cache store factory, please check IGNITE-11708 attachments. >> >>>>>> Regarding IgniteCacheConfigVariationsFullApiTest#testGetOutTx* >> >> tests: >> >>>>>> from my point of view, test scenarios make no sense. We perform >> >> get() >> >>>>>> operation from ATOMIC caches and expect that entries will be >> >> locked. I >> >>>>>> don't understand why we should lock entries on ATOMIC get, >> >> therefore I >> >>>>>> suppose to remove part of code where we listen and check >> >>>>>> EVT_CACHE_OBJECT_LOCKED/UNLOCKED events. >> >>>>>> >> >>>>>> Best Regards, >> >>>>>> Ivan Rakov >> >>>>>> >> >>>>>>> On 17.04.2019 22:05, Ivan Rakov wrote: >> >>>>>>> Hi Ivan, >> >>>>>>> >> >>>>>>> I've checked your branch. Seems like these tests fail due to real >> >>>>>>> issue in functionality. >> >>>>>>> I'll take a look. >> >>>>>>> >> >>>>>>> Best Regards, >> >>>>>>> Ivan Rakov >> >>>>>>> >> >>>>>>>> On 17.04.2019 13:54, Ivan Fedotov wrote: >> >>>>>>>> Hi, Igniters! >> >>>>>>>> >> >>>>>>>> During work on iep-30[1] I discovered that >> >>>>>>>> IgniteConfigVariationsAbstractTest subclasses - it is about >> 15_000 >> >>>>>>>> tests[2] >> >>>>>>>> - do not work. >> >>>>>>>> You can check it just run one of the tests with log output, for >> >>>> example >> >>>>>>>> ConfigVariationsTestSuiteBuilderTest#LegacyLifecycleTest#test1 >> >> [3]. >> >>>>>>>> There is no warning notification in the console. The same >> >> situation >> >>>> with >> >>>>>>>> other IgniteConfigVariationsAbstractTest subclasses - tests run, >> >> but >> >>>>>>>> they >> >>>>>>>> simply represent empty code. >> >>>>>>>> >> >>>>>>>> So, I created a ticket on such issue [4] and it turned out that >> >> the >> >>>>>>>> problem >> >>>>>>>> is with ruleChain in IgniteConfigVariationsAbstractTest [5]. >> >>>>>>>> The rule that is responsible for running a test statement does >> not >> >>>> start >> >>>>>>>> indeed [6] under ruleChain runRule. I suggested a solution - move >> >>>>>>>> testsCfg >> >>>>>>>> initialization to >> >>>>>>>> IgniteConfigVariationsAbstractTest#beforeTestsStarted method. >> >> After >> >>>> such >> >>>>>>>> changes ruleChain becomes not necessary. >> >>>>>>>> >> >>>>>>>> But I faced another problem - multiple failures on TeamCity [7]. >> >> From >> >>>>>>>> logs, >> >>>>>>>> it seems that failures are related to what tests check, but not >> >> JUnit >> >>>>>>>> error. >> >>>>>>>> I can not track TeamCity history on that fact were tests failed >> or >> >>>>>>>> not on >> >>>>>>>> the previous JUnit version - the oldest log is dated the start of >> >> the >> >>>>>>>> March >> >>>>>>>> when JUnit4 >> >>>>>>>> already was implemented (for example, this [8] test). >> >>>>>>>> Moreover, there are not so much failed tests, but because of >> >> running >> >>>>>>>> with >> >>>>>>>> multiple configurations >> >>>>>>>> (InterceptorCacheConfigVariationsFullApiTestSuite_0 >> >>>>>>>> ..._95) it turns out about 400 failed tests. TeamCity results >> also >> >>>>>>>> confirm >> >>>>>>>> that tests do not work in the master branch - duration time is >> >> less >> >>>> than >> >>>>>>>> 1ms. Now all tests are green and that is not surprising - under >> >> @Test >> >>>>>>>> annotation, nothing happens. >> >>>>>>>> >> >>>>>>>> Could some of us confirm or disprove my guess that tests are red >> >>>>>>>> because of >> >>>>>>>> its functionality, but not JUnit implementation? >> >>>>>>>> And if it is true, how should I take such fact into account in >> >> this >> >>>>>>>> ticket? >> >>>>>>>> >> >>>>>>>> [1] >> >>>>>>>> >> >>>> >> >> >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-30%3A+Migration+to+JUnit+5 >> >>>>>>>> >> >>>>>>>> [2] >> >>>>>>>> >> >>>> >> >> >> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testsuites/InterceptorCacheConfigVariationsFullApiTestSuite.java >> >>>>>>>> >> >>>>>>>> [3] >> >>>>>>>> >> >>>> >> >> >> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/test/ConfigVariationsTestSuiteBuilderTest.java#L434 >> >>>>>>>> >> >>>>>>>> [4] https://issues.apache.org/jira/browse/IGNITE-11708 >> >>>>>>>> [5] >> >>>>>>>> >> >>>> >> >> >> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteConfigVariationsAbstractTest.java#L62 >> >>>>>>>> >> >>>>>>>> [6] >> >>>>>>>> >> >>>> >> >> >> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/GridAbstractTest.java#L181 >> >>>>>>>> >> >>>>>>>> [7] >> >>>>>>>> >> >>>> >> >> >> https://mtcga.gridgain.com/pr.html?serverId=apache&suiteId=IgniteTests24Java8_RunAll&branchForTc=pull/6434/head&action=Latest >> >>>>>>>> >> >>>>>>>> [8] >> >>>>>>>> >> >>>> >> >> >> https://ci.ignite.apache.org/project.html?tab=testDetails&projectId=IgniteTests24Java8&testNameId=-9037806478172035481&page=8 >> >>>>>>>> >> >>>>>>>> >> >>>>> >> >>>>> >> >>>> >> >>> >> >>> >> >>> -- >> >>> Ivan Fedotov. >> >>> >> >>> [email protected] >> >> >> >> >> >> >> >> -- >> >> Best regards, >> >> Ivan Pavlukhin >> >> >> > >> > >> > -- >> > Ivan Fedotov. >> > >> > [email protected] >> > > > -- > Ivan Fedotov. > > [email protected] > -- Ivan Fedotov. [email protected]
