Hi Igniters! I found the solution for how to run IgniteConfigVariationsAbstractTest. I removed garbage injection [1] and static variable [2]. Now assigning of VariationsTestsConfig proceeds in constructor class which is created dynamically [3].
But I faced with the problem that a certain amount of tests are failed. It seems that they failed because of the real reasons, not because of the test workflow. Despite those fact that a big amount of tests on TC became red, really failed tests are not so much. Derivatives tests appear from different configs. Could some of us take a look on failed tests? I am going to ignore them in the context of the current ticket [5] and create separate issues. [1] https://github.com/apache/ignite/pull/6434/files#diff-cd3a07ce10f21d396c1eac0c328850e0L102 [2] https://github.com/apache/ignite/pull/6434/files#diff-cd3a07ce10f21d396c1eac0c328850e0L84 [3] https://github.com/apache/ignite/pull/6434/files#diff-3da5dbb6f22e5bf3bf18734933f1bfc5R434 [4] https://ci.ignite.apache.org/viewLog.html?buildId=3978807&buildTypeId=IgniteTests24Java8_RunAllNightly [5]https://issues.apache.org/jira/browse/IGNITE-11708 вт, 14 мая 2019 г. в 16:31, Ivan Pavlukhina <vololo...@gmail.com>: > Ivan F., > > Agree with referring tickets in @Ignore annotation. > > Currently I have no access to a computer. Will be able to look at updated > PR next week. > > Sent from my iPhone > > > On 14 May 2019, at 10:55, Ivan Fedotov <ivanan...@gmail.com> wrote: > > > > Ivan P., > > I managed with IgniteConfigVariationsAbstractTest - now subclasses enable > > [1]. > > I ignored all the failed tests that were mentioned in our conversation. > If > > you don't mind, I can create appropriate tickets and mention them in > Ignore > > annotation. > > > > [1] https://github.com/apache/ignite/pull/6434/files > > > > ср, 1 мая 2019 г. в 15:29, Павлухин Иван <vololo...@gmail.com>: > > > >> Ivan F., > >> > >> I think that it is better to enable IgniteConfigVariationsAbstractTest > >> subclasses first, so no new broken tests will appear. After that we > >> can fix IgniteConfigVariationsAbstractTest subclasses one by one in > >> separate ticket(s). > >> > >> вт, 30 апр. 2019 г. в 13:23, Ivan Fedotov <ivanan...@gmail.com>: > >>> > >>> Ivan R., Ivan P., thank you for the suggestions, I took a look at them. > >>> > >>> According to checking CacheAtomicityMode on null in > >>> IgniteCacheConfigVariationsAbstractTest#atomicityMode - are you sure > >>> that checking should proceed on test level? May be it will be better to > >>> make default cache value in case if cc.getAtomicityMode() == null > >>> in CacheConfiguration constructor [1]? > >>> > >>> Also let me suggest one minor change according cache specification in > >>> IgniteCacheReadThroughEvictionSelfTest. The same logic is used in > >>> GridCacheAbstractSelfTest#cacheConfiguration [2]. > >>> I think we can excract this code block in a separate static methods > >>> (initCacheConfig, for example) in > IgniteCacheReadThroughEvictionSelfTest > >> and > >>> invoke it in IgniteCacheReadThroughEvictionSelfTest#variationConfig and > >>> GridCacheAbstractSelfTest#cacheConfiguration methods. > >>> > >>> In addition, I want to draw your attention to > >>> IgniteContinuousQueryConfigVariationsSuite test runs [3]. > >>> CacheContinuousQueryVariationsTest are generated dynamically. > >>> The first batch (12 tests) works fine, but on the next runs we got NPE > in > >>> IgniteCacheConfigVariationsAbstractTest#afterTest - default cache does > >> not > >>> exist and we can not > >>> invoke unwrap() on this [4]. > >>> Seems that the problem is in > >>> > >> > IgniteCacheConfigVariationsAbstractTest#beforeTestsStarted/afterTestsStopped > >>> methods, cache is not properly created (or already existed cache is > >>> destroyed). > >>> > >>> By the way, should I resolve these issues in the context of > IGNITE-11708 > >> or > >>> create a separate ticket(s)? > >>> > >>> [1] > >>> > >> > https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/configuration/CacheConfiguration.java#L434 > >>> [2] > >>> > >> > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/GridCacheAbstractSelfTest.java#L235 > >>> [3] > >>> > >> > https://ci.ignite.apache.org/viewLog.html?buildId=3701865&buildTypeId=IgniteTests24Java8_RunAllNightly > >>> [4] > >>> > >> > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteCacheConfigVariationsAbstractTest.java#L212 > >>> > >>> пт, 26 апр. 2019 г. в 18:01, Ivan Rakov <ivan.glu...@gmail.com>: > >>> > >>>> Ivan P., > >>>> > >>>> Good catch, thanks. > >>>> > >>>> I was wrong, test scenario is correct. The problem was in > >>>> atomicityMode() method - it could have returned null (which was okay > >> for > >>>> config generation, but wasn't expected in the test code). > >>>> Please take a look at tx_out_test_fixed.patch (attached to > >>>> IGNITE-11708). To sum it up, both issues should be fixed now. > >>>> > >>>> Best Regards, > >>>> Ivan Rakov > >>>> > >>>>> On 26.04.2019 14:40, Павлухин Иван wrote: > >>>>> Ivan R., > >>>>> > >>>>> As I can see IgniteCacheConfigVariationsFullApiTest#testGetOutTx does > >>>>> not expect lock/unlock events due to line: > >>>>> if (atomicityMode() == ATOMIC) > >>>>> return lockEvtCnt.get() == 0; > >>>>> > >>>>> Could you please elaborate? > >>>>> > >>>>> пт, 26 апр. 2019 г. в 13:32, Ivan Rakov <ivan.glu...@gmail.com>: > >>>>>> Ivan, > >>>>>> > >>>>>> Seems like IgniteCacheReadThroughEvictionSelfTest is broken. Test > >>>>>> scenario assumes that even after expiration entry will be present in > >>>>>> IgniteCache as per it will be loaded from CacheStore. However, > >>>>>> CacheStore is not specified in node config. I've added patch that > >>>>>> enables cache store factory, please check IGNITE-11708 attachments. > >>>>>> Regarding IgniteCacheConfigVariationsFullApiTest#testGetOutTx* > >> tests: > >>>>>> from my point of view, test scenarios make no sense. We perform > >> get() > >>>>>> operation from ATOMIC caches and expect that entries will be > >> locked. I > >>>>>> don't understand why we should lock entries on ATOMIC get, > >> therefore I > >>>>>> suppose to remove part of code where we listen and check > >>>>>> EVT_CACHE_OBJECT_LOCKED/UNLOCKED events. > >>>>>> > >>>>>> Best Regards, > >>>>>> Ivan Rakov > >>>>>> > >>>>>>> On 17.04.2019 22:05, Ivan Rakov wrote: > >>>>>>> Hi Ivan, > >>>>>>> > >>>>>>> I've checked your branch. Seems like these tests fail due to real > >>>>>>> issue in functionality. > >>>>>>> I'll take a look. > >>>>>>> > >>>>>>> Best Regards, > >>>>>>> Ivan Rakov > >>>>>>> > >>>>>>>> On 17.04.2019 13:54, Ivan Fedotov wrote: > >>>>>>>> Hi, Igniters! > >>>>>>>> > >>>>>>>> During work on iep-30[1] I discovered that > >>>>>>>> IgniteConfigVariationsAbstractTest subclasses - it is about 15_000 > >>>>>>>> tests[2] > >>>>>>>> - do not work. > >>>>>>>> You can check it just run one of the tests with log output, for > >>>> example > >>>>>>>> ConfigVariationsTestSuiteBuilderTest#LegacyLifecycleTest#test1 > >> [3]. > >>>>>>>> There is no warning notification in the console. The same > >> situation > >>>> with > >>>>>>>> other IgniteConfigVariationsAbstractTest subclasses - tests run, > >> but > >>>>>>>> they > >>>>>>>> simply represent empty code. > >>>>>>>> > >>>>>>>> So, I created a ticket on such issue [4] and it turned out that > >> the > >>>>>>>> problem > >>>>>>>> is with ruleChain in IgniteConfigVariationsAbstractTest [5]. > >>>>>>>> The rule that is responsible for running a test statement does not > >>>> start > >>>>>>>> indeed [6] under ruleChain runRule. I suggested a solution - move > >>>>>>>> testsCfg > >>>>>>>> initialization to > >>>>>>>> IgniteConfigVariationsAbstractTest#beforeTestsStarted method. > >> After > >>>> such > >>>>>>>> changes ruleChain becomes not necessary. > >>>>>>>> > >>>>>>>> But I faced another problem - multiple failures on TeamCity [7]. > >> From > >>>>>>>> logs, > >>>>>>>> it seems that failures are related to what tests check, but not > >> JUnit > >>>>>>>> error. > >>>>>>>> I can not track TeamCity history on that fact were tests failed or > >>>>>>>> not on > >>>>>>>> the previous JUnit version - the oldest log is dated the start of > >> the > >>>>>>>> March > >>>>>>>> when JUnit4 > >>>>>>>> already was implemented (for example, this [8] test). > >>>>>>>> Moreover, there are not so much failed tests, but because of > >> running > >>>>>>>> with > >>>>>>>> multiple configurations > >>>>>>>> (InterceptorCacheConfigVariationsFullApiTestSuite_0 > >>>>>>>> ..._95) it turns out about 400 failed tests. TeamCity results also > >>>>>>>> confirm > >>>>>>>> that tests do not work in the master branch - duration time is > >> less > >>>> than > >>>>>>>> 1ms. Now all tests are green and that is not surprising - under > >> @Test > >>>>>>>> annotation, nothing happens. > >>>>>>>> > >>>>>>>> Could some of us confirm or disprove my guess that tests are red > >>>>>>>> because of > >>>>>>>> its functionality, but not JUnit implementation? > >>>>>>>> And if it is true, how should I take such fact into account in > >> this > >>>>>>>> ticket? > >>>>>>>> > >>>>>>>> [1] > >>>>>>>> > >>>> > >> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-30%3A+Migration+to+JUnit+5 > >>>>>>>> > >>>>>>>> [2] > >>>>>>>> > >>>> > >> > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testsuites/InterceptorCacheConfigVariationsFullApiTestSuite.java > >>>>>>>> > >>>>>>>> [3] > >>>>>>>> > >>>> > >> > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/test/ConfigVariationsTestSuiteBuilderTest.java#L434 > >>>>>>>> > >>>>>>>> [4] https://issues.apache.org/jira/browse/IGNITE-11708 > >>>>>>>> [5] > >>>>>>>> > >>>> > >> > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteConfigVariationsAbstractTest.java#L62 > >>>>>>>> > >>>>>>>> [6] > >>>>>>>> > >>>> > >> > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/GridAbstractTest.java#L181 > >>>>>>>> > >>>>>>>> [7] > >>>>>>>> > >>>> > >> > https://mtcga.gridgain.com/pr.html?serverId=apache&suiteId=IgniteTests24Java8_RunAll&branchForTc=pull/6434/head&action=Latest > >>>>>>>> > >>>>>>>> [8] > >>>>>>>> > >>>> > >> > https://ci.ignite.apache.org/project.html?tab=testDetails&projectId=IgniteTests24Java8&testNameId=-9037806478172035481&page=8 > >>>>>>>> > >>>>>>>> > >>>>> > >>>>> > >>>> > >>> > >>> > >>> -- > >>> Ivan Fedotov. > >>> > >>> ivanan...@gmail.com > >> > >> > >> > >> -- > >> Best regards, > >> Ivan Pavlukhin > >> > > > > > > -- > > Ivan Fedotov. > > > > ivanan...@gmail.com > -- Ivan Fedotov. ivanan...@gmail.com