Re: IgniteConfigVariationsAbstractTest subclasses do not run
Igniters, I created corresponding tickets [1][2][3][4] in Jira and outlined description of problems in a nutshell. In the context of the current ticket (IGNITE-11708), I ignored them. If some of us have a comprehension of the problem why tests are failed, please let know here or in the tickets. [1] https://issues.apache.org/jira/browse/IGNITE-11883 [2] https://issues.apache.org/jira/browse/IGNITE-11884 [3] https://issues.apache.org/jira/browse/IGNITE-11885 [4] https://issues.apache.org/jira/browse/IGNITE-11886 чт, 30 мая 2019 г. в 14:55, Ivan Fedotov : > Hi Igniters! > > I found the solution for how to run IgniteConfigVariationsAbstractTest. I > removed garbage injection [1] and static variable [2]. > Now assigning of VariationsTestsConfig proceeds in constructor class which > is created dynamically [3]. > > But I faced with the problem that a certain amount of tests are failed. It > seems that they failed because of the real reasons, not because > of the test workflow. Despite those fact that a big amount of tests on TC > became red, really failed tests are not so much. Derivatives tests appear > from > different configs. > > Could some of us take a look on failed tests? I am going to ignore them in > the context of the current ticket [5] and create separate > issues. > > [1] > https://github.com/apache/ignite/pull/6434/files#diff-cd3a07ce10f21d396c1eac0c328850e0L102 > [2] > https://github.com/apache/ignite/pull/6434/files#diff-cd3a07ce10f21d396c1eac0c328850e0L84 > [3] > https://github.com/apache/ignite/pull/6434/files#diff-3da5dbb6f22e5bf3bf18734933f1bfc5R434 > [4] > https://ci.ignite.apache.org/viewLog.html?buildId=3978807&buildTypeId=IgniteTests24Java8_RunAllNightly > [5]https://issues.apache.org/jira/browse/IGNITE-11708 > > вт, 14 мая 2019 г. в 16:31, Ivan Pavlukhina : > >> Ivan F., >> >> Agree with referring tickets in @Ignore annotation. >> >> Currently I have no access to a computer. Will be able to look at updated >> PR next week. >> >> Sent from my iPhone >> >> > On 14 May 2019, at 10:55, Ivan Fedotov wrote: >> > >> > Ivan P., >> > I managed with IgniteConfigVariationsAbstractTest - now subclasses >> enable >> > [1]. >> > I ignored all the failed tests that were mentioned in our conversation. >> If >> > you don't mind, I can create appropriate tickets and mention them in >> Ignore >> > annotation. >> > >> > [1] https://github.com/apache/ignite/pull/6434/files >> > >> > ср, 1 мая 2019 г. в 15:29, Павлухин Иван : >> > >> >> Ivan F., >> >> >> >> I think that it is better to enable IgniteConfigVariationsAbstractTest >> >> subclasses first, so no new broken tests will appear. After that we >> >> can fix IgniteConfigVariationsAbstractTest subclasses one by one in >> >> separate ticket(s). >> >> >> >> вт, 30 апр. 2019 г. в 13:23, Ivan Fedotov : >> >>> >> >>> Ivan R., Ivan P., thank you for the suggestions, I took a look at >> them. >> >>> >> >>> According to checking CacheAtomicityMode on null in >> >>> IgniteCacheConfigVariationsAbstractTest#atomicityMode - are you sure >> >>> that checking should proceed on test level? May be it will be better >> to >> >>> make default cache value in case if cc.getAtomicityMode() == null >> >>> in CacheConfiguration constructor [1]? >> >>> >> >>> Also let me suggest one minor change according cache specification in >> >>> IgniteCacheReadThroughEvictionSelfTest. The same logic is used in >> >>> GridCacheAbstractSelfTest#cacheConfiguration [2]. >> >>> I think we can excract this code block in a separate static methods >> >>> (initCacheConfig, for example) in >> IgniteCacheReadThroughEvictionSelfTest >> >> and >> >>> invoke it in IgniteCacheReadThroughEvictionSelfTest#variationConfig >> and >> >>> GridCacheAbstractSelfTest#cacheConfiguration methods. >> >>> >> >>> In addition, I want to draw your attention to >> >>> IgniteContinuousQueryConfigVariationsSuite test runs [3]. >> >>> CacheContinuousQueryVariationsTest are generated dynamically. >> >>> The first batch (12 tests) works fine, but on the next runs we got >> NPE in >> >>> IgniteCacheConfigVariationsAbstractTest#afterTest - default cache does >> >> not >> >>> exist and we can not >> >>> invoke unwrap() on this [4]. >> >>> Seems that the problem is in >> >>> >> >> >> IgniteCacheConfigVariationsAbstractTest#beforeTestsStarted/afterTestsStopped >> >>> methods, cache is not properly created (or already existed cache is >> >>> destroyed). >> >>> >> >>> By the way, should I resolve these issues in the context of >> IGNITE-11708 >> >> or >> >>> create a separate ticket(s)? >> >>> >> >>> [1] >> >>> >> >> >> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/configuration/CacheConfiguration.java#L434 >> >>> [2] >> >>> >> >> >> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/GridCacheAbstractSelfTest.java#L235 >> >>> [3] >> >>> >> >> >> https://ci.ignite.apache.org/viewLog.html?buildId=3701865&buildTypeId=Ignit
Re: IgniteConfigVariationsAbstractTest subclasses do not run
Hi Igniters! I found the solution for how to run IgniteConfigVariationsAbstractTest. I removed garbage injection [1] and static variable [2]. Now assigning of VariationsTestsConfig proceeds in constructor class which is created dynamically [3]. But I faced with the problem that a certain amount of tests are failed. It seems that they failed because of the real reasons, not because of the test workflow. Despite those fact that a big amount of tests on TC became red, really failed tests are not so much. Derivatives tests appear from different configs. Could some of us take a look on failed tests? I am going to ignore them in the context of the current ticket [5] and create separate issues. [1] https://github.com/apache/ignite/pull/6434/files#diff-cd3a07ce10f21d396c1eac0c328850e0L102 [2] https://github.com/apache/ignite/pull/6434/files#diff-cd3a07ce10f21d396c1eac0c328850e0L84 [3] https://github.com/apache/ignite/pull/6434/files#diff-3da5dbb6f22e5bf3bf18734933f1bfc5R434 [4] https://ci.ignite.apache.org/viewLog.html?buildId=3978807&buildTypeId=IgniteTests24Java8_RunAllNightly [5]https://issues.apache.org/jira/browse/IGNITE-11708 вт, 14 мая 2019 г. в 16:31, Ivan Pavlukhina : > Ivan F., > > Agree with referring tickets in @Ignore annotation. > > Currently I have no access to a computer. Will be able to look at updated > PR next week. > > Sent from my iPhone > > > On 14 May 2019, at 10:55, Ivan Fedotov wrote: > > > > Ivan P., > > I managed with IgniteConfigVariationsAbstractTest - now subclasses enable > > [1]. > > I ignored all the failed tests that were mentioned in our conversation. > If > > you don't mind, I can create appropriate tickets and mention them in > Ignore > > annotation. > > > > [1] https://github.com/apache/ignite/pull/6434/files > > > > ср, 1 мая 2019 г. в 15:29, Павлухин Иван : > > > >> Ivan F., > >> > >> I think that it is better to enable IgniteConfigVariationsAbstractTest > >> subclasses first, so no new broken tests will appear. After that we > >> can fix IgniteConfigVariationsAbstractTest subclasses one by one in > >> separate ticket(s). > >> > >> вт, 30 апр. 2019 г. в 13:23, Ivan Fedotov : > >>> > >>> Ivan R., Ivan P., thank you for the suggestions, I took a look at them. > >>> > >>> According to checking CacheAtomicityMode on null in > >>> IgniteCacheConfigVariationsAbstractTest#atomicityMode - are you sure > >>> that checking should proceed on test level? May be it will be better to > >>> make default cache value in case if cc.getAtomicityMode() == null > >>> in CacheConfiguration constructor [1]? > >>> > >>> Also let me suggest one minor change according cache specification in > >>> IgniteCacheReadThroughEvictionSelfTest. The same logic is used in > >>> GridCacheAbstractSelfTest#cacheConfiguration [2]. > >>> I think we can excract this code block in a separate static methods > >>> (initCacheConfig, for example) in > IgniteCacheReadThroughEvictionSelfTest > >> and > >>> invoke it in IgniteCacheReadThroughEvictionSelfTest#variationConfig and > >>> GridCacheAbstractSelfTest#cacheConfiguration methods. > >>> > >>> In addition, I want to draw your attention to > >>> IgniteContinuousQueryConfigVariationsSuite test runs [3]. > >>> CacheContinuousQueryVariationsTest are generated dynamically. > >>> The first batch (12 tests) works fine, but on the next runs we got NPE > in > >>> IgniteCacheConfigVariationsAbstractTest#afterTest - default cache does > >> not > >>> exist and we can not > >>> invoke unwrap() on this [4]. > >>> Seems that the problem is in > >>> > >> > IgniteCacheConfigVariationsAbstractTest#beforeTestsStarted/afterTestsStopped > >>> methods, cache is not properly created (or already existed cache is > >>> destroyed). > >>> > >>> By the way, should I resolve these issues in the context of > IGNITE-11708 > >> or > >>> create a separate ticket(s)? > >>> > >>> [1] > >>> > >> > https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/configuration/CacheConfiguration.java#L434 > >>> [2] > >>> > >> > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/GridCacheAbstractSelfTest.java#L235 > >>> [3] > >>> > >> > https://ci.ignite.apache.org/viewLog.html?buildId=3701865&buildTypeId=IgniteTests24Java8_RunAllNightly > >>> [4] > >>> > >> > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteCacheConfigVariationsAbstractTest.java#L212 > >>> > >>> пт, 26 апр. 2019 г. в 18:01, Ivan Rakov : > >>> > Ivan P., > > Good catch, thanks. > > I was wrong, test scenario is correct. The problem was in > atomicityMode() method - it could have returned null (which was okay > >> for > config generation, but wasn't expected in the test code). > Please take a look at tx_out_test_fixed.patch (attached to > IGNITE-11708). To sum it up, both issues should be fixed now. > > Best Regards, > Iv
Re: IgniteConfigVariationsAbstractTest subclasses do not run
Ivan F., Agree with referring tickets in @Ignore annotation. Currently I have no access to a computer. Will be able to look at updated PR next week. Sent from my iPhone > On 14 May 2019, at 10:55, Ivan Fedotov wrote: > > Ivan P., > I managed with IgniteConfigVariationsAbstractTest - now subclasses enable > [1]. > I ignored all the failed tests that were mentioned in our conversation. If > you don't mind, I can create appropriate tickets and mention them in Ignore > annotation. > > [1] https://github.com/apache/ignite/pull/6434/files > > ср, 1 мая 2019 г. в 15:29, Павлухин Иван : > >> Ivan F., >> >> I think that it is better to enable IgniteConfigVariationsAbstractTest >> subclasses first, so no new broken tests will appear. After that we >> can fix IgniteConfigVariationsAbstractTest subclasses one by one in >> separate ticket(s). >> >> вт, 30 апр. 2019 г. в 13:23, Ivan Fedotov : >>> >>> Ivan R., Ivan P., thank you for the suggestions, I took a look at them. >>> >>> According to checking CacheAtomicityMode on null in >>> IgniteCacheConfigVariationsAbstractTest#atomicityMode - are you sure >>> that checking should proceed on test level? May be it will be better to >>> make default cache value in case if cc.getAtomicityMode() == null >>> in CacheConfiguration constructor [1]? >>> >>> Also let me suggest one minor change according cache specification in >>> IgniteCacheReadThroughEvictionSelfTest. The same logic is used in >>> GridCacheAbstractSelfTest#cacheConfiguration [2]. >>> I think we can excract this code block in a separate static methods >>> (initCacheConfig, for example) in IgniteCacheReadThroughEvictionSelfTest >> and >>> invoke it in IgniteCacheReadThroughEvictionSelfTest#variationConfig and >>> GridCacheAbstractSelfTest#cacheConfiguration methods. >>> >>> In addition, I want to draw your attention to >>> IgniteContinuousQueryConfigVariationsSuite test runs [3]. >>> CacheContinuousQueryVariationsTest are generated dynamically. >>> The first batch (12 tests) works fine, but on the next runs we got NPE in >>> IgniteCacheConfigVariationsAbstractTest#afterTest - default cache does >> not >>> exist and we can not >>> invoke unwrap() on this [4]. >>> Seems that the problem is in >>> >> IgniteCacheConfigVariationsAbstractTest#beforeTestsStarted/afterTestsStopped >>> methods, cache is not properly created (or already existed cache is >>> destroyed). >>> >>> By the way, should I resolve these issues in the context of IGNITE-11708 >> or >>> create a separate ticket(s)? >>> >>> [1] >>> >> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/configuration/CacheConfiguration.java#L434 >>> [2] >>> >> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/GridCacheAbstractSelfTest.java#L235 >>> [3] >>> >> https://ci.ignite.apache.org/viewLog.html?buildId=3701865&buildTypeId=IgniteTests24Java8_RunAllNightly >>> [4] >>> >> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteCacheConfigVariationsAbstractTest.java#L212 >>> >>> пт, 26 апр. 2019 г. в 18:01, Ivan Rakov : >>> Ivan P., Good catch, thanks. I was wrong, test scenario is correct. The problem was in atomicityMode() method - it could have returned null (which was okay >> for config generation, but wasn't expected in the test code). Please take a look at tx_out_test_fixed.patch (attached to IGNITE-11708). To sum it up, both issues should be fixed now. Best Regards, Ivan Rakov > On 26.04.2019 14:40, Павлухин Иван wrote: > Ivan R., > > As I can see IgniteCacheConfigVariationsFullApiTest#testGetOutTx does > not expect lock/unlock events due to line: > if (atomicityMode() == ATOMIC) > return lockEvtCnt.get() == 0; > > Could you please elaborate? > > пт, 26 апр. 2019 г. в 13:32, Ivan Rakov : >> Ivan, >> >> Seems like IgniteCacheReadThroughEvictionSelfTest is broken. Test >> scenario assumes that even after expiration entry will be present in >> IgniteCache as per it will be loaded from CacheStore. However, >> CacheStore is not specified in node config. I've added patch that >> enables cache store factory, please check IGNITE-11708 attachments. >> Regarding IgniteCacheConfigVariationsFullApiTest#testGetOutTx* >> tests: >> from my point of view, test scenarios make no sense. We perform >> get() >> operation from ATOMIC caches and expect that entries will be >> locked. I >> don't understand why we should lock entries on ATOMIC get, >> therefore I >> suppose to remove part of code where we listen and check >> EVT_CACHE_OBJECT_LOCKED/UNLOCKED events. >> >> Best Regards, >> Ivan Rakov >> >>> On 17.04.2019 22:05, Ivan Rakov wrote: >>> Hi Ivan, >>> >>> I've checked your branch. Seems li
Re: IgniteConfigVariationsAbstractTest subclasses do not run
Ivan P., I managed with IgniteConfigVariationsAbstractTest - now subclasses enable [1]. I ignored all the failed tests that were mentioned in our conversation. If you don't mind, I can create appropriate tickets and mention them in Ignore annotation. [1] https://github.com/apache/ignite/pull/6434/files ср, 1 мая 2019 г. в 15:29, Павлухин Иван : > Ivan F., > > I think that it is better to enable IgniteConfigVariationsAbstractTest > subclasses first, so no new broken tests will appear. After that we > can fix IgniteConfigVariationsAbstractTest subclasses one by one in > separate ticket(s). > > вт, 30 апр. 2019 г. в 13:23, Ivan Fedotov : > > > > Ivan R., Ivan P., thank you for the suggestions, I took a look at them. > > > > According to checking CacheAtomicityMode on null in > > IgniteCacheConfigVariationsAbstractTest#atomicityMode - are you sure > > that checking should proceed on test level? May be it will be better to > > make default cache value in case if cc.getAtomicityMode() == null > > in CacheConfiguration constructor [1]? > > > > Also let me suggest one minor change according cache specification in > > IgniteCacheReadThroughEvictionSelfTest. The same logic is used in > > GridCacheAbstractSelfTest#cacheConfiguration [2]. > > I think we can excract this code block in a separate static methods > > (initCacheConfig, for example) in IgniteCacheReadThroughEvictionSelfTest > and > > invoke it in IgniteCacheReadThroughEvictionSelfTest#variationConfig and > > GridCacheAbstractSelfTest#cacheConfiguration methods. > > > > In addition, I want to draw your attention to > > IgniteContinuousQueryConfigVariationsSuite test runs [3]. > > CacheContinuousQueryVariationsTest are generated dynamically. > > The first batch (12 tests) works fine, but on the next runs we got NPE in > > IgniteCacheConfigVariationsAbstractTest#afterTest - default cache does > not > > exist and we can not > > invoke unwrap() on this [4]. > > Seems that the problem is in > > > IgniteCacheConfigVariationsAbstractTest#beforeTestsStarted/afterTestsStopped > > methods, cache is not properly created (or already existed cache is > > destroyed). > > > > By the way, should I resolve these issues in the context of IGNITE-11708 > or > > create a separate ticket(s)? > > > > [1] > > > https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/configuration/CacheConfiguration.java#L434 > > [2] > > > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/GridCacheAbstractSelfTest.java#L235 > > [3] > > > https://ci.ignite.apache.org/viewLog.html?buildId=3701865&buildTypeId=IgniteTests24Java8_RunAllNightly > > [4] > > > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteCacheConfigVariationsAbstractTest.java#L212 > > > > пт, 26 апр. 2019 г. в 18:01, Ivan Rakov : > > > > > Ivan P., > > > > > > Good catch, thanks. > > > > > > I was wrong, test scenario is correct. The problem was in > > > atomicityMode() method - it could have returned null (which was okay > for > > > config generation, but wasn't expected in the test code). > > > Please take a look at tx_out_test_fixed.patch (attached to > > > IGNITE-11708). To sum it up, both issues should be fixed now. > > > > > > Best Regards, > > > Ivan Rakov > > > > > > On 26.04.2019 14:40, Павлухин Иван wrote: > > > > Ivan R., > > > > > > > > As I can see IgniteCacheConfigVariationsFullApiTest#testGetOutTx does > > > > not expect lock/unlock events due to line: > > > > if (atomicityMode() == ATOMIC) > > > > return lockEvtCnt.get() == 0; > > > > > > > > Could you please elaborate? > > > > > > > > пт, 26 апр. 2019 г. в 13:32, Ivan Rakov : > > > >> Ivan, > > > >> > > > >> Seems like IgniteCacheReadThroughEvictionSelfTest is broken. Test > > > >> scenario assumes that even after expiration entry will be present in > > > >> IgniteCache as per it will be loaded from CacheStore. However, > > > >> CacheStore is not specified in node config. I've added patch that > > > >> enables cache store factory, please check IGNITE-11708 attachments. > > > >> Regarding IgniteCacheConfigVariationsFullApiTest#testGetOutTx* > tests: > > > >> from my point of view, test scenarios make no sense. We perform > get() > > > >> operation from ATOMIC caches and expect that entries will be > locked. I > > > >> don't understand why we should lock entries on ATOMIC get, > therefore I > > > >> suppose to remove part of code where we listen and check > > > >> EVT_CACHE_OBJECT_LOCKED/UNLOCKED events. > > > >> > > > >> Best Regards, > > > >> Ivan Rakov > > > >> > > > >> On 17.04.2019 22:05, Ivan Rakov wrote: > > > >>> Hi Ivan, > > > >>> > > > >>> I've checked your branch. Seems like these tests fail due to real > > > >>> issue in functionality. > > > >>> I'll take a look. > > > >>> > > > >>> Best Regards, > > > >>> Ivan Rakov > > > >>> > > > >>> On 17.04.2019 13:54, Ivan Fedotov wrote: > > >
Re: IgniteConfigVariationsAbstractTest subclasses do not run
Ivan F., I think that it is better to enable IgniteConfigVariationsAbstractTest subclasses first, so no new broken tests will appear. After that we can fix IgniteConfigVariationsAbstractTest subclasses one by one in separate ticket(s). вт, 30 апр. 2019 г. в 13:23, Ivan Fedotov : > > Ivan R., Ivan P., thank you for the suggestions, I took a look at them. > > According to checking CacheAtomicityMode on null in > IgniteCacheConfigVariationsAbstractTest#atomicityMode - are you sure > that checking should proceed on test level? May be it will be better to > make default cache value in case if cc.getAtomicityMode() == null > in CacheConfiguration constructor [1]? > > Also let me suggest one minor change according cache specification in > IgniteCacheReadThroughEvictionSelfTest. The same logic is used in > GridCacheAbstractSelfTest#cacheConfiguration [2]. > I think we can excract this code block in a separate static methods > (initCacheConfig, for example) in IgniteCacheReadThroughEvictionSelfTest and > invoke it in IgniteCacheReadThroughEvictionSelfTest#variationConfig and > GridCacheAbstractSelfTest#cacheConfiguration methods. > > In addition, I want to draw your attention to > IgniteContinuousQueryConfigVariationsSuite test runs [3]. > CacheContinuousQueryVariationsTest are generated dynamically. > The first batch (12 tests) works fine, but on the next runs we got NPE in > IgniteCacheConfigVariationsAbstractTest#afterTest - default cache does not > exist and we can not > invoke unwrap() on this [4]. > Seems that the problem is in > IgniteCacheConfigVariationsAbstractTest#beforeTestsStarted/afterTestsStopped > methods, cache is not properly created (or already existed cache is > destroyed). > > By the way, should I resolve these issues in the context of IGNITE-11708 or > create a separate ticket(s)? > > [1] > https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/configuration/CacheConfiguration.java#L434 > [2] > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/GridCacheAbstractSelfTest.java#L235 > [3] > https://ci.ignite.apache.org/viewLog.html?buildId=3701865&buildTypeId=IgniteTests24Java8_RunAllNightly > [4] > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteCacheConfigVariationsAbstractTest.java#L212 > > пт, 26 апр. 2019 г. в 18:01, Ivan Rakov : > > > Ivan P., > > > > Good catch, thanks. > > > > I was wrong, test scenario is correct. The problem was in > > atomicityMode() method - it could have returned null (which was okay for > > config generation, but wasn't expected in the test code). > > Please take a look at tx_out_test_fixed.patch (attached to > > IGNITE-11708). To sum it up, both issues should be fixed now. > > > > Best Regards, > > Ivan Rakov > > > > On 26.04.2019 14:40, Павлухин Иван wrote: > > > Ivan R., > > > > > > As I can see IgniteCacheConfigVariationsFullApiTest#testGetOutTx does > > > not expect lock/unlock events due to line: > > > if (atomicityMode() == ATOMIC) > > > return lockEvtCnt.get() == 0; > > > > > > Could you please elaborate? > > > > > > пт, 26 апр. 2019 г. в 13:32, Ivan Rakov : > > >> Ivan, > > >> > > >> Seems like IgniteCacheReadThroughEvictionSelfTest is broken. Test > > >> scenario assumes that even after expiration entry will be present in > > >> IgniteCache as per it will be loaded from CacheStore. However, > > >> CacheStore is not specified in node config. I've added patch that > > >> enables cache store factory, please check IGNITE-11708 attachments. > > >> Regarding IgniteCacheConfigVariationsFullApiTest#testGetOutTx* tests: > > >> from my point of view, test scenarios make no sense. We perform get() > > >> operation from ATOMIC caches and expect that entries will be locked. I > > >> don't understand why we should lock entries on ATOMIC get, therefore I > > >> suppose to remove part of code where we listen and check > > >> EVT_CACHE_OBJECT_LOCKED/UNLOCKED events. > > >> > > >> Best Regards, > > >> Ivan Rakov > > >> > > >> On 17.04.2019 22:05, Ivan Rakov wrote: > > >>> Hi Ivan, > > >>> > > >>> I've checked your branch. Seems like these tests fail due to real > > >>> issue in functionality. > > >>> I'll take a look. > > >>> > > >>> Best Regards, > > >>> Ivan Rakov > > >>> > > >>> On 17.04.2019 13:54, Ivan Fedotov wrote: > > Hi, Igniters! > > > > During work on iep-30[1] I discovered that > > IgniteConfigVariationsAbstractTest subclasses - it is about 15_000 > > tests[2] > > - do not work. > > You can check it just run one of the tests with log output, for > > example > > ConfigVariationsTestSuiteBuilderTest#LegacyLifecycleTest#test1 [3]. > > There is no warning notification in the console. The same situation > > with > > other IgniteConfigVariationsAbstractTest subclasses - tests run, but > > they > > simply represent empty code. > > >
Re: IgniteConfigVariationsAbstractTest subclasses do not run
Ivan R., Ivan P., thank you for the suggestions, I took a look at them. According to checking CacheAtomicityMode on null in IgniteCacheConfigVariationsAbstractTest#atomicityMode - are you sure that checking should proceed on test level? May be it will be better to make default cache value in case if cc.getAtomicityMode() == null in CacheConfiguration constructor [1]? Also let me suggest one minor change according cache specification in IgniteCacheReadThroughEvictionSelfTest. The same logic is used in GridCacheAbstractSelfTest#cacheConfiguration [2]. I think we can excract this code block in a separate static methods (initCacheConfig, for example) in IgniteCacheReadThroughEvictionSelfTest and invoke it in IgniteCacheReadThroughEvictionSelfTest#variationConfig and GridCacheAbstractSelfTest#cacheConfiguration methods. In addition, I want to draw your attention to IgniteContinuousQueryConfigVariationsSuite test runs [3]. CacheContinuousQueryVariationsTest are generated dynamically. The first batch (12 tests) works fine, but on the next runs we got NPE in IgniteCacheConfigVariationsAbstractTest#afterTest - default cache does not exist and we can not invoke unwrap() on this [4]. Seems that the problem is in IgniteCacheConfigVariationsAbstractTest#beforeTestsStarted/afterTestsStopped methods, cache is not properly created (or already existed cache is destroyed). By the way, should I resolve these issues in the context of IGNITE-11708 or create a separate ticket(s)? [1] https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/configuration/CacheConfiguration.java#L434 [2] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/GridCacheAbstractSelfTest.java#L235 [3] https://ci.ignite.apache.org/viewLog.html?buildId=3701865&buildTypeId=IgniteTests24Java8_RunAllNightly [4] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteCacheConfigVariationsAbstractTest.java#L212 пт, 26 апр. 2019 г. в 18:01, Ivan Rakov : > Ivan P., > > Good catch, thanks. > > I was wrong, test scenario is correct. The problem was in > atomicityMode() method - it could have returned null (which was okay for > config generation, but wasn't expected in the test code). > Please take a look at tx_out_test_fixed.patch (attached to > IGNITE-11708). To sum it up, both issues should be fixed now. > > Best Regards, > Ivan Rakov > > On 26.04.2019 14:40, Павлухин Иван wrote: > > Ivan R., > > > > As I can see IgniteCacheConfigVariationsFullApiTest#testGetOutTx does > > not expect lock/unlock events due to line: > > if (atomicityMode() == ATOMIC) > > return lockEvtCnt.get() == 0; > > > > Could you please elaborate? > > > > пт, 26 апр. 2019 г. в 13:32, Ivan Rakov : > >> Ivan, > >> > >> Seems like IgniteCacheReadThroughEvictionSelfTest is broken. Test > >> scenario assumes that even after expiration entry will be present in > >> IgniteCache as per it will be loaded from CacheStore. However, > >> CacheStore is not specified in node config. I've added patch that > >> enables cache store factory, please check IGNITE-11708 attachments. > >> Regarding IgniteCacheConfigVariationsFullApiTest#testGetOutTx* tests: > >> from my point of view, test scenarios make no sense. We perform get() > >> operation from ATOMIC caches and expect that entries will be locked. I > >> don't understand why we should lock entries on ATOMIC get, therefore I > >> suppose to remove part of code where we listen and check > >> EVT_CACHE_OBJECT_LOCKED/UNLOCKED events. > >> > >> Best Regards, > >> Ivan Rakov > >> > >> On 17.04.2019 22:05, Ivan Rakov wrote: > >>> Hi Ivan, > >>> > >>> I've checked your branch. Seems like these tests fail due to real > >>> issue in functionality. > >>> I'll take a look. > >>> > >>> Best Regards, > >>> Ivan Rakov > >>> > >>> On 17.04.2019 13:54, Ivan Fedotov wrote: > Hi, Igniters! > > During work on iep-30[1] I discovered that > IgniteConfigVariationsAbstractTest subclasses - it is about 15_000 > tests[2] > - do not work. > You can check it just run one of the tests with log output, for > example > ConfigVariationsTestSuiteBuilderTest#LegacyLifecycleTest#test1 [3]. > There is no warning notification in the console. The same situation > with > other IgniteConfigVariationsAbstractTest subclasses - tests run, but > they > simply represent empty code. > > So, I created a ticket on such issue [4] and it turned out that the > problem > is with ruleChain in IgniteConfigVariationsAbstractTest [5]. > The rule that is responsible for running a test statement does not > start > indeed [6] under ruleChain runRule. I suggested a solution - move > testsCfg > initialization to > IgniteConfigVariationsAbstractTest#beforeTestsStarted method. After > such > changes ruleChain becomes not necessary. > > >>>
Re: IgniteConfigVariationsAbstractTest subclasses do not run
Ivan P., Good catch, thanks. I was wrong, test scenario is correct. The problem was in atomicityMode() method - it could have returned null (which was okay for config generation, but wasn't expected in the test code). Please take a look at tx_out_test_fixed.patch (attached to IGNITE-11708). To sum it up, both issues should be fixed now. Best Regards, Ivan Rakov On 26.04.2019 14:40, Павлухин Иван wrote: Ivan R., As I can see IgniteCacheConfigVariationsFullApiTest#testGetOutTx does not expect lock/unlock events due to line: if (atomicityMode() == ATOMIC) return lockEvtCnt.get() == 0; Could you please elaborate? пт, 26 апр. 2019 г. в 13:32, Ivan Rakov : Ivan, Seems like IgniteCacheReadThroughEvictionSelfTest is broken. Test scenario assumes that even after expiration entry will be present in IgniteCache as per it will be loaded from CacheStore. However, CacheStore is not specified in node config. I've added patch that enables cache store factory, please check IGNITE-11708 attachments. Regarding IgniteCacheConfigVariationsFullApiTest#testGetOutTx* tests: from my point of view, test scenarios make no sense. We perform get() operation from ATOMIC caches and expect that entries will be locked. I don't understand why we should lock entries on ATOMIC get, therefore I suppose to remove part of code where we listen and check EVT_CACHE_OBJECT_LOCKED/UNLOCKED events. Best Regards, Ivan Rakov On 17.04.2019 22:05, Ivan Rakov wrote: Hi Ivan, I've checked your branch. Seems like these tests fail due to real issue in functionality. I'll take a look. Best Regards, Ivan Rakov On 17.04.2019 13:54, Ivan Fedotov wrote: Hi, Igniters! During work on iep-30[1] I discovered that IgniteConfigVariationsAbstractTest subclasses - it is about 15_000 tests[2] - do not work. You can check it just run one of the tests with log output, for example ConfigVariationsTestSuiteBuilderTest#LegacyLifecycleTest#test1 [3]. There is no warning notification in the console. The same situation with other IgniteConfigVariationsAbstractTest subclasses - tests run, but they simply represent empty code. So, I created a ticket on such issue [4] and it turned out that the problem is with ruleChain in IgniteConfigVariationsAbstractTest [5]. The rule that is responsible for running a test statement does not start indeed [6] under ruleChain runRule. I suggested a solution - move testsCfg initialization to IgniteConfigVariationsAbstractTest#beforeTestsStarted method. After such changes ruleChain becomes not necessary. But I faced another problem - multiple failures on TeamCity [7]. From logs, it seems that failures are related to what tests check, but not JUnit error. I can not track TeamCity history on that fact were tests failed or not on the previous JUnit version - the oldest log is dated the start of the March when JUnit4 already was implemented (for example, this [8] test). Moreover, there are not so much failed tests, but because of running with multiple configurations (InterceptorCacheConfigVariationsFullApiTestSuite_0 ..._95) it turns out about 400 failed tests. TeamCity results also confirm that tests do not work in the master branch - duration time is less than 1ms. Now all tests are green and that is not surprising - under @Test annotation, nothing happens. Could some of us confirm or disprove my guess that tests are red because of its functionality, but not JUnit implementation? And if it is true, how should I take such fact into account in this ticket? [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-30%3A+Migration+to+JUnit+5 [2] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testsuites/InterceptorCacheConfigVariationsFullApiTestSuite.java [3] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/test/ConfigVariationsTestSuiteBuilderTest.java#L434 [4] https://issues.apache.org/jira/browse/IGNITE-11708 [5] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteConfigVariationsAbstractTest.java#L62 [6] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/GridAbstractTest.java#L181 [7] https://mtcga.gridgain.com/pr.html?serverId=apache&suiteId=IgniteTests24Java8_RunAll&branchForTc=pull/6434/head&action=Latest [8] https://ci.ignite.apache.org/project.html?tab=testDetails&projectId=IgniteTests24Java8&testNameId=-9037806478172035481&page=8
Re: IgniteConfigVariationsAbstractTest subclasses do not run
Ivan R., As I can see IgniteCacheConfigVariationsFullApiTest#testGetOutTx does not expect lock/unlock events due to line: if (atomicityMode() == ATOMIC) return lockEvtCnt.get() == 0; Could you please elaborate? пт, 26 апр. 2019 г. в 13:32, Ivan Rakov : > > Ivan, > > Seems like IgniteCacheReadThroughEvictionSelfTest is broken. Test > scenario assumes that even after expiration entry will be present in > IgniteCache as per it will be loaded from CacheStore. However, > CacheStore is not specified in node config. I've added patch that > enables cache store factory, please check IGNITE-11708 attachments. > Regarding IgniteCacheConfigVariationsFullApiTest#testGetOutTx* tests: > from my point of view, test scenarios make no sense. We perform get() > operation from ATOMIC caches and expect that entries will be locked. I > don't understand why we should lock entries on ATOMIC get, therefore I > suppose to remove part of code where we listen and check > EVT_CACHE_OBJECT_LOCKED/UNLOCKED events. > > Best Regards, > Ivan Rakov > > On 17.04.2019 22:05, Ivan Rakov wrote: > > Hi Ivan, > > > > I've checked your branch. Seems like these tests fail due to real > > issue in functionality. > > I'll take a look. > > > > Best Regards, > > Ivan Rakov > > > > On 17.04.2019 13:54, Ivan Fedotov wrote: > >> Hi, Igniters! > >> > >> During work on iep-30[1] I discovered that > >> IgniteConfigVariationsAbstractTest subclasses - it is about 15_000 > >> tests[2] > >> - do not work. > >> You can check it just run one of the tests with log output, for example > >> ConfigVariationsTestSuiteBuilderTest#LegacyLifecycleTest#test1 [3]. > >> There is no warning notification in the console. The same situation with > >> other IgniteConfigVariationsAbstractTest subclasses - tests run, but > >> they > >> simply represent empty code. > >> > >> So, I created a ticket on such issue [4] and it turned out that the > >> problem > >> is with ruleChain in IgniteConfigVariationsAbstractTest [5]. > >> The rule that is responsible for running a test statement does not start > >> indeed [6] under ruleChain runRule. I suggested a solution - move > >> testsCfg > >> initialization to > >> IgniteConfigVariationsAbstractTest#beforeTestsStarted method. After such > >> changes ruleChain becomes not necessary. > >> > >> But I faced another problem - multiple failures on TeamCity [7]. From > >> logs, > >> it seems that failures are related to what tests check, but not JUnit > >> error. > >> I can not track TeamCity history on that fact were tests failed or > >> not on > >> the previous JUnit version - the oldest log is dated the start of the > >> March > >> when JUnit4 > >> already was implemented (for example, this [8] test). > >> Moreover, there are not so much failed tests, but because of running > >> with > >> multiple configurations > >> (InterceptorCacheConfigVariationsFullApiTestSuite_0 > >> ..._95) it turns out about 400 failed tests. TeamCity results also > >> confirm > >> that tests do not work in the master branch - duration time is less than > >> 1ms. Now all tests are green and that is not surprising - under @Test > >> annotation, nothing happens. > >> > >> Could some of us confirm or disprove my guess that tests are red > >> because of > >> its functionality, but not JUnit implementation? > >> And if it is true, how should I take such fact into account in this > >> ticket? > >> > >> [1] > >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-30%3A+Migration+to+JUnit+5 > >> > >> [2] > >> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testsuites/InterceptorCacheConfigVariationsFullApiTestSuite.java > >> > >> [3] > >> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/test/ConfigVariationsTestSuiteBuilderTest.java#L434 > >> > >> [4] https://issues.apache.org/jira/browse/IGNITE-11708 > >> [5] > >> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteConfigVariationsAbstractTest.java#L62 > >> > >> [6] > >> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/GridAbstractTest.java#L181 > >> > >> [7] > >> https://mtcga.gridgain.com/pr.html?serverId=apache&suiteId=IgniteTests24Java8_RunAll&branchForTc=pull/6434/head&action=Latest > >> > >> [8] > >> https://ci.ignite.apache.org/project.html?tab=testDetails&projectId=IgniteTests24Java8&testNameId=-9037806478172035481&page=8 > >> > >> -- Best regards, Ivan Pavlukhin
Re: IgniteConfigVariationsAbstractTest subclasses do not run
Ivan, Seems like IgniteCacheReadThroughEvictionSelfTest is broken. Test scenario assumes that even after expiration entry will be present in IgniteCache as per it will be loaded from CacheStore. However, CacheStore is not specified in node config. I've added patch that enables cache store factory, please check IGNITE-11708 attachments. Regarding IgniteCacheConfigVariationsFullApiTest#testGetOutTx* tests: from my point of view, test scenarios make no sense. We perform get() operation from ATOMIC caches and expect that entries will be locked. I don't understand why we should lock entries on ATOMIC get, therefore I suppose to remove part of code where we listen and check EVT_CACHE_OBJECT_LOCKED/UNLOCKED events. Best Regards, Ivan Rakov On 17.04.2019 22:05, Ivan Rakov wrote: Hi Ivan, I've checked your branch. Seems like these tests fail due to real issue in functionality. I'll take a look. Best Regards, Ivan Rakov On 17.04.2019 13:54, Ivan Fedotov wrote: Hi, Igniters! During work on iep-30[1] I discovered that IgniteConfigVariationsAbstractTest subclasses - it is about 15_000 tests[2] - do not work. You can check it just run one of the tests with log output, for example ConfigVariationsTestSuiteBuilderTest#LegacyLifecycleTest#test1 [3]. There is no warning notification in the console. The same situation with other IgniteConfigVariationsAbstractTest subclasses - tests run, but they simply represent empty code. So, I created a ticket on such issue [4] and it turned out that the problem is with ruleChain in IgniteConfigVariationsAbstractTest [5]. The rule that is responsible for running a test statement does not start indeed [6] under ruleChain runRule. I suggested a solution - move testsCfg initialization to IgniteConfigVariationsAbstractTest#beforeTestsStarted method. After such changes ruleChain becomes not necessary. But I faced another problem - multiple failures on TeamCity [7]. From logs, it seems that failures are related to what tests check, but not JUnit error. I can not track TeamCity history on that fact were tests failed or not on the previous JUnit version - the oldest log is dated the start of the March when JUnit4 already was implemented (for example, this [8] test). Moreover, there are not so much failed tests, but because of running with multiple configurations (InterceptorCacheConfigVariationsFullApiTestSuite_0 ..._95) it turns out about 400 failed tests. TeamCity results also confirm that tests do not work in the master branch - duration time is less than 1ms. Now all tests are green and that is not surprising - under @Test annotation, nothing happens. Could some of us confirm or disprove my guess that tests are red because of its functionality, but not JUnit implementation? And if it is true, how should I take such fact into account in this ticket? [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-30%3A+Migration+to+JUnit+5 [2] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testsuites/InterceptorCacheConfigVariationsFullApiTestSuite.java [3] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/test/ConfigVariationsTestSuiteBuilderTest.java#L434 [4] https://issues.apache.org/jira/browse/IGNITE-11708 [5] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteConfigVariationsAbstractTest.java#L62 [6] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/GridAbstractTest.java#L181 [7] https://mtcga.gridgain.com/pr.html?serverId=apache&suiteId=IgniteTests24Java8_RunAll&branchForTc=pull/6434/head&action=Latest [8] https://ci.ignite.apache.org/project.html?tab=testDetails&projectId=IgniteTests24Java8&testNameId=-9037806478172035481&page=8
Re: IgniteConfigVariationsAbstractTest subclasses do not run
Hi Dmitry, Maybe if it will turn out that all tests fail because of functionality we should ignore or mute these tests in the context of the ticket IGNITE-11708? ср, 17 апр. 2019 г. в 23:20, Dmitriy Pavlov : > Hi Ivan F. > > Thank you for finding it and bringing it here. > > Please feel free to fix test to run (and fail) if the code it tests is > faulty now. If the community knows the moment when tests run will be > enabled, it is absolutely not an issue, that TeamCity Bot will complain > about new failures. I strongly believe that we should see a true picture of > tests output rather than having these tests not running. > > Sincerely, > Dmitriy Pavlov > > ср, 17 апр. 2019 г. в 22:05, Ivan Rakov : > > > Hi Ivan, > > > > I've checked your branch. Seems like these tests fail due to real issue > > in functionality. > > I'll take a look. > > > > Best Regards, > > Ivan Rakov > > > > On 17.04.2019 13:54, Ivan Fedotov wrote: > > > Hi, Igniters! > > > > > > During work on iep-30[1] I discovered that > > > IgniteConfigVariationsAbstractTest subclasses - it is about 15_000 > > tests[2] > > > - do not work. > > > You can check it just run one of the tests with log output, for example > > > ConfigVariationsTestSuiteBuilderTest#LegacyLifecycleTest#test1 [3]. > > > There is no warning notification in the console. The same situation > with > > > other IgniteConfigVariationsAbstractTest subclasses - tests run, but > they > > > simply represent empty code. > > > > > > So, I created a ticket on such issue [4] and it turned out that the > > problem > > > is with ruleChain in IgniteConfigVariationsAbstractTest [5]. > > > The rule that is responsible for running a test statement does not > start > > > indeed [6] under ruleChain runRule. I suggested a solution - move > > testsCfg > > > initialization to > > > IgniteConfigVariationsAbstractTest#beforeTestsStarted method. After > such > > > changes ruleChain becomes not necessary. > > > > > > But I faced another problem - multiple failures on TeamCity [7]. From > > logs, > > > it seems that failures are related to what tests check, but not JUnit > > error. > > > I can not track TeamCity history on that fact were tests failed or not > on > > > the previous JUnit version - the oldest log is dated the start of the > > March > > > when JUnit4 > > > already was implemented (for example, this [8] test). > > > Moreover, there are not so much failed tests, but because of running > with > > > multiple configurations > > (InterceptorCacheConfigVariationsFullApiTestSuite_0 > > > ..._95) it turns out about 400 failed tests. TeamCity results also > > confirm > > > that tests do not work in the master branch - duration time is less > than > > > 1ms. Now all tests are green and that is not surprising - under @Test > > > annotation, nothing happens. > > > > > > Could some of us confirm or disprove my guess that tests are red > because > > of > > > its functionality, but not JUnit implementation? > > > And if it is true, how should I take such fact into account in this > > ticket? > > > > > > [1] > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-30%3A+Migration+to+JUnit+5 > > > [2] > > > > > > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testsuites/InterceptorCacheConfigVariationsFullApiTestSuite.java > > > [3] > > > > > > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/test/ConfigVariationsTestSuiteBuilderTest.java#L434 > > > [4] https://issues.apache.org/jira/browse/IGNITE-11708 > > > [5] > > > > > > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteConfigVariationsAbstractTest.java#L62 > > > [6] > > > > > > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/GridAbstractTest.java#L181 > > > [7] > > > > > > https://mtcga.gridgain.com/pr.html?serverId=apache&suiteId=IgniteTests24Java8_RunAll&branchForTc=pull/6434/head&action=Latest > > > [8] > > > > > > https://ci.ignite.apache.org/project.html?tab=testDetails&projectId=IgniteTests24Java8&testNameId=-9037806478172035481&page=8 > > > > > > -- Ivan Fedotov. ivanan...@gmail.com
Re: IgniteConfigVariationsAbstractTest subclasses do not run
Hi Ivan F. Thank you for finding it and bringing it here. Please feel free to fix test to run (and fail) if the code it tests is faulty now. If the community knows the moment when tests run will be enabled, it is absolutely not an issue, that TeamCity Bot will complain about new failures. I strongly believe that we should see a true picture of tests output rather than having these tests not running. Sincerely, Dmitriy Pavlov ср, 17 апр. 2019 г. в 22:05, Ivan Rakov : > Hi Ivan, > > I've checked your branch. Seems like these tests fail due to real issue > in functionality. > I'll take a look. > > Best Regards, > Ivan Rakov > > On 17.04.2019 13:54, Ivan Fedotov wrote: > > Hi, Igniters! > > > > During work on iep-30[1] I discovered that > > IgniteConfigVariationsAbstractTest subclasses - it is about 15_000 > tests[2] > > - do not work. > > You can check it just run one of the tests with log output, for example > > ConfigVariationsTestSuiteBuilderTest#LegacyLifecycleTest#test1 [3]. > > There is no warning notification in the console. The same situation with > > other IgniteConfigVariationsAbstractTest subclasses - tests run, but they > > simply represent empty code. > > > > So, I created a ticket on such issue [4] and it turned out that the > problem > > is with ruleChain in IgniteConfigVariationsAbstractTest [5]. > > The rule that is responsible for running a test statement does not start > > indeed [6] under ruleChain runRule. I suggested a solution - move > testsCfg > > initialization to > > IgniteConfigVariationsAbstractTest#beforeTestsStarted method. After such > > changes ruleChain becomes not necessary. > > > > But I faced another problem - multiple failures on TeamCity [7]. From > logs, > > it seems that failures are related to what tests check, but not JUnit > error. > > I can not track TeamCity history on that fact were tests failed or not on > > the previous JUnit version - the oldest log is dated the start of the > March > > when JUnit4 > > already was implemented (for example, this [8] test). > > Moreover, there are not so much failed tests, but because of running with > > multiple configurations > (InterceptorCacheConfigVariationsFullApiTestSuite_0 > > ..._95) it turns out about 400 failed tests. TeamCity results also > confirm > > that tests do not work in the master branch - duration time is less than > > 1ms. Now all tests are green and that is not surprising - under @Test > > annotation, nothing happens. > > > > Could some of us confirm or disprove my guess that tests are red because > of > > its functionality, but not JUnit implementation? > > And if it is true, how should I take such fact into account in this > ticket? > > > > [1] > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-30%3A+Migration+to+JUnit+5 > > [2] > > > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testsuites/InterceptorCacheConfigVariationsFullApiTestSuite.java > > [3] > > > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/test/ConfigVariationsTestSuiteBuilderTest.java#L434 > > [4] https://issues.apache.org/jira/browse/IGNITE-11708 > > [5] > > > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteConfigVariationsAbstractTest.java#L62 > > [6] > > > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/GridAbstractTest.java#L181 > > [7] > > > https://mtcga.gridgain.com/pr.html?serverId=apache&suiteId=IgniteTests24Java8_RunAll&branchForTc=pull/6434/head&action=Latest > > [8] > > > https://ci.ignite.apache.org/project.html?tab=testDetails&projectId=IgniteTests24Java8&testNameId=-9037806478172035481&page=8 > > >
Re: IgniteConfigVariationsAbstractTest subclasses do not run
Hi Ivan, I've checked your branch. Seems like these tests fail due to real issue in functionality. I'll take a look. Best Regards, Ivan Rakov On 17.04.2019 13:54, Ivan Fedotov wrote: Hi, Igniters! During work on iep-30[1] I discovered that IgniteConfigVariationsAbstractTest subclasses - it is about 15_000 tests[2] - do not work. You can check it just run one of the tests with log output, for example ConfigVariationsTestSuiteBuilderTest#LegacyLifecycleTest#test1 [3]. There is no warning notification in the console. The same situation with other IgniteConfigVariationsAbstractTest subclasses - tests run, but they simply represent empty code. So, I created a ticket on such issue [4] and it turned out that the problem is with ruleChain in IgniteConfigVariationsAbstractTest [5]. The rule that is responsible for running a test statement does not start indeed [6] under ruleChain runRule. I suggested a solution - move testsCfg initialization to IgniteConfigVariationsAbstractTest#beforeTestsStarted method. After such changes ruleChain becomes not necessary. But I faced another problem - multiple failures on TeamCity [7]. From logs, it seems that failures are related to what tests check, but not JUnit error. I can not track TeamCity history on that fact were tests failed or not on the previous JUnit version - the oldest log is dated the start of the March when JUnit4 already was implemented (for example, this [8] test). Moreover, there are not so much failed tests, but because of running with multiple configurations (InterceptorCacheConfigVariationsFullApiTestSuite_0 ..._95) it turns out about 400 failed tests. TeamCity results also confirm that tests do not work in the master branch - duration time is less than 1ms. Now all tests are green and that is not surprising - under @Test annotation, nothing happens. Could some of us confirm or disprove my guess that tests are red because of its functionality, but not JUnit implementation? And if it is true, how should I take such fact into account in this ticket? [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-30%3A+Migration+to+JUnit+5 [2] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testsuites/InterceptorCacheConfigVariationsFullApiTestSuite.java [3] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/test/ConfigVariationsTestSuiteBuilderTest.java#L434 [4] https://issues.apache.org/jira/browse/IGNITE-11708 [5] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteConfigVariationsAbstractTest.java#L62 [6] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/GridAbstractTest.java#L181 [7] https://mtcga.gridgain.com/pr.html?serverId=apache&suiteId=IgniteTests24Java8_RunAll&branchForTc=pull/6434/head&action=Latest [8] https://ci.ignite.apache.org/project.html?tab=testDetails&projectId=IgniteTests24Java8&testNameId=-9037806478172035481&page=8
IgniteConfigVariationsAbstractTest subclasses do not run
Hi, Igniters! During work on iep-30[1] I discovered that IgniteConfigVariationsAbstractTest subclasses - it is about 15_000 tests[2] - do not work. You can check it just run one of the tests with log output, for example ConfigVariationsTestSuiteBuilderTest#LegacyLifecycleTest#test1 [3]. There is no warning notification in the console. The same situation with other IgniteConfigVariationsAbstractTest subclasses - tests run, but they simply represent empty code. So, I created a ticket on such issue [4] and it turned out that the problem is with ruleChain in IgniteConfigVariationsAbstractTest [5]. The rule that is responsible for running a test statement does not start indeed [6] under ruleChain runRule. I suggested a solution - move testsCfg initialization to IgniteConfigVariationsAbstractTest#beforeTestsStarted method. After such changes ruleChain becomes not necessary. But I faced another problem - multiple failures on TeamCity [7]. From logs, it seems that failures are related to what tests check, but not JUnit error. I can not track TeamCity history on that fact were tests failed or not on the previous JUnit version - the oldest log is dated the start of the March when JUnit4 already was implemented (for example, this [8] test). Moreover, there are not so much failed tests, but because of running with multiple configurations (InterceptorCacheConfigVariationsFullApiTestSuite_0 ..._95) it turns out about 400 failed tests. TeamCity results also confirm that tests do not work in the master branch - duration time is less than 1ms. Now all tests are green and that is not surprising - under @Test annotation, nothing happens. Could some of us confirm or disprove my guess that tests are red because of its functionality, but not JUnit implementation? And if it is true, how should I take such fact into account in this ticket? [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-30%3A+Migration+to+JUnit+5 [2] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testsuites/InterceptorCacheConfigVariationsFullApiTestSuite.java [3] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/test/ConfigVariationsTestSuiteBuilderTest.java#L434 [4] https://issues.apache.org/jira/browse/IGNITE-11708 [5] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteConfigVariationsAbstractTest.java#L62 [6] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/GridAbstractTest.java#L181 [7] https://mtcga.gridgain.com/pr.html?serverId=apache&suiteId=IgniteTests24Java8_RunAll&branchForTc=pull/6434/head&action=Latest [8] https://ci.ignite.apache.org/project.html?tab=testDetails&projectId=IgniteTests24Java8&testNameId=-9037806478172035481&page=8 -- Ivan Fedotov. ivanan...@gmail.com
