[jira] [Created] (IGNITE-11345) remove deprecation for beforeTestsStarted and afterTestsStopped
Oleg Ignatenko created IGNITE-11345: --- Summary: remove deprecation for beforeTestsStarted and afterTestsStopped Key: IGNITE-11345 URL: https://issues.apache.org/jira/browse/IGNITE-11345 Project: Ignite Issue Type: Task Affects Versions: 2.8 Reporter: Oleg Ignatenko This task is created per discussion at dev list: [beforeTestsStarted() was deprecated, what should we use instead?|http://apache-ignite-developers.2346864.n4.nabble.com/beforeTestsStarted-was-deprecated-what-should-we-use-instead-td40625.html] We are going to remove deprecation from {{beforeTestsStarted}} and {{afterTestsStopped}}. The reason for this is that the way recommended instead of these methods makes it less convenient to start and stop the grid. Note that regarding the mentioned inconvenient way, we also plan to improve it (IGNITE-11240). After it is done, we need to consider recovering deprecation for these methods because as follows from above, inconvenience of using the recommended way is the only reason to remove it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11240) provide more convenient way to start and stop grid from static context of @BeforeClass and @AfterClass
Oleg Ignatenko created IGNITE-11240: --- Summary: provide more convenient way to start and stop grid from static context of @BeforeClass and @AfterClass Key: IGNITE-11240 URL: https://issues.apache.org/jira/browse/IGNITE-11240 Project: Ignite Issue Type: Bug Affects Versions: 2.8 Reporter: Oleg Ignatenko Related discussion at dev list: [beforeTestsStarted() was deprecated, what should we use instead?|http://apache-ignite-developers.2346864.n4.nabble.com/beforeTestsStarted-was-deprecated-what-should-we-use-instead-tt40625.html] {quote}...the method should be *static* to be annotated with @BeforeClass. And I see a problem here since you're not able to use non-static methods like GridAbstractTest#startGrid() at method annotated with @BeforeClass.{quote} The way to achieve above is currently not quite straightforward: {{MyTestClass.class.newInstance().startGrid()}} and {{MyTestClass.class.newInstance().stopGrid()}}. This task is for Ignite testframework to provide more convenient way to do that. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11158) Accommodate some of the recent tests to changes made per IGNITE-10179
Oleg Ignatenko created IGNITE-11158: --- Summary: Accommodate some of the recent tests to changes made per IGNITE-10179 Key: IGNITE-11158 URL: https://issues.apache.org/jira/browse/IGNITE-11158 Project: Ignite Issue Type: Bug Affects Versions: 2.8 Reporter: Oleg Ignatenko Changes made per IGNITE-10179 can impact any test inheriting {{GridAbstractTest}}. Some of recent tests were added / enabled too late to properly accommodate to these changes. This task is to investigate such tests and do corrections if needed. Per failure report of Teamcity bot, list of involved tests is as follows: {quote} * cache.IgniteTopologyValidatorNearPartitionedTxCacheGroupsTest * eviction.GridCacheConcurrentEvictionConsistencySelfTest * cache.IgniteTopologyValidatorNearPartitionedTxCacheTest * eviction.DhtAndNearEvictionTest * cache.CacheMetricsForClusterGroupSelfTest * eviction.GridCacheEvictionLockUnlockSelfTest * replicated.GridCacheReplicatedMetricsSelfTest * eviction.GridCacheEvictionTouchSelfTest * eviction.GridCacheConcurrentEvictionsSelfTest * local.GridCacheLocalMetricsSelfTest * near.GridCachePartitionedMetricsSelfTest * local.GridCacheLocalIteratorsSelfTest * internal.TransactionMetricsMxBeanImplTest * cache.GridEvictionPolicyMBeansTest * cache.GridCacheTxPartitionedLocalStoreSelfTest * cache.IgniteCacheNearLockValueSelfTest * cache.IgnitePutAllLargeBatchSelfTest * jdbc.CacheJdbcPojoStoreBinaryMarshallerStoreKeepBinarySelfTest * cache.IgnitePutAllUpdateNonPreloadedPartitionSelfTest * jdbc.CacheJdbcPojoStoreBinaryMarshallerSelfTest * near.GridCacheGetStoreErrorSelfTest * distributed.IgniteCrossCacheTxStoreSelfTest * cache.GridCacheColocatedTxStoreExceptionSelfTest * cache.IgniteCacheEntryListenerTxTest * cache.GridCacheLocalTxStoreExceptionSelfTest * near.GridCachePartitionedStorePutSelfTest * local.GridCacheLocalTxExceptionSelfTest * cache.GridCacheNearTxStoreExceptionSelfTest * cache.GridCacheReplicatedTxStoreExceptionSelfTest * distributed.CacheTxNearUpdateTopologyChangeTest * dht.GridCacheGlobalLoadTest * cache.GridCacheTtlManagerSelfTest * cache.IgniteCacheEntryListenerEagerTtlDisabledTest * store.CacheTransactionalStoreReadFromBackupTest * jdbc.CacheJdbcPojoStoreBinaryMarshallerStoreKeepBinaryWithSqlEscapeSelfTest * cache.IgniteCacheEntryListenerTxReplicatedTest * store.GridCacheLoadOnlyStoreAdapterSelfTest * cache.IgniteCacheEntryListenerTxLocalTest * cache.CachePutEventListenerErrorSelfTest * cache.GridCacheObjectToStringSelfTest * near.GridCacheNearTxExceptionSelfTest * jdbc.CacheJdbcPojoStoreBinaryMarshallerWithSqlEscapeSelfTest * cache.IgniteCacheTxLocalInvokeTest * jdbc.GridCacheJdbcBlobStoreMultithreadedSelfTest {quote} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11116) replace Assert in particular test classes from junit.framework (JUnit 3) to org.junit (JUnit 4)
Oleg Ignatenko created IGNITE-6: --- Summary: replace Assert in particular test classes from junit.framework (JUnit 3) to org.junit (JUnit 4) Key: IGNITE-6 URL: https://issues.apache.org/jira/browse/IGNITE-6 Project: Ignite Issue Type: Sub-task Affects Versions: 2.7 Reporter: Oleg Ignatenko Some tests start failing on Teamcity when Assert is replaced to one from {{org.junit}} package (note I couldn't reliably reproduce these failures locally). Per my observations it looks like troublesome code involves {{assertEquals}} that compares double / float values with non-zero delta. This issue was discovered in the course of IGNITE-10927. Search in the [PR 5866|https://github.com/apache/ignite/pull/5866] linked to IGNITE-10927 for import of {{junit.framework.Assert}} shows that this involves ZookeeperDiscoverySpiTestHelper, IoStatisticsMetricsLocalMXBeanImplSelfTest, BinaryMarshallerSelfTest, RestMemcacheProtocolSelfTest. Search in this PR for static import of {{junit.framework.Assert}} shows 12 more involved test classes, CassandraConfigTest, ClientCacheFlagsCodecTest etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11038) recover IgniteCacheTestSuite3 that was lost because of changes made per IGNITE-10796
Oleg Ignatenko created IGNITE-11038: --- Summary: recover IgniteCacheTestSuite3 that was lost because of changes made per IGNITE-10796 Key: IGNITE-11038 URL: https://issues.apache.org/jira/browse/IGNITE-11038 Project: Ignite Issue Type: Bug Affects Versions: 2.8 Reporter: Oleg Ignatenko This bug was reported at dev list [here|http://apache-ignite-developers.2346864.n4.nabble.com/IgniteCacheTestSuite3-on-TeamCity-tt40240.html]. Need to recover IgniteCacheTestSuite3 that was lost because of changes made per IGNITE-10796. Specifically, in file {{IgniteBinaryObjectsCacheTestSuite3.java}} line 64 currently reads: {code}List> suite = new ArrayList<>(); {code} Above is wrong, correct code at this line should be: {code}List> suite = new ArrayList<>(IgniteCacheTestSuite3.suite(ignoredTests)); {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10988) Zookeeper test case testCommunicationFailureResolve_KillRandom frequently fails in GridAbstractTest.waitForTopology
Oleg Ignatenko created IGNITE-10988: --- Summary: Zookeeper test case testCommunicationFailureResolve_KillRandom frequently fails in GridAbstractTest.waitForTopology Key: IGNITE-10988 URL: https://issues.apache.org/jira/browse/IGNITE-10988 Project: Ignite Issue Type: Bug Affects Versions: 2.7 Reporter: Oleg Ignatenko Zookeeper test case testCommunicationFailureResolve_KillRandom frequently fails in GridAbstractTest.waitForTopology. Teamcity test history in master currently shows "Test runs: 625 total / 251 failures" ([link|https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-3849782382784091413&tab=testDetails). Need to find out why is that and whether test case can be improved and made more reliable. Typical failure message looks as follows: {noformat} [2019-01-18 06:57:23,489][ERROR][main][root] Test failed. junit.framework.AssertionFailedError at junit.framework.Assert.fail(Assert.java:55) at junit.framework.Assert.assertTrue(Assert.java:22) at junit.framework.Assert.assertTrue(Assert.java:31) at org.apache.ignite.testframework.junits.GridAbstractTest.waitForTopology(GridAbstractTest.java:2294) at org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoverySpiTest.waitForTopology(ZookeeperDiscoverySpiTest.java:5103) at org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoverySpiTest.testCommunicationFailureResolve_KillRandom(ZookeeperDiscoverySpiTest.java:3145) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.apache.ignite.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2088) at java.lang.Thread.run(Thread.java:748){noformat} Side note per IGNITE-10777 this test case has moved to another class, from {{ZookeeperDiscoverySpiTest}} to {{ZookeeperDiscoverySpiTest5}} but that didn't change anything: it still fails frequently and failure message is the same, with the only difference in the new test class name. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10951) migration from Junit 3 to 4 phase 2 (follow-up to IGNITE-10173)
Oleg Ignatenko created IGNITE-10951: --- Summary: migration from Junit 3 to 4 phase 2 (follow-up to IGNITE-10173) Key: IGNITE-10951 URL: https://issues.apache.org/jira/browse/IGNITE-10951 Project: Ignite Issue Type: Task Affects Versions: 2.8 Reporter: Oleg Ignatenko This is an umbrella ticket for those tasks related to migration from JUnit 3 to 4 which assume that all JUnit 3 related code has been already cleaned from the project in prior phase per IGNITE-10173 and IGNITE-10762. This involves such tasks as update guidelines as suggested in comments of IGNITE-10178, study of whether to migrate to JUnit 5, removal of scaffolding annotations etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10927) Relieve JUnit3TestLegacySupport from inheriting deprecated junit.framework.Assert (follow-up to IGNITE-10177)
Oleg Ignatenko created IGNITE-10927: --- Summary: Relieve JUnit3TestLegacySupport from inheriting deprecated junit.framework.Assert (follow-up to IGNITE-10177) Key: IGNITE-10927 URL: https://issues.apache.org/jira/browse/IGNITE-10927 Project: Ignite Issue Type: Sub-task Affects Versions: 2.7 Reporter: Oleg Ignatenko {{JUnit3TestLegacySupport}} currently inherits deprecated {{junit.framework.Assert}}. This was done only in order to minimize risky code changes when tests were migrating from Junit 3, after {{GridAbstractTest}} has dropped inheriting {{junit.framework.TestCase}}. Now that migration is over it is less risky to cleanup project from deprecated assert methods and drop the harmful inheritance. In order to make this smoother and minimize amount of test changes, after inheritance is dropped, {{JUnit3TestLegacySupport}} should be extended with a set of "temporary patch" methods that would delegate most popular assertions used by subclasses to respective methods of {{org.junit.Assert}}. Mentioned temporary patch methods, in turn, should get {{@Deprecated}} annotation and respective {{2deprecation}} noticed in javadocs that would encourage developers to (safely and gradually) change them to direct invocations and static imports of respective Assert methods instead of using those inherited from superclass. These patch methods should be declared protected final or protected static final, to minimize their applicability and prevent them spreading more than intended. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10796) migrate from JUnit 3 to 4 suites involving IgniteTestSuite
Oleg Ignatenko created IGNITE-10796: --- Summary: migrate from JUnit 3 to 4 suites involving IgniteTestSuite Key: IGNITE-10796 URL: https://issues.apache.org/jira/browse/IGNITE-10796 Project: Ignite Issue Type: Sub-task Affects Versions: 2.8 Reporter: Oleg Ignatenko This task is to migrate from JUnit 3 to 4 test suites suites involving {{IgniteTestSuite}} API that was introduced per IGNITE-3658. If needed, refer parent task comments for more details. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10776) migrate from JUnit 3 to 4 config variations test suites
Oleg Ignatenko created IGNITE-10776: --- Summary: migrate from JUnit 3 to 4 config variations test suites Key: IGNITE-10776 URL: https://issues.apache.org/jira/browse/IGNITE-10776 Project: Ignite Issue Type: Sub-task Affects Versions: 2.8 Reporter: Oleg Ignatenko This task is to migrate config variations test suites from JUnit 3 to 4. If needed, refer parent task comments for more details. Note this might be somewhat tough nut to crack design wise since {{JUnit4TestAdapter}} plays a special role in config variations tests, see IGNITE-10739. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10775) migrate from JUnit 3 to 4 test suites of simple dynamic lists that use addTestIfNeeded API
Oleg Ignatenko created IGNITE-10775: --- Summary: migrate from JUnit 3 to 4 test suites of simple dynamic lists that use addTestIfNeeded API Key: IGNITE-10775 URL: https://issues.apache.org/jira/browse/IGNITE-10775 Project: Ignite Issue Type: Sub-task Affects Versions: 2.8 Reporter: Oleg Ignatenko This task is to migrate from JUnit 3 to 4 test suites of simple dynamic lists that use {{GridTestUtils.addTestIfNeeded}} API. If needed, refer parent task comments for more details. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10777) cleanup remainders of JUnit4TestAdapter and other JUnit 3 API involving test suites
Oleg Ignatenko created IGNITE-10777: --- Summary: cleanup remainders of JUnit4TestAdapter and other JUnit 3 API involving test suites Key: IGNITE-10777 URL: https://issues.apache.org/jira/browse/IGNITE-10777 Project: Ignite Issue Type: Sub-task Affects Versions: 2.8 Reporter: Oleg Ignatenko This task is to cleanup remainders of JUnit4TestAdapter and other JUnit 3 API involving test suites that may be missed in prior sub-tasks under the parent task. If needed, refer to parent task comments for more details. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10774) migrate test suites that are fixed lists of test classes from Junit 3 to 4
Oleg Ignatenko created IGNITE-10774: --- Summary: migrate test suites that are fixed lists of test classes from Junit 3 to 4 Key: IGNITE-10774 URL: https://issues.apache.org/jira/browse/IGNITE-10774 Project: Ignite Issue Type: Sub-task Affects Versions: 2.8 Reporter: Oleg Ignatenko This task is to migrate test suites that are fixed lists of test classes from Junit 3 to 4 that can be handled by {{@SuiteClasses}} API. If needed, find more details in the comments of the parent task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10773) migrate examples testsuites from Junit 3 to 4
Oleg Ignatenko created IGNITE-10773: --- Summary: migrate examples testsuites from Junit 3 to 4 Key: IGNITE-10773 URL: https://issues.apache.org/jira/browse/IGNITE-10773 Project: Ignite Issue Type: Sub-task Affects Versions: 2.8 Reporter: Oleg Ignatenko This taks is to migrate examples testsuites from Junit 3 to 4. If needed, find more details in comments of parent task -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10758) remove from tests scaffolding annotations "@RunWith(JUnit4.class)" and their respective imports
Oleg Ignatenko created IGNITE-10758: --- Summary: remove from tests scaffolding annotations "@RunWith(JUnit4.class)" and their respective imports Key: IGNITE-10758 URL: https://issues.apache.org/jira/browse/IGNITE-10758 Project: Ignite Issue Type: Sub-task Affects Versions: 2.8 Reporter: Oleg Ignatenko In the course of IGNITE-10175 and IGNITE-10176 many test classes were annotated {{@RunWith(JUnit4.class)}}. This was necessary to allow gradual switching to JUnit 4 to let classes run under the version that worked for these. After IGNITE-10177 is over, these scaffolding annotations will be not needed anymore (and will become even somewhat damaging by misleading readers to think that they serve some real purpose). The task is to remove these annotations and respective import statements after IGNITE-10177 is merged to master. Note it was initially planned as a part of IGNITE-10177 but per discussion with [~EdShangGG] we decided that it will be more convenient to do this in a separate ticket because this will make changes much easier to review, test and merge. (!) Thing worth paying attention is that mentioned annotations should be kept in one case - namely in root test class {{GridAbstractTest}} because over there these are necessary until JUnit 3 dependencies are wiped out from remaining cases listed in IGNITE-10739. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10739) get rid of using JUnit 3 API in IgniteConfigVariationsAbstractTest
Oleg Ignatenko created IGNITE-10739: --- Summary: get rid of using JUnit 3 API in IgniteConfigVariationsAbstractTest Key: IGNITE-10739 URL: https://issues.apache.org/jira/browse/IGNITE-10739 Project: Ignite Issue Type: Task Affects Versions: 2.8 Reporter: Oleg Ignatenko {{IgniteConfigVariationsAbstractTest}} and part of test framework that uses this class very heavily rely on JUnit 3 specific features. The task is to get rid of these and use newer versions instead - preferably JUnit 4, although JUnit 5 is also an option This will allow consistent JUnit framework version across Ignite tests which in turn is expected to simplify maintenance and development of the tests (including ignoring tests, workiing on Teamcity related things etc). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10701) add missing import to CacheQueryEntityWithJsr310Java8DateTimeApiFieldsTest
Oleg Ignatenko created IGNITE-10701: --- Summary: add missing import to CacheQueryEntityWithJsr310Java8DateTimeApiFieldsTest Key: IGNITE-10701 URL: https://issues.apache.org/jira/browse/IGNITE-10701 Project: Ignite Issue Type: Task Affects Versions: 2.8 Reporter: Oleg Ignatenko As of now, [CacheQueryEntityWithJsr310Java8DateTimeApiFieldsTest|https://github.com/apache/ignite/blob/master/modules/indexing/src/test/java8/org/apache/ignite/internal/processors/query/h2/CacheQueryEntityWithJsr310Java8DateTimeApiFieldsTest.java] lacks imports for {{@RunWith(JUnit4.class)}}. Problem was discovered by manual check because this file is in "java8" directory and appears to be ignored by Teamcity build check (also I could not find Maven profile to activate its usage) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10670) investigate why Cassandra modules don't have testsuite(s)
Oleg Ignatenko created IGNITE-10670: --- Summary: investigate why Cassandra modules don't have testsuite(s) Key: IGNITE-10670 URL: https://issues.apache.org/jira/browse/IGNITE-10670 Project: Ignite Issue Type: Task Affects Versions: 2.8 Reporter: Oleg Ignatenko Unlike most if not all other Ignite modules, [cassandra|https://github.com/apache/ignite/tree/master/modules/cassandra] currently doesn't have testsuites. Find out why is that and if it is a matter of simple omission, add testsuites and integrate them in Teamcity. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10633) CacheEntryProcessorExternalizableFailedTest fails on Teamcity when executed under JUnit 4
Oleg Ignatenko created IGNITE-10633: --- Summary: CacheEntryProcessorExternalizableFailedTest fails on Teamcity when executed under JUnit 4 Key: IGNITE-10633 URL: https://issues.apache.org/jira/browse/IGNITE-10633 Project: Ignite Issue Type: Bug Affects Versions: 2.8 Reporter: Oleg Ignatenko {{CacheEntryProcessorExternalizableFailedTest}} fails on Teamcity when executed under JUnit 4 (I couldn't reproduce that locally. Link to example failures at TC: [here|https://ci.ignite.apache.org/viewLog.html?buildId=2516404&tab=buildResultsDiv&buildTypeId=IgniteTests24Java8_Cache1] Example snippet from error log: {noformat} org.apache.ignite.IgniteIllegalStateException: Ignite instance with provided name doesn't exist. Did you call Ignition.start(..) to start an Ignite instance? [name=continuous.CacheEntryProcessorExternalizableFailedTest3] at org.apache.ignite.internal.processors.cache.query.continuous.CacheEntryProcessorExternalizableFailedTest.doTestInvokeTest(CacheEntryProcessorExternalizableFailedTest.java:625) at org.apache.ignite.internal.processors.cache.query.continuous.CacheEntryProcessorExternalizableFailedTest.testOptimistic(CacheEntryProcessorExternalizableFailedTest.java:155) --- Stdout: --- [2018-12-11 00:46:36,826][INFO ][test-runner-#4280%continuous.CacheEntryProcessorExternalizableFailedTest%][root] >>> Starting test: CacheEntryProcessorExternalizableFailedTest#testOptimistic <<< [2018-12-11 00:46:36,826][INFO ][test-runner-#4280%continuous.CacheEntryProcessorExternalizableFailedTest%][root] >>> Stopping test: CacheEntryProcessorExternalizableFailedTest#testOptimistic in 0 ms <<< --- Stderr: --- [2018-12-11 00:46:36,827][ERROR][main][root] Test failed. class org.apache.ignite.IgniteIllegalStateException: Ignite instance with provided name doesn't exist. Did you call Ignition.start(..) to start an Ignite instance? [name=continuous.CacheEntryProcessorExternalizableFailedTest3] at org.apache.ignite.internal.IgnitionEx.grid(IgnitionEx.java:1392) at org.apache.ignite.Ignition.ignite(Ignition.java:529) at org.apache.ignite.testframework.junits.GridAbstractTest.grid(GridAbstractTest.java:1285) at org.apache.ignite.testframework.junits.GridAbstractTest.grid(GridAbstractTest.java:1301) at org.apache.ignite.internal.processors.cache.query.continuous.CacheEntryProcessorExternalizableFailedTest.doTestInvokeTest(CacheEntryProcessorExternalizableFailedTest.java:625) at org.apache.ignite.internal.processors.cache.query.continuous.CacheEntryProcessorExternalizableFailedTest.testOptimistic(CacheEntryProcessorExternalizableFailedTest.java:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at org.apache.ignite.testframework.junits.GridAbstractTest$7.run(GridAbstractTest.java:2119) at java.lang.Thread.run(Thread.java:748) {noformat} - Possible reason for failure is that some code is missing in {{beforeTest}} because it is placed in {{beforeTestsStarted}} instead. I think so because per prior experience of migrating tests from Junit 3 to 4 we discovered that handling of these methods noticeably differs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10629) Migration follow up: check for old style tests that could be slipped through in transition period
Oleg Ignatenko created IGNITE-10629: --- Summary: Migration follow up: check for old style tests that could be slipped through in transition period Key: IGNITE-10629 URL: https://issues.apache.org/jira/browse/IGNITE-10629 Project: Ignite Issue Type: Sub-task Affects Versions: 2.8 Reporter: Oleg Ignatenko We need to account for risk that while tests are migrating some commits may by mistake slip in old style test cases - that will be ignored by JUnit 4. In order to address possible issues of that kind, do the following a week or two after IGNITE-10177 is merged to master: run the IntelliJ inspection called "old style Junit test method in JUnit 4 class", review report and fix discovered problems if there are any. For the reference, my version of IDE explains this inspection as follows: {quote}Reports JUnit 3 style test methods which are located inside a class which does not extend the abstract JUnit 3 class TestCase and contains JUnit 4/JUnit 5 @Test annotated methods.{quote} (note concerns mentioned in this ticket were originally raised at dev list: [here|http://apache-ignite-developers.2346864.n4.nabble.com/Is-it-time-to-move-forward-to-JUnit4-5-tp29608p39300.html]) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10614) investigate blocking of WalCompactionTest.testCompressorToleratesEmptyWalSegmentsFsync at stopping grids when migrating from JUnit 3 to 4
Oleg Ignatenko created IGNITE-10614: --- Summary: investigate blocking of WalCompactionTest.testCompressorToleratesEmptyWalSegmentsFsync at stopping grids when migrating from JUnit 3 to 4 Key: IGNITE-10614 URL: https://issues.apache.org/jira/browse/IGNITE-10614 Project: Ignite Issue Type: Sub-task Affects Versions: 2.8 Reporter: Oleg Ignatenko {{WalCompactionTest.testCompressorToleratesEmptyWalSegmentsFsync}} runs OK under JUnit 3 but blocks at stopping grids when migrating to JUnit 4. Workaround for that is to run stopAllGrids in separate thread: {code} @Override protected void afterTest() throws Exception { Thread thread = new Thread(this::stopAllGrids); thread.start(); thread.join(getTestTimeout()); cleanPersistenceDir(); } {code} If needed, [here is a diff at Github|https://github.com/apache/ignite/pull/5615/commits/1b342ee4c8c772b0e57583ee7f8721fc2c070bcf] with more context. Need to find out what is going on there and if something needs to be changed in this test or maybe in parent test classes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10553) Investigate differences in test lists prior and after migration of core module tests to Junit 4 (follow-up to IGNITE-10175)
Oleg Ignatenko created IGNITE-10553: --- Summary: Investigate differences in test lists prior and after migration of core module tests to Junit 4 (follow-up to IGNITE-10175) Key: IGNITE-10553 URL: https://issues.apache.org/jira/browse/IGNITE-10553 Project: Ignite Issue Type: Sub-task Affects Versions: 2.8 Reporter: Oleg Ignatenko Investigate differences in test lists prior and after migration of core tests to Junit 4 and if some lost tests are discovered recover these. Particular report generated based on comparing test lists obtained from teamcity builds [#2467986|https://ci.ignite.apache.org/viewLog.html?buildId=2467986&tab=buildResultsDiv&buildTypeId=IgniteTests24Java8_RunAll] (prior to migration) and [#2462054|https://ci.ignite.apache.org/viewLog.html?buildId=2462054&buildTypeId=IgniteTests24Java8_RunAll] (after migration) suggests that there are 145 test cases worth giving a closer look, {{HibernateL2CacheConfigurationSelfTest.testDefaultCache}} etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10450) In Ignite code style inspections increase level for those used at Teamcity build checks
Oleg Ignatenko created IGNITE-10450: --- Summary: In Ignite code style inspections increase level for those used at Teamcity build checks Key: IGNITE-10450 URL: https://issues.apache.org/jira/browse/IGNITE-10450 Project: Ignite Issue Type: Improvement Affects Versions: 2.6 Reporter: Oleg Ignatenko Attachments: IDEA.inspections.TC-bot.png Some of [Ignite code style|https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines] inspections are verified at Teamcity per IGNITE-9983. (Currently TC inspections are "SizeReplaceableByIsEmpty", "UnusedImport", "MissingOverrideAnnotation", "MissortedModifiers", "RedundantSuppression".) Per discussion of issue IGNITE-10399 it looks like there is a room for improvement here. Specifically, the problem occurred because it was too difficult to find a new deviation that made TC inspections check fail because it was buried among multiple similar looking but non-critical deviations in a particular piece of old code ([PageMemoryImpl.java|https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/pagemem/PageMemoryImpl.java]). It would be more convenient if programmer could easier see subset of checks that are used at Teamcity because this would allow to fix these earlier and avoid cumbersome TC runs and failure reports analysis. Technically this could be achieved by editing inspections config file and increasing respective inspections level to {{ERROR}}. I briefly checked how it would work in a "sandbox" project on my machine and it looked quite convenient: violations of inspections used by TC were shown as red in Error Stripe while the rest remained yellow, easy to see. (It's probably not very important but for the sake of completeness a thing I noticed when testing is that having red inspections didn't block compilation and execution of the code.) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10335) move ML examples datasets files to resources
Oleg Ignatenko created IGNITE-10335: --- Summary: move ML examples datasets files to resources Key: IGNITE-10335 URL: https://issues.apache.org/jira/browse/IGNITE-10335 Project: Ignite Issue Type: Improvement Components: ml Affects Versions: 2.7 Reporter: Oleg Ignatenko ML examples datasets files were moved from resources to some folder under src/main/java in the course of IGNITE-10209. This was done as a quick workaround to unblock another change. However, resources is a more appropriate location for this kind of content and in the course of IGNITE-10234 we discovered that there is a way to make things work with dataset files in this folder. Solution appears to be to rework {{SandboxMLCache}} to use {{IgniteUtils.resolveIgnitePath}} API like it is done in [SqlJdbcCopyExample|https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/sql/SqlJdbcCopyExample.java]. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10249) TcpDiscoveryMultiThreadedTest#testCustomEventNodeRestart: Getting affinity for topology version earlier than affinity is calculated
Oleg Ignatenko created IGNITE-10249: --- Summary: TcpDiscoveryMultiThreadedTest#testCustomEventNodeRestart: Getting affinity for topology version earlier than affinity is calculated Key: IGNITE-10249 URL: https://issues.apache.org/jira/browse/IGNITE-10249 Project: Ignite Issue Type: Bug Affects Versions: 2.6 Reporter: Oleg Ignatenko TcpDiscoveryMultiThreadedTest#testCustomEventNodeRestart (in current codebase muted by renaming to {{_testCustomEventNodeRestart}}) fails on Teamcity with unexpected exception (could not reproduce locally): {noformat} org.apache.ignite.IgniteException: Getting affinity for topology version earlier than affinity is calculated [locNode=TcpDiscoveryNode [id=ae293efb-8b41-4b86-b32b-74707081, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47502], discPort=47502, order=14, intOrder=10, lastExchangeTime=1542188985763, loc=true, ver=2.7.0#20181113-sha1:b186327e, isClient=false], grp=default, topVer=AffinityTopologyVersion [topVer=17, minorTopVer=0], head=AffinityTopologyVersion [topVer=18, minorTopVer=0], history=[AffinityTopologyVersion [topVer=14, minorTopVer=0], AffinityTopologyVersion [topVer=15, minorTopVer=0], AffinityTopologyVersion [topVer=18, minorTopVer=0]]] Caused by: javax.cache.CacheException: Getting affinity for topology version earlier than affinity is calculated [locNode=TcpDiscoveryNode [id=ae293efb-8b41-4b86-b32b-74707081, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10209) some tests in IgniteExamplesMLTestSuite fail with FileNotFoundException at SandboxMLCache#fillCacheWith(MLSandboxDatasets)
Oleg Ignatenko created IGNITE-10209: --- Summary: some tests in IgniteExamplesMLTestSuite fail with FileNotFoundException at SandboxMLCache#fillCacheWith(MLSandboxDatasets) Key: IGNITE-10209 URL: https://issues.apache.org/jira/browse/IGNITE-10209 Project: Ignite Issue Type: Bug Components: ml Affects Versions: 2.8 Reporter: Oleg Ignatenko Tests for examples that use {{SandboxMLCache#fillCacheWith(MLSandboxDatasets)}} fail with FNFE: {noformat} [ERROR] Tests run: 34, Failures: 0, Errors: 13, Skipped: 0, Time elapsed: 638.752 s <<< FAILURE! - in org.apache.ignite.testsuites.IgniteExamplesMLTestSuite [ERROR] testExample(org.apache.ignite.examples.ml.KMeansClusterizationExampleSelfName) Time elapsed: 21.701 s <<< ERROR! java.io.FileNotFoundException: examples\src\main\resources\datasets\two_classed_iris.csv (Системе не удается найти указанный путь) at org.apache.ignite.examples.ml.KMeansClusterizationExampleSelfName.testExample(KMeansClusterizationExampleSelfName.java) [ERROR] testExample(org.apache.ignite.examples.ml.KNNClassificationExampleSelfName) Time elapsed: 13.81 s <<< ERROR! java.io.FileNotFoundException: examples\src\main\resources\datasets\iris.txt (Системе не удается найти указанный путь) at org.apache.ignite.examples.ml.KNNClassificationExampleSelfName.testExample(KNNClassificationExampleSelfName.java) [ERROR] testExample(org.apache.ignite.examples.ml.KNNRegressionExampleSelfName) Time elapsed: 13.633 s <<< ERROR! java.io.FileNotFoundException: examples\src\main\resources\datasets\cleared_machines.csv (Системе не удается найти указанный путь) at org.apache.ignite.examples.ml.KNNRegressionExampleSelfName.testExample(KNNRegressionExampleSelfName.java) [ERROR] testExample(org.apache.ignite.examples.ml.LinearRegressionLSQRTrainerExampleSelfName) Time elapsed: 13.71 s <<< ERROR! java.io.FileNotFoundException: examples\src\main\resources\datasets\mortalitydata.csv (Системе не удается найти указанный путь) at org.apache.ignite.examples.ml.LinearRegressionLSQRTrainerExampleSelfName.testExample(LinearRegressionLSQRTrainerExampleSelfName.java) [ERROR] testExample(org.apache.ignite.examples.ml.LinearRegressionLSQRTrainerWithMinMaxScalerExampleSelfName) Time elapsed: 13.728 s <<< ERROR! java.io.FileNotFoundException: examples\src\main\resources\datasets\mortalitydata.csv (Системе не удается найти указанный путь) at org.apache.ignite.examples.ml.LinearRegressionLSQRTrainerWithMinMaxScalerExampleSelfName.testExample(LinearRegressionLSQRTrainerWithMinMaxScalerExampleSelfName.java) [ERROR] testExample(org.apache.ignite.examples.ml.LinearRegressionSGDTrainerExampleSelfName) Time elapsed: 13.874 s <<< ERROR! java.io.FileNotFoundException: examples\src\main\resources\datasets\mortalitydata.csv (Системе не удается найти указанный путь) at org.apache.ignite.examples.ml.LinearRegressionSGDTrainerExampleSelfName.testExample(LinearRegressionSGDTrainerExampleSelfName.java) [ERROR] testExample(org.apache.ignite.examples.ml.LogisticRegressionSGDTrainerExampleSelfName) Time elapsed: 13.694 s <<< ERROR! java.io.FileNotFoundException: examples\src\main\resources\datasets\two_classed_iris.csv (Системе не удается найти указанный путь) at org.apache.ignite.examples.ml.LogisticRegressionSGDTrainerExampleSelfName.testExample(LogisticRegressionSGDTrainerExampleSelfName.java) [ERROR] testExample(org.apache.ignite.examples.ml.LogRegressionMultiClassClassificationExampleSelfName) Time elapsed: 13.823 s <<< ERROR! java.io.FileNotFoundException: examples\src\main\resources\datasets\glass_identification.csv (Системе не удается найти указанный путь) at org.apache.ignite.examples.ml.LogRegressionMultiClassClassificationExampleSelfName.testExample(LogRegressionMultiClassClassificationExampleSelfName.java) [ERROR] testExample(org.apache.ignite.examples.ml.TrainTestDatasetSplitterExampleSelfName) Time elapsed: 12.875 s <<< ERROR! java.io.FileNotFoundException: examples\src\main\resources\datasets\mortalitydata.csv (Системе не удается найти указанный путь) at org.apache.ignite.examples.ml.TrainTestDatasetSplitterExampleSelfName.testExample(TrainTestDatasetSplitterExampleSelfName.java) [ERROR] testExample(org.apache.ignite.examples.ml.SVMBinaryClassificationExampleSelfName) Time elapsed: 12.795 s <<< ERROR! java.io.FileNotFoundException: examples\src\main\resources\datasets\two_classed_iris.csv (Системе не удается найти указанный путь) at org.apache.ignite.examples.ml.SVMBinaryClassificationExampleSelfName.testExample(SVMBinaryClassificationExampleSelfName.java) [ERROR] testExample(org.apache.ignite.examples.ml.SVMMultiClassClassificationExampleSelfName) Time elapsed: 12.948 s <<< ERROR! java.io.FileNotFoundException: examples\src\main\resources\dat
[jira] [Created] (IGNITE-10208) Verify list of tests after migration to Junit 4 against some prior reference (follow-up to IGNITE-10177)
Oleg Ignatenko created IGNITE-10208: --- Summary: Verify list of tests after migration to Junit 4 against some prior reference (follow-up to IGNITE-10177) Key: IGNITE-10208 URL: https://issues.apache.org/jira/browse/IGNITE-10208 Project: Ignite Issue Type: Sub-task Reporter: Oleg Ignatenko Migration from Junit 3 to 4 involves manually adding {{@Test}} annotation to existing test cases. Since Ignite contains many thousands test cases there is a substantial risk that we may miss some of these in such a transition. In order to mitigate the risk, suggest to do a check after completion of IGNITE-10177 - somehow generate list of tests in the project and compare it against similar list of tests of some "known good" prior project version (at this point, 2.7 release branch looks like a good candidate for such a reference). If comparison shows that some test cases from older version are missed in newer one, open a ticket to investigate that. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10193) IgniteBaselineAffinityTopologyActivationTest#testBaselineTopologyHistoryIsDeletedOnBaselineDelete fails asserting bltHist.history().size()
Oleg Ignatenko created IGNITE-10193: --- Summary: IgniteBaselineAffinityTopologyActivationTest#testBaselineTopologyHistoryIsDeletedOnBaselineDelete fails asserting bltHist.history().size() Key: IGNITE-10193 URL: https://issues.apache.org/jira/browse/IGNITE-10193 Project: Ignite Issue Type: Bug Affects Versions: 2.6 Reporter: Oleg Ignatenko IgniteBaselineAffinityTopologyActivationTest#testBaselineTopologyHistoryIsDeletedOnBaselineDelete (in current codebase muted by renaming to {{_testBaselineTopologyHistoryIsDeletedOnBaselineDelete}}) fails. Test javadoc says: "Restore this test when requirements for BaselineTopology deletion are clarified and this feature is covered with more tests." (javadoc appears to be giving proper reason) Failure message: {noformat} junit.framework.AssertionFailedError: Expected :0 Actual :2 at junit.framework.Assert.fail(Assert.java:57) at junit.framework.Assert.failNotEquals(Assert.java:329) at junit.framework.Assert.assertEquals(Assert.java:78) at junit.framework.Assert.assertEquals(Assert.java:234) at junit.framework.Assert.assertEquals(Assert.java:241) at junit.framework.TestCase.assertEquals(TestCase.java:409) at org.apache.ignite.internal.processors.cache.persistence.IgniteBaselineAffinityTopologyActivationTest$17.verify(IgniteBaselineAffinityTopologyActivationTest.java:1041) at org.apache.ignite.internal.processors.cache.persistence.IgniteBaselineAffinityTopologyActivationTest.verifyBaselineTopologyHistoryOnNodes(IgniteBaselineAffinityTopologyActivationTest.java:693) at org.apache.ignite.internal.processors.cache.persistence.IgniteBaselineAffinityTopologyActivationTest.testBaselineTopologyHistoryIsDeletedOnBaselineDelete(IgniteBaselineAffinityTopologyActivationTest.java:1082) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at junit.framework.TestCase.runTest(TestCase.java:176) at org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2208) at org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:144) at org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:2124) at java.lang.Thread.run(Thread.java:745) {noformat} Snippet of code referred to from above message: {code}assertEquals(0, bltHist.history().size());{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10192) OptimizedMarshallerTest#testAllocationOverflow throws OOME instead of expected IOE
Oleg Ignatenko created IGNITE-10192: --- Summary: OptimizedMarshallerTest#testAllocationOverflow throws OOME instead of expected IOE Key: IGNITE-10192 URL: https://issues.apache.org/jira/browse/IGNITE-10192 Project: Ignite Issue Type: Bug Reporter: Oleg Ignatenko OptimizedMarshallerTest#_testAllocationOverflow (in current codebase muted by renaming to {{_testAllocationOverflow}}) throws OOME instead of expected IOE when trying to marshal {{HugeObject}} - which in turn writes 4 times {{new byte\[1 << 31 - 2]}}. Note test javadocs say: "WARNING! Requires a lot of heap space. Should not be run on CI." -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10191) Incorrect comparison of lists in RendezvousAffinityFunctionSimpleBenchmark#testAffinityCompatibility
Oleg Ignatenko created IGNITE-10191: --- Summary: Incorrect comparison of lists in RendezvousAffinityFunctionSimpleBenchmark#testAffinityCompatibility Key: IGNITE-10191 URL: https://issues.apache.org/jira/browse/IGNITE-10191 Project: Ignite Issue Type: Bug Affects Versions: 2.6 Reporter: Oleg Ignatenko RendezvousAffinityFunctionSimpleBenchmark#testAffinityCompatibility (in current codebase muted by renaming to {{_testAffinityCompatibility}}) looks troublesome: apparent bug is incorrect comparison of lists expecting elements to be always in the same order which doesn't look like the case for the tested API: {code}List> assignment0 = assignPartitions(aff0, nodes, null, backups, 0).get2(); List> assignment1 = assignPartitions(aff1, nodes, null, backups, 0).get2(); assertEquals (assignment0, assignment1); {code} Though test kept failing even after I experimented with replacing comparison to one that was insensitive to the order of list elements. Brief checking of code intended to be tested suggests that maybe it isn't even supposed to be deterministic - in case if this is correct test should be very thoroughly redesigned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10198) test cases fail in TcpDiscoveryMultiThreadedTest: testCustomEventOnJoinCoordinatorStop and testClientContinuousQueryCoordinatorStop
Oleg Ignatenko created IGNITE-10198: --- Summary: test cases fail in TcpDiscoveryMultiThreadedTest: testCustomEventOnJoinCoordinatorStop and testClientContinuousQueryCoordinatorStop Key: IGNITE-10198 URL: https://issues.apache.org/jira/browse/IGNITE-10198 Project: Ignite Issue Type: Bug Affects Versions: 2.6 Reporter: Oleg Ignatenko * TcpDiscoveryMultiThreadedTest#testCustomEventOnJoinCoordinatorStop (in current codebase muted by renaming to {{_testCustomEventOnJoinCoordinatorStop}}) fails to start client cache: {noformat} class org.apache.ignite.IgniteCheckedException: Failed to start client cache (a cache with the given name is not started): default at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7425) at org.apache.ignite.internal.util.future.GridFutureAdapter.resolve(GridFutureAdapter.java:261) at org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:172) at org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141) at org.apache.ignite.testframework.GridTestUtils.lambda$runMultiThreadedAsync$96d302c5$1(GridTestUtils.java:856) at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:395) at org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:349) at org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:337) at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:507) at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:486) at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:474) at org.apache.ignite.testframework.GridTestUtils.lambda$runAsync$2(GridTestUtils.java:1011) at org.apache.ignite.testframework.GridTestUtils$7.call(GridTestUtils.java:1299) at org.apache.ignite.testframework.GridTestThread.run(GridTestThread.java:84) Caused by: org.apache.ignite.cache.CacheExistsException: Failed to start client cache (a cache with the given name is not started): default at org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheChangeRequest(GridCacheProcessor.java:5287) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.lambda$dynamicStartCache$20(GridCacheProcessor.java:3606) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.dynamicStartCache(GridCacheProcessor.java:3635) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.dynamicStartCache(GridCacheProcessor.java:3543) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.publicJCache(GridCacheProcessor.java:4680) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.publicJCache(GridCacheProcessor.java:4651) at org.apache.ignite.internal.IgniteKernal.getOrCreateCache(IgniteKernal.java:3313) at org.apache.ignite.spi.discovery.tcp.TcpDiscoveryMultiThreadedTest$7.call(TcpDiscoveryMultiThreadedTest.java:548) ... 1 more{noformat} * TcpDiscoveryMultiThreadedTest#testClientContinuousQueryCoordinatorStop (in current codebase muted by renaming to {{_testClientContinuousQueryCoordinatorStop}}) fails because client node disconnected: {noformat} class org.apache.ignite.IgniteCheckedException: Client node disconnected. at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7425) at org.apache.ignite.internal.util.future.GridFutureAdapter.resolve(GridFutureAdapter.java:261) at org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:172) at org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141) at org.apache.ignite.testframework.GridTestUtils.lambda$runMultiThreadedAsync$96d302c5$1(GridTestUtils.java:856) at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:395) at org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:349) at org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:337) at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:507) at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:486) at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:474) at org.apache.ignite.testframework.GridTestUtils.lambda$runAsync$2(GridTestUtils.java:1011) at org.apache.ignite.testframework.GridTestUtils$7.call(GridTestUtils.java:1299) at
[jira] [Created] (IGNITE-10197) unexpected IllegalArgumentException in IgniteDbPutGetAbstractTest#testRandomPutGetRemove
Oleg Ignatenko created IGNITE-10197: --- Summary: unexpected IllegalArgumentException in IgniteDbPutGetAbstractTest#testRandomPutGetRemove Key: IGNITE-10197 URL: https://issues.apache.org/jira/browse/IGNITE-10197 Project: Ignite Issue Type: Bug Affects Versions: 2.6 Reporter: Oleg Ignatenko IgniteDbPutGetAbstractTest#testRandomPutGetRemove (in current codebase muted by renaming to {{_testRandomPutGetRemove}}) fails with IAE: {noformat}java.lang.IllegalArgumentException: Ouch! Argument is invalid: Cache name must not be null or empty. at org.apache.ignite.internal.util.GridArgumentCheck.ensure(GridArgumentCheck.java:109) at org.apache.ignite.internal.processors.cache.GridCacheUtils.validateCacheName(GridCacheUtils.java:1600) at org.apache.ignite.internal.IgniteKernal.cache(IgniteKernal.java:2901) at org.apache.ignite.internal.processors.database.IgniteDbPutGetAbstractTest.cache(IgniteDbPutGetAbstractTest.java:74) at org.apache.ignite.internal.processors.database.IgniteDbPutGetAbstractTest.testRandomPutGetRemove(IgniteDbPutGetAbstractTest.java:814) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at junit.framework.TestCase.runTest(TestCase.java:176) at org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2208) at org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:144) at org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:2124) at java.lang.Thread.run(Thread.java:745){noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10200) GridCacheSetAbstractSelfTest#testMultipleStructuresInDifferentGroups and some other tests are incorrectly muted (by underscore prefix in name)
Oleg Ignatenko created IGNITE-10200: --- Summary: GridCacheSetAbstractSelfTest#testMultipleStructuresInDifferentGroups and some other tests are incorrectly muted (by underscore prefix in name) Key: IGNITE-10200 URL: https://issues.apache.org/jira/browse/IGNITE-10200 Project: Ignite Issue Type: Bug Affects Versions: 2.6 Reporter: Oleg Ignatenko The following tests are incorrectly muted: - GridCacheSetAbstractSelfTest#_testMultipleStructuresInDifferentGroups This test was muted long ago and currently passes, needs to be unmuted ([commit d6c64a8|https://github.com/apache/ignite/commit/d6c64a82ac738ca4acad6785c87b6de28e9760b1]). - TcpDiscoveryMultiThreadedTest#_testCustomEventNodeRestart This test was muted long ago and currently passes, needs to be unmuted. Git history refers IGNITE-3418, [commit a5e965e|https://github.com/apache/ignite/commit/2e65beb4f628e0ba138abc2dfab707bc9a5e965e] - ServiceDeploymentOnActivationTest#_testClientsStaticConfigWithoutPersistence Test name should be corrected, it needs to be failed and muted per IGNITE-8279 - RendezvousAffinityFunctionSimpleBenchmark#_testAffinityCompatibility Test name should be corrected, it needs to be failed and muted per IGNITE-10191 - OptimizedMarshallerTest#_testAllocationOverflow Test name should be corrected, it needs to be failed and muted per IGNITE-10192 - IgniteBaselineAffinityTopologyActivationTest#_testBaselineTopologyHistoryIsDeletedOnBaselineDelete Test name should be corrected, it needs to be failed and muted per IGNITE-10193 - IgniteDbPutGetAbstractTest#_testRandomPutGetRemove Test name should be corrected, it needs to be failed and muted per IGNITE-10197 - TcpDiscoveryMultiThreadedTest#_testCustomEventOnJoinCoordinatorStop Test name should be corrected, it needs to be failed and muted per IGNITE-10198 - TcpDiscoveryMultiThreadedTest#_testClientContinuousQueryCoordinatorStop Test name should be corrected, it needs to be failed and muted per IGNITE-10198 - IgniteSqlSplitterSelfTest##testMergeJoin Test name should be corrected, it needs to be failed and muted per IGNITE-10199 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10199) unexpected result from QueryCursor.getAll() in IgniteSqlSplitterSelfTest#testMergeJoin
Oleg Ignatenko created IGNITE-10199: --- Summary: unexpected result from QueryCursor.getAll() in IgniteSqlSplitterSelfTest#testMergeJoin Key: IGNITE-10199 URL: https://issues.apache.org/jira/browse/IGNITE-10199 Project: Ignite Issue Type: Bug Affects Versions: 2.6 Reporter: Oleg Ignatenko IgniteSqlSplitterSelfTest#testMergeJoin (in current codebase muted by renaming to {{_testMergeJoin}}) fails with message: {noformat} junit.framework.AssertionFailedError at junit.framework.Assert.fail(Assert.java:55) at junit.framework.Assert.assertTrue(Assert.java:22) at junit.framework.Assert.assertTrue(Assert.java:31) at junit.framework.TestCase.assertTrue(TestCase.java:201) at org.apache.ignite.internal.processors.query.IgniteSqlSplitterSelfTest.testMergeJoin(IgniteSqlSplitterSelfTest.java:182) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at junit.framework.TestCase.runTest(TestCase.java:176) at org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2208) at org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:144) at org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:2124) at java.lang.Thread.run(Thread.java:745) {noformat} Test code referred by above message is last assert in below snippet: {code} String qry = "select o1.* from Org o1, " + "(select max(o.name) as name, o.id from Org o group by o.id) o2 " + "where o1.id = o2.id"; List> plan = c.query(new SqlFieldsQuery("explain " + qry) .setEnforceJoinOrder(true)).getAll(); X.println("Plan: " + plan); String map0 = (String)plan.get(0).get(0); String map1 = (String)plan.get(1).get(0); String rdc = (String)plan.get(2).get(0); assertTrue(map0.contains("ORDER BY")); assertTrue(map1.contains("ORDER BY")); {code} Note git history appears to refer mute in [commit 58e57fd|https://github.com/apache/ignite/commit/70eed75422ea50a7bb9dbe539e2b9a62458e57fd]. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10180) Investigate migration from Junit 4 to 5
Oleg Ignatenko created IGNITE-10180: --- Summary: Investigate migration from Junit 4 to 5 Key: IGNITE-10180 URL: https://issues.apache.org/jira/browse/IGNITE-10180 Project: Ignite Issue Type: Sub-task Reporter: Oleg Ignatenko If needed, refer parent task for more details. Find out if Junit 5 is mature enough - specifically why maven still uses Junit 4 as default, are there serious reasons for that or not. If it looks okay, create a new JIRA task to migrate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10179) Change new tests root to use @BeforeClass and @AfterClass instead of isFirstTest() and isLastTest()
Oleg Ignatenko created IGNITE-10179: --- Summary: Change new tests root to use @BeforeClass and @AfterClass instead of isFirstTest() and isLastTest() Key: IGNITE-10179 URL: https://issues.apache.org/jira/browse/IGNITE-10179 Project: Ignite Issue Type: Sub-task Reporter: Oleg Ignatenko If needed, refer parent task for more details. isFirstTest() and isLastTest() homebrew methods seem to be in GridAbstractTest only because Junit 3 lacked necessary functionality; after migration to Junit 4 these would better changed to use standard API of this framework. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10177) cleanup Junit 3 from the project
Oleg Ignatenko created IGNITE-10177: --- Summary: cleanup Junit 3 from the project Key: IGNITE-10177 URL: https://issues.apache.org/jira/browse/IGNITE-10177 Project: Ignite Issue Type: Sub-task Reporter: Oleg Ignatenko If needed, refer parent task for more details. 1) remove deprecated API of GridAbstractTest and its subclasses 2) remove dependencies from Junit 3 in Maven 3) migrate tests that were missed at prior steps, if there are any -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10178) change tests that fail("Ignite JIRA ticket URL") to @Ignore("Ignite JIRA ticket URL")
Oleg Ignatenko created IGNITE-10178: --- Summary: change tests that fail("Ignite JIRA ticket URL") to @Ignore("Ignite JIRA ticket URL") Key: IGNITE-10178 URL: https://issues.apache.org/jira/browse/IGNITE-10178 Project: Ignite Issue Type: Sub-task Reporter: Oleg Ignatenko If needed, refer parent task for more details. Note this step would better be coordinated with Teamcity and TC bot maintainers because it may substantially impact them -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10176) migrate non-core modules tests from Junit 3 to 4
Oleg Ignatenko created IGNITE-10176: --- Summary: migrate non-core modules tests from Junit 3 to 4 Key: IGNITE-10176 URL: https://issues.apache.org/jira/browse/IGNITE-10176 Project: Ignite Issue Type: Sub-task Reporter: Oleg Ignatenko If needed, refer parent task for more details. Migrate using new API introduced and tested at prior step. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10175) migrate core module tests from Junit 3 to 4
Oleg Ignatenko created IGNITE-10175: --- Summary: migrate core module tests from Junit 3 to 4 Key: IGNITE-10175 URL: https://issues.apache.org/jira/browse/IGNITE-10175 Project: Ignite Issue Type: Sub-task Reporter: Oleg Ignatenko If needed, refer parent task for more details. Migrate using new API introduced at prior step. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10174) migrate examples tests from Junit 3 to 4
Oleg Ignatenko created IGNITE-10174: --- Summary: migrate examples tests from Junit 3 to 4 Key: IGNITE-10174 URL: https://issues.apache.org/jira/browse/IGNITE-10174 Project: Ignite Issue Type: Sub-task Reporter: Oleg Ignatenko If needed, refer parent task for more details. This is first task because examples are most publicly visible, relativelly small and least risky part of unit tests in the project. Steps: 1) create Junit-4 based twin of GridAbstractTest (and all needed subclasses), 2) declare GridAbstractTest deprecated with reference to use newer API instead 3) change unit tests in examples to use new API -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10173) Gradually move unit tests from Junit 3 to newer version
Oleg Ignatenko created IGNITE-10173: --- Summary: Gradually move unit tests from Junit 3 to newer version Key: IGNITE-10173 URL: https://issues.apache.org/jira/browse/IGNITE-10173 Project: Ignite Issue Type: Task Affects Versions: 2.6 Reporter: Oleg Ignatenko (i) Related dev list discussion: [Is it time to move forward to JUnit4 (5)?|http://apache-ignite-developers.2346864.n4.nabble.com/Is-it-time-to-move-forward-to-JUnit4-5-td29608.html] Currently unit tests in the project are mix of two versions Junit 3 and 4. This makes it hard to develop and maintain. Another reason why migration to newer version is desirable is Junit 4 provides developer an option to conveniently mute tests by Ignore annotation while with Junit 3 this could only be done by adding fail and manually muting the test in Teamcity (the latter turned out to be extremely painful when I had to do some things per IGNITE-9884). This task is to migrate all tests to single uniform version Junit 4 (with further option to migrate to Junit 5, see also note below). The difficulty of migration is that too many tests depend on Junit3-based [GridAbstractTest|https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/GridAbstractTest.java] and its subclasses so that making them all change in a single move would be very tedious and risky. A way to work around this is to create a parallel Junit-4 based "twin" of GridAbstractTest (tentative name {{IgniteAbstractTest}}) and using is move to Junit 4 gradually, smoothly and safely. Step by step plan of changes is below (to be further made to sub-tasks under this "umbrella" ticket): # migrate examples tests from Junit 3 to 4 This is first task because examples are most publicly visible, relativelly small and least risky part of unit tests in the project. Steps: 1) create Junit-4 based twin of GridAbstractTest (and all needed subclasses), 2) declare GridAbstractTest deprecated with reference to use newer API instead 3) change unit tests in examples to use new API # migrate core module tests from Junit 3 to 4 using new API introduced and tested at prior step # migrate non-core modules tests from Junit 3 to 4 using new API introduced and tested at prior step # cleanup Junit 3 from the project 1) remove deprecated API of GridAbstractTest and its subclasses 2) remove dependencies from Junit 3 in Maven 3) migrate tests that were missed at prior steps, if there are any # change tests that fail("Ignite JIRA ticket URL") to @Ignore("Ignite JIRA ticket URL") Note this step would better be coordinated with Teamcity and TC bot maintainers because it may substantially impact them # Change new tests root to use @BeforeClass and @AfterClass instead of isFirstTest() and isLastTest() these homebrew methods seem to be in GridAbstractTest only because Junit 3 lacked necessary functionality; after migration to Junit 4 these would better changed to use standard API of this framework. # Investigate migration from Junit 4 to 5 Find out if Junit 5 is mature enough - specifically why maven still uses Junit 4 as default, are there serious reasons for that or not. If it looks okay, create a new JIRA task to migrate. Note on migrating to Junit 5. While preparing this task I considered an option to migrate from Junit 3 to 5 instead of 4. I dropped that primarily because migration from Junit 3 requires quite a lot of manual changes in the code (changing imports and adding annotations), as opposed to migration from Junit 4 to 5 for which there seem to be options to make most changes automatically (eg IntelliJ seem to provide such an option). Because of that it looks more convenient to split it to separate steps, so that after all tests in the project get to Junit 4 we could do an automated migration to newer version. Another thing that made me prefer this way is that I wanted to avoid having three different versions Junit in the project (3, 4, 5), even if that would be temporarily (note that migration from Junit 3 is expected to be manual and quite lengthy, so "temporarily" may mean quite a lot of time really). Also note that as pointed in above list of steps we better make some preliminary assessment on whether time has really come to migrate to Junit 5. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10162) IgniteCacheAtomicQuerySelfTest#testTwoObjectsTextSearch fails with IgniteCheckedException: Failed to find SQL table for type: ObjectValue
Oleg Ignatenko created IGNITE-10162: --- Summary: IgniteCacheAtomicQuerySelfTest#testTwoObjectsTextSearch fails with IgniteCheckedException: Failed to find SQL table for type: ObjectValue Key: IGNITE-10162 URL: https://issues.apache.org/jira/browse/IGNITE-10162 Project: Ignite Issue Type: Bug Affects Versions: 2.6 Reporter: Oleg Ignatenko IgniteCacheAtomicQuerySelfTest#testTwoObjectsTextSearch (in current codebase muted by renaming to {{_testTwoObjectsTextSearch}}) fails: {noformat} [2018-11-07 14:37:32,728][INFO ][grid-nio-worker-tcp-comm-1-#91%near.IgniteCacheAtomicQuerySelfTest1%][IgniteCachePartitionedQuerySelfTest$TestTcpCommunicationSpi] Accepted incoming communication connection [locAddr=/127.0.0.1:47101, rmtAddr=/127.0.0.1:57395][2018-11-07 14:37:32,728][WARN ][sys-stripe-1-#61%near.IgniteCacheAtomicQuerySelfTest2%][GridQueryProcessor] Key-value pair is not inserted into any SQL table [cacheName=Object-Object, valType=o.a.i.i.processors.cache.IgniteCacheAbstractQuerySelfTest$ObjectValue] [2018-11-07 14:37:32,728][WARN ][sys-stripe-1-#61%near.IgniteCacheAtomicQuerySelfTest2%][GridQueryProcessor] ^-- Value type(s) are specified via CacheConfiguration.indexedTypes or CacheConfiguration.queryEntities [2018-11-07 14:37:32,728][WARN ][sys-stripe-1-#61%near.IgniteCacheAtomicQuerySelfTest2%][GridQueryProcessor] ^-- Make sure that same type(s) used when adding Object or BinaryObject to cache [2018-11-07 14:37:32,728][WARN ][sys-stripe-1-#61%near.IgniteCacheAtomicQuerySelfTest2%][GridQueryProcessor] ^-- Otherwise, entries will be stored in cache, but not appear as SQL Table rows [2018-11-07 14:37:32,744][INFO ][grid-nio-worker-tcp-comm-1-#94%near.IgniteCacheAtomicQuerySelfTest2%][IgniteCachePartitionedQuerySelfTest$TestTcpCommunicationSpi] Established outgoing communication connection [locAddr=/127.0.0.1:57395, rmtAddr=/127.0.0.1:47101] [2018-11-07 14:37:32,766][WARN ][sys-stripe-6-#66%near.IgniteCacheAtomicQuerySelfTest2%][GridQueryProcessor] Key-value pair is not inserted into any SQL table [cacheName=Object-Object, valType=o.a.i.i.processors.cache.IgniteCacheAbstractQuerySelfTest$ObjectValueOther] [2018-11-07 14:37:32,867][ERROR][query-#150%near.IgniteCacheAtomicQuerySelfTest2%][GridCacheDistributedQueryManager] Failed to run query [qry=GridCacheQueryInfo [loc=false, trans=null, rdc=null, qry=GridCacheQueryAdapter [type=TEXT, clsName=ObjectValue, clause=str, filter=null, transform=null, part=null, incMeta=false, metrics=null, pageSize=1024, timeout=0, incBackups=false, forceLocal=false, dedup=false, prj=null, keepBinary=false, subjId=964a4e0a-38e6-47db-b91c-95d3c410, taskHash=0, mvccSnapshot=null], locFut=null, sndId=964a4e0a-38e6-47db-b91c-95d3c410, reqId=27, incMeta=false, all=false], node=5c1facf8-cf09-498f-bdf0-e75f9942] class org.apache.ignite.IgniteCheckedException: Failed to find SQL table for type: ObjectValue at org.apache.ignite.internal.processors.query.GridQueryProcessor.typeName(GridQueryProcessor.java:2675) at org.apache.ignite.internal.processors.query.GridQueryProcessor.access$500(GridQueryProcessor.java:129) at org.apache.ignite.internal.processors.query.GridQueryProcessor$9.applyx(GridQueryProcessor.java:2617) at org.apache.ignite.internal.processors.query.GridQueryProcessor$9.applyx(GridQueryProcessor.java:2615) at org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36) at org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2696) at org.apache.ignite.internal.processors.query.GridQueryProcessor.queryText(GridQueryProcessor.java:2614) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.executeQuery(GridCacheQueryManager.java:614) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.queryResult(GridCacheQueryManager.java:1472) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.runQuery(GridCacheQueryManager.java:1106) at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager.processQueryRequest(GridCacheDistributedQueryManager.java:234) at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$2.apply(GridCacheDistributedQueryManager.java:109) at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$2.apply(GridCacheDistributedQueryManager.java:107) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1054) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:579) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(Grid
[jira] [Created] (IGNITE-10159) IgniteCacheAbstractQuerySelfTest.testObjectQueryWithSwap fails non-null assertion at localPeek
Oleg Ignatenko created IGNITE-10159: --- Summary: IgniteCacheAbstractQuerySelfTest.testObjectQueryWithSwap fails non-null assertion at localPeek Key: IGNITE-10159 URL: https://issues.apache.org/jira/browse/IGNITE-10159 Project: Ignite Issue Type: Bug Affects Versions: 2.6 Reporter: Oleg Ignatenko IgniteCacheAbstractQuerySelfTest.testObjectQueryWithSwap (in current codebase muted by renaming to {{_testObjectQueryWithSwap}}) fails with the following error: {noformat} junit.framework.AssertionFailedError at junit.framework.Assert.fail(Assert.java:55) at junit.framework.Assert.assertTrue(Assert.java:22) at junit.framework.Assert.assertNotNull(Assert.java:256) at junit.framework.Assert.assertNotNull(Assert.java:248) at junit.framework.TestCase.assertNotNull(TestCase.java:417) at org.apache.ignite.internal.processors.cache.IgniteCacheAbstractQuerySelfTest.testObjectQueryWithSwap(IgniteCacheAbstractQuerySelfTest.java:960) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ...{noformat} Code referred to above is: {code}assertNotNull(c.localPeek(i, CachePeekMode.ONHEAP));{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10158) Some tests in IgniteCacheAbstractQuerySelfTest are incorrectly muted
Oleg Ignatenko created IGNITE-10158: --- Summary: Some tests in IgniteCacheAbstractQuerySelfTest are incorrectly muted Key: IGNITE-10158 URL: https://issues.apache.org/jira/browse/IGNITE-10158 Project: Ignite Issue Type: Bug Affects Versions: 2.6 Reporter: Oleg Ignatenko Some tests in [IgniteCacheAbstractQuerySelfTest|https://github.com/apache/ignite/blob/master/modules/indexing/src/test/java/org/apache/ignite/internal/processors/cache/IgniteCacheAbstractQuerySelfTest.java] are muted by renaming (prefixing with underscore, {{_test...}} and refer invalid JIRA URL in fail parameter ("http://atlassian.gridgain.com/jira/browse/GG-11216";). - _testDifferentKeyTypes this test should change expectation to opposite and after that recovered - _testObjectQueryWithSwap and _testTwoObjectsTextSearch Need to be properly muted and further investigated, per separate tickets. Per my preliminary checks tests fail because of wrong cache configuration, although there is also a chance that test design is wrong and these should be deleted. There is also a dead code there, a private class {{EmptyObject}} - it needs to be deleted. Code that was using this class was removed per [IGNITE-1232|https://issues.apache.org/jira/browse/IGNITE-1232] ([commit 79b8b08|https://github.com/gridgain/apache-ignite/commit/68891e89dd0e0f19321d6a4d45ae7372279b8b08#diff-a2f35b3aa70a70b98ce0cd6a1381d1f7]) but this private class was forgotten. I also searched project code for other occurrences of mentioned troublesome fail parameter "GG-11216" and found yet another incorrectly muted test: {{IgniteCacheQueryMultiThreadedSelfTest#_testMultiThreadedSwapUnswapLongString}} This test should be recovered. It passed on my machine and per my comparison with similar test cases {{testMultiThreadedSwapUnswapLong}} and {{testMultiThreadedSwapUnswapString}} its design looks fairly reasonable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10021) IgniteInterruptedCheckedException: sleep interrupted in GridServiceProcessorBatchDeploySelfTest#testCancelAllTopologyChange
Oleg Ignatenko created IGNITE-10021: --- Summary: IgniteInterruptedCheckedException: sleep interrupted in GridServiceProcessorBatchDeploySelfTest#testCancelAllTopologyChange Key: IGNITE-10021 URL: https://issues.apache.org/jira/browse/IGNITE-10021 Project: Ignite Issue Type: Bug Affects Versions: 2.6 Reporter: Oleg Ignatenko Attachments: GridServiceProcessorBatchDeploySelfTest.testCancelAllTopologyChange.log Test {{GridServiceProcessorBatchDeploySelfTest#testCancelAllTopologyChange}} failed at TC (couldn't reproduce locally), error message: {noformat} [2018-10-23 15:40:25,651][ERROR][srvc-deploy-#121506%extra-node-1%][TcpCommunicationSpi] Failed to send message to remote node [node=TcpDiscoveryNode [id=ce88f2ef-5c88-46f1-83d4-50dd8163, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47503], discPort=47503, order=4, intOrder=4, lastExchangeTime=1540309225078, loc=false, ver=2.7.0#20181023-sha1:d09f1d47, isClient=false], msg=GridIoMessage [plc=5, topic=TOPIC_CACHE, topicOrd=8, ordered=false, timeout=0, skipOnTimeout=false, msg=GridCacheIdMessage [cacheId=0]GridDistributedBaseMessage [ver=GridCacheVersion [topVer=151789232, order=1540309225927, nodeOrder=7], committedVers=null, rolledbackVers=null, cnt=0, super=]GridDistributedTxPrepareRequest [threadId=133839, concurrency=OPTIMISTIC, isolation=READ_COMMITTED, writeVer=GridCacheVersion [topVer=151789232, order=1540309225928, nodeOrder=7], timeout=0, reads=null, writes=PredicateCollectionView [IgniteTxEntry [key=KeyCacheObjectImpl [part=9, val=GridServiceAssignmentsKey [name=testService-175], hasValBytes=true], cacheId=-2100569601, txKey=IgniteTxKey [key=KeyCacheObjectImpl [part=9, val=GridServiceAssignmentsKey [name=testService-175], hasValBytes=true], cacheId=-2100569601], val=[op=DELETE, val=null], prevVal=[op=NOOP, val=null], oldVal=[op=NOOP, val=null], entryProcessorsCol=null, ttl=-1, conflictExpireTime=-1, conflictVer=null, explicitVer=null, dhtVer=null, filters=CacheEntryPredicate[] [], filtersPassed=false, filtersSet=true, entry=GridCacheMapEntry [key=KeyCacheObjectImpl [part=9, val=GridServiceAssignmentsKey [name=testService-175], hasValBytes=true], val=CacheObjectImpl [val=GridServiceAssignments [nodeId=bdeeb063-5558-4dab-90c3-b61b91b338f0, topVer=6, cfg=LazyServiceConfiguration [srvcClsName=org.apache.ignite.internal.processors.service.DummyService, svcCls=, nodeFilterCls=AttributeFilter], assigns=HashMap {0069f119-2516-48ee-866c-8a3ba55f8000=0, dcf0a942-d7d0-4574-870a-6b251740=0, ce88f2ef-5c88-46f1-83d4-50dd8163=1, 3264d65e-695f-433c-b028-7bff8b82=0, 4427ca0c-24c1-4e61-9f04-bbd37d11=0}], hasValBytes=true], ver=GridCacheVersion [topVer=151789229, order=1540309225560, nodeOrder=1], hash=1705888709, extras=GridCacheMvccEntryExtras [mvcc=GridCacheMvcc [locs=LinkedList [GridCacheMvccCandidate [nodeId=100f5b42-d915-409b-980c-726f1945a001, ver=GridCacheVersion [topVer=151789232, order=1540309225927, nodeOrder=7], threadId=133839, id=335933, topVer=AffinityTopologyVersion [topVer=9, minorTopVer=0], reentry=null, otherNodeId=100f5b42-d915-409b-980c-726f1945a001, otherVer=GridCacheVersion [topVer=151789232, order=1540309225927, nodeOrder=7], mappedDhtNodes=null, mappedNearNodes=null, ownerVer=null, serOrder=null, key=KeyCacheObjectImpl [part=9, val=GridServiceAssignmentsKey [name=testService-175], hasValBytes=true], masks=local=1|owner=1|ready=1|reentry=0|used=0|tx=1|single_implicit=1|dht_local=1|near_local=0|removed=0|read=0, prevVer=null, nextVer=null]], rmts=null]], flags=2]GridDistributedCacheEntry [super=]GridDhtCacheEntry [rdrs=ReaderId[] [], part=9, super=], prepared=1, locked=false, nodeId=100f5b42-d915-409b-980c-726f1945a001, locMapped=false, expiryPlc=null, transferExpiryPlc=false, flags=0, partUpdateCntr=0, serReadVer=null, xidVer=GridCacheVersion [topVer=151789232, order=1540309225927, nodeOrder=7]]], dhtVers=null, txSize=0, plc=5, txState=null, flags=retVal|last|sys, super=]GridDhtTxPrepareRequest [nearNodeId=100f5b42-d915-409b-980c-726f1945a001, futId=f0a1fb1a661-0ff0152a-8da5-4117-8c29-4ba138e30072, miniId=1, topVer=AffinityTopologyVersion [topVer=9, minorTopVer=0], invalidateNearEntries={}, nearWrites=null, owned=null, nearXidVer=GridCacheVersion [topVer=151789232, order=1540309225927, nodeOrder=7], subjId=100f5b42-d915-409b-980c-726f1945a001, taskNameHash=0, preloadKeys=null, mvccSnapshot=null, skipCompletedVers=true, super=]]] class org.apache.ignite.internal.IgniteInterruptedCheckedException: sleep interrupted at org.apache.ignite.internal.util.IgniteUtils.sleep(IgniteUtils.java:7782) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2888) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:2713) at
[jira] [Created] (IGNITE-9884) Some tests are incorrectly muted
Oleg Ignatenko created IGNITE-9884: -- Summary: Some tests are incorrectly muted Key: IGNITE-9884 URL: https://issues.apache.org/jira/browse/IGNITE-9884 Project: Ignite Issue Type: Bug Affects Versions: 2.6 Reporter: Oleg Ignatenko The following tests pass when unmuted. It looks like the reasons that made them fail in the past were somehow resolved by later code changes but tests were forgotten to unmute: {panel} GridCacheStoreManagerDeserializationTest#_testBinaryStream IgniteCacheConfigVariationsFullApiTest#_testDeletedEntriesFlag IgniteCacheConfigVariationsFullApiTest#_testEvictExpired GridCacheStoreManagerDeserializationTest#_testBinaryStream PageIdDistributionTest#_testRealHistory GridEventConsumeSelfTest#_testMultithreadedWithNodeRestart BPlusTreeSelfTest#_testBenchInvoke IgniteClientReconnectMassiveShutdownTest#_testMassiveServersShutdown1 IgniteClientReconnectMassiveShutdownTest#_testMassiveServersShutdown3 TcpDiscoveryMultiThreadedTest#_testMultiThreadedServersRestart (passes) GridServiceProcessorBatchDeploySelfTest#_testDeployAllTopologyChange GridServiceProcessorBatchDeploySelfTest#_testDeployAllTopologyChangeFail GridServiceProcessorBatchDeploySelfTest#_testCancelAllTopologyChange {panel} The following tests are muted by renaming (prefixing with underscore, {{_test...}}, should use standard muting by known JIRA tickets instead: {panel} - GridCacheAbstractFullApiSelfTest#_testInvokeAllMultithreaded should fail with the reference to IGNITE-4380 - CacheMvccTransactionsTest#_testNodesRestartNoHang should fail with the reference to IGNITE-5935 {panel} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9754) Increase timeout waiting for commitLatch in test TxRollbackAsyncWithPersistenceTest.testRollbackOnTopologyLockPessimistic
Oleg Ignatenko created IGNITE-9754: -- Summary: Increase timeout waiting for commitLatch in test TxRollbackAsyncWithPersistenceTest.testRollbackOnTopologyLockPessimistic Key: IGNITE-9754 URL: https://issues.apache.org/jira/browse/IGNITE-9754 Project: Ignite Issue Type: Bug Affects Versions: 2.6 Reporter: Oleg Ignatenko Current timeout for {{commitLatch}} in test {{TxRollbackAsyncWithPersistenceTest.testRollbackOnTopologyLockPessimistic}} is 10 seconds. This looks insufficient to make a compelling reason that test failure indicates a problem in the implementation. For example log of this [recent failure at Teamcity|https://ci.ignite.apache.org/viewLog.html?buildId=1970040&tab=buildResultsDiv&buildTypeId=IgniteTests24Java8_Cache7] suggests that there were just a couple of non-critical delays for a few seconds that made test fail. Suggest to increase timeout from 10 seconds to 60, so that in case if test fails waiting on the latch one could assume with reasonable certainty that something is wrong with the implementation or infrastructure. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9740) [ML] Remove IgniteThread wrapper from ml unit test EvaluatorTest (follow up to IGNITE-9711)
Oleg Ignatenko created IGNITE-9740: -- Summary: [ML] Remove IgniteThread wrapper from ml unit test EvaluatorTest (follow up to IGNITE-9711) Key: IGNITE-9740 URL: https://issues.apache.org/jira/browse/IGNITE-9740 Project: Ignite Issue Type: Bug Components: ml Affects Versions: 2.6 Reporter: Oleg Ignatenko [EvaluatorTest|https://github.com/apache/ignite/blob/master/modules/ml/src/test/java/org/apache/ignite/ml/selection/scoring/evaluator/EvaluatorTest.java] involves {{IgniteThread}} which is in fact not needed there and should be removed. {{IgniteThread}} usage is a remainder / copy-paste from older tests and examples that were using API requiring it. This API has been removed and there is no need for wrapping like that anymore. For the reference on how to perform suggested cleanup check changes made to ml examples per IGNITE-9711. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9721) NPE in IgniteAuthenticationProcessor$RefreshUsersStorageWorker.body
Oleg Ignatenko created IGNITE-9721: -- Summary: NPE in IgniteAuthenticationProcessor$RefreshUsersStorageWorker.body Key: IGNITE-9721 URL: https://issues.apache.org/jira/browse/IGNITE-9721 Project: Ignite Issue Type: Bug Affects Versions: 2.6 Reporter: Oleg Ignatenko Tests at Teamcity sometimes fail with NPE in {{IgniteAuthenticationProcessor}}. Example of failure (in test {{AuthenticationProcessorNodeRestartTest.test1kUsersNodeRestartServer}}) is [here|https://ci.ignite.apache.org/viewLog.html?buildId=1945459&tab=buildResultsDiv&buildTypeId=IgniteTests24Java8_Cache7], and stacktrace looks as follows: {noformat}[2018-09-26 07:06:36,732][ERROR][auth-#6589%authentication.AuthenticationProcessorNodeRestartTest0%][IgniteAuthenticationProcessor] Runtime error caught during grid runnable execution: GridWorker [name=refresh-store, igniteInstanceName=authentication.AuthenticationProcessorNodeRestartTest0, finished=false, heartbeatTs=1537945595104, hashCode=733597684, interrupted=true, runner=auth-#6589%authentication.AuthenticationProcessorNodeRestartTest0%] java.lang.NullPointerException at org.apache.ignite.internal.processors.authentication.IgniteAuthenticationProcessor$RefreshUsersStorageWorker.body(IgniteAuthenticationProcessor.java:1346) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748){noformat} This might be either coding error (missing null check) or data race. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9691) AuthenticationProcessorNodeRestartTest.testConcurrentAuthorize uses outdated assumption about exception message
Oleg Ignatenko created IGNITE-9691: -- Summary: AuthenticationProcessorNodeRestartTest.testConcurrentAuthorize uses outdated assumption about exception message Key: IGNITE-9691 URL: https://issues.apache.org/jira/browse/IGNITE-9691 Project: Ignite Issue Type: Bug Affects Versions: 2.6 Reporter: Oleg Ignatenko Test {{AuthenticationProcessorNodeRestartTest.testConcurrentAuthorize}} that was introduced per IGNITE-7436 uses particular assumption about exception message thrown from method [GridIoManager.send|https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/managers/communication/GridIoManager.java]: {code} // Skip exception if server down. if (!e.getMessage().contains("Failed to send message (node may have left the grid or " + "TCP connection cannot be established due to firewall issues)")) { e.printStackTrace(); fail("Unexpected exception: " + e.getMessage()); } // ...{code} This expectation appears to be broken by changes introduced per IGNITE-4191 which added yet another exception message that may occur in above piece of test code: {code} if (!ctx.discovery().alive(node)) throw new ClusterTopologyCheckedException("Failed to send message, node left: " + node.id(), e);{code} (above code was added at line 1664 in {{GridIoManager.java}}) Regression wasn't immediately discovered because of indeterministic test scenario which made new failures appear randomly and mixed with passes when particular condition was missed in the course of test execution. Test needs to be updated to accommodate the changes in codebase. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9585) Error message sometimes refers nonexisting log file when remote node fails to start
Oleg Ignatenko created IGNITE-9585: -- Summary: Error message sometimes refers nonexisting log file when remote node fails to start Key: IGNITE-9585 URL: https://issues.apache.org/jira/browse/IGNITE-9585 Project: Ignite Issue Type: Bug Affects Versions: 2.6 Reporter: Oleg Ignatenko Teamcity build logs sometimes refer to remote node log files that aren't present in build artifacts ([example|https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteTests24Java8_StartNodes&buildId=1849937&branch_IgniteTests24Java8_StartNodes=%3Cdefault%3E]). I managed to reproduce this on my machine (details below) and it looks like typically the root cause of this is error message from [StartNodeCallableImpl|https://github.com/apache/ignite/blob/master/modules/ssh/src/main/java/org/apache/ignite/internal/util/nodestart/StartNodeCallableImpl.java] referring readers to file that doesn't exist (and it wasn't even created to start with). {code:java} return new ClusterStartNodeResultImpl(spec.host(), false, "Remote node could not start. " + "See log for details: " + scriptOutputPath); {code} This is quite painful when one tries to investigate node launching failures because the misleading message causes one to waste time investigating the problem that doesn't exist (it appears as if log file was there but somehow disappeared for some mysterious reason). To reproduce the issue locally one can do as follows: first, modify file [start-nodes-custom.sh|https://github.com/apache/ignite/blob/master/modules/core/src/test/bin/start-nodes-custom.sh] by replacing {{ignite.sh}} with the name of script that doesn't exist, eg {{noignite.sh}}. After that, execute unit test [IgniteProjectionStartStopRestartSelfTest|https://github.com/apache/ignite/blob/master/modules/ssh/src/test/java/org/apache/ignite/internal/IgniteProjectionStartStopRestartSelfTest.java] and study its results and logs. You will find that {{testCustomScript}} fails - which is expected because name of the script intended to be executed has changed to one that doesn't exist. Also you will find that log file for respective node hasn't been created - which is also expected because shell command fails before creating it. But in the same time test log will refer to mentioned file as if it exists. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9545) IgniteProjectionStartStopRestartSelfTest: misleading javadocs, required conditions are not described, inconvenient to configure locally
Oleg Ignatenko created IGNITE-9545: -- Summary: IgniteProjectionStartStopRestartSelfTest: misleading javadocs, required conditions are not described, inconvenient to configure locally Key: IGNITE-9545 URL: https://issues.apache.org/jira/browse/IGNITE-9545 Project: Ignite Issue Type: Bug Affects Versions: 2.6 Reporter: Oleg Ignatenko This test has been [reported as flaky|https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&tab=testDetails&testNameId=2278553016619338221#analysis] at Teamcity. Looking closer shows that there are some issues with the test. [IgniteProjectionStartStopRestartSelfTest|https://github.com/apache/ignite/blob/master/modules/ssh/src/test/java/org/apache/ignite/internal/IgniteProjectionStartStopRestartSelfTest.java] class javadocs provide instructions on how to configure test which have nothing to do with the way how it is actually configured. Not only the way is different but even respective property names are incorrect, which is easy to see from very first 3 statements in test code that initialize configuration. Checking git history of this file shows that root cause for this issue is a change made about 4 years ago when obtaining test properties has changed from {{GridTestProperties.getProperty}} to {{System.getenv}} (back then, also property names have changed) but test javadoc was not updated to reflect that. Another issue with javadoc which makes it unnecessarily difficult to investigate failures is that it doesn't explain that test expects configured target host to run ssh server and accept connections at configured port from user with specified credentials. Javadocs need to be corrected. Another issue with the test is the way it obtains the config (username and password): when I tried to do some quick experiments on my machine it turned out fairly difficult to set to what I wanted. When I tried to change test code to obtain config in the way how it was in the past (via {{GridTestProperties}}) it went much easier. One good thing of the current way is it has proven to work well on Teamcity and because of that it makes sense to keep it. But on the other hand it looks desirable to augment it with fallback to the way that is more convenient for local experimenting. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9461) Implement random subspace method and provide an option to combine it with bagging
Oleg Ignatenko created IGNITE-9461: -- Summary: Implement random subspace method and provide an option to combine it with bagging Key: IGNITE-9461 URL: https://issues.apache.org/jira/browse/IGNITE-9461 Project: Ignite Issue Type: Task Components: ml Affects Versions: 2.6 Reporter: Oleg Ignatenko Implement random subspace method (aka attribute bagging or feature bagging) to give ML API users more options to address overfitting. Also provide an option to combine this method with bagging. References: * [Wikipedia article|https://en.wikipedia.org/wiki/Random_subspace_method] {quote}Informally, this causes individual learners to not over-focus on features that appear highly predictive/descriptive in the training set, but fail to be as predictive for points outside that set. For this reason, random subspaces are an attractive choice for problems where the number of features is much larger than the number of training points, such as learning from fMRI data or gene expression data...{quote} * [Combining Bagging and Random Subspaces to Create Better Ensembles|https://pdfs.semanticscholar.org/d38f/979ad85d59fc93058279010efc73a24a712c.pdf] * [Bagging and the Random Subspace Method for Redundant Feature Spaces|https://link.springer.com/chapter/10.1007/3-540-48219-9_1] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9423) unreliable test: SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase[Data divided on 4 partitions, training with batch size {1}]
Oleg Ignatenko created IGNITE-9423: -- Summary: unreliable test: SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase[Data divided on 4 partitions, training with batch size {1}] Key: IGNITE-9423 URL: https://issues.apache.org/jira/browse/IGNITE-9423 Project: Ignite Issue Type: Task Components: ml Affects Versions: 2.6 Reporter: Oleg Ignatenko [Teamcity here|https://ci.ignite.apache.org/viewLog.html?buildId=1755119&;] complains that {{SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase}} is unreliable: {quote}*This test looks flaky:* Frequent test status changes: 39 changes out of 128 invocations Test status change in build without changes: from successful to failed [View test history »|https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&tab=testDetails&testNameId=5118678405024105955#analysis]{quote} Output for most recent test failure: {noformat} SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase[Data divided on 4 partitions, training with batch size \{1}] java.lang.AssertionError: expected:<0.0> but was:<1.0> at org.apache.ignite.ml.svm.SVMMultiClassTrainerTest .testTrainWithTheLinearlySeparableCase(SVMMultiClassTrainerTest.java:53){noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9348) ML examples improvements, follow-up to IGNITE-9297
Oleg Ignatenko created IGNITE-9348: -- Summary: ML examples improvements, follow-up to IGNITE-9297 Key: IGNITE-9348 URL: https://issues.apache.org/jira/browse/IGNITE-9348 Project: Ignite Issue Type: Task Components: ml Affects Versions: 2.6 Reporter: Oleg Ignatenko After IGNITE-9297 was resolved, ML examples were discussed with some folks ([~skozlov], [~avolkov], [~chief]) in order to check if some other improvements may be desired. This ticket is to address obtained feedback. List of points that look worth taking care of includes (but is not limited to): * some examples javadocs still looks like missing details (eg random forest) * some mentions of algorithms (eg kNN, gradient boosting etc) are better to be linked to respective introductory articles in order to help less experienced users easier grasp the examples * presence of some examples doesn't look justified (eg LocalDatasetExample) * logging in some examples looks sub-optimal, either too brief or too lengthy -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9316) Analyse and improve code coverage in ML module (follow-up to IGNITE-9146)
Oleg Ignatenko created IGNITE-9316: -- Summary: Analyse and improve code coverage in ML module (follow-up to IGNITE-9146) Key: IGNITE-9316 URL: https://issues.apache.org/jira/browse/IGNITE-9316 Project: Ignite Issue Type: Task Components: ml Affects Versions: 2.6 Reporter: Oleg Ignatenko Studying of coverage reports attached in IGNITE-9146 suggests that many default methods in interfaces are not covered by tests. In particular the default methods in the following interfaces look like worth covering in tests: {noformat} org.apache.ignite.ml.Model org.apache.ignite.ml.math.functions.IgniteTriFunction org.apache.ignite.ml.dataset.PartitionDataBuilder org.apache.ignite.ml.dataset.PartitionContextBuilder org.apache.ignite.ml.dataset.Dataset org.apache.ignite.ml.environment.parallelism.Promise org.apache.ignite.ml.composition.predictionsaggregator.PredictionsAggregator{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9297) Review ML examples javadocs as of release 2.6
Oleg Ignatenko created IGNITE-9297: -- Summary: Review ML examples javadocs as of release 2.6 Key: IGNITE-9297 URL: https://issues.apache.org/jira/browse/IGNITE-9297 Project: Ignite Issue Type: Task Components: examples, ml Affects Versions: 2.6 Reporter: Oleg Ignatenko There are many ML examples added and changed in latest releases, and some of them have rather brief javadocs and may lack useful details. Review the examples and where needed improve javadocs, preferably using some well established example as a reference (eg [ComputeClientBinaryTaskExecutionExample|https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/binary/computegrid/ComputeClientBinaryTaskExecutionExample.java]). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9146) Analyse and improve code coverage in ML module
Oleg Ignatenko created IGNITE-9146: -- Summary: Analyse and improve code coverage in ML module Key: IGNITE-9146 URL: https://issues.apache.org/jira/browse/IGNITE-9146 Project: Ignite Issue Type: Task Components: ml Affects Versions: 2.6 Reporter: Oleg Ignatenko Run code coverage analysis, study results and add missing tests where needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9124) Remove some dead code in math.exceptions and optimization packages of ML module
Oleg Ignatenko created IGNITE-9124: -- Summary: Remove some dead code in math.exceptions and optimization packages of ML module Key: IGNITE-9124 URL: https://issues.apache.org/jira/browse/IGNITE-9124 Project: Ignite Issue Type: Task Components: ml Affects Versions: 2.5 Reporter: Oleg Ignatenko Doing code coverage analysis discovered a bit of unused code: # in package [math.exceptions|https://github.com/apache/ignite/tree/master/modules/ml/src/main/java/org/apache/ignite/ml/math/exceptions] - classes ConvergenceException / MathIllegalStateException, InsufficientDataException, MathArithmeticException, MathIllegalNumberException / NumberIsTooSmallException, NonPositiveDefiniteMatrixException, NonSymmetricMatrixException, NullArgumentException, SingularMatrixException, UnknownProviderException. In very early releases of ML these exceptions were used by some linear algebra code. This code has been very deeply reworked since but outdated exceptions were left hanging unattended, making rather confusing impression of the current API. # in [optimization|https://github.com/apache/ignite/tree/master/modules/ml/src/main/java/org/apache/ignite/ml/optimization] - interface Updater and two classes implementing it: BarzilaiBorweinUpdater and SimpleUpdater. This code has left hanging after cleanup that was made for release 2.5 (IGNITE-8232). Per my discussion with [~dmitrievanthony] (who most actively maintains this package) it would be better to remove this code. This task is to remove code mentioned above. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7593) Improve data used in DecisionTreesExample
Oleg Ignatenko created IGNITE-7593: -- Summary: Improve data used in DecisionTreesExample Key: IGNITE-7593 URL: https://issues.apache.org/jira/browse/IGNITE-7593 Project: Ignite Issue Type: Task Components: ml Affects Versions: 2.4 Reporter: Oleg Ignatenko Data currently used in {{DecisionTreesExample}} looks not quite optimal: # It is large, as evidenced in the warning in javadocs: "It is recommended to start at least one node prior to launching this example if you intend to run it with default memory settings." # It makes example run for quite a long time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7559) Adjust documentation for ML module at readme.io
Oleg Ignatenko created IGNITE-7559: -- Summary: Adjust documentation for ML module at readme.io Key: IGNITE-7559 URL: https://issues.apache.org/jira/browse/IGNITE-7559 Project: Ignite Issue Type: Sub-task Components: documentation, ml Affects Versions: 2.4 Reporter: Oleg Ignatenko Reflect changes made to github per IGNITE-7509 in respective docs at readme.io. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7488) fix javadoc and license check failures introduced per IGNITE-7284
Oleg Ignatenko created IGNITE-7488: -- Summary: fix javadoc and license check failures introduced per IGNITE-7284 Key: IGNITE-7488 URL: https://issues.apache.org/jira/browse/IGNITE-7488 Project: Ignite Issue Type: Bug Affects Versions: 2.4 Reporter: Oleg Ignatenko fix javadoc and license check failures introduced per IGNITE-7284. One example is: {noformat} [15:22:32][Step 7/7] [WARNING] Javadoc Warnings [15:22:32][Step 7/7] [WARNING] /data/teamcity/work/bd85361428dcdb1/modules/core/src/main/java/org/apache/ignite/IgniteLogger.java:97: warning - @implSpec is an unknown tag. [15:22:32][Step 7/7] [WARNING] /data/teamcity/work/bd85361428dcdb1/modules/core/src/main/java/org/apache/ignite/IgniteLogger.java:117: warning - @implSpec is an unknown tag. [15:22:32][Step 7/7] [WARNING] /data/teamcity/work/bd85361428dcdb1/modules/core/src/main/java/org/apache/ignite/IgniteLogger.java:137: warning - @implSpec is an unknown tag. [15:22:32][Step 7/7] [WARNING] /data/teamcity/work/bd85361428dcdb1/modules/core/src/main/java/org/apache/ignite/IgniteLogger.java:168: warning - @implSpec is an unknown tag. [15:22:32][Step 7/7] [WARNING] /data/teamcity/work/bd85361428dcdb1/modules/core/src/main/java/org/apache/ignite/IgniteLogger.java:199: warning - @implSpec is an unknown tag. {noformat} and there may be others. Link for Teamcity task to verify that there are no violations: https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_LicensesJavadoc -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7454) Wrong package in IgniteExamplesMLTestSuite after it was moved per IGNITE-7203
Oleg Ignatenko created IGNITE-7454: -- Summary: Wrong package in IgniteExamplesMLTestSuite after it was moved per IGNITE-7203 Key: IGNITE-7454 URL: https://issues.apache.org/jira/browse/IGNITE-7454 Project: Ignite Issue Type: Bug Components: examples, ml Reporter: Oleg Ignatenko Wrong package in IgniteExamplesMLTestSuite after it was moved per IGNITE-7203: \{{org.apache.ignite.ml.testsuites}}. Also, it is not added to the list in {{IgniteExamplesSelfTestSuite{{ which is supposed to run all examples self-tests. Change to correct package: \{{org.apache.ignite.testsuites}} and add to main testsuite. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7348) Create MLP benchmarks
Oleg Ignatenko created IGNITE-7348: -- Summary: Create MLP benchmarks Key: IGNITE-7348 URL: https://issues.apache.org/jira/browse/IGNITE-7348 Project: Ignite Issue Type: Sub-task Components: ml, yardstick Reporter: Oleg Ignatenko We want to start tracking our performance to avoid performance degradation. Also we need some performance comparison with other ml libs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7331) create MLP examples
Oleg Ignatenko created IGNITE-7331: -- Summary: create MLP examples Key: IGNITE-7331 URL: https://issues.apache.org/jira/browse/IGNITE-7331 Project: Ignite Issue Type: Sub-task Components: examples, ml Reporter: Oleg Ignatenko create examples for local and distributed MLP in order to help project users see how to use these features -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7214) performance measurement for FCM and KNN algorithms
Oleg Ignatenko created IGNITE-7214: -- Summary: performance measurement for FCM and KNN algorithms Key: IGNITE-7214 URL: https://issues.apache.org/jira/browse/IGNITE-7214 Project: Ignite Issue Type: Task Components: ml, yardstick Reporter: Oleg Ignatenko We want to start tracking our performance of Fuzzy c-means (FCM) and k nearest neighbours (KNN) to avoid performance degradation. Also we need some performance comparison with other ml libs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7118) performance measurement for distributed k-means clustering
Oleg Ignatenko created IGNITE-7118: -- Summary: performance measurement for distributed k-means clustering Key: IGNITE-7118 URL: https://issues.apache.org/jira/browse/IGNITE-7118 Project: Ignite Issue Type: Bug Components: ml, yardstick Reporter: Oleg Ignatenko We want to start tracking our performance to avoid performance degradation. Also we need some performance comparison with other ml libs. Initial draft for this benchmark was made per IGNITE-6123 (class {{IgniteKMeansDistributedClustererBenchmark}}) but it currently fails because of the implementation issues reported in IGNITE-6849 so it is excluded. Enable this benchmark and re-test it using codebase that has has corrected implementation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7097) performance measurement for SparseDistributedMatrix multiplication
Oleg Ignatenko created IGNITE-7097: -- Summary: performance measurement for SparseDistributedMatrix multiplication Key: IGNITE-7097 URL: https://issues.apache.org/jira/browse/IGNITE-7097 Project: Ignite Issue Type: Task Components: ml, yardstick Reporter: Oleg Ignatenko We want to start tracking our performance to avoid performance degradation. Also we need some performance comparison with other ml libs. Initial draft for this benchmark was made per IGNITE-6123 (class {{IgniteSparseDistributedMatrixMulBenchmark}}) but it currently hangs so it is excluded. Find a way to do it right. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6872) Linear regression should implement Model API
Oleg Ignatenko created IGNITE-6872: -- Summary: Linear regression should implement Model API Key: IGNITE-6872 URL: https://issues.apache.org/jira/browse/IGNITE-6872 Project: Ignite Issue Type: Task Security Level: Public (Viewable by anyone) Components: ml Reporter: Oleg Ignatenko Assignee: Oleg Ignatenko Fix For: 2.4 When linear regression was originally implemented per IGNITE-5012 we had no Model API. Now that this API is available (merged into master with IGNITE-5218) lin regression needs to adapt to implement it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6495) performance measurement of decision trees algorithms
Oleg Ignatenko created IGNITE-6495: -- Summary: performance measurement of decision trees algorithms Key: IGNITE-6495 URL: https://issues.apache.org/jira/browse/IGNITE-6495 Project: Ignite Issue Type: Improvement Components: ml Reporter: Oleg Ignatenko Assignee: Oleg Ignatenko We want to start tracking our performance to avoid performance degradation. Also we need some performance comparison with other ml libs. This task is "extracted" from IGNITE-6123 because implementation of decision trees is not yet integrated into master (IGNITE-5218). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6394) Can't create off-heap matrix with size over 2 Gb (int overflow in DenseOffHeapMatrixStorage.allocateMemory)
Oleg Ignatenko created IGNITE-6394: -- Summary: Can't create off-heap matrix with size over 2 Gb (int overflow in DenseOffHeapMatrixStorage.allocateMemory) Key: IGNITE-6394 URL: https://issues.apache.org/jira/browse/IGNITE-6394 Project: Ignite Issue Type: Bug Components: ml Reporter: Oleg Ignatenko Assignee: Yury Babak In order to reproduce, try to create off-heap matrix with size larger than 2Gb, eg like this: {code}new DenseLocalOffHeapMatrix(17_000, 17_000){code} Above fails: {noformat} java.lang.IllegalArgumentException: null at sun.misc.Unsafe.allocateMemory(Native Method) at org.apache.ignite.internal.util.GridUnsafe.allocateMemory(GridUnsafe.java:1068) at org.apache.ignite.ml.math.impls.storage.matrix.DenseOffHeapMatrixStorage.allocateMemory(DenseOffHeapMatrixStorage.java:219) ... {noformat} (you may need to allow Java get more than 2Gb off-heap memory for that with VM argument like {{-XX:MaxDirectMemorySize=1000g}}) The reason for failure is that int arguments aren't converted to long prior to multiplying in {{DenseOffHeapMatrixStorage.allocateMemory}}: {code}ptr = GridUnsafe.allocateMemory((long)rows * cols * Double.BYTES);{code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6329) add Teamcity task to verify the build of ml examples
Oleg Ignatenko created IGNITE-6329: -- Summary: add Teamcity task to verify the build of ml examples Key: IGNITE-6329 URL: https://issues.apache.org/jira/browse/IGNITE-6329 Project: Ignite Issue Type: Task Components: build, examples Reporter: Oleg Ignatenko Assignee: Oleg Ostanin Add a teamcity task to verify that ml examples build successfully. For an example of how to build refer [this doc at readme.io|https://apacheignite.readme.io/docs/machine-learning]: {quote} * ...Make sure you're using Java 8 or later. * Build and install Apache Ignite Data Fabric from the project's root directory: {noformat}mvn clean install -DskipTests -Dmaven.javadoc.skip=true -P java8{noformat} * Build and install ML Grid from the project's root directory: {noformat}mvn install -Pml -DskipTests -U -pl modules/ml -am{noformat} * ...If you need to build the ML Grid examples from sources execute the commands below: {noformat}cd examples mvn clean package -DskipTests -Pml{noformat} {quote} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6328) in ML examples, class ExampleMatrixStorage doesn't compile
Oleg Ignatenko created IGNITE-6328: -- Summary: in ML examples, class ExampleMatrixStorage doesn't compile Key: IGNITE-6328 URL: https://issues.apache.org/jira/browse/IGNITE-6328 Project: Ignite Issue Type: Bug Components: examples Reporter: Oleg Ignatenko Assignee: Yury Babak As of now compilation fails with message: {noformat}Error:(33, 1) java: org.apache.ignite.examples.ml.math.matrix.ExampleMatrixStorage is not abstract and does not override abstract method accessMode() in org.apache.ignite.ml.math.MatrixStorage{noformat} Per preliminary discussion it was likely broken by [commit 518238473c0a12380cb9ed96c67597d1fe42652c|https://github.com/apache/ignite/commit/518238473c0a12380cb9ed96c67597d1fe42652c]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5018) review and improve javadocs in ML module
Oleg Ignatenko created IGNITE-5018: -- Summary: review and improve javadocs in ML module Key: IGNITE-5018 URL: https://issues.apache.org/jira/browse/IGNITE-5018 Project: Ignite Issue Type: Task Reporter: Oleg Ignatenko Assignee: Oleg Ignatenko Priority: Minor Fix For: 2.0 Review and improve javadocs in Ignite ML module (added per IGNITE-4572). To name a few, add descriptions for constructor parameters in classes {{CacheMatrix}}, {{CacheMatrixStorage}}, {{RandomVector}}. Etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)