Hi Igniters! I made a research into tests that aren't included in any test suite. As TeamCity runs tests by suites so there could be tests that never run on TC. So I tried implementing a simple check for such tests and include it in Ignite's travis config.
The check runs while "mvn test" command and piggy-backs on the maven surefire plugin. I replaced the junit provider with a custom one that checks if a class is a test or a suite (there are some Ignite specific stuff), marks tests that are in suites and raises an exception if there are non-suited tests. It's implemented as a part of maven command so it runs for every module separately. I've prepared draft PR with this check: https://github.com/apache/ignite/pull/8367 Travis check report is here: https://travis-ci.org/github/apache/ignite/jobs/737046387 As It's a draft, so I skip some maven configuration steps for a while. Also I run the check only for the core module. But I have some results that want to discuss before continue the work: 1. Currently in the core module there are 53 tests that aren't part of any test suite. I'm not sure about the reason for every test. So I just put below a list of the tests and last contributor to a file that contains a test. 2. Some tests are located in the core module, but suites are in a different, for example ignite-indexing suite IgniteCacheQuerySelfTestSuite3 contains only tests written in the core module, and none from the indexing module. Also there are suites in spring, uri-deploy, zookeeper modules. In my PR I've just copied the test suites to the core module. 3. Some test classes are named with the "Abstract" suffix but don't have the corresponding modifier (for example, IgniteTxTimeoutAbstractTest). So, I add the modifier for every such file if it's not a part of any suite. What do you think about this check? If Ignite needs it, let's discuss next things: 1. Mark tests that should never be in any suite by some reason; 2. Fix the missed tests; 3. How to declare suites that contains tests from a different module; 4. How to check if TC runs all suites. List of non-suited tests in the core module: [email protected]: GridTcpCommunicationSpiLogTest [email protected]: IgniteCacheClientMultiNodeUpdateTopologyLockTest CacheClientsConcurrentStartTest IgniteOutOfMemoryPropagationTest GridCacheP2PUndeploySelfTest GridCacheRebalancingOrderingTest IgniteMassLoadSandboxTest PageLockTrackerMXBeanImplTest IgniteBinaryMetadataUpdateNodeRestartTest CacheLockCandidatesThreadTest GridMBeanBaselineTest RendezvousAffinityFunctionSimpleBenchmark [email protected]: IgnitePdsNoSpaceLeftOnDeviceTest [email protected]: GridCacheOnCopyFlagReplicatedSelfTest GridCacheOnCopyFlagLocalSelfTest GridCacheReplicatedAtomicReferenceMultiNodeTest GridCacheReplicatedMarshallerTxTest GridCacheReplicatedTxConcurrentGetTest GridCacheOnCopyFlagTxPartitionedSelfTest GridCacheReplicatedTxReadTest GridCachePartitionedAtomicReferenceMultiNodeTest GridCacheOnCopyFlagAtomicSelfTest [email protected]: GridActivateExtensionTest IgniteChangeGlobalStateCacheTest IgniteChangeGlobalStateTest IgniteChangeGlobalStateServiceTest IgniteChangeGlobalStateDataStructureTest [email protected]: CacheEntryProcessorCopySelfTest MemoryLeaksOnRestartNodeTest GridCacheAtomicPreloadSelfTest WalCompactionAfterRestartTest IgniteCacheConcurrentPutGetRemove GridIoManagerBenchmark0 [email protected]: GridLongRunningInitNewCrdFutureDiagnosticsTest GridCacheMultithreadedFailoverAbstractTest [email protected]: GridCacheBinaryObjectsAtomicOnheapSelfTest GridCacheBinaryObjectsAtomicNearDisabledOnheapSelfTest GridCacheBinaryObjectsPartitionedOnheapSelfTest GridCacheBinaryObjectsPartitionedNearDisabledOnheapSelfTest [email protected]: IgnitePartitionedLockSelfTest [email protected]: IgniteStableBaselineCachePutAllFailoverTest IgniteStableBaselineCacheRemoveFailoverTest [email protected]: IgniteCacheAtomicOnheapExpiryPolicyTest IgniteCacheAtomicLocalOnheapExpiryPolicyTest GridCacheReplicatedOnheapFullApiSelfTest GridCacheBinaryObjectsLocalOnheapSelfTest [email protected]: GridCacheTtlManagerEvictionSelfTest [email protected]: CommonPoolStarvationCheckpointTest [email protected]: RemoveAllDeadlockTest [email protected]: FullyConnectedComponentSearcherTest [email protected]: IgniteDataStructuresNoClassOnServerTest [email protected]: ReliableChannelTest ThinClientPartitionAwarenessDiscoveryTest
