Re: Full API coverage enhancement

2016-03-01 Thread Artem Shutak
>> Is this suite turned on by default? Yes, see "Ignite Cache Full API Basic Config Variations" [1]. [1] http://204.14.53.151/viewType.html?buildTypeId=IgniteTests_IgniteCacheFullApiNewBasicCfgDynamicCacheStartNotReady&branch_IgniteTests=%3Cdefault%3E&tab=buildTypeStatusDiv Thanks, -- Artem --

Re: Full API coverage enhancement

2016-03-01 Thread Artem Shutak
-- Artem -- On Tue, Mar 1, 2016 at 1:31 AM, Dmitriy Setrakyan wrote: > Thanks Artem! I think this test suite should go a long way towards removing > issues with different configuration property permutations. > > Is this suite turned on by default? > > D. > > On Mon, Feb 29, 2016 at 7:07 AM, Arte

Re: Full API coverage enhancement

2016-02-29 Thread Dmitriy Setrakyan
Thanks Artem! I think this test suite should go a long way towards removing issues with different configuration property permutations. Is this suite turned on by default? D. On Mon, Feb 29, 2016 at 7:07 AM, Artem Shutak wrote: > Igniters, > > I'm glad to announce that Configuration Variations

Re: Full API coverage enhancement

2016-02-29 Thread Artem Shutak
Igniters, I'm glad to announce that Configuration Variations test framework has been merged to master and is ready to use. The framework provides an opportunity to write a test-case once and to run it in many Ignite and Cache configuration variations, for different Ignite topologies (with and wit

Re: Full API coverage enhancement

2016-02-10 Thread Dmitriy Setrakyan
Agree about separation. I think we need to define some internal permutations that keep coming up with bugs. I can start the list here: 1. Serializable, Externalizable, neither. 1. We should run the whole suite 3 times, once for each serialization mode. Having 2 or 3 nodes in the cluste

Re: Full API coverage enhancement

2016-02-10 Thread Alexey Goncharuk
Dmitriy, the size of the circular buffer can be decreased by setting IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE system property. I was wondering what our next steps will be after implementing the framework. From the initial discussion I thought we want to convert some older tests to this new framewor

Re: Full API coverage enhancement

2016-02-08 Thread Dmitriy Setrakyan
t; > > Date: Mon, 8 Feb 2016 17:45:33 +0300 > > Subject: Re: Full API coverage enhancement > > From: ashu...@gridgain.com > > To: dev@ignite.apache.org > > > > Sergey, > > > > I think we should start more caches, like 1000 in one time. But we have >

RE: Full API coverage enhancement

2016-02-08 Thread Andrey Kornev
he size is set to 1.5M entries? Thanks Andrey > Date: Mon, 8 Feb 2016 17:45:33 +0300 > Subject: Re: Full API coverage enhancement > From: ashu...@gridgain.com > To: dev@ignite.apache.org > > Sergey, > > I think we should start more caches, like 1000 in one time. But we have

Re: Full API coverage enhancement

2016-02-08 Thread Artem Shutak
Alexey, First of all, it's strange to have disk-dependent tests for in-memory data fabric ;) I'm sure we can solve this issue without bottleneck. I see many technical issues with that approach. Another bad thing about the approach is we don't know how to find test result by configuration. I thin

Re: Full API coverage enhancement

2016-02-08 Thread Alexey Kuznetsov
How about such approach: Generate all permutation descriptors and save them to database. On each agent start some process that will grab from DB some descriptors (lets say 100) mark them as "in progress" and execute them. Other agents will grab remaining "not in progress" descriptors and do the sa

Re: Full API coverage enhancement

2016-02-08 Thread Sergey Kozlov
1000 caches x 50MB = 50GB heap. Do we really have >50GB RAM on each agents? On Mon, Feb 8, 2016 at 5:45 PM, Artem Shutak wrote: > Sergey, > > I think we should start more caches, like 1000 in one time. But we have to > have enough memory on our TC agents. As I know, empty cache is require > abou

Re: Full API coverage enhancement

2016-02-08 Thread Artem Shutak
Sergey, I think we should start more caches, like 1000 in one time. But we have to have enough memory on our TC agents. As I know, empty cache is require about 50 mb (without indexing), am I right? You are right, I keep in mind that *backups* and *REPLICATED* mode make no sense together, but we s

Re: Full API coverage enhancement

2016-02-08 Thread Sergey Kozlov
Hi Artem It's good idea to create 20-30 cache configurations at once and then to iterate tests over those caches in parallel (but make sure that cache names are unique). Another point that some combinations make no sense like *backups *and *REPLICATED *cache On Mon, Feb 8, 2016 at 5:07 PM, Artem

Re: Full API coverage enhancement

2016-02-08 Thread Artem Shutak
Hi all, I have an update. I've started from *CacheConfiguration* permutations. I wrote out a list with all CacheConfiguration setters and filtered it with Alexey G. Finally we have: 1. CacheMode - 3 variants 2. CacheAtomicityMode - 2 variants 3. CacheMemoryMode - 3 variants 4. setLo

Re: Full API coverage enhancement

2016-02-03 Thread Semyon Boikov
Artem, One more thing for new tests: I think test should start both server and client nodes and use Ignite API from all nodes. On Wed, Feb 3, 2016 at 6:40 PM, Artem Shutak wrote: > Dmitriy, > > Actually, I don't have a list with all the permutations. > > At first, we need to split in our discus

Re: Full API coverage enhancement

2016-02-03 Thread Dmitriy Setrakyan
Thanks Artem! In your view, how many combinations of the configuration properties will be tested together. I don’t think we can afford to test every possible combination. On Wed, Feb 3, 2016 at 7:40 AM, Artem Shutak wrote: > Dmitriy, > > Actually, I don't have a list with all the permutations. >

Re: Full API coverage enhancement

2016-02-03 Thread Alexey Kuznetsov
Artem, could you describe (in general) how are you going to describe permutations? Are you going to run tests with all possible combinations of all possible properties? Along time ago I implemented smth. similar (not in java) and I created framework where I could describe nested rules of what I

Re: Full API coverage enhancement

2016-02-03 Thread Artem Shutak
Dmitriy, Actually, I don't have a list with all the permutations. At first, we need to split in our discussion test cases and Ignite configuration which should be covered. For example, new Full Api test cases for cache are based on old Full Api test cases. So, it need to think what the test case

Re: Full API coverage enhancement

2016-02-03 Thread Dmitriy Setrakyan
Artem, I think it is best to specify all the permutations here, so others can make additional suggestions. Otherwise, we cannot get a full picture. Thanks, D. On Wed, Feb 3, 2016 at 2:02 AM, Artem Shutak wrote: > Igniters, > > I thought a little bit more and think we need to add a support for t

Re: Full API coverage enhancement

2016-02-03 Thread Artem Shutak
Igniters, I thought a little bit more and think we need to add a support for the following permutations too (I've added these to the jira description): - We should also test with Serializable, Externalizable, and plain Pojos for keys and values. - The Pojo in the above test should contain an enum

Re: Full API coverage enhancement

2016-02-02 Thread Artem Shutak
Dmitriy, There is a branch at my fork and a pull request at Ignite. See comment about pull request at the ticket (PR-446). But I have to notice that the branch under hard development and you it can not work (have compilation or test errors) at some moments. Good luck! -- Artem -- On Tue, Feb 2

Re: Full API coverage enhancement

2016-02-02 Thread Dmitriy Setrakyan
Artem, This is great. I have noticed from the ticket that you have created some initial suite already. Is there a branch I can look at it? D. On Tue, Feb 2, 2016 at 10:02 AM, Artem Shutak wrote: > Igniters, > > I'm working on an enhancement of Full API coverage [1] [2]. > > Ignite already has

Full API coverage enhancement

2016-02-02 Thread Artem Shutak
Igniters, I'm working on an enhancement of Full API coverage [1] [2]. Ignite already has Full API test, but currently it's hard to test all configuration combinations. Feel free to add comments in the jira if you have any thought. [1] https://issues.apache.org/jira/browse/IGNITE-2521 [2] https: