Andrey, I think the reason for the waste is some sort of internal ring-buffer allocation which can be turned off.
Alexey G, can you please comment? D. On Mon, Feb 8, 2016 at 10:21 AM, Andrey Kornev <andrewkor...@hotmail.com> wrote: > 50MB for an empty cache! That's a lot of emptiness! > > The Ignite team's recommendation to have one key/value type per cache now > seems a little more dubious and difficult to follow in practice. > > Just curious, what is the reason for such waste? Is it because by default > the initial cache size is set to 1.5M entries? > > Thanks > Andrey > > > Date: Mon, 8 Feb 2016 17:45:33 +0300 > > Subject: Re: Full API coverage enhancement > > From: ashu...@gridgain.com > > To: dev@ignite.apache.org > > > > Sergey, > > > > I think we should start more caches, like 1000 in one time. But we have > to > > have enough memory on our TC agents. As I know, empty cache is require > > about 50 mb (without indexing), am I right? > > > > You are right, I keep in mind that *backups* and *REPLICATED* mode make > no > > sense together, but we still have to test it in one node and multi node > > casesю > > > > Any other *no sense* combinations? > > > > I forgot about custom BinaryConfiguration at IgniteConfiguration for > > BinaryMarshaller. So, at least 6 IgniteConfigurations. > > > > -- Artem -- > > > > On Mon, Feb 8, 2016 at 5:17 PM, Sergey Kozlov <skoz...@gridgain.com> > wrote: > > > > > Hi Artem > > > > > > It's good idea to create 20-30 cache configurations at once and then to > > > iterate tests over those caches in parallel (but make sure that cache > names > > > are unique). > > > Another point that some combinations make no sense like *backups *and > > > *REPLICATED > > > *cache > > > > > > > > > On Mon, Feb 8, 2016 at 5:07 PM, Artem Shutak <ashu...@gridgain.com> > wrote: > > > > > > > Hi all, > > > > > > > > I have an update. > > > > > > > > I've started from *CacheConfiguration* permutations. I wrote out a > list > > > > with all CacheConfiguration setters and filtered it with Alexey G. > > > > > > > > Finally we have: > > > > > > > > 1. CacheMode - 3 variants > > > > 2. CacheAtomicityMode - 2 variants > > > > 3. CacheMemoryMode - 3 variants > > > > 4. setLoadPreviousValue - 2 variants > > > > 5. setReadFromBackup - 2 variants > > > > 6. setStoreKeepBinary - 2 variants > > > > 7. setRebalanceMode - SYNC and ASYNC (2 variants) > > > > 8. setSwapEnabled - 2 variants > > > > 9. setCopyOnRead - 2 variants > > > > 10. NearConfiguration disabled / default NearConfiguration / > custom > > > > NearConfiguration - 3 variants > > > > 11. With and without a complex parameter. The complex parameter > > > defines > > > > not-default Eviction policy and filter, cache store configuration > > > > (storeFactory and storeSessionListenerFactory), rebalancing > > > > configuration, > > > > affinity function, offHeapMaxMemory, interceptor, topology > validator > > > and > > > > CacheEntryListener. > > > > > > > > I've run 123 Cache Full Api test cases for all permutations of > parameters > > > > 1-9 and got 256896 test cases (1152 configuration variants * 123 test > > > > cases). All these tests take 4 hours 40 minutes. Not all tests pass, > so > > > MAY > > > > BE when all tests will pass it will take less time (3,5 hours for > > > example). > > > > > > > > As we can see the tests take a lot of time. > > > > > > > > The following permutation should be supported too: > > > > > > > > 1. Nodes count and Bakups count - 1 node and 0 backup, 3 nodes > and 1 > > > > backups, 4 nodes and 2 backups - 3 variants > > > > 2. Client and Server nodes - 2 variants > > > > 3. Indexing enabled and disabled for cache - 2 variants > > > > 4. IgniteConfiguration permutations - how many variants? I see at > > > least > > > > 4 (2 Marshallers, P2P). > > > > > > > > Plus we need add new test cases to test different key and value > types and > > > > etc. > > > > > > > > So, we need multiply more then 3,5-4,5 hours on ~250. If we will > split > > > all > > > > tests on 250 suites and run on all 30 TC agents it will take about > 30-40 > > > > hours. Ok, we can do it during weekends. > > > > > > > > I think it will take too much time. > > > > > > > > As an option we can start a cache for each configuration and run > tests > > > > concurrently. But we need to implement this opportunity in our test > > > > framework. > > > > > > > > Any other thoughts how we can decrease time for tests? > > > > > > > > Thanks, > > > > -- Artem -- > > > > > > > > On Thu, Feb 4, 2016 at 8:43 AM, Semyon Boikov <sboi...@gridgain.com> > > > > wrote: > > > > > > > > > Artem, > > > > > > > > > > One more thing for new tests: I think test should start both > server and > > > > > client nodes and use Ignite API from all nodes. > > > > > > > > > > On Wed, Feb 3, 2016 at 6:40 PM, Artem Shutak <ashu...@gridgain.com > > > > > > wrote: > > > > > > > > > > > Dmitriy, > > > > > > > > > > > > Actually, I don't have a list with all the permutations. > > > > > > > > > > > > At first, we need to split in our discussion test cases and > Ignite > > > > > > configuration which should be covered. > > > > > > > > > > > > For example, new Full Api test cases for cache are based on old > Full > > > > Api > > > > > > test cases. So, it need to think what the test cases was not > covered > > > > > > before. > > > > > > > > > > > > About Ignite configurations, I'm going to add permutation for > each > > > > > > IgniteConfiguration and CacheConfiguration property. > > > > > > > > > > > > By the way, the jira contains the following list of permutation > (feel > > > > > free > > > > > > to add something): > > > > > > > > > > > > The following tests should be added (for functional blocks): > > > > > > > > > > > > 1. Interceptor > > > > > > 2. Queries: continuous, scan, SQL, fields and text queries. > > > > > > 3. cache events > > > > > > 4. We should also test with Serializable, Externalizable, and > > > plain > > > > > > Pojos for keys and values. > > > > > > 5. The Pojo in the above test should contain an enum value > > > > > > 6. We should also test Enums as keys and Enums as values > > > > > > 7. All operations should have single-key and multi-key > operations > > > > > > > > > > > > New tests should cover all combinations for following properties: > > > > > > > > > > > > 1. cache modes > > > > > > 2. operation from client nodes and server nodes > > > > > > 3. store enabled/disabled > > > > > > 4. evicts sycn/non-sync > > > > > > 5. eviction policies > > > > > > 6. near on/off > > > > > > 7. marshallers (+ Binary marshaller with different mappers) > > > > > > 8. keys and values - externalizable, serializable, > binaryzable, > > > > "none > > > > > of > > > > > > previous" > > > > > > 9. classes available on servers: true/false > > > > > > 10. Peer loading on/off > > > > > > 11. Affinity functions > > > > > > 12. expiry policies > > > > > > > > > > > > > > > > > > Thanks, > > > > > > -- Artem -- > > > > > > > > > > > > On Wed, Feb 3, 2016 at 6:14 PM, Dmitriy Setrakyan < > > > > dsetrak...@apache.org > > > > > > > > > > > > wrote: > > > > > > > > > > > > > Artem, I think it is best to specify all the permutations > here, so > > > > > others > > > > > > > can make additional suggestions. Otherwise, we cannot get a > full > > > > > picture. > > > > > > > > > > > > > > Thanks, > > > > > > > D. > > > > > > > > > > > > > > On Wed, Feb 3, 2016 at 2:02 AM, Artem Shutak < > ashu...@gridgain.com > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > > > I thought a little bit more and think we need to add a > support > > > for > > > > > the > > > > > > > > following permutations too (I've added these to the jira > > > > > description): > > > > > > > > - We should also test with Serializable, Externalizable, and > > > plain > > > > > > Pojos > > > > > > > > for keys and values. > > > > > > > > - The Pojo in the above test should contain an enum value > > > > > > > > - We should also test Enums as keys and Enums as values > > > > > > > > - All operations should have single-key and multi-key > operations > > > > > > > > > > > > > > > > Maybe someone see any other permutation to be supported? > > > > > > > > > > > > > > > > -- Artem -- > > > > > > > > > > > > > > > > On Tue, Feb 2, 2016 at 10:05 PM, Artem Shutak < > > > > ashu...@gridgain.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Dmitriy, > > > > > > > > > > > > > > > > > > There is a branch at my fork and a pull request at Ignite. > See > > > > > > comment > > > > > > > > > about pull request at the ticket (PR-446). > > > > > > > > > > > > > > > > > > But I have to notice that the branch under hard > development and > > > > you > > > > > > it > > > > > > > > can > > > > > > > > > not work (have compilation or test errors) at some moments. > > > > > > > > > > > > > > > > > > Good luck! > > > > > > > > > > > > > > > > > > -- Artem -- > > > > > > > > > > > > > > > > > > On Tue, Feb 2, 2016 at 9:45 PM, Dmitriy Setrakyan < > > > > > > > dsetrak...@apache.org > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > >> Artem, > > > > > > > > >> > > > > > > > > >> This is great. I have noticed from the ticket that you > have > > > > > created > > > > > > > some > > > > > > > > >> initial suite already. Is there a branch I can look at it? > > > > > > > > >> > > > > > > > > >> D. > > > > > > > > >> > > > > > > > > >> On Tue, Feb 2, 2016 at 10:02 AM, Artem Shutak < > > > > > ashu...@gridgain.com > > > > > > > > > > > > > > > >> wrote: > > > > > > > > >> > > > > > > > > >> > Igniters, > > > > > > > > >> > > > > > > > > > >> > I'm working on an enhancement of Full API coverage [1] > [2]. > > > > > > > > >> > > > > > > > > > >> > Ignite already has Full API test, but currently it's > hard to > > > > > test > > > > > > > all > > > > > > > > >> > configuration combinations. > > > > > > > > >> > > > > > > > > > >> > Feel free to add comments in the jira if you have any > > > thought. > > > > > > > > >> > > > > > > > > > >> > [1] https://issues.apache.org/jira/browse/IGNITE-2521 > > > > > > > > >> > [2] > > > > > > > > >> > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/Implementing+Tests > > > > > > > > >> > > > > > > > > > >> > Thanks, > > > > > > > > >> > -- Artem -- > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Sergey Kozlov > > > GridGain Systems > > > www.gridgain.com > > > > >
