Creation of each cache requires creation and initialization of internal data structures that leads to increased pressure to GC. Could you enable GC logs and look at result. I think you will find long GC pauses.
In order to reduce memory consumption by created caches on creation stage we can do the following: - Decrease cache start size (see cacheConfiguration.setStartSize(), default value is 1 500 000). It safe but can lead to some performance penalty during dynamic increasing cache size; - For atomic caches we can decrease size of deferred delete queue size via JVM parameter (-DIGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE=<your value>, default value is 200 000 entries per partition). It's static configuration and can't be changed at runtime; - Partitions number also affect memory consumption and cache creation time. But it affect also data distribution so should be configured carefully. Caches with default configuration uses RendevousAffinityFunction ith 1024 partitions. This parameter can't be changed after cache creation and start. Does it work for you? On Tue, Apr 11, 2017 at 12:53 PM, ctranxuan <cedric.tranx...@streamdata.io> wrote: > Hi, > We are trying for test purposes to create lots of cache on a machine. > Something like: > > for (int i = 0; i < 10000; i++) { > IgniteCache<Long, Snapshot> cache; > cache = ignite.<Long, Snapshot>getOrCreateCache("cache-" + > i).withAsync(); > LOGGER.info("starting to read cache #" + cache.getName()); > } > > What we are noticing is that after ~200 caches created, the cache creation > becomes slower and slower. For instance, when reaching the 2000th cache, it > takes between 2-3 seconds. For instance, here some logs: > > 09:34:27.857 starting to read cache #cache-2087 > 09:34:29.621 starting to read cache #cache-2088 > 09:34:31.450 starting to read cache #cache-2089 > 09:34:33.127 starting to read cache #cache-2090 > > That's may be a naive way of doing that and may be a naive question: but is > it normal that the more cache is created, the more time the creation takes? > > The program is run on a 12GB RAM machine with 4 CPUs with a Java 8 > (1.8.0_121). > > Thanks in advance for the answers! > > > > > -- > View this message in context: > http://apache-ignite-users.70518.x6.nabble.com/Lots-of-cache-creation-become-slow-tp11875.html > Sent from the Apache Ignite Users mailing list archive at Nabble.com.