Hi Fvyaba,
In order to avoid memory overhead per table, you should create all tables
as part of the same cache group:
https://apacheignite.readme.io/docs/cache-groups
D.
On Mon, Mar 26, 2018 at 7:26 AM, aealexsandrov <aealexsand...@gmail.com>
wrote:
> Hi Fvyaba,
>
> I investigated your example. In your code you are going to create new cache
> every time when you are going to create new table. Every new cache will
> have
> some memory overhead. Next code can help you to get the average allocated
> memory:
>
> try (IgniteCache<?, ?> cache =
> ignite.getOrCreateCache(defaultCacheCfg)) {
> for(int i = 1; i < 100; i++) {
> cache.query(new SqlFieldsQuery(String.format(
> "CREATE TABLE TBL_%s (id BIGINT,uid VARCHAR,PRIMARY
> KEY(id))", i)));
> System.out.println("Count " + i + "
> -------------------------------------------------------------");
> for (DataRegionMetrics metrics :
> ignite.dataRegionMetrics()) {
> System.out.println(">>> Memory Region Name: " +
> metrics.getName());
> System.out.println(">>> Allocation Rate: " +
> metrics.getAllocationRate());
> System.out.println(">>> Allocated Size Full: " +
> metrics.getTotalAllocatedSize());
> System.out.println(">>> Allocated Size avg: " +
> metrics.getTotalAllocatedSize() / i);
> System.out.println(">>> Physical Memory Size: " +
> metrics.getPhysicalMemorySize());
> }
> }
> }
>
> On my machine with default settings I got next:
>
> >>> Memory Region Name: Default_Region
> >>> Allocation Rate: 3419.9666
> >>> Allocated Size Full: 840491008
> >>> Allocated Size avg: 8489808
> >>> Physical Memory Size: 840491008
>
> So it's about 8mb per cache (so if you will have 3.2 GB then you can create
> about 400 caches). I am not sure is it ok but you can do next to avoid
> org.apache.ignite.IgniteCheckedException: Out of memory in data region:
>
> 1)Increase the max value of available off-heap memory:
>
>
> <property name="dataStorageConfiguration">
> <bean
> class="org.apache.ignite.configuration.DataStorageConfiguration">
>
> <property name="defaultDataRegionConfiguration">
> <bean
> class="org.apache.ignite.configuration.DataRegionConfiguration">
> <property name="name" value="Default_Region"/>
> <property name="maxSize" value="#{1L * 1024 * 1024
> *
> 1024}"/> //HERE
> <property name="metricsEnabled" value="true"/>
> </bean>
> </property>
> </bean>
> </property>
>
> 2)Use persistence (or swaping space):
>
> <property name="dataStorageConfiguration">
> <bean
> class="org.apache.ignite.configuration.DataStorageConfiguration">
>
> <property name="defaultDataRegionConfiguration">
> <bean
> class="org.apache.ignite.configuration.DataRegionConfiguration">
> <property name="name" value="Default_Region"/>
> <property name="maxSize" value="#{1L * 1024 * 1024
> *
> 1024}"/>
> <property name="metricsEnabled" value="true"/>
> <property name="persistenceEnabled" value="true"/>
> //THIS ONE
> </bean>
> </property>
> </bean>
> </property>
>
> Read more about it you can here:
>
> https://apacheignite.readme.io/docs/distributed-persistent-store
> https://apacheignite.readme.io/v1.0/docs/off-heap-memory
>
> Please try to test next code:
>
> 1) add this to your config:
>
>
> <property name="dataStorageConfiguration">
> <bean
> class="org.apache.ignite.configuration.DataStorageConfiguration">
>
> <property name="defaultDataRegionConfiguration">
> <bean
> class="org.apache.ignite.configuration.DataRegionConfiguration">
> <property name="name" value="Default_Region"/>
> <property name="maxSize" value="#{1L * 1024 * 1024
> *
> 1024}"/>
> <property name="metricsEnabled" value="true"/>
> <property name="persistenceEnabled" value="true"/>
> </bean>
> </property>
> </bean>
> </property>
>
> 2)Run next:
>
> public class example {
> public static void main(String[] args) throws IgniteException {
> try (Ignite ignite =
> Ignition.start("examples/config/example-ignite.xml")) {
> ignite.cluster().active(true);
>
> CacheConfiguration<?, ?> defaultCacheCfg = new
> CacheConfiguration<>("Default_cache").setSqlSchema("PUBLIC");
>
> defaultCacheCfg.setDataRegionName("Default_Region");
>
> try (IgniteCache<?, ?> cache =
> ignite.getOrCreateCache(defaultCacheCfg)) {
> for(int i = 1; i < 1000; i++) {
> //remove old table cache just in case
> cache.query(new SqlFieldsQuery(String.format(
> "DROP TABLE TBL_%s", i)));
> //create new table
> cache.query(new SqlFieldsQuery(String.format(
> "CREATE TABLE TBL_%s (id BIGINT,uid VARCHAR,PRIMARY
> KEY(id))", i)));
> System.out.println("Count " + i + "
> -------------------------------------------------------------");
> for (DataRegionMetrics metrics :
> ignite.dataRegionMetrics()) {
> System.out.println(">>> Memory Region Name: " +
> metrics.getName());
> System.out.println(">>> Allocation Rate: " +
> metrics.getAllocationRate());
> System.out.println(">>> Allocated Size Full: " +
> metrics.getTotalAllocatedSize());
> System.out.println(">>> Allocated Size avg: " +
> metrics.getTotalAllocatedSize() / i);
> System.out.println(">>> Physical Memory Size: " +
> metrics.getPhysicalMemorySize());
> }
> }
> }
>
> ignite.cluster().active(false);
> }
> }
> }
>
>
>
>
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
