I am developing a datagrid using Ignite 2.1 to store a large size table (~25G). I will be using 8 nodes with 12G RAM. I am currently developing in my local machine and trying to understand the cache load mechanism, printed metrics and eviction policy.
Based on the below metrics, the heap size keeps increasing. I thought the default is off-heap and expected the Non heap memory to increase. I haven't used setOnheapCacheEnabled in code, so that shouldnt be enabled. *Metrics for local node (to disable set 'metricsLogFrequency' to 0)* * ^-- Node [id=782e9494, name=somegrid, uptime=00:09:30:317]* * ^-- H/N/C [hosts=1, nodes=1, CPUs=4]* * ^-- CPU [cur=1.7%, avg=1.43%, GC=0%]* * ^-- PageMemory [pages=2250]* * ^-- Heap [used=4921MB, free=19.9%, comm=6144MB]* * ^-- Non heap [used=76MB, free=-1%, comm=78MB]* * ^-- Public thread pool [active=0, idle=0, qSize=0]* * ^-- System thread pool [active=0, idle=6, qSize=0]* * ^-- Outbound messages queue [size=0]* 1) Am I missing something about off-heap vs on-heap or is my interpretation on the printed metrics incorrect? 2) Another question is about Eviction policy during initial data load - Since I am running in my local machine(to test code before pushing to multiple servers), I dont have RAM capacity to load all of the data. I ran the application in local to see if eviction policy would kick-in during initial data load, but I received java.lang.OutOfMemoryError: Java heap space error. Shouldn't eviction process remove data from page when 80% is reached? Is eviction policy monitor not started at loadCache or is there an issue with my configuration (given below)? Code snippet-- The undisclosed section is just trivial part of the settings CacheConfiguration<String, String> cacheConfiguration = new CacheConfiguration<>(); cacheConfiguration.setName(CACHE_NAME) .setCacheMode(CacheMode.PARTITIONED) .setMemoryPolicyName(DEFAULT_MEMORY_POLICY) .................... .setCacheStoreFactory(FactoryBuilder.factoryOf(dataLoader)); MemoryPolicyConfiguration memoryPolicyConfiguration = new MemoryPolicyConfiguration() .setName(DEFAULT_MEMORY_POLICY) .setInitialSize(STARTUP_OFFHEAP_MEMORY_SIZE) //3G .setPageEvictionMode(DataPageEvictionMode.RANDOM_2_LRU); MemoryConfiguration memCfg = new MemoryConfiguration() .setDefaultMemoryPolicyName(DEFAULT_MEMORY_POLICY) .setMemoryPolicies(memoryPolicyConfiguration); public class DataLoader extends CacheStoreAdapter<String, String> implements Serializable { @Autowired private transient JdbcTemplate jdbcTemplate; @Value("${load.all.query}") //select key, vaue from table private String loadAllQuery; @Value("${store.one.query}") private String storeOneQuery; ................. @Override public void loadCache(IgniteBiInClosure<String, String> closure, Object... args) { log.info("In loadcache"); jdbcTemplate.query(loadAllQuery, (rs, rownNum) -> { closure.apply(rs.getString(1), rs.getString(2)); return null; }); } } Thanks, Yasser