I am developing a datagrid using Ignite 2.1 to store a large size table
(~25G).
I will be using 8 nodes with 12G RAM.
I am currently developing in my local machine and trying to understand the
cache load mechanism, printed metrics and eviction policy.

Based on the below metrics, the heap size keeps increasing. I thought the
default is off-heap and expected the Non heap memory to increase.
I haven't used setOnheapCacheEnabled in code, so that shouldnt be enabled.
*Metrics for local node (to disable set 'metricsLogFrequency' to 0)*
*    ^-- Node [id=782e9494, name=somegrid, uptime=00:09:30:317]*
*    ^-- H/N/C [hosts=1, nodes=1, CPUs=4]*
*    ^-- CPU [cur=1.7%, avg=1.43%, GC=0%]*
*    ^-- PageMemory [pages=2250]*
*    ^-- Heap [used=4921MB, free=19.9%, comm=6144MB]*
*    ^-- Non heap [used=76MB, free=-1%, comm=78MB]*
*    ^-- Public thread pool [active=0, idle=0, qSize=0]*
*    ^-- System thread pool [active=0, idle=6, qSize=0]*
*    ^-- Outbound messages queue [size=0]*

1) Am I missing something about off-heap vs on-heap or is my interpretation
on the printed metrics incorrect?
2) Another question is about Eviction policy during initial data load -
Since I am running in my local machine(to test code before pushing to
multiple servers), I dont have RAM capacity to load all of the data.
I ran the application in local to see if eviction policy would kick-in
during initial data load, but I received java.lang.OutOfMemoryError: Java
heap space error.
Shouldn't eviction process remove data from page when 80% is reached?
Is eviction policy monitor not started at loadCache or is there an issue
with my configuration (given below)?

Code snippet-- The undisclosed section is just trivial part of the settings

CacheConfiguration<String, String> cacheConfiguration = new
CacheConfiguration<>();
        cacheConfiguration.setName(CACHE_NAME)
                .setCacheMode(CacheMode.PARTITIONED)
                .setMemoryPolicyName(DEFAULT_MEMORY_POLICY)
                ....................
                .setCacheStoreFactory(FactoryBuilder.factoryOf(dataLoader));

        MemoryPolicyConfiguration memoryPolicyConfiguration = new
MemoryPolicyConfiguration()
                .setName(DEFAULT_MEMORY_POLICY)
                .setInitialSize(STARTUP_OFFHEAP_MEMORY_SIZE) //3G
                .setPageEvictionMode(DataPageEvictionMode.RANDOM_2_LRU);

        MemoryConfiguration memCfg = new MemoryConfiguration()
                .setDefaultMemoryPolicyName(DEFAULT_MEMORY_POLICY)
                .setMemoryPolicies(memoryPolicyConfiguration);
public class DataLoader extends CacheStoreAdapter<String, String>
implements Serializable {

    @Autowired
    private transient JdbcTemplate jdbcTemplate;

    @Value("${load.all.query}") //select key, vaue from table
    private String loadAllQuery;

    @Value("${store.one.query}")
    private String storeOneQuery;

 .................

    @Override
    public void loadCache(IgniteBiInClosure<String, String> closure,
Object... args) {
        log.info("In loadcache");
        jdbcTemplate.query(loadAllQuery, (rs, rownNum) -> {
            closure.apply(rs.getString(1), rs.getString(2));
            return null;
        });
    }
}

Thanks,
Yasser

Reply via email to