Hello!
When you get IgniteOOM, your node is now in incorrect state where behavior is
undefined. You should plan carefully to avoid getting IOOM on nodes, or enable
persistence/page eviction.
Regards,
On 2019/08/15 13:16:26, colinc <colinmcass...@yahoo.co.uk> wrote:
> I am using the Ignite metrics API to monitor memory usage levels in an
> attempt to avoid OOM conditions.
>
> I have found that the metrics API appears to provide reasonably accurate
> figures as entries are being written to the cache - but usage levels do not
> come down again when entries are removed. I have tried removing all entries,
> individual entries and even destroying the cache. I have also tried waiting
> for a significant period of time.
>
> Even when the cache is destroyed, the memory usage figure does not appear to
> drop. In fact, removing entries can even cause an increase in the figure.
> Despite the metrics, it is possible to insert new entries following removal
> of the old ones - indicating that space is in fact available.
>
> A reproducer is below and produces results like:
>
> Out of memory: CachePartialUpdateException after 89MB
> Memory used: 99.39624%
> Memory used: 99.41027%
> Memory used: 99.41406%
>
>
>
> package mytest;
>
> import org.apache.ignite.DataRegionMetrics;
> import org.apache.ignite.Ignite;
> import org.apache.ignite.IgniteCache;
> import org.apache.ignite.Ignition;
> import org.apache.ignite.configuration.DataRegionConfiguration;
> import org.apache.ignite.configuration.DataStorageConfiguration;
> import org.apache.ignite.configuration.IgniteConfiguration;
> import org.apache.ignite.failure.NoOpFailureHandler;
> import org.junit.Test;
>
> import javax.cache.CacheException;
>
> public class MemoryTest {
>
> private static final String CACHE_NAME = "cache";
> private static final String DEFAULT_MEMORY_REGION = "Default_Region";
> private static final long MEM_SIZE = 100L * 1024 * 1024;
>
>
> @Test
> public void testOOM() throws InterruptedException {
> try (Ignite ignite = startIgnite("IgniteMemoryMonitorTest1")) {
> fillDataRegion(ignite);
> IgniteCache<Object, Object> cache =
> ignite.getOrCreateCache(CACHE_NAME);
>
> // Clear all entries from the cache to free up memory
> memUsed(ignite);
> cache.clear(); // Fails here
> cache.put("Key", "Value");
> memUsed(ignite);
>
> cache.destroy();
> Thread.sleep(5000);
> memUsed(ignite);
> }
> }
>
>
> private Ignite startIgnite(String instanceName) {
> IgniteConfiguration cfg = new IgniteConfiguration();
> cfg.setIgniteInstanceName(instanceName);
> cfg.setDataStorageConfiguration(createDataStorageConfiguration());
> cfg.setFailureHandler(new NoOpFailureHandler());
> return Ignition.start(cfg);
> }
>
> private DataStorageConfiguration createDataStorageConfiguration() {
> return new DataStorageConfiguration()
> .setDefaultDataRegionConfiguration(
> new DataRegionConfiguration()
> .setName(DEFAULT_MEMORY_REGION)
> .setInitialSize(MEM_SIZE)
> .setMaxSize(MEM_SIZE)
> .setMetricsEnabled(true));
> }
>
>
> private void fillDataRegion(Ignite ignite) {
> byte[] megabyte = new byte[1024 * 1024];
>
> int storedDataMB = 0;
> try {
> IgniteCache<Object, Object> cache =
> ignite.getOrCreateCache(CACHE_NAME);
> for (int i = 0; i < 200; i++) {
> cache.put(i, megabyte);
> storedDataMB++;
>
> memUsed(ignite);
> }
> } catch (CacheException e) {
> System.out.println("Out of memory: " +
> e.getClass().getSimpleName() + " after " + storedDataMB + "MB");
> }
> }
>
> private void memUsed(Ignite ignite) {
> DataRegionConfiguration defaultDataRegionCfg =
> ignite.configuration()
> .getDataStorageConfiguration()
> .getDefaultDataRegionConfiguration();
> String regionName = defaultDataRegionCfg.getName();
> DataRegionMetrics metrics = ignite.dataRegionMetrics(regionName);
> float usedMem = metrics.getPagesFillFactor() *
> metrics.getTotalAllocatedPages() * metrics.getPageSize();
> float pctUsed = 100 * usedMem / defaultDataRegionCfg.getMaxSize();
> System.out.println("Memory used: " + pctUsed + "%");
> }
> }
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
