Hi, everyone. I started a program 3 days ago on a server with 1GB RAM.
Everything was running smoothly until a couple of hours ago, when the service wasn't responding. When we go check the logs, there is an OOMError with an Ignite stack-trace tagging along. All that I store on that server is stored on Ignite (with read-through and write-through to a MySQL DB). It's fairly light stuff, just some objects with 10+ fields, and it's just 1 cache right now. However, in the logs, the bulk of the operations preceding the OOM error were read-operations that neither created nor updated the data on Ignite. Just performed some queries. And yet, for some reason, there was a HashMap being updated somewhere, which triggered the OOM error. Check it out: Apr 23, 2016 4:35:44 PM org.apache.ignite.logger.java.JavaLogger error SEVERE: Caught unhandled exception in NIO worker thread (restart the node). java.lang.OutOfMemoryError: Java heap space at java.util.HashMap$KeySet.iterator(HashMap.java:912) at java.util.HashSet.iterator(HashSet.java:172) at java.util.Collections$UnmodifiableCollection$1.<init>(Collections.java:1039) at java.util.Collections$UnmodifiableCollection.iterator(Collections.java:1038) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.checkIdle(GridNioServer.java:1489) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:1406) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1280) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:745) Apr 23, 2016 4:35:45 PM org.apache.ignite.logger.java.JavaLogger error SEVERE: Runtime error caught during grid runnable execution: GridWorker [name=grid-nio-worker-0, gridName=null, finished=false, isCancelled=false, hashCode=1645791145, interrupted=false, runner=grid-nio-worker-0-#40%null%] java.lang.OutOfMemoryError: Java heap space at java.util.HashMap$KeySet.iterator(HashMap.java:912) at java.util.HashSet.iterator(HashSet.java:172) at java.util.Collections$UnmodifiableCollection$1.<init>(Collections.java:1039) at java.util.Collections$UnmodifiableCollection.iterator(Collections.java:1038) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.checkIdle(GridNioServer.java:1489) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:1406) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1280) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:745) Apr 23, 2016 4:35:46 PM org.apache.ignite.logger.java.JavaLogger error SEVERE: Runtime error caught during grid runnable execution: GridWorker [name=partition-exchanger, gridName=null, finished=false, isCancelled=false, hashCode=685490869, interrupted=false, runner=exchange-worker-#45%null%] java.lang.OutOfMemoryError: Java heap space at java.util.HashMap.newNode(HashMap.java:1734) at java.util.HashMap.putVal(HashMap.java:630) at java.util.HashMap.put(HashMap.java:611) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionMap2.put(GridDhtPartitionMap2.java:112) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionMap2.<init>(GridDhtPartitionMap2.java:96) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionFullMap.<init>(GridDhtPartitionFullMap.java:107) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl.partitionMap(GridDhtPartitionTopologyImpl.java:841) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.sendAllPartitions(GridCachePartitionExchangeManager.java:747) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.refreshPartitions(GridCachePartitionExchangeManager.java:698) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.refreshPartitions(GridCachePartitionExchangeManager.java:724) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.access$1600(GridCachePartitionExchangeManager.java:107) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:1267) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:745) Exception in thread "exchange-worker-#45%null%" Exception in thread "grid-nio-worker-0-#40%null%" java.lang.OutOfMemoryError: Java heap space at java.util.HashMap.newNode(HashMap.java:1734) at java.util.HashMap.putVal(HashMap.java:630) at java.util.HashMap.put(HashMap.java:611) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionMap2.put(GridDhtPartitionMap2.java:112) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionMap2.<init>(GridDhtPartitionMap2.java:96) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionFullMap.<init>(GridDhtPartitionFullMap.java:107) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl.partitionMap(GridDhtPartitionTopologyImpl.java:841) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.sendAllPartitions(GridCachePartitionExchangeManager.java:747) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.refreshPartitions(GridCachePartitionExchangeManager.java:698) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.refreshPartitions(GridCachePartitionExchangeManager.java:724) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.access$1600(GridCachePartitionExchangeManager.java:107) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:1267) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:745) java.lang.OutOfMemoryError: Java heap space at java.util.HashMap$KeySet.iterator(HashMap.java:912) at java.util.HashSet.iterator(HashSet.java:172) at java.util.Collections$UnmodifiableCollection$1.<init>(Collections.java:1039) at java.util.Collections$UnmodifiableCollection.iterator(Collections.java:1038) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.checkIdle(GridNioServer.java:1489) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:1406) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1280) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:745) --------------------------------------------------- I'm mostly using the default configuration for the cache. I don't know if maybe I might be missing something to get the most out of Ignite. Any tips and suggestions are welcome, as I cannot have Ignite dying on me (specially after just 3 days). Thanks in advance.
