Hi Andrey Mashenkov,
Thanks, I will have a try and apply these locally. Do you know 1.8’s release plan? Thanks Shawn 发件人: Andrey Mashenkov [mailto:amashen...@gridgain.com] 发送时间: 2016年10月26日 20:53 收件人: user@ignite.apache.org 主题: Re: 答复: ignite used too much memory Hi, Shawn Du It seems you faced next 2 bugs. First bug: High memory utilization using OffHeap with ExpirePolicy. Issue has a fix, but it is not merged to master yet. See: https://issues.apache.org/jira/browse/IGNITE-3840. Second bug: TTL Manager continue track evicted (and removed) entries, https://issues.apache.org/jira/browse/IGNITE-3948, it seems to be ok for merge, but still is not present in master. You can try to merge them locally or wait until they will be available in master. On Wed, Oct 26, 2016 at 2:35 PM, Shawn Du <shawn...@neulion.com.cn <mailto:shawn...@neulion.com.cn> > wrote: Hi experts, Can anyone help to explain ignite memory model? Now I tried following ways, but no effect. 1) Remove all indexes. 2) enable swap. I see more than 800M’s data are stored in swapspace directory. 3) Don’t cache short life entries. All cache with fifo evict policy and max size is 10k. After running for a while, I have a check the caches, there were only 20k entries, but the memory still grow up? Please help. Thanks Shawn 发件人: Shawn Du [mailto:shawn...@neulion.com.cn <mailto:shawn...@neulion.com.cn> ] 发送时间: 2016年10月26日 17:17 收件人: user@ignite.apache.org <mailto:user@ignite.apache.org> 主题: 答复: ignite used too much memory Hi This is the output of jmap histo:live <pid>. Any useful information? There are about 300k cache entries. Also I enable swap. Each entry costs 10K memory in average. But each entry only has 50 bytes at most. Any help will be appreciated. num #instances #bytes class name ---------------------------------------------- 1: 21642240 519413760 org.apache.ignite.internal.util.GridCircularBuffer$Item 2: 49027 405647408 [Lorg.jsr166.ConcurrentHashMap8$Node; 3: 6056069 304233144 [B 4: 5754322 279563064 [C 5: 11376526 273036624 java.util.concurrent.ConcurrentSkipListMap$Node 6: 5688514 227540560 org.apache.ignite.internal.processors.query.h2.opt.GridH2KeyValueRowOnheap 7: 5684293 181897376 org.apache.ignite.internal.processors.cache.GridCacheTtlManager$EntryWrapper 8: 5333102 170659264 java.util.concurrent.ConcurrentHashMap$Node 9: 5325117 170403744 org.apache.ignite.spi.swapspace.file.FileSwapSpaceSpi$SwapValue 10: 5754728 138113472 java.lang.String 11: 5689577 136549848 java.util.concurrent.ConcurrentSkipListMap$Index 12: 5686353 136472472 org.apache.ignite.internal.processors.cache.KeyCacheObjectImpl 13: 5325117 127802808 org.apache.ignite.spi.swapspace.SwapKey 14: 5690452 91047232 org.h2.value.ValueString 15: 85112 87930752 [Lorg.apache.ignite.internal.util.GridCircularBuffer$Item; 16: 1515799 48505568 java.util.HashMap$Node 17: 268818 42628512 [Ljava.lang.Object; 18: 13467 35031504 [Ljava.util.concurrent.ConcurrentHashMap$Node; 19: 361155 23113920 org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCacheEntry 20: 1226922 19630752 java.lang.Integer 21: 102183 15207336 [Ljava.util.HashMap$Node; 22: 363397 14535880 org.apache.ignite.internal.binary.BinaryObjectImpl 23: 395583 12658656 org.jsr166.ConcurrentHashMap8$Node 24: 359176 11493632 org.apache.ignite.internal.processors.cache.extras.GridCacheTtlEntryExtras 25: 363397 8721528 org.apache.ignite.internal.processors.query.h2.opt.GridH2ValueCacheObject 26: 257615 6182760 java.util.concurrent.atomic.AtomicLong 27: 170775 5464800 java.util.concurrent.locks.ReentrantLock$NonfairSync 28: 85112 5447168 org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtLocalPartition 29: 85112 5447168 org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtLocalPartition$1 30: 103188 4953024 java.util.HashMap 31: 87751 3510040 org.jsr166.ConcurrentHashMap8 32: 136382 3273168 org.jsr166.ConcurrentLinkedDeque8$Node 33: 90699 2902368 org.jsr166.LongAdder8 34: 87277 2792864 java.lang.ref.WeakReference 35: 170663 2730608 java.util.concurrent.locks.ReentrantLock 36: 85112 2723584 org.apache.ignite.internal.processors.cache.GridCacheConcurrentMapImpl 37: 85112 2723584 org.apache.ignite.internal.util.GridCircularBuffer 38: 134255 2148080 org.apache.ignite.internal.processors.cache.CacheEvictableEntryImpl 39: 85304 2047296 java.util.concurrent.CopyOnWriteArrayList 40: 10900 1693000 [Lorg.jsr166.ConcurrentLinkedHashMap$HashEntry; 41: 96594 1545504 java.util.HashSet 42: 86086 1377376 java.util.concurrent.atomic.AtomicInteger 43: 85435 1366960 java.util.HashMap$KeySet 44: 85002 1360032 org.apache.ignite.internal.processors.query.h2.opt.GridH2AbstractKeyValueRow$WeakValue 45: 44295 1063080 java.util.ArrayList 46: 13513 864832 java.util.concurrent.ConcurrentHashMap 47: 21090 843600 org.apache.ignite.internal.processors.cache.version.GridCacheVersion 48: 160 653824 [Lorg.apache.ignite.internal.processors.cache.distributed.dht.GridDhtLocalPartition; 49: 5664 637120 java.lang.Class 50: 7047 620136 java.lang.reflect.Method 51: 10900 610400 org.jsr166.ConcurrentLinkedHashMap$Segment 52: 12528 601344 java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync 53: 11566 555168 org.h2.expression.ExpressionColumn 54: 16698 400752 java.lang.Long 55: 12522 400704 javax.management.MBeanAttributeInfo 56: 7878 381504 [I 57: 1767 310992 org.h2.command.dml.Select 58: 6162 295776 org.apache.ignite.internal.processors.affinity.GridAffinityAssignment 59: 7223 288920 java.util.LinkedHashMap$Entry 60: 9276 222624 org.h2.expression.ValueExpression 61: 6912 221184 org.h2.expression.Comparison 62: 3042 219024 java.lang.reflect.Field 63: 12529 200464 java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock 64: 12529 200464 java.util.concurrent.locks.ReentrantReadWriteLock$Sync$ThreadLocalHoldCounter 65: 12529 200464 java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock 66: 1767 197904 org.h2.table.TableFilter 67: 2116 186208 org.apache.ignite.internal.processors.jobmetrics.GridJobMetricsSnapshot 68: 1767 169632 org.h2.jdbc.JdbcPreparedStatement 69: 5145 164640 org.h2.expression.ConditionAndOr 70: 7674 162472 [Ljava.lang.Class; 71: 4654 148928 org.h2.expression.Alias 72: 1726 137328 [S 73: 1767 127224 org.h2.index.IndexCursor 74: 7800 124800 java.lang.Object 75: 1360 119680 org.apache.ignite.internal.util.StripedCompositeReadWriteLock$ReadLock 76: 1767 113088 org.h2.jdbc.JdbcResultSet 77: 1992 111552 java.util.LinkedHashMap 78: 4228 101472 org.h2.value.ValueLong 79: 2804 89728 java.lang.ThreadLocal$ThreadLocalMap$Entry 80: 309 88992 org.apache.ignite.configuration.CacheConfiguration 81: 1767 84816 org.h2.command.CommandContainer 82: 1172 84384 org.h2.expression.Aggregate 83: 777 80808 org.h2.table.Column 84: 2502 80064 java.util.concurrent.ConcurrentSkipListMap$HeadIndex 85: 2939 79912 [Lorg.h2.expression.Expression; 86: 1870 73152 [Lorg.h2.table.IndexColumn; 87: 1798 71920 java.lang.ref.SoftReference 88: 17 69904 [Ljava.nio.ByteBuffer; 89: 2166 69312 org.apache.ignite.internal.GridLoggerProxy 90: 1715 68600 org.h2.expression.Operation 91: 64 66560 [Lorg.apache.ignite.internal.processors.jobmetrics.GridJobMetricsSnapshot; 92: 20 65920 [Ljava.nio.channels.SelectionKey; 发件人: Shawn Du [mailto:shawn...@neulion.com.cn] 发送时间: 2016年10月26日 13:25 收件人: user@ignite.apache.org <mailto:user@ignite.apache.org> 主题: ignite used too much memory Hi, In my ignite server, I have several caches, each cache has about 10k entries. I build the entry using binary object. Each entry just has 3 or 4 fields, each field is short, less than 20 bytes. But I enable index for each field. Most entry has set expired time. The expired time is short, about 90 seconds. After run for 2 hours, 8G memory are used, and the ignite run out of memory. I build the ignite from source code, and use yesterdays’ github master branch code. My questions: 1) Does the expired data release the memory? 2) How ignite build the index? How much memory it cost? Thanks Shawn
