Hi My Cache key class is java.util.UUID I am not using any collocation affinity, could you please elaborate how can i use constant affinity to check if cache entry still exists on backup?
Thanks On Mon, Nov 14, 2016 at 4:50 PM, Andrey Mashenkov <amashen...@gridgain.com> wrote: > Hi, > > On remove operation entry should be removed from primary node and backup > nodes as well. > Can you reproduce the issue? Can you check if entry was removed only from > primary node and exists on backup, e.g. using constant affinity function? > > I think its possible that the backup is not being cleaned, due to key > serialization issues. Would you provide key class implementation? > > On Sun, Nov 13, 2016 at 4:14 PM, Isaeed Mohanna <isae...@gmail.com> wrote: > >> Hi >> There is no eviction policy since entries in the caches are removed by my >> application. (calling IgniteCache.remove ). >> >> Digging through the core dump I can see that most resident items are >> cache entries were the cachecontext.cachename points to EventsCache, as i >> have mentioned before this cache has very frequent writes and deletions of >> events (i am using remove to delete the events), however this cache is also >> atomic,partitioned and have a backup of at least one so in case a node >> fails the event is not lost. when calling remove on a cache, is the backup >> of an entry removed as well? is it possible that the backup is not being >> cleaned? >> >> Currently i am using the default garbage collector settings, i can't see >> any spikes in performance due to GC, since i experience memory outage in >> several days i am not sure i am collecting data more than the GC is able to >> claim, I will try manually performing a GC when the system is about to >> crash to see wither forcing a GC will clean the memory. >> >> Thank you for ur help >> >> On Fri, Nov 11, 2016 at 11:59 AM, Andrey Mashenkov < >> amashen...@gridgain.com> wrote: >> >>> Hi Isaeed Mohanna, >>> >>> I don't see any eviction or expired policy configured. Is entry deletion >>> performed by you application? >>> >>> Have you try to detect which of caches id grows unexpectedly? >>> Have you analyse GC logs or tried to tune GC? Actually, you can putting >>> data faster as garbage is collecting. This page may be helpful >>> http://apacheignite.gridgain.org/v1.7/docs/performan >>> ce-tips#tune-garbage-collection. >>> >>> Also you can get profile (with e.g. JavaFlightRecorder) of grid under >>> load to understand what is really going on. >>> >>> Please let me know, if there are any issues. >>> >>> >>> >>> On Thu, Nov 10, 2016 at 10:10 AM, Isaeed Mohanna <isae...@gmail.com> >>> wrote: >>> >>>> Hi >>>> My cache configurations appear below. >>>> >>>> // Cache 1 - a cache of ~15 entities that has a date stamp that is >>>> updated every 30 - 120 seconds >>>> CacheConfiguration<?, ?> Cache1Cfg = new CacheConfiguration<>(); >>>> Cache1Cfg cheCfg.setName("Cache1Name"); >>>> Cache1Cfg .setCacheMode(CacheMode.REPLICATED); >>>> Cache1Cfg .setAtomicityMode(CacheAtomicityMode.ATOMIC); >>>> Cache1Cfg .setStartSize(50); >>>> >>>> // Cache 2 - A cache used as an ignite queue with frequent inserts and >>>> removal from the queue >>>> CacheConfiguration<?, ?> Cache2Cfg = new CacheConfiguration<>(); >>>> Cache2Cfg .setName("Cache2Name"); >>>> Cache2Cfg .setCacheMode(CacheMode.REPLICATED); >>>> Cache2Cfg .setAtomicityMode(CacheAtomicityMode.ATOMIC); >>>> >>>> // Cache 3 - hundreds of entities updated daily >>>> CacheConfiguration<?, ?> Cache3Cfg = new CacheConfiguration<>(); >>>> Cache3Cfg .setName("Cache3Name"); >>>> Cache3Cfg .setCacheMode(CacheMode.REPLICATED); >>>> Cache3Cfg .setAtomicityMode(CacheAtomicityMode.ATOMIC); >>>> Cache3Cfg .setIndexedTypes(UUID.class, SomeClass.class); >>>> >>>> // Cache 4 - Cache with very few writes and reads >>>> CacheConfiguration<?, ?> Cache4Cfg = new CacheConfiguration<>(); >>>> Cache4Cfg .setName("Cache4Name"); >>>> Cache4Cfg .setCacheMode(CacheMode.REPLICATED); >>>> Cache4Cfg .setAtomicityMode(CacheAtomicityMode.ATOMIC); >>>> >>>> // Events Cache - cache with very frequent writes and delete, acts as >>>> events queue >>>> CacheConfiguration<?, ?> eventsCacheConfig= new CacheConfiguration<>(); >>>> eventsCacheConfig.setName("EventsCache"); >>>> eventsCacheConfig.setCacheMode(CacheMode.PARTITIONED); >>>> eventsCacheConfig.setAtomicityMode(CacheAtomicityMode.ATOMIC); >>>> eventsCacheConfig.setIndexedTypes(UUID.class, SomeClass.class); >>>> eventsCacheConfig.setBackups(1); >>>> eventsCacheConfig.setOffHeapMaxMemory(0); >>>> >>>> // Failed Events Cache - cache with less writes and reads stores failed >>>> events >>>> CacheConfiguration<?, ?> failedEventsCacheConfig = new >>>> CacheConfiguration<>(); >>>> failedEventsCacheConfig.setName("FailedEventsCache"); >>>> failedEventsCacheConfig.setCacheMode(CacheMode.PARTITIONED); >>>> failedEventsCacheConfig.setAtomicityMode(CacheAtomicityMode.ATOMIC); >>>> failedEventsCacheConfig.setIndexedTypes(UUID.class, EventEntity.class); >>>> failedEventsCacheConfig.setBackups(1); >>>> failedEventsCacheConfig.setOffHeapMaxMemory(0); >>>> >>>> // In addition i have one atomic reference >>>> AtomicConfiguration atomicCfg = new AtomicConfiguration(); >>>> atomicCfg.setCacheMode(CacheMode.REPLICATED); >>>> Thanks again >>>> >>>> On Wed, Nov 9, 2016 at 5:26 PM, Andrey Mashenkov < >>>> amashen...@gridgain.com> wrote: >>>> >>>>> Hi Isaeed Mohanna, >>>>> >>>>> Would you please provide your cache configurations? >>>>> >>>>> >>>>> On Wed, Nov 9, 2016 at 5:37 PM, Isaeed Mohanna <isae...@gmail.com> >>>>> wrote: >>>>> >>>>>> Hi >>>>>> i have an ignite 1.7.0 cluster with 3 nodes running , i have 3 >>>>>> PARTITIONED >>>>>> ATOMIC CACHES and 2 REPLICATED ATOMIC CACHES, Most of these caches are >>>>>> populated with events data, so each cache entry is short lived its >>>>>> inserted, >>>>>> processed later by some task and removed. so the caches are pretty >>>>>> much very >>>>>> dynamic. >>>>>> Recently the load in our system has increased (more events were >>>>>> received and >>>>>> generated) and we started experiencing out of memory fails once in >>>>>> while >>>>>> (several days depending on machine size). >>>>>> I have created several heap dumps and noticed the largest retained >>>>>> objects >>>>>> in memory is by the following classes: GridDhtLocalPartition, >>>>>> ConccurentHashMap8,ConccurentHashMap8$Node[]. >>>>>> I can see the GridDhtLocalPartition has a ConccurentHashMap8 so most >>>>>> likely >>>>>> all three reference the same thing. >>>>>> My question what is this class and why does it retain memory, >>>>>> entities in my >>>>>> caches are usually short lived (several minutes in most caches) so i >>>>>> would >>>>>> expect the memory to be released? any hints on how to continue my >>>>>> investigation would be great. >>>>>> Thanks >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> View this message in context: http://apache-ignite-users.705 >>>>>> 18.x6.nabble.com/Cache-Memory-Behavior-GridDhtLocalPartition >>>>>> -tp8835.html >>>>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com. >>>>>> >>>>> >>>>> >>>> >>> >> >
