Hi Charly, There is no special thread in charge of clearing the queue. Entry is cleared when you try to add new entry to the queue, but it is already full. So, once queue becomes full, it will always stay full, and lowering queue size obviously reduces heap consumption. If cache is destroyed, the queue should be cleared immediately.
However, if queue size is too low, it may lead to data inconsistency on unstable topology. Here is an example: Node A just joined the cluster, became primary node for some partitions and started preloading from node B, which is backup. Node B sends value V for key K, but before it was received and processed on node A user executed cache.remove(K). - If queue size is 0, node A now doesn't have any information that entry was just removed and has to store (K, V) pair from node B. On the other hand, node B will remove K and now primary and backup nodes have different data. - If queue size is N > 0, node A will save a tombstone entry for K. It will have all necessary information to understand that (K, V) pair from node B is older than current value and don't store it. Tombstone will be cleared after N other keys are removed. Hope this helps. On Thu, Jun 22, 2017 at 2:11 AM, charly <cdess...@appnexus.com> wrote: > Hey everyone, > > We use Ignite 1.8 and see a difference in heap used when lowering the value > for IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE. I'm not sure to understand > what > it's going to happen if we decrease that value on production. Could someone > clear that up for me please? From what I understand, deleted entries go in > that queue for cleaning, where one thread per cache is in charge of > cleaning > the queue. But what if the queue is full? I might have missed that > information in the documentation, my bad if I did. > > Maybe on the same subject, is it going to slow down the free up of the heap > after cache.destroy()? We have a use-case where we delete 4 caches at the > same time containing ~500k entries heavily indexed. When doing so, it takes > few minutes for the heap to decrease as you can see here > http://imgur.com/a/kPsSc . We would not want to extend the time to free up > memory even more. > > More information about our setup: > - on heap (we'll eventually move to off heap but back then we could not > make Ignite 1.8 to free up memory at all with our version of glibc. It > would > work with jmalloc but that was not permitted on production at that time). > - no backup > - atomicity mode: atomic > - cache mode: partitioned > - cluster of 4 nodes > - jvm options: -J-DIGNITE_LONG_OPERATIONS_DUMP_TIMEOUT=300000 -J-Xms12g > -J-Xmx12g -J-server -J-XX:+UseParNewGC -J-XX:+UseConcMarkSweepGC > -J-XX:+UseTLAB -J-XX:NewSize=128m -J-XX:MaxNewSize=128m > -J-XX:MaxTenuringThreshold=0 -J-XX:SurvivorRatio=1024 > -J-XX:+UseCMSInitiatingOccupancyOnly -J-XX:CMSInitiatingOccupancyFraction > =10 > -J-XX:MaxGCPauseMillis=1000 -J-XX:InitiatingHeapOccupancyPercent=10 > -J-XX:+UseCompressedOops -J-XX:ParallelGCThreads=8 -J-XX:ConcGCThreads=8 > -J-XX:+PrintGCDetails -J-XX:+PrintGCTimeStamps -J-XX:+PrintGCDateStamps > -J-XX:+UseGCLogFileRotation -J-XX:NumberOfGCLogFiles=10 > -J-XX:GCLogFileSize=100M -J-Xloggc:/tmp/ignite-gc.log > -J-Dcom.sun.management.jmxremote.port=49130 > > Thanks for your help, > Charly > > > > > > -- > View this message in context: http://apache-ignite-users. > 70518.x6.nabble.com/Small-value-for-IGNITE-ATOMIC-CACHE- > DELETE-HISTORY-SIZE-tp14037.html > Sent from the Apache Ignite Users mailing list archive at Nabble.com. > -- Best regards, Ilya
