[ https://issues.apache.org/jira/browse/SOLR-8241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137111#comment-15137111 ]
Shawn Heisey commented on SOLR-8241: ------------------------------------ If this proves to be advantageous, it would be my recommendation to replace LFUCache, not add an entirely new implementation. I would also suggest that we build something into the tests that will help us evaluate the performance of the various cache implementations, to determine if a later step should be to change the example configs to use LFUCache. In general, I believe LFU to be a more viable eviction method for Solr than LRU, inherently more capable of reaching a higher hit ratio, but we need an efficient implementation. It looks like this is a master-only patch. I can see some code that looks like it's for Java 8 only, both before and after the patch. I do not understand this part of the patch, because I do not have any idea how to use the new capabilities in Java 8, or what those new capabilities actually do: {code} - RemovalListener<BlockCacheKey,BlockCacheLocation> listener = - notification -> releaseLocation(notification.getValue()); + RemovalListener<BlockCacheKey,BlockCacheLocation> listener = + (key, value, cause) -> releaseLocation(value); {code} The code that is patched above does not exist in branch_5x. It's not a problem to have a master-only patch, just something notable. As for the actual implementation: That is going to take me longer to digest. My free time is in short supply. > Evaluate W-TinyLfu cache > ------------------------ > > Key: SOLR-8241 > URL: https://issues.apache.org/jira/browse/SOLR-8241 > Project: Solr > Issue Type: Wish > Components: search > Reporter: Ben Manes > Priority: Minor > Attachments: SOLR-8241.patch > > > SOLR-2906 introduced an LFU cache and in-progress SOLR-3393 makes it O(1). > The discussions seem to indicate that the higher hit rate (vs LRU) is offset > by the slower performance of the implementation. An original goal appeared to > be to introduce ARC, a patented algorithm that uses ghost entries to retain > history information. > My analysis of Window TinyLfu indicates that it may be a better option. It > uses a frequency sketch to compactly estimate an entry's popularity. It uses > LRU to capture recency and operate in O(1) time. When using available > academic traces the policy provides a near optimal hit rate regardless of the > workload. > I'm getting ready to release the policy in Caffeine, which Solr already has a > dependency on. But, the code is fairly straightforward and a port into Solr's > caches instead is a pragmatic alternative. More interesting is what the > impact would be in Solr's workloads and feedback on the policy's design. > https://github.com/ben-manes/caffeine/wiki/Efficiency -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org