[
https://issues.apache.org/jira/browse/SOLR-8241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237993#comment-15237993
]
Shawn Heisey commented on SOLR-8241:
------------------------------------
Those are Lucene caches. The first one is in Lucene core. The other two are
in the Lucene facet module -- which is a completely different implementation
from Solr's faceting.
Recently I found some instances of entire external source files being copied
*into* the Lucene core module. When I asked about it, I was told that this is
to comply with a strict rule about Lucene core remaining 100 percent
dependency-free. I have not confirmed this rule, but it would not surprise me.
I'm all for replacing LFUCache in Solr with your implementation, and I'm
willing to do the work to make it happen. We have quite a few external
dependencies already.
Working on Lucene will require a separate issue, and even though I have write
access to the code, I'm going to leave that to people who really understand
Lucene and who've been around the code a lot longer than I have.
> Evaluate W-TinyLfu cache
> ------------------------
>
> Key: SOLR-8241
> URL: https://issues.apache.org/jira/browse/SOLR-8241
> Project: Solr
> Issue Type: Wish
> Components: search
> Reporter: Ben Manes
> Priority: Minor
> Attachments: SOLR-8241.patch
>
>
> SOLR-2906 introduced an LFU cache and in-progress SOLR-3393 makes it O(1).
> The discussions seem to indicate that the higher hit rate (vs LRU) is offset
> by the slower performance of the implementation. An original goal appeared to
> be to introduce ARC, a patented algorithm that uses ghost entries to retain
> history information.
> My analysis of Window TinyLfu indicates that it may be a better option. It
> uses a frequency sketch to compactly estimate an entry's popularity. It uses
> LRU to capture recency and operate in O(1) time. When using available
> academic traces the policy provides a near optimal hit rate regardless of the
> workload.
> I'm getting ready to release the policy in Caffeine, which Solr already has a
> dependency on. But, the code is fairly straightforward and a port into Solr's
> caches instead is a pragmatic alternative. More interesting is what the
> impact would be in Solr's workloads and feedback on the policy's design.
> https://github.com/ben-manes/caffeine/wiki/Efficiency
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
