jgq2008303393 commented on a change in pull request #940: LUCENE-9002: Query
caching leads to absurdly slow queries
URL: https://github.com/apache/lucene-solr/pull/940#discussion_r334223157
##########
File path: lucene/core/src/java/org/apache/lucene/search/LRUQueryCache.java
##########
@@ -732,8 +741,39 @@ public ScorerSupplier scorerSupplier(LeafReaderContext
context) throws IOExcepti
if (docIdSet == null) {
if (policy.shouldCache(in.getQuery())) {
- docIdSet = cache(context);
- putIfAbsent(in.getQuery(), docIdSet, cacheHelper);
+ final ScorerSupplier supplier = in.scorerSupplier(context);
+ if (supplier == null) {
+ putIfAbsent(in.getQuery(), DocIdSet.EMPTY, cacheHelper);
+ return null;
+ }
+
+ final long cost = supplier.cost();
+ return new ScorerSupplier() {
+ @Override
+ public Scorer get(long leadCost) throws IOException {
+ // skip cache operation which would slow query down too much
+ if ((cost > skipCacheCost || cost > leadCost * skipCacheFactor)
Review comment:
Yes. If query has a 250x higher cost, it’s good enough now without
_skipCacheCost_.
Also we found that _skipCacheCost_ could be done through the following code.
Although we think that the allowed cache entry size is a little large, we are
not going to modify it now. If the feature is necessary in the future, we will
open a new PR.
https://github.com/apache/lucene-solr/blob/61a00822e78a02f14ac060dabf66c2b5053c12d7/lucene/core/src/java/org/apache/lucene/search/LRUQueryCache.java#L679-L691
I will submit a new patch that remove _skipCacheCost_.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
