[ https://issues.apache.org/jira/browse/SOLR-16693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17725557#comment-17725557 ]
David Smiley commented on SOLR-16693: ------------------------------------- Unfortunately retaining the old behavior would add a *new* performance penalty for everyone *not* using timeAllowed due to Lucene's design change :-/. So I'm definitely not in favor of continuing with ExitableDirectoryReader unless maybe if it could be optional per-query. Maybe it could. First, let's explore what does the behavior look like before this change for such a feature? BTW there is a test "ExitableDirectoryReaderTest" in Solr that perhaps should be named "TimeAllowedTest" instead of the implementation detail of how it's implemented. It doesn't use faceting but maybe it should? And/or spellcheck? We might end up adding a check before each faceting step (e.g. between fields) instead of doing a slow term-by-term check. > Use TimeLimitingBulkScorer; stop using ExitableDirectoryReader > -------------------------------------------------------------- > > Key: SOLR-16693 > URL: https://issues.apache.org/jira/browse/SOLR-16693 > Project: Solr > Issue Type: Improvement > Affects Versions: 9.2 > Reporter: David Smiley > Assignee: David Smiley > Priority: Blocker > Fix For: 9.3 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > While reviewing Lucene 9.5 changes coming into Solr, I noticed some changes > relating to the ability to specify timeAllowed in Solr on a search query. > Solr uses both ExitableDirectoryReader and TimeLimitingCollector from Lucene > for this (complementary to each other). Unfortunately, changes in Lucene > will make the cost of ExitableDirectoryReader wrapping happen for all queries > into Solr, even those not using timeAllowed. Options to keep EDR aren't good > -- fork it basically. Anecdotally, I think I've heard the overhead is not > trivial and my intuition thinks likewise. Meanwhile, Lucene 9.3 added a new > TimeLimitingBulkScorer which even gets first class integration into > IndexSearcher which has a timeout. It's been incrementally improved, and I > really like its approach, probable performance, and simplicity. It should be > straightforward to integrate this into SolrIndexSearcher and also only do so > for queries specifying timeAllowed. I'm not sure TimeLimitingCollector > offers much value to using TLBS other than additional precision on > timeAllowed at some cost to unselective queries. > I think doing this should block Solr 9.2 using Lucene 9.5. Alternatively, > someone might benchmark the state of things and see that things aren't so bad > as they may seem. But that takes work too. > [1] QueryTimeout.isTimeoutEnabled is gone: > https://github.com/apache/lucene/pull/11954 > [2] TimeLimitingBulkScorer in LUCENE-10151 > https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/search/TimeLimitingBulkScorer.java -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org