[jira] [Commented] (LUCENE-10151) Add timeout support to IndexSearcher
[
https://issues.apache.org/jira/browse/LUCENE-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17571374#comment-17571374
]
Michael Sokolov commented on LUCENE-10151:
--
oh, too bad. Well this feature is new, so at least no existing usage will be
broken.
> Add timeout support to IndexSearcher
>
>
> Key: LUCENE-10151
> URL: https://issues.apache.org/jira/browse/LUCENE-10151
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Greg Miller
>Priority: Minor
> Fix For: 9.3
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> I'd like to explore adding optional "timeout" capabilities to
> {{IndexSearcher}}. This would enable users to (optionally) specify a maximum
> time budget for search execution. If the search "times out", partial results
> would be available.
> This idea originated on the dev list (thanks [~jpountz] for the suggestion).
> Thread for reference:
> [http://mail-archives.apache.org/mod_mbox/lucene-dev/202110.mbox/%3CCAL8PwkZdNGmYJopPjeXYK%3DF7rvLkWon91UEXVxMM4MeeJ3UHxQ%40mail.gmail.com%3E]
>
> A couple things to watch out for with this change:
> # We want to make sure it's robust to a two-phase query evaluation scenario
> where the "approximate" step matches a large number of candidates but the
> "confirmation" step matches very few (or none). This is a particularly tricky
> case.
> # We want to make sure the {{TotalHits#Relation}} reported by {{TopDocs}} is
> {{GREATER_THAN_OR_EQUAL_TO}} if the query times out
> # We want to make sure it plays nice with the {{LRUCache}} since it iterates
> the query to pre-populate a {{BitSet}} when caching. That step shouldn't be
> allowed to overrun the timeout. The proper way to handle this probably needs
> some thought.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-10151) Add timeout support to IndexSearcher
[
https://issues.apache.org/jira/browse/LUCENE-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17571370#comment-17571370
]
Adrien Grand commented on LUCENE-10151:
---
I just noticed that the push of my backport had failed, so it will be in 9.4,
not 9.3. I don't think it's worth respinning for it.
> Add timeout support to IndexSearcher
>
>
> Key: LUCENE-10151
> URL: https://issues.apache.org/jira/browse/LUCENE-10151
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Greg Miller
>Priority: Minor
> Fix For: 9.3
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> I'd like to explore adding optional "timeout" capabilities to
> {{IndexSearcher}}. This would enable users to (optionally) specify a maximum
> time budget for search execution. If the search "times out", partial results
> would be available.
> This idea originated on the dev list (thanks [~jpountz] for the suggestion).
> Thread for reference:
> [http://mail-archives.apache.org/mod_mbox/lucene-dev/202110.mbox/%3CCAL8PwkZdNGmYJopPjeXYK%3DF7rvLkWon91UEXVxMM4MeeJ3UHxQ%40mail.gmail.com%3E]
>
> A couple things to watch out for with this change:
> # We want to make sure it's robust to a two-phase query evaluation scenario
> where the "approximate" step matches a large number of candidates but the
> "confirmation" step matches very few (or none). This is a particularly tricky
> case.
> # We want to make sure the {{TotalHits#Relation}} reported by {{TopDocs}} is
> {{GREATER_THAN_OR_EQUAL_TO}} if the query times out
> # We want to make sure it plays nice with the {{LRUCache}} since it iterates
> the query to pre-populate a {{BitSet}} when caching. That step shouldn't be
> allowed to overrun the timeout. The proper way to handle this probably needs
> some thought.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-10151) Add timeout support to IndexSearcher
[
https://issues.apache.org/jira/browse/LUCENE-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17571367#comment-17571367
]
ASF subversion and git services commented on LUCENE-10151:
--
Commit be81cd79346e869da94d9db89e1b863bfaabbd65 in lucene's branch
refs/heads/branch_9x from Adrien Grand
[ https://gitbox.apache.org/repos/asf?p=lucene.git;h=be81cd79346 ]
LUCENE-10151: Some fixes to query timeouts. (#996)
I noticed some minor bugs in the original PR #927 that this PR should fix:
- When a timeout is set, we would no longer catch
`CollectionTerminatedException`.
- I added randomization to `LuceneTestCase` to randomly set a timeout, it
would have caught the above bug.
- Fixed visibility of `TimeLimitingBulkScorer`.
> Add timeout support to IndexSearcher
>
>
> Key: LUCENE-10151
> URL: https://issues.apache.org/jira/browse/LUCENE-10151
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Greg Miller
>Priority: Minor
> Fix For: 9.3
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> I'd like to explore adding optional "timeout" capabilities to
> {{IndexSearcher}}. This would enable users to (optionally) specify a maximum
> time budget for search execution. If the search "times out", partial results
> would be available.
> This idea originated on the dev list (thanks [~jpountz] for the suggestion).
> Thread for reference:
> [http://mail-archives.apache.org/mod_mbox/lucene-dev/202110.mbox/%3CCAL8PwkZdNGmYJopPjeXYK%3DF7rvLkWon91UEXVxMM4MeeJ3UHxQ%40mail.gmail.com%3E]
>
> A couple things to watch out for with this change:
> # We want to make sure it's robust to a two-phase query evaluation scenario
> where the "approximate" step matches a large number of candidates but the
> "confirmation" step matches very few (or none). This is a particularly tricky
> case.
> # We want to make sure the {{TotalHits#Relation}} reported by {{TopDocs}} is
> {{GREATER_THAN_OR_EQUAL_TO}} if the query times out
> # We want to make sure it plays nice with the {{LRUCache}} since it iterates
> the query to pre-populate a {{BitSet}} when caching. That step shouldn't be
> allowed to overrun the timeout. The proper way to handle this probably needs
> some thought.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-10151) Add timeout support to IndexSearcher
[
https://issues.apache.org/jira/browse/LUCENE-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17568757#comment-17568757
]
Adrien Grand commented on LUCENE-10151:
---
I backported #996 too.
> Add timeout support to IndexSearcher
>
>
> Key: LUCENE-10151
> URL: https://issues.apache.org/jira/browse/LUCENE-10151
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Greg Miller
>Priority: Minor
> Fix For: 9.3
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> I'd like to explore adding optional "timeout" capabilities to
> {{IndexSearcher}}. This would enable users to (optionally) specify a maximum
> time budget for search execution. If the search "times out", partial results
> would be available.
> This idea originated on the dev list (thanks [~jpountz] for the suggestion).
> Thread for reference:
> [http://mail-archives.apache.org/mod_mbox/lucene-dev/202110.mbox/%3CCAL8PwkZdNGmYJopPjeXYK%3DF7rvLkWon91UEXVxMM4MeeJ3UHxQ%40mail.gmail.com%3E]
>
> A couple things to watch out for with this change:
> # We want to make sure it's robust to a two-phase query evaluation scenario
> where the "approximate" step matches a large number of candidates but the
> "confirmation" step matches very few (or none). This is a particularly tricky
> case.
> # We want to make sure the {{TotalHits#Relation}} reported by {{TopDocs}} is
> {{GREATER_THAN_OR_EQUAL_TO}} if the query times out
> # We want to make sure it plays nice with the {{LRUCache}} since it iterates
> the query to pre-populate a {{BitSet}} when caching. That step shouldn't be
> allowed to overrun the timeout. The proper way to handle this probably needs
> some thought.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-10151) Add timeout support to IndexSearcher
[
https://issues.apache.org/jira/browse/LUCENE-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17567330#comment-17567330
]
Michael Sokolov commented on LUCENE-10151:
--
> Did you forget to push to branch_9x? I cannot see the change there.
Yes! Thanks - pushed now
> Add timeout support to IndexSearcher
>
>
> Key: LUCENE-10151
> URL: https://issues.apache.org/jira/browse/LUCENE-10151
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Greg Miller
>Priority: Minor
> Fix For: 9.3
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> I'd like to explore adding optional "timeout" capabilities to
> {{IndexSearcher}}. This would enable users to (optionally) specify a maximum
> time budget for search execution. If the search "times out", partial results
> would be available.
> This idea originated on the dev list (thanks [~jpountz] for the suggestion).
> Thread for reference:
> [http://mail-archives.apache.org/mod_mbox/lucene-dev/202110.mbox/%3CCAL8PwkZdNGmYJopPjeXYK%3DF7rvLkWon91UEXVxMM4MeeJ3UHxQ%40mail.gmail.com%3E]
>
> A couple things to watch out for with this change:
> # We want to make sure it's robust to a two-phase query evaluation scenario
> where the "approximate" step matches a large number of candidates but the
> "confirmation" step matches very few (or none). This is a particularly tricky
> case.
> # We want to make sure the {{TotalHits#Relation}} reported by {{TopDocs}} is
> {{GREATER_THAN_OR_EQUAL_TO}} if the query times out
> # We want to make sure it plays nice with the {{LRUCache}} since it iterates
> the query to pre-populate a {{BitSet}} when caching. That step shouldn't be
> allowed to overrun the timeout. The proper way to handle this probably needs
> some thought.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-10151) Add timeout support to IndexSearcher
[
https://issues.apache.org/jira/browse/LUCENE-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17567328#comment-17567328
]
ASF subversion and git services commented on LUCENE-10151:
--
Commit aa082b46f669f71cd0deb2e409c62be863f17091 in lucene's branch
refs/heads/branch_9x from Deepika0510
[ https://gitbox.apache.org/repos/asf?p=lucene.git;h=aa082b46f66 ]
LUCENE-10151: Adding Timeout Support to IndexSearcher (#927)
Authored-by: Deepika Sharma
> Add timeout support to IndexSearcher
>
>
> Key: LUCENE-10151
> URL: https://issues.apache.org/jira/browse/LUCENE-10151
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Greg Miller
>Priority: Minor
> Fix For: 9.3
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> I'd like to explore adding optional "timeout" capabilities to
> {{IndexSearcher}}. This would enable users to (optionally) specify a maximum
> time budget for search execution. If the search "times out", partial results
> would be available.
> This idea originated on the dev list (thanks [~jpountz] for the suggestion).
> Thread for reference:
> [http://mail-archives.apache.org/mod_mbox/lucene-dev/202110.mbox/%3CCAL8PwkZdNGmYJopPjeXYK%3DF7rvLkWon91UEXVxMM4MeeJ3UHxQ%40mail.gmail.com%3E]
>
> A couple things to watch out for with this change:
> # We want to make sure it's robust to a two-phase query evaluation scenario
> where the "approximate" step matches a large number of candidates but the
> "confirmation" step matches very few (or none). This is a particularly tricky
> case.
> # We want to make sure the {{TotalHits#Relation}} reported by {{TopDocs}} is
> {{GREATER_THAN_OR_EQUAL_TO}} if the query times out
> # We want to make sure it plays nice with the {{LRUCache}} since it iterates
> the query to pre-populate a {{BitSet}} when caching. That step shouldn't be
> allowed to overrun the timeout. The proper way to handle this probably needs
> some thought.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-10151) Add timeout support to IndexSearcher
[
https://issues.apache.org/jira/browse/LUCENE-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17567329#comment-17567329
]
ASF subversion and git services commented on LUCENE-10151:
--
Commit 5cd6eda8caba5a93eeaf60215885ec3171707449 in lucene's branch
refs/heads/branch_9x from Michael Sokolov
[ https://gitbox.apache.org/repos/asf?p=lucene.git;h=5cd6eda8cab ]
CHANGES entry for LUCENE-10151
> Add timeout support to IndexSearcher
>
>
> Key: LUCENE-10151
> URL: https://issues.apache.org/jira/browse/LUCENE-10151
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Greg Miller
>Priority: Minor
> Fix For: 9.3
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> I'd like to explore adding optional "timeout" capabilities to
> {{IndexSearcher}}. This would enable users to (optionally) specify a maximum
> time budget for search execution. If the search "times out", partial results
> would be available.
> This idea originated on the dev list (thanks [~jpountz] for the suggestion).
> Thread for reference:
> [http://mail-archives.apache.org/mod_mbox/lucene-dev/202110.mbox/%3CCAL8PwkZdNGmYJopPjeXYK%3DF7rvLkWon91UEXVxMM4MeeJ3UHxQ%40mail.gmail.com%3E]
>
> A couple things to watch out for with this change:
> # We want to make sure it's robust to a two-phase query evaluation scenario
> where the "approximate" step matches a large number of candidates but the
> "confirmation" step matches very few (or none). This is a particularly tricky
> case.
> # We want to make sure the {{TotalHits#Relation}} reported by {{TopDocs}} is
> {{GREATER_THAN_OR_EQUAL_TO}} if the query times out
> # We want to make sure it plays nice with the {{LRUCache}} since it iterates
> the query to pre-populate a {{BitSet}} when caching. That step shouldn't be
> allowed to overrun the timeout. The proper way to handle this probably needs
> some thought.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-10151) Add timeout support to IndexSearcher
[
https://issues.apache.org/jira/browse/LUCENE-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17562247#comment-17562247
]
Adrien Grand commented on LUCENE-10151:
---
For reference, I opened new JIRA issues for suggested follow-ups: LUCENE-10640,
LUCENE-10641.
> Add timeout support to IndexSearcher
>
>
> Key: LUCENE-10151
> URL: https://issues.apache.org/jira/browse/LUCENE-10151
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Greg Miller
>Priority: Minor
> Fix For: 9.3
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> I'd like to explore adding optional "timeout" capabilities to
> {{IndexSearcher}}. This would enable users to (optionally) specify a maximum
> time budget for search execution. If the search "times out", partial results
> would be available.
> This idea originated on the dev list (thanks [~jpountz] for the suggestion).
> Thread for reference:
> [http://mail-archives.apache.org/mod_mbox/lucene-dev/202110.mbox/%3CCAL8PwkZdNGmYJopPjeXYK%3DF7rvLkWon91UEXVxMM4MeeJ3UHxQ%40mail.gmail.com%3E]
>
> A couple things to watch out for with this change:
> # We want to make sure it's robust to a two-phase query evaluation scenario
> where the "approximate" step matches a large number of candidates but the
> "confirmation" step matches very few (or none). This is a particularly tricky
> case.
> # We want to make sure the {{TotalHits#Relation}} reported by {{TopDocs}} is
> {{GREATER_THAN_OR_EQUAL_TO}} if the query times out
> # We want to make sure it plays nice with the {{LRUCache}} since it iterates
> the query to pre-populate a {{BitSet}} when caching. That step shouldn't be
> allowed to overrun the timeout. The proper way to handle this probably needs
> some thought.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-10151) Add timeout support to IndexSearcher
[
https://issues.apache.org/jira/browse/LUCENE-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17562219#comment-17562219
]
Adrien Grand commented on LUCENE-10151:
---
bq. I've merged this now to main and backported to 9.x
Did you forget to push to branch_9x? I cannot see the change there.
> Add timeout support to IndexSearcher
>
>
> Key: LUCENE-10151
> URL: https://issues.apache.org/jira/browse/LUCENE-10151
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Greg Miller
>Priority: Minor
> Fix For: 9.3
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> I'd like to explore adding optional "timeout" capabilities to
> {{IndexSearcher}}. This would enable users to (optionally) specify a maximum
> time budget for search execution. If the search "times out", partial results
> would be available.
> This idea originated on the dev list (thanks [~jpountz] for the suggestion).
> Thread for reference:
> [http://mail-archives.apache.org/mod_mbox/lucene-dev/202110.mbox/%3CCAL8PwkZdNGmYJopPjeXYK%3DF7rvLkWon91UEXVxMM4MeeJ3UHxQ%40mail.gmail.com%3E]
>
> A couple things to watch out for with this change:
> # We want to make sure it's robust to a two-phase query evaluation scenario
> where the "approximate" step matches a large number of candidates but the
> "confirmation" step matches very few (or none). This is a particularly tricky
> case.
> # We want to make sure the {{TotalHits#Relation}} reported by {{TopDocs}} is
> {{GREATER_THAN_OR_EQUAL_TO}} if the query times out
> # We want to make sure it plays nice with the {{LRUCache}} since it iterates
> the query to pre-populate a {{BitSet}} when caching. That step shouldn't be
> allowed to overrun the timeout. The proper way to handle this probably needs
> some thought.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-10151) Add timeout support to IndexSearcher
[
https://issues.apache.org/jira/browse/LUCENE-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17562216#comment-17562216
]
ASF subversion and git services commented on LUCENE-10151:
--
Commit 81d4a7a69f1c9085e40df412be87de22d0aa8cd6 in lucene's branch
refs/heads/main from Adrien Grand
[ https://gitbox.apache.org/repos/asf?p=lucene.git;h=81d4a7a69f1 ]
LUCENE-10151: Some fixes to query timeouts. (#996)
I noticed some minor bugs in the original PR #927 that this PR should fix:
- When a timeout is set, we would no longer catch
`CollectionTerminatedException`.
- I added randomization to `LuceneTestCase` to randomly set a timeout, it
would have caught the above bug.
- Fixed visibility of `TimeLimitingBulkScorer`.
> Add timeout support to IndexSearcher
>
>
> Key: LUCENE-10151
> URL: https://issues.apache.org/jira/browse/LUCENE-10151
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Greg Miller
>Priority: Minor
> Fix For: 9.3
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> I'd like to explore adding optional "timeout" capabilities to
> {{IndexSearcher}}. This would enable users to (optionally) specify a maximum
> time budget for search execution. If the search "times out", partial results
> would be available.
> This idea originated on the dev list (thanks [~jpountz] for the suggestion).
> Thread for reference:
> [http://mail-archives.apache.org/mod_mbox/lucene-dev/202110.mbox/%3CCAL8PwkZdNGmYJopPjeXYK%3DF7rvLkWon91UEXVxMM4MeeJ3UHxQ%40mail.gmail.com%3E]
>
> A couple things to watch out for with this change:
> # We want to make sure it's robust to a two-phase query evaluation scenario
> where the "approximate" step matches a large number of candidates but the
> "confirmation" step matches very few (or none). This is a particularly tricky
> case.
> # We want to make sure the {{TotalHits#Relation}} reported by {{TopDocs}} is
> {{GREATER_THAN_OR_EQUAL_TO}} if the query times out
> # We want to make sure it plays nice with the {{LRUCache}} since it iterates
> the query to pre-populate a {{BitSet}} when caching. That step shouldn't be
> allowed to overrun the timeout. The proper way to handle this probably needs
> some thought.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-10151) Add timeout support to IndexSearcher
Title: Message Title Michael Sokolov commented on LUCENE-10151 Re: Add timeout support to IndexSearcher Thanks, Deepika Sharma I've merged this now to main and backported to 9.x One oddity I noticed was a linter failure that happened on 9.x only, but not on main? I don't know if we may have relaxed some checks on main? In any case I added a patch for both branches, which is this change: https://gitbox.apache.org/repos/asf?p=lucene.git;a=commit;h=e078bc1cd9c1e647f963fbdd55cbcd4ec59fac94 Add Comment This message was sent by Atlassian Jira (v8.20.10#820010-sha1:ace47f9)
[jira] [Commented] (LUCENE-10151) Add timeout support to IndexSearcher
Title: Message Title ASF subversion and git services commented on LUCENE-10151 Re: Add timeout support to IndexSearcher Commit 95de554b65bece9697396eeb4a5e78a8352f58d0 in lucene's branch refs/heads/main from Michael Sokolov [ https://gitbox.apache.org/repos/asf?p=lucene.git;h=95de554b65b ] CHANGES entry for LUCENE-10151 Add Comment This message was sent by Atlassian Jira (v8.20.10#820010-sha1:ace47f9)
[jira] [Commented] (LUCENE-10151) Add timeout support to IndexSearcher
Title: Message Title ASF subversion and git services commented on LUCENE-10151 Re: Add timeout support to IndexSearcher Commit af05550ebfe3dc1bc40aeb2318c132a9b12e37a2 in lucene's branch refs/heads/main from Deepika0510 [ https://gitbox.apache.org/repos/asf?p=lucene.git;h=af05550ebfe ] LUCENE-10151: Adding Timeout Support to IndexSearcher (#927) Authored-by: Deepika Sharma Add Comment This message was sent by Atlassian Jira (v8.20.10#820010-sha1:ace47f9)
[jira] [Commented] (LUCENE-10151) Add timeout support to IndexSearcher
[
https://issues.apache.org/jira/browse/LUCENE-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17543885#comment-17543885
]
Deepika Sharma commented on LUCENE-10151:
-
I have raised a PR for the `BulkScorer` approach. Please take a look and let me
know your thoughts.
> Add timeout support to IndexSearcher
>
>
> Key: LUCENE-10151
> URL: https://issues.apache.org/jira/browse/LUCENE-10151
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Greg Miller
>Priority: Minor
>
> I'd like to explore adding optional "timeout" capabilities to
> {{IndexSearcher}}. This would enable users to (optionally) specify a maximum
> time budget for search execution. If the search "times out", partial results
> would be available.
> This idea originated on the dev list (thanks [~jpountz] for the suggestion).
> Thread for reference:
> [http://mail-archives.apache.org/mod_mbox/lucene-dev/202110.mbox/%3CCAL8PwkZdNGmYJopPjeXYK%3DF7rvLkWon91UEXVxMM4MeeJ3UHxQ%40mail.gmail.com%3E]
>
> A couple things to watch out for with this change:
> # We want to make sure it's robust to a two-phase query evaluation scenario
> where the "approximate" step matches a large number of candidates but the
> "confirmation" step matches very few (or none). This is a particularly tricky
> case.
> # We want to make sure the {{TotalHits#Relation}} reported by {{TopDocs}} is
> {{GREATER_THAN_OR_EQUAL_TO}} if the query times out
> # We want to make sure it plays nice with the {{LRUCache}} since it iterates
> the query to pre-populate a {{BitSet}} when caching. That step shouldn't be
> allowed to overrun the timeout. The proper way to handle this probably needs
> some thought.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-10151) Add timeout support to IndexSearcher
[
https://issues.apache.org/jira/browse/LUCENE-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17532223#comment-17532223
]
Deepika Sharma commented on LUCENE-10151:
-
I am exploring adding timeout support to the {{IndexSearcher}} by using
{{ExitableDirectoryReader.}} However, one issue with
{{ExitableDirectoryReader}} is that it enforces timeout checking at the time of
instantiating {{BulkScorer}} and doesn't actually enforce it once you start
iterating postings/impacts. This is being discussed in
[LUCENE-10544|https://issues.apache.org/jira/browse/LUCENE-10544]
I want to ask if there are any suggestions on alternative ways to approach this
problem that I should consider?{{{}{}}}
> Add timeout support to IndexSearcher
>
>
> Key: LUCENE-10151
> URL: https://issues.apache.org/jira/browse/LUCENE-10151
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Greg Miller
>Priority: Minor
>
> I'd like to explore adding optional "timeout" capabilities to
> {{IndexSearcher}}. This would enable users to (optionally) specify a maximum
> time budget for search execution. If the search "times out", partial results
> would be available.
> This idea originated on the dev list (thanks [~jpountz] for the suggestion).
> Thread for reference:
> [http://mail-archives.apache.org/mod_mbox/lucene-dev/202110.mbox/%3CCAL8PwkZdNGmYJopPjeXYK%3DF7rvLkWon91UEXVxMM4MeeJ3UHxQ%40mail.gmail.com%3E]
>
> A couple things to watch out for with this change:
> # We want to make sure it's robust to a two-phase query evaluation scenario
> where the "approximate" step matches a large number of candidates but the
> "confirmation" step matches very few (or none). This is a particularly tricky
> case.
> # We want to make sure the {{TotalHits#Relation}} reported by {{TopDocs}} is
> {{GREATER_THAN_OR_EQUAL_TO}} if the query times out
> # We want to make sure it plays nice with the {{LRUCache}} since it iterates
> the query to pre-populate a {{BitSet}} when caching. That step shouldn't be
> allowed to overrun the timeout. The proper way to handle this probably needs
> some thought.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-10151) Add timeout support to IndexSearcher
[
https://issues.apache.org/jira/browse/LUCENE-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17469557#comment-17469557
]
Greg Miller commented on LUCENE-10151:
--
Just so I don't loose track of this thought, we'll probably also want the
blocking call to {{Future#get}} to specify a timeout as well if the user has
specified one (here:
https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/search/IndexSearcher.java#L721).
> Add timeout support to IndexSearcher
>
>
> Key: LUCENE-10151
> URL: https://issues.apache.org/jira/browse/LUCENE-10151
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Greg Miller
>Priority: Minor
>
> I'd like to explore adding optional "timeout" capabilities to
> {{IndexSearcher}}. This would enable users to (optionally) specify a maximum
> time budget for search execution. If the search "times out", partial results
> would be available.
> This idea originated on the dev list (thanks [~jpountz] for the suggestion).
> Thread for reference:
> [http://mail-archives.apache.org/mod_mbox/lucene-dev/202110.mbox/%3CCAL8PwkZdNGmYJopPjeXYK%3DF7rvLkWon91UEXVxMM4MeeJ3UHxQ%40mail.gmail.com%3E]
>
> A couple things to watch out for with this change:
> # We want to make sure it's robust to a two-phase query evaluation scenario
> where the "approximate" step matches a large number of candidates but the
> "confirmation" step matches very few (or none). This is a particularly tricky
> case.
> # We want to make sure the {{TotalHits#Relation}} reported by {{TopDocs}} is
> {{GREATER_THAN_OR_EQUAL_TO}} if the query times out
> # We want to make sure it plays nice with the {{LRUCache}} since it iterates
> the query to pre-populate a {{BitSet}} when caching. That step shouldn't be
> allowed to overrun the timeout. The proper way to handle this probably needs
> some thought.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-10151) Add timeout support to IndexSearcher
[
https://issues.apache.org/jira/browse/LUCENE-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17425746#comment-17425746
]
Adrien Grand commented on LUCENE-10151:
---
Another question I have is how to signal to the user that their query timed out.
> Add timeout support to IndexSearcher
>
>
> Key: LUCENE-10151
> URL: https://issues.apache.org/jira/browse/LUCENE-10151
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Greg Miller
>Priority: Minor
>
> I'd like to explore adding optional "timeout" capabilities to
> {{IndexSearcher}}. This would enable users to (optionally) specify a maximum
> time budget for search execution. If the search "times out", partial results
> would be available.
> This idea originated on the dev list (thanks [~jpountz] for the suggestion).
> Thread for reference:
> [http://mail-archives.apache.org/mod_mbox/lucene-dev/202110.mbox/%3CCAL8PwkZdNGmYJopPjeXYK%3DF7rvLkWon91UEXVxMM4MeeJ3UHxQ%40mail.gmail.com%3E]
>
> A couple things to watch out for with this change:
> # We want to make sure it's robust to a two-phase query evaluation scenario
> where the "approximate" step matches a large number of candidates but the
> "confirmation" step matches very few (or none). This is a particularly tricky
> case.
> # We want to make sure the {{TotalHits#Relation}} reported by {{TopDocs}} is
> {{GREATER_THAN_OR_EQUAL_TO}} if the query times out
> # We want to make sure it plays nice with the {{LRUCache}} since it iterates
> the query to pre-populate a {{BitSet}} when caching. That step shouldn't be
> allowed to overrun the timeout. The proper way to handle this probably needs
> some thought.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
