[
https://issues.apache.org/jira/browse/LUCENE-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16847377#comment-16847377
]
Atri Sharma commented on LUCENE-8808:
-------------------------------------
[~jpountz] I further tested this by trying out a 900 GB dataset with a large
query (around 6 billion hits) on a 32 vCPU machine. With 8 concurrent clients
doing the same query and sharing the same threadpool, the backlog of threads on
the threadpool was 3-5X larger than the case when there was a capping of 16
threads per query. Overall, the P50 improved with the cap since the extra
dequeuing and thread context switching costs were saved. However, P99.9
worsened due to lesser concurrency.
Since this is an optional parameter, users will have a choice, so is
configurable per workload.
WDYT?
> Introduce Optional Cap on Number Of Threads Per Query
> -----------------------------------------------------
>
> Key: LUCENE-8808
> URL: https://issues.apache.org/jira/browse/LUCENE-8808
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
> Reporter: Atri Sharma
> Priority: Major
>
> With the presence of https://issues.apache.org/jira/browse/LUCENE-8757 , a
> natural progression is to allow advanced users to specify a cap on the number
> of threads a query can use. This is especially useful for long polled
> IndexSearcher instances, where the same instance is used to fire queries for
> multiple runs and there are multiple concurrent IndexSearcher instances on
> the same node.
>
> This will be an optional parameter and local only to the IndexSearcher
> instance being configured during construction.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
