[jira] [Commented] (LUCENE-8808) Introduce Optional Cap on Number Of Threads Per Query
[ https://issues.apache.org/jira/browse/LUCENE-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847742#comment-16847742 ] Atri Sharma commented on LUCENE-8808: - [~jpountz] Essentially, no. My proposal is basically to have a last level limit to ensure that no single query overloads the entire system i.e. a query does not decide to use 72 threads on a 32 core machine. We could potentially not even have a configurable cap, and always set it to number of cores. I am not married to this approach, so happy to hear ideas. > Introduce Optional Cap on Number Of Threads Per Query > - > > Key: LUCENE-8808 > URL: https://issues.apache.org/jira/browse/LUCENE-8808 > Project: Lucene - Core > Issue Type: Improvement > Components: core/search >Reporter: Atri Sharma >Priority: Major > > With the presence of https://issues.apache.org/jira/browse/LUCENE-8757 , a > natural progression is to allow advanced users to specify a cap on the number > of threads a query can use. This is especially useful for long polled > IndexSearcher instances, where the same instance is used to fire queries for > multiple runs and there are multiple concurrent IndexSearcher instances on > the same node. > > This will be an optional parameter and local only to the IndexSearcher > instance being configured during construction. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8808) Introduce Optional Cap on Number Of Threads Per Query
[ https://issues.apache.org/jira/browse/LUCENE-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847715#comment-16847715 ] Adrien Grand commented on LUCENE-8808: -- Sorry I don't. This is a hard problem. Your approach makes the assumption that it's ok to make slow queries slower if that helps fast queries remain fast, which I don't think would be the right assumption for most users? > Introduce Optional Cap on Number Of Threads Per Query > - > > Key: LUCENE-8808 > URL: https://issues.apache.org/jira/browse/LUCENE-8808 > Project: Lucene - Core > Issue Type: Improvement > Components: core/search >Reporter: Atri Sharma >Priority: Major > > With the presence of https://issues.apache.org/jira/browse/LUCENE-8757 , a > natural progression is to allow advanced users to specify a cap on the number > of threads a query can use. This is especially useful for long polled > IndexSearcher instances, where the same instance is used to fire queries for > multiple runs and there are multiple concurrent IndexSearcher instances on > the same node. > > This will be an optional parameter and local only to the IndexSearcher > instance being configured during construction. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8808) Introduce Optional Cap on Number Of Threads Per Query
[ https://issues.apache.org/jira/browse/LUCENE-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847430#comment-16847430 ] Atri Sharma commented on LUCENE-8808: - Sure. Do you have any ideas of how could we control that large queries do not go out of hand in terms of resource consumption? Happy to discuss them > Introduce Optional Cap on Number Of Threads Per Query > - > > Key: LUCENE-8808 > URL: https://issues.apache.org/jira/browse/LUCENE-8808 > Project: Lucene - Core > Issue Type: Improvement > Components: core/search >Reporter: Atri Sharma >Priority: Major > > With the presence of https://issues.apache.org/jira/browse/LUCENE-8757 , a > natural progression is to allow advanced users to specify a cap on the number > of threads a query can use. This is especially useful for long polled > IndexSearcher instances, where the same instance is used to fire queries for > multiple runs and there are multiple concurrent IndexSearcher instances on > the same node. > > This will be an optional parameter and local only to the IndexSearcher > instance being configured during construction. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8808) Introduce Optional Cap on Number Of Threads Per Query
[ https://issues.apache.org/jira/browse/LUCENE-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847400#comment-16847400 ] Adrien Grand commented on LUCENE-8808: -- In general I'm reluctant to adding more configuration options. It increases the API surface area, which in turn makes Lucene harder to use. My gut feeling right now is that this new setting would not help enough to warrant the introduction of a new option? > Introduce Optional Cap on Number Of Threads Per Query > - > > Key: LUCENE-8808 > URL: https://issues.apache.org/jira/browse/LUCENE-8808 > Project: Lucene - Core > Issue Type: Improvement > Components: core/search >Reporter: Atri Sharma >Priority: Major > > With the presence of https://issues.apache.org/jira/browse/LUCENE-8757 , a > natural progression is to allow advanced users to specify a cap on the number > of threads a query can use. This is especially useful for long polled > IndexSearcher instances, where the same instance is used to fire queries for > multiple runs and there are multiple concurrent IndexSearcher instances on > the same node. > > This will be an optional parameter and local only to the IndexSearcher > instance being configured during construction. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8808) Introduce Optional Cap on Number Of Threads Per Query
[ https://issues.apache.org/jira/browse/LUCENE-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847377#comment-16847377 ] Atri Sharma commented on LUCENE-8808: - [~jpountz] I further tested this by trying out a 900 GB dataset with a large query (around 6 billion hits) on a 32 vCPU machine. With 8 concurrent clients doing the same query and sharing the same threadpool, the backlog of threads on the threadpool was 3-5X larger than the case when there was a capping of 16 threads per query. Overall, the P50 improved with the cap since the extra dequeuing and thread context switching costs were saved. However, P99.9 worsened due to lesser concurrency. Since this is an optional parameter, users will have a choice, so is configurable per workload. WDYT? > Introduce Optional Cap on Number Of Threads Per Query > - > > Key: LUCENE-8808 > URL: https://issues.apache.org/jira/browse/LUCENE-8808 > Project: Lucene - Core > Issue Type: Improvement > Components: core/search >Reporter: Atri Sharma >Priority: Major > > With the presence of https://issues.apache.org/jira/browse/LUCENE-8757 , a > natural progression is to allow advanced users to specify a cap on the number > of threads a query can use. This is especially useful for long polled > IndexSearcher instances, where the same instance is used to fire queries for > multiple runs and there are multiple concurrent IndexSearcher instances on > the same node. > > This will be an optional parameter and local only to the IndexSearcher > instance being configured during construction. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8808) Introduce Optional Cap on Number Of Threads Per Query
[ https://issues.apache.org/jira/browse/LUCENE-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845843#comment-16845843 ] Atri Sharma commented on LUCENE-8808: - [~jpountz] Let me elaborate by taking an example: Consider a scenario where the underlying hardware is a 16 vCPU machine. There is a single large index (approx 450 GB). Now, N concurrent IndexSearchers sharing the same executor are firing different queries. This could mean that some queries could have X number of slices, where X >> 16. Other queries could need number of slices << 16. If we had a hard cap of the number of slices a single query can have, that will mean that the large query will be bound by the N. I agree that if there is a sizeable percentage of the overall query set which will ideally need slices >> than the number of cores present, then eventually the executor's queue will build up. However, that seems more like a problem of hardware underprovisioning than a limitation in Lucene? The essential idea is to have a best-effort way to limit the amount of resources a single query can use. As you highlighted, this will not cover all edge cases, but should be better than nothing. WDYT? > Introduce Optional Cap on Number Of Threads Per Query > - > > Key: LUCENE-8808 > URL: https://issues.apache.org/jira/browse/LUCENE-8808 > Project: Lucene - Core > Issue Type: Improvement > Components: core/search >Reporter: Atri Sharma >Priority: Major > > With the presence of https://issues.apache.org/jira/browse/LUCENE-8757 , a > natural progression is to allow advanced users to specify a cap on the number > of threads a query can use. This is especially useful for long polled > IndexSearcher instances, where the same instance is used to fire queries for > multiple runs and there are multiple concurrent IndexSearcher instances on > the same node. > > This will be an optional parameter and local only to the IndexSearcher > instance being configured during construction. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8808) Introduce Optional Cap on Number Of Threads Per Query
[ https://issues.apache.org/jira/browse/LUCENE-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845828#comment-16845828 ] Adrien Grand commented on LUCENE-8808: -- I don't think it really fixes the problem, it only makes it occur a bit later? If there is a single bad query, it makes the issue less problematic for the well-behaved queries, but what if 10% of your queries are bad, then these queries would likely still hog all available threads (and probably even fill the queue) and none of the well-behaved queries may run? > Introduce Optional Cap on Number Of Threads Per Query > - > > Key: LUCENE-8808 > URL: https://issues.apache.org/jira/browse/LUCENE-8808 > Project: Lucene - Core > Issue Type: Improvement > Components: core/search >Reporter: Atri Sharma >Priority: Major > > With the presence of https://issues.apache.org/jira/browse/LUCENE-8757 , a > natural progression is to allow advanced users to specify a cap on the number > of threads a query can use. This is especially useful for long polled > IndexSearcher instances, where the same instance is used to fire queries for > multiple runs and there are multiple concurrent IndexSearcher instances on > the same node. > > This will be an optional parameter and local only to the IndexSearcher > instance being configured during construction. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8808) Introduce Optional Cap on Number Of Threads Per Query
[ https://issues.apache.org/jira/browse/LUCENE-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845822#comment-16845822 ] Atri Sharma commented on LUCENE-8808: - [~jpountz] If a shared executor is used across all IndexSearchers, then there is a probability that a few odd large queries will hog all available threads on the executor, and the remaining queries can be queued in the executor's queue. This parameter is an effort to ensure that a small subset of queries does not overload the shared resource. > Introduce Optional Cap on Number Of Threads Per Query > - > > Key: LUCENE-8808 > URL: https://issues.apache.org/jira/browse/LUCENE-8808 > Project: Lucene - Core > Issue Type: Improvement > Components: core/search >Reporter: Atri Sharma >Priority: Major > > With the presence of https://issues.apache.org/jira/browse/LUCENE-8757 , a > natural progression is to allow advanced users to specify a cap on the number > of threads a query can use. This is especially useful for long polled > IndexSearcher instances, where the same instance is used to fire queries for > multiple runs and there are multiple concurrent IndexSearcher instances on > the same node. > > This will be an optional parameter and local only to the IndexSearcher > instance being configured during construction. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8808) Introduce Optional Cap on Number Of Threads Per Query
[ https://issues.apache.org/jira/browse/LUCENE-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845802#comment-16845802 ] Adrien Grand commented on LUCENE-8808: -- Can you explain the use-case a bit more? What does it buy compared to having an executor with a fixed number of threads that is shared across all IndexSearcher instances? > Introduce Optional Cap on Number Of Threads Per Query > - > > Key: LUCENE-8808 > URL: https://issues.apache.org/jira/browse/LUCENE-8808 > Project: Lucene - Core > Issue Type: Improvement > Components: core/search >Reporter: Atri Sharma >Priority: Major > > With the presence of https://issues.apache.org/jira/browse/LUCENE-8757 , a > natural progression is to allow advanced users to specify a cap on the number > of threads a query can use. This is especially useful for long polled > IndexSearcher instances, where the same instance is used to fire queries for > multiple runs and there are multiple concurrent IndexSearcher instances on > the same node. > > This will be an optional parameter and local only to the IndexSearcher > instance being configured during construction. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8808) Introduce Optional Cap on Number Of Threads Per Query
[ https://issues.apache.org/jira/browse/LUCENE-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845796#comment-16845796 ] Atri Sharma commented on LUCENE-8808: - [~jpountz] Does this seem like a useful thing? > Introduce Optional Cap on Number Of Threads Per Query > - > > Key: LUCENE-8808 > URL: https://issues.apache.org/jira/browse/LUCENE-8808 > Project: Lucene - Core > Issue Type: Improvement > Components: core/search >Reporter: Atri Sharma >Priority: Major > > With the presence of https://issues.apache.org/jira/browse/LUCENE-8757 , a > natural progression is to allow advanced users to specify a cap on the number > of threads a query can use. This is especially useful for long polled > IndexSearcher instances, where the same instance is used to fire queries for > multiple runs and there are multiple concurrent IndexSearcher instances on > the same node. > > This will be an optional parameter and local only to the IndexSearcher > instance being configured during construction. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
