Re: Enabling concurrent search only for certain queries

2023-07-18 Thread Alexander Lukyanchikov
if that helps. > > > Le mar. 18 juil. 2023, 23:59, Alexander Lukyanchikov < > alexanderlukyanchi...@gmail.com> a écrit : > >> Hi everyone, >> We performed testing of the concurrent rewrite for knn vector queries in >> Lucene 9.7 and the results look great, we s

Enabling concurrent search only for certain queries

2023-07-18 Thread Alexander Lukyanchikov
Hi everyone, We performed testing of the concurrent rewrite for knn vector queries in Lucene 9.7 and the results look great, we see up to x9 improvement on large datasets. Our current implementation for intra-query concurrency relies on a single IndexSearcher per index which is always configured

Slow disjunction query on a large dataset

2023-04-28 Thread Alexander Lukyanchikov
Hello everyone, I am running a simple boolean `should` over 3 Term queries on the same field, ~500M docs dataset. The query takes around 20 seconds. Explain shows that each individual Term query is relatively fast (up to 250 ms), but the Boolean query match phase takes 17 seconds, scoring 1.5

Re: Boolean query regression after migrating from Lucene 8.5 to 9.2

2022-08-22 Thread Alexander Lukyanchikov
ed). This > works by having huger blocks of virtual memory (currently limited to 1 > Gigabyte per mapping) => https://github.com/apache/lucene/pull/912 > > Uwe > > > Thank you, > Alex > > > On Fri, Aug 19, 2022 at 2:42 AM Robert Muir wrote: > >> On Thu, Aug

Re: Boolean query regression after migrating from Lucene 8.5 to 9.2

2022-08-20 Thread Alexander Lukyanchikov
to avoid MMAPing files with the random access pattern on the most recent Lucene and JVM versions? Thank you, Alex On Fri, Aug 19, 2022 at 2:42 AM Robert Muir wrote: > On Thu, Aug 18, 2022 at 1:47 PM Alexander Lukyanchikov > wrote: > > > > > Currently we are trying to av

Re: Boolean query regression after migrating from Lucene 8.5 to 9.2

2022-08-18 Thread Alexander Lukyanchikov
, Aug 9, 2022 at 6:34 AM Alexander Lukyanchikov < alexanderlukyanchi...@gmail.com> wrote: > Hello everyone, > We have a use-case which shows about 10 times higher latency for boolean > queries after migrating to Lucene 9.2. Each query contains 3 filter clauses > and up to a th

Boolean query regression after migrating from Lucene 8.5 to 9.2

2022-08-09 Thread Alexander Lukyanchikov
Hello everyone, We have a use-case which shows about 10 times higher latency for boolean queries after migrating to Lucene 9.2. Each query contains 3 filter clauses and up to a thousand single-term should clauses. They usually return less than 5 documents with a single stored field and used to

Re: 30% query performance degradation for documents with small stored fields

2022-06-07 Thread Alexander Lukyanchikov
ch/stored_fields_benchmarks.html > . > > - If you're seeing a 30% performance degradation with recent changes to > stored fields, there are good chances that you could improve the > performance of this workload significantly with a custom codec that is > lighter on compression.

30% query performance degradation for documents with small stored fields

2022-06-06 Thread Alexander Lukyanchikov
Hello everyone, We are in the process of upgrading from Lucene 8.5.0 and on the latest version our query performance tests show significant latency degradation for one of the important use cases. In this test, each query retrieves a relatively large dataset of 40k documents with a small stored