Sorry for the confusion. To explain, I use a local ann-benchmarks set-up
that makes use of KnnGraphTester. It is a bit hacky and I accidentally
included the warm-ups in the final timings. So the change to warm-up
explains why we saw different results in our tests. This is great
motivation to solidi
I'm confused, since warming should not be counted in the timings. Are you
saying that the recall was affected??
On Mon, Sep 19, 2022 at 6:12 PM Julie Tibshirani
wrote:
> Using the ann-benchmarks framework, I still saw a similar regression as
> Mayya between 9.3 and 9.4. I investigated and found
Using the ann-benchmarks framework, I still saw a similar regression as
Mayya between 9.3 and 9.4. I investigated and found it was due to
"KnnGraphTester to use KnnVectorQuery" (
https://github.com/apache/lucene/pull/796), specifically the change to the
warm-up strategy. If I revert it, the results
Thanks for your speedy testing! I am observing comparable latencies *when
the index geometry (ie number of segments)* is unchanged. Agree we can
leave this for a later day. I'll proceed to cut 9.4 artifacts
On Mon, Sep 19, 2022 at 11:02 AM Mayya Sharipova
wrote:
> It would be great if you all ar
>
> It would be great if you all are able to test again with
> https://github.com/apache/lucene/pull/11781/ applied
I ran the ann benchmarks with this change, and was happy to confirm that
in my test recall with this PR is the same as in 9.3 branch, although QPS
is lower, but we can investigate
