jegentile commented on PR #16200: URL: https://github.com/apache/lucene/pull/16200#issuecomment-4624793619
Thanks for the review, @benwtrent! **Re: splitting into separate changes:** I've split out the int4 change. This PR now only contains the cosine norm precomputation. I'll submit the int4 optimization as a separate PR targeting Lucene104. **Re: whether the cosine optimization is necessary:** That's a fair point. Normalizing vectors and using DOT_PRODUCT is certainly the recommended best practice. I think the case for this change is that since Lucene offers COSINE as a first-class similarity function, it would be nice for it to perform as well as it reasonably can. The current implementation recomputes the query vector's squared norm on every `score()` call even though the query doesn't change between calls, and caching it is a small, straightforward fix. Benchmarks show 10-35% throughput improvement for byte vector cosine scoring (larger vectors benefit more). The change adds a single `cosine(byte[], int, byte[])` overload to VectorUtilSupport and uses it in the existing scorers. That said, I completely understand if the project prefers not to optimize this path. Happy to defer to your judgment on whether it's worth the added surface area. **Re: Lucene99 vs Lucene104:** Good to know, thank you for pointing that out. The cosine optimization here actually applies to `DefaultFlatVectorScorer` and the MemorySegment scorer supplier, which are shared infrastructure rather than Lucene99-specific. The int4 change was the one that touched Lucene99 code, and I'll target Lucene104 for that in the follow-up PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
