tveasey commented on PR #15903: URL: https://github.com/apache/lucene/pull/15903#issuecomment-4170318031
> Wow, what an impressive genai example! I also know nearly nothing about TQ, and only scratch surfaces in understanding OSQ. I am curious how the two compare. E.g. does OSQ also not alter the quantization per-segment (merge of flat vectors could optimized copyBytes (the hardest function in the world to implement correctly/performantly!))? Do we get a 3 bit option with OSQ? So they are actually very similar in conception. If you notice Equation (4) in their [paper](https://arxiv.org/pdf/2504.19874) is almost exactly equal to our [initialisation procedure](https://www.elastic.co/search-labs/blog/scalar-quantization-optimization#initializing-the-quantization-interval). The only difference is they allow for a non-uniform grid. Whereas for 2 bits we put centroids at [−1.493, -0.7465, 0.7465, 1.493] they put them at [-1.51, -0.453, 0.453, 1.51]. If you abandon uniform grid spacing you can no longer implement the dot product via integer arithmetic. This is actually a huge performance hit, IIRC we get 4-8x performance vs float arithmetic for well crafted SIMD variants of low bit integer dot products. The final implementation is table lookup (centroid positions) followed by floating point arithmetic. They also do a bias correction based on QJL. Since we optimise the dot product in the direction of document vector in our [full implementation](https://www.elastic.co/search-labs/blog/scalar-quantization-optimization#refining-the-quantization-interval) I don't think this will actually help OSQ, but I will try this out. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
