mccullocht commented on issue #13251: URL: https://github.com/apache/lucene/issues/13251#issuecomment-4328699540
There was some discussion of this on https://github.com/apache/lucene/pull/15903 since the proposed format is data blind. * OSQ quantizes the residual against a mean vector computed from the input segment. You would want to break this dependency to discard the raw vectors at ingestion time and rotation might help cover some of the losses. I could also see discarding the raw vectors at some later point if we believe the mean isn't moving but that might be trickier than dropping them at ingestion. * A technique that OSQ doesn't implement but appears in one of the source papers (Intel LVQ) is quantizing the residual, e.g. for some vector `v` quantize `v - dequantize(quantize(v))` and use this for re-ranking. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
