kaivalnp opened a new issue, #15697: URL: https://github.com/apache/lucene/issues/15697
### Description 4-bit quantized vectors are stored as nibbles, with two nibbles for consecutive dimensions packed into a single byte (e.g. an 8-dimension vector takes up 4 bytes, with the first byte containing dimension 1+2, second byte containing dimension 3+4, and so on). For performing individual vector computations, we "unpack" the nibbles into a `byte[]` -- [where we take the second nibble from each byte and place it at the end](https://github.com/apache/lucene/blob/448c7d8e2414eaae43b68cc398fcb3e2191b4132/lucene/core/src/java/org/apache/lucene/codecs/lucene104/OffHeapScalarQuantizedVectorValues.java#L188-L194) (e.g. the byte-wise dimensions in the above vector are now [1, 3, 5, 7, 2, 4, 6, 8]). Now for individual dot-product computations b/w a packed and unpacked vector: we [read the first byte of the packed vector (dimensions 1, 2) and multiply individual nibbles against byte number 1 (dimension 1) and 5 (dimension 2) of the unpacked vector](https://github.com/apache/lucene/blob/448c7d8e2414eaae43b68cc398fcb3e2191b4132/lucene/core/src/java/org/apache/lucene/internal/vectorization/DefaultVectorUtilSupport.java#L171-L182). For higher dimension vectors, can this lead to CPU cache line misses? If so, can we "unpack" the nibbles in dimension order for performance gains? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
