mikemccand opened a new pull request, #13874:
URL: https://github.com/apache/lucene/pull/13874
This PR:
1. Fixes a pre-existing BWC testing bug where our `int8_hnsw*.zip` bwc
test files failed to actually use the scalar quantization codec ... they were
just ordinary `float32` HNSW since they were using Lucene's default codec to
write the bwc indices. This is why no bwc tests failed with my original PR
(#13767), sigh. I also renamed these files and the test class itself to
`int7*` since they are actually all 7 bit not 8 bit quantization.
1. Regenerates the bwc indices for 9.10.0, 9.11.0, and 9.11.1 (the binary
zip files). This entailed checking out each release tag for these releases,
backporting the fix above, regenerating the zip files and copying them into
this PR. It was a bit tricky for 9.10.0 ([see details
here](https://github.com/apache/lucene/issues/13867#issuecomment-2400927997)).
1. Fixes the actual bug from #13767 using the suggested fix from @parnmatt
(thank you!). The bug happened because scalar quantized HNSW indices written
with 9.11.0 or 9.11.1 claimed `compress=true` (written into the index) yet
vector compression is silently `false` unless `bits` is 7 or 8. This trap is
fixed for newly created indices in 9.12.0, but 9.11.x indices bake in the trap.
I confirmed that after the 2nd step above, the bwc test was indeed failing
properly (because of the #13867 bug), and then upon applying the fix in the
third step above, bwc tests (and all tests) now pass.
Thank you @neio-ionut for discovering and opening this issue and @parnmatt
and @benwtrent for digging and proposing the root cause fix.
Closes #13867
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
