msokolov commented on code in PR #16415:
URL: https://github.com/apache/lucene/pull/16415#discussion_r3677430640
##########
lucene/test-framework/src/java/org/apache/lucene/tests/index/BaseKnnVectorsFormatTestCase.java:
##########
@@ -2071,6 +2071,94 @@ public void testReadQuantizedVectorWithEmptyRawVectors()
throws Exception {
}
}
+ /**
+ * Tests that dropping the raw full-precision vectors does not change
scoring. {@link
+ * FloatVectorValues#scorer(float[])} is documented to score against the
quantized vectors when
+ * the underlying format quantizes, so a quantized index must produce
identical scores before and
+ * after its raw vector file is emptied.
+ */
+ public void testScoresUnchangedWithEmptyRawVectors() throws Exception {
+ assumeTrue("Test only applies to scalar quantized formats",
supportsFloatVectorFallback());
+
+ String vectorFieldName = "vec1";
+ int numVectors = 1 + random().nextInt(50);
+ int dim = random().nextInt(64) + 1;
+ if (dim % 2 == 1) {
+ dim++;
+ }
+ VectorSimilarityFunction similarityFunction = randomSimilarity();
+ boolean normalize = similarityFunction == VectorSimilarityFunction.COSINE;
Review Comment:
I thought normalization was required for `DOT_PRODUCT` but not for `COSINE`?
##########
lucene/test-framework/src/java/org/apache/lucene/tests/index/BaseKnnVectorsFormatTestCase.java:
##########
@@ -2071,6 +2071,94 @@ public void testReadQuantizedVectorWithEmptyRawVectors()
throws Exception {
}
}
+ /**
+ * Tests that dropping the raw full-precision vectors does not change
scoring. {@link
+ * FloatVectorValues#scorer(float[])} is documented to score against the
quantized vectors when
+ * the underlying format quantizes, so a quantized index must produce
identical scores before and
+ * after its raw vector file is emptied.
+ */
+ public void testScoresUnchangedWithEmptyRawVectors() throws Exception {
Review Comment:
I assume this test failed without the above fix? Did you try it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
