jtibshirani commented on code in PR #1054:
URL: https://github.com/apache/lucene/pull/1054#discussion_r950631270
##########
lucene/core/src/java/org/apache/lucene/util/VectorUtil.java:
##########
@@ -213,4 +243,48 @@ public static void add(float[] u, float[] v) {
u[i] += v[i];
}
}
+
+ /**
+ * Dot product computed over signed bytes.
+ *
+ * @param a bytes containing a vector
+ * @param b bytes containing another vector, of the same dimension
+ * @return the value of the dot product of the two vectors
+ */
+ public static float dotProduct(BytesRef a, BytesRef b) {
+ assert a.length == b.length;
+ int total = 0;
+ int aOffset = a.offset, bOffset = b.offset;
+ for (int i = 0; i < a.length; i++) {
+ total += a.bytes[aOffset++] * b.bytes[bOffset++];
+ }
+ return total;
+ }
+
+ /**
+ * Dot product score computed over signed bytes, scaled to be in [0, 1].
+ *
+ * @param a bytes containing a vector
+ * @param b bytes containing another vector, of the same dimension
+ * @return the value of the similarity function applied to the two vectors
+ */
+ public static float dotProductScore(BytesRef a, BytesRef b) {
+ // divide by 2 * 2^14 (maximum absolute value of product of 2 signed
bytes) * len
+ return (1 + dotProduct(a, b)) / (float) (a.length * (1 << 15));
+ }
+
+ /**
+ * Convert a floating point vector to an array of bytes using casting; the
vector values should be
+ * in [-128,127]
+ *
+ * @param vector a vector
+ * @return a new BytesRef containing the vector's values cast to byte.
+ */
+ public static BytesRef toBytesRef(float[] vector) {
Review Comment:
Then maybe we could do a check in `Lucene94HnswVectorsReader`? The main
thing I had in mind was if a user accidentally provides a query where the
elements don't fall in the range [-128, 127]. We would silently cast the floats
to bytes, which could result in incorrect search results.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
