benwtrent commented on code in PR #14256:
URL: https://github.com/apache/lucene/pull/14256#discussion_r1967569988
##########
lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraphBuilder.java:
##########
@@ -229,25 +230,32 @@ public void addGraphNode(int node,
UpdateableRandomVectorScorer scorer) throws I
// NOTE: the entry node and max level may not be paired, but because we
get the level first
// we ensure that the entry node we get later will always exist on the
curMaxLevel
int[] eps = new int[] {hnsw.entryNode()};
+ float[] epsScores = new float[] {scorer.score(eps[0])};
// we first do the search from top to bottom
// for levels > nodeLevel search with topk = 1
GraphBuilderKnnCollector candidates = entryCandidates;
for (int level = curMaxLevel; level > nodeLevel; level--) {
candidates.clear();
- graphSearcher.searchLevel(candidates, scorer, level, eps, hnsw, null);
+ graphSearcher.searchLevel(candidates, scorer, level, eps, epsScores,
hnsw, null);
+ epsScores[0] = candidates.queue.topScore();
eps[0] = candidates.popNode();
}
// for levels <= nodeLevel search with topk = beamWidth, and add
connections
candidates = beamCandidates;
NeighborArray[] scratchPerLevel =
new NeighborArray[Math.min(nodeLevel, curMaxLevel) -
lowestUnsetLevel + 1];
+ int epCount = eps.length;
for (int i = scratchPerLevel.length - 1; i >= 0; i--) {
int level = i + lowestUnsetLevel;
candidates.clear();
- graphSearcher.searchLevel(candidates, scorer, level, eps, hnsw, null);
- eps = candidates.popUntilNearestKNodes();
+ graphSearcher.searchLevel(candidates, scorer, level, eps, epsScores,
epCount, hnsw, null);
+ candidates.popUntilNearestKNodes();
+ eps = ArrayUtil.growExact(eps, candidates.size());
+ epsScores = ArrayUtil.growExact(epsScores, candidates.size());
+ epCount = candidates.size();
+ candidates.collectNodesAndScores(eps, epsScores);
Review Comment:
Yes, we were recalculating entry point scores every time we add a node to a
layer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
