[
https://issues.apache.org/jira/browse/LUCENE-10391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492885#comment-17492885
]
Julie Tibshirani commented on LUCENE-10391:
-------------------------------------------
I've been keeping an eye on nightly benchmarks and don't see a noticeable
improvement in search or indexing performance. The updated heap profiles do
look better though. Apparently allocating the results queue is still
non-trivial, so it could be worth trying to share that as well (which my
initial PR did not do):
{code}
8.39% 43814M org.apache.lucene.util.LongHeap#()
at org.apache.lucene.util.hnsw.NeighborQueue#()
at
org.apache.lucene.util.hnsw.HnswGraphSearcher#searchLevel()
at
org.apache.lucene.util.hnsw.HnswGraphBuilder#addGraphNode()
{code}
> Reuse data structures across HnswGraph invocations
> --------------------------------------------------
>
> Key: LUCENE-10391
> URL: https://issues.apache.org/jira/browse/LUCENE-10391
> Project: Lucene - Core
> Issue Type: Task
> Reporter: Adrien Grand
> Assignee: Julie Tibshirani
> Priority: Minor
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Creating HNSW graphs involves doing many repeated calls to HnswGraph#search.
> Profiles from nightly benchmarks suggest that allocating data-structures
> incurs both lots of heap allocations
> ([http://people.apache.org/~mikemccand/lucenebench/2022.01.23.18.03.17.html#profiler_1kb_indexing_vectors_4_heap)]
> and CPU usage
> ([http://people.apache.org/~mikemccand/lucenebench/2022.01.23.18.03.17.html#profiler_1kb_indexing_vectors_4_cpu).]
> It looks like reusing data structures across invocations would be a
> low-hanging fruit that could help save significant CPU?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
