msokolov commented on issue #15904: URL: https://github.com/apache/lucene/issues/15904#issuecomment-4166163922
The original argument we had for keeping it on heap today is to support random access and modification. Lucene's index files are write-once so we can't use them while building the graph. But there's no reason we couldn't create a temporary memory-mapped or other file to store the graph. One idea could be to create a space for a full set of neighbors (max-conn of them) for each node as it is appended. This will make the file bigger than it will ultimately need to be since the average out-degree of our graphs seems to be about maxconn/2. Maybe that's worth it though. Certainly it could enable graphs bigger than RAM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
