Hi all, We extensively use Lucene and HNSW graph search capability for ANN searches. One issue we have been running into is long build times with higher dimensional vectors. To address this, we are exploring ways where we can build the hnsw index on the GPU and merge it into an existing Lucene index to serve queries. For example, Nvidia's cuvs library supports building a CAGRA <https://arxiv.org/pdf/2308.15136> index and transforming it into a hnswlib graph.
My idea is - once the hnswgraph is built on the GPUs, we can import the graph. We need the graph vertices and their connections. We can then write it to a lucene compatible segment file format. We also map the docids to embeddings and update the fieldinfos. I would like feedback from the community on whether this sounds feasible and any implementation pointers you might have. Thanks, Anand Kotriwal
