Lucene, I have talked to OpenSearch, Datastax, Elastic, Lucidworks, and Atlas about integrating NVIDIA RAFT for GPU accelerated VSS. They all use Lucene at some level. Rather than integrating into each, we would like to integrate foundationally into Lucene.
Could we set up some time and make some introductions? You can read more about RAFT here: Accelerating Vector Search: Using GPU-Powered Indexes with RAPIDS RAFT | NVIDIA Technical Blog<https://developer.nvidia.com/blog/accelerating-vector-search-using-gpu-powered-indexes-with-rapids-raft/> Accelerating Vector Search: Fine-Tuning GPU Index Algorithms | NVIDIA Technical Blog<https://developer.nvidia.com/blog/accelerating-vector-search-fine-tuning-gpu-index-algorithms/> Thanks, Nathan ________________________________ From: Nathan Stephens <nsteph...@nvidia.com> Sent: Monday, August 14, 2023 9:32 AM To: dev@lucene.apache.org <dev@lucene.apache.org> Subject: RAPIDS RAFT NVIDIA has created accelerated vector search algorithms for IVF-Flat, IVF-PQ, and CAGRA (a new graph based algorithm specifically designed for GPU's). These algorithms are part of RAPIDS RAFT. RAFT contains fundamental widely-used algorithms and primitives for machine learning and information retrieval. The algorithms are CUDA-accelerated and form building blocks for more easily writing high performance applications. API's exist for Python and C++ (we do not currently have a Java API). RAFT has been integrated into Milvus, Redis, and Faiss. We are looking for other integration partners. I wanted to see if there was an interest in the Lucene community. https://github.com/rapidsai/raft Nathan Nathan Stephens (He/Him/His) Enterprise Products: Data Science, AI/ML NVIDIA<http://www.nvidia.com/>
