Vivek Narang created SOLR-17892:
-----------------------------------
Summary: Add support for cuVS-Lucene as a pluggable codec in Solr
Key: SOLR-17892
URL: https://issues.apache.org/jira/browse/SOLR-17892
Project: Solr
Issue Type: New Feature
Components: vector-search
Affects Versions: 10.0
Reporter: Vivek Narang
This issue proposes adding *cuVS-Lucene* as a pluggable codec in Solr to enable
GPU-accelerated vector indexing.
*Background* * *cuVS-Lucene* is a new NVIDIA project that integrates GPU
acceleration into Lucene for vector search.
* It supports building HNSW graphs on GPUs via the state-of-the-art *CAGRA*
algorithm.
* The first official release of cuVS-Lucene is planned for {*}early October{*}.
* At present, artifacts are not yet published on Maven Central. For early
adoption, development, and testing, SearchScale is publishing temporary
artifacts to its Maven repository. Once released, official artifacts will be
available on Maven Central, and the PR will be updated to use the released
artifact accordingly.
*Usage in Solr*
This change introduces the ability to configure Solr to use the
{{*Lucene101AcceleratedHNSWVectorsFormat*}} provided by the cuVS-Lucene. Users
can opt to use the GPU-accelerated indexing by selecting the {{CuVSCodec}}
codec, while retaining compatibility with existing CPU-based codecs.
Documentation will include steps on how to enable and configure this format
within Solr.
**
*Testing Strategy* * NVIDIA is dedicating GPU resources for testing cuVS-Lucene
directly.
* In Solr’s test framework:
** Tests for cuVS-Lucene will be skipped on non-GPU machines.
** On GPU-enabled machines, the tests will run fully, validating integration.
* We are also exploring the possibility of contributing GPU resources to
Apache Solr’s CI infrastructure for continuous GPU test coverage (to be
discussed further with the community).
*Benchmarks*
We will publish benchmark results shortly to demonstrate the performance
improvements of GPU-accelerated HNSW graph construction compared to CPU-only
implementations.
*Motivation for Solr 10*
Including cuVS-Lucene support in *Solr 10* would highlight Solr’s adoption of
first-class GPU acceleration, delivering significant performance improvements
for vector search and positioning Solr as a leader in large-scale AI/ML search
workloads.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
