[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] miklosovic updated CASSANDRA-18715: --- Attachment: signature.asc Would you mind to take over and deliver / merge it, please? You are more aware of what should be there than me. Thank you in advance. Sent from ProtonMail mobile \ > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-alpha2, 5.0, 5.1 > > Attachments: signature.asc > > Time Spent: 25h 40m > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a DiskANN instance > * VectorSegmentBuilder - that writes a DiskANN graph to the following > on-disk components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-18715: --- Fix Version/s: 5.0-alpha2 5.0 5.1 (was: 5.x) (was: 5.0-beta) Source Control Link: https://github.com/apache/cassandra/commit/949b760f5516c139591473038917247b1fd7f500 Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed as https://github.com/apache/cassandra/commit/949b760f5516c139591473038917247b1fd7f500 > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-alpha2, 5.0, 5.1 > > Time Spent: 25h 10m > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a DiskANN instance > * VectorSegmentBuilder - that writes a DiskANN graph to the following > on-disk components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-18715: --- Status: Ready to Commit (was: Review In Progress) > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-beta, 5.x > > Time Spent: 25h > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a DiskANN instance > * VectorSegmentBuilder - that writes a DiskANN graph to the following > on-disk components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18715: - Description: The patch associated with this ticket adds a new vector index to SAI. This introduces the following new elements and changes to SAI: * VectorMemtableIndex - the in-memory representation of the vector indexes that writes data to a DiskANN instance * VectorSegmentBuilder - that writes a DiskANN graph to the following on-disk components: ** VECTOR - contains the floating point vectors associated with the graph ** TERMS - contains the HNSW graph on-disk representation written by a HnswGraphWriter ** POSTINGS - contains the index postings as written by a VectorPostingsWriter * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph was: The patch associated with this ticket adds a new vector index to SAI. This introduces the following new elements and changes to SAI: * VectorMemtableIndex - the in-memory representation of the vector indexes that writes data to a CassandraOnHeapHnsw instance * VectorSegmentBuilder - that writes a HNSW graph to the following on-disk components: ** VECTOR - contains the floating point vectors associated with the graph ** TERMS - contains the HNSW graph on-disk representation written by a HnswGraphWriter ** POSTINGS - contains the index postings as written by a VectorPostingsWriter * VectorIndexSegmentSearcher - used to search the on-disk HNSW index > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-beta, 5.x > > Time Spent: 24h 10m > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a DiskANN instance > * VectorSegmentBuilder - that writes a DiskANN graph to the following > on-disk components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-18715: -- Status: Review In Progress (was: Patch Available) > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-beta, 5.x > > Time Spent: 18h 50m > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a CassandraOnHeapHnsw instance > * VectorSegmentBuilder - that writes a HNSW graph to the following on-disk > components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk HNSW index -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-18715: --- Reviewers: Andres de la Peña, Jonathan Ellis (was: Andres de la Peña) > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-beta, 5.x > > Time Spent: 12h 10m > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a CassandraOnHeapHnsw instance > * VectorSegmentBuilder - that writes a HNSW graph to the following on-disk > components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk HNSW index -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-18715: --- Reviewers: Andres de la Peña > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Time Spent: 12h 10m > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a CassandraOnHeapHnsw instance > * VectorSegmentBuilder - that writes a HNSW graph to the following on-disk > components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk HNSW index -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-18715: --- Fix Version/s: 5.0-beta 5.x > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-beta, 5.x > > Time Spent: 12h 10m > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a CassandraOnHeapHnsw instance > * VectorSegmentBuilder - that writes a HNSW graph to the following on-disk > components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk HNSW index -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18715: - Test and Documentation Plan: Current test run: https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/288/workflows/e102cc8b-303a-400e-8265-f49cadf08eb5/jobs/19867 Status: Patch Available (was: In Progress) > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a CassandraOnHeapHnsw instance > * VectorSegmentBuilder - that writes a HNSW graph to the following on-disk > components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk HNSW index -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18715: - Description: The patch associated with this ticket adds a new vector index to SAI. This introduces the following new elements and changes to SAI: * VectorMemtableIndex - the in-memory representation of the vector indexes that writes data to a CassandraOnHeapHnsw instance * VectorSegmentBuilder - that writes a HNSW graph to the following on-disk components: ** VECTOR - contains the floating point vectors associated with the graph ** TERMS - contains the HNSW graph on-disk representation written by a HnswGraphWriter ** POSTINGS - contains the index postings as written by a VectorPostingsWriter * VectorIndexSegmentSearcher - used to search the on-disk HNSW index > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a CassandraOnHeapHnsw instance > * VectorSegmentBuilder - that writes a HNSW graph to the following on-disk > components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk HNSW index -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18715: - Summary: Add support for vector search in SAI (was: Add support for vector search in SAi) > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAi
[
https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-18715:
-
Description: (was: Approximate nearest neighbour search results are
scored. As such, it is important that the results are returned in score order.
In order to do this proposal is that the ANN search is expressed in the grammar
as:
{code:java}
ORDER BY ANN OF {code}
rather than including the ANN search in the where clause.)
> Add support for vector search in SAi
>
>
> Key: CASSANDRA-18715
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18715
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Vector Search
>Reporter: Mike Adamson
>Priority: Normal
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAi
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18715: - Change Category: Operability Complexity: Challenging Status: Open (was: Triage Needed) > Add support for vector search in SAi > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAi
[
https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-18715:
-
Summary: Add support for vector search in SAi (was: Add support for ORDER
BY ANN OF )
> Add support for vector search in SAi
>
>
> Key: CASSANDRA-18715
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18715
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Vector Search
>Reporter: Mike Adamson
>Priority: Normal
>
> Approximate nearest neighbour search results are scored. As such, it is
> important that the results are returned in score order. In order to do this
> proposal is that the ANN search is expressed in the grammar as:
> {code:java}
> ORDER BY ANN OF {code}
> rather than including the ANN search in the where clause.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
