davidradl commented on code in PR #27216:
URL: https://github.com/apache/flink/pull/27216#discussion_r2509599497
##########
docs/content/docs/dev/table/sourcesSinks.md:
##########
@@ -223,6 +224,20 @@ for more information.
The runtime implementation of a `LookupTableSource` is a `TableFunction` or
`AsyncTableFunction`. The function
will be called with values for the given lookup keys during runtime.
+#### Vector Search Table Source
+
+A `VectorSearchTableSource` searches an external storage system using an input
vector and returns the most similar top-K rows during runtime. Users
+can determine which algorithm to use to calculate the similarity between the
input data and data stored in the external system. In general, most
+vector databases support using Euclidean distance or Cosine distance to
calculate similarity.
+
+Compared to `ScanTableSource`, a `VectorSearchTableSource` currently only
supports emitting insert-only changes.
Review Comment:
I think it would be useful to explain more what this means including why are
we emitting inserts for a read only search. How does this compare to the lookup
processing in this respect.
I wonder if we could put together a table with as many relevant capabilities
and how they effect lookup scan and Vector Search Table. This would make it
really easy to see the similarities and differences. I would suggest you
include example SQL that the planner would use to would target each of these
behaviours.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
