This is an automated email from the ASF dual-hosted git repository.
wenjin272 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/flink-agents.git
The following commit(s) were added to refs/heads/main by this push:
new 8c17640a [hotfix][vector-store][java] Update Elasticsearch filters
documentation (#997)
8c17640a is described below
commit 8c17640a45b02d1a8975050a9164e9c1565dcf18
Author: kaiwangleo <[email protected]>
AuthorDate: Tue Aug 25 16:37:14 2026 +0800
[hotfix][vector-store][java] Update Elasticsearch filters documentation
(#997)
Co-authored-by: Leo Wang <[email protected]>
---
.../elasticsearch/ElasticsearchVectorStore.java | 95 +++++++++++++---------
1 file changed, 56 insertions(+), 39 deletions(-)
diff --git
a/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java
b/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java
index 5d66e479..4be3d6be 100644
---
a/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java
+++
b/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java
@@ -112,7 +112,7 @@ public class ElasticsearchVectorStore extends
BaseVectorStore
/** Default vector dimensionality used when {@code dims} is not provided.
*/
public static final int DEFAULT_DIMENSION = 768;
- /** The maximum number of documents that can be retrieved in get. */
+ /** Default request size used by {@code get} when no limit is provided. */
public static final int MAX_RESULT_WINDOW = 10000;
public static final String DEFAULT_METADATA_FIELD = "_metadata";
@@ -336,23 +336,31 @@ public class ElasticsearchVectorStore extends
BaseVectorStore
/**
* Retrieve documents from the vector store.
*
- * <p>If ids is not provided, this method will retrieve documents
according to {@code limit},
- * {@code offset}, and {@code filter_query} in additional arguments. If
{@code limit} is null,
- * up to {@link ElasticsearchVectorStore#MAX_RESULT_WINDOW} documents are
returned (an
- * Elasticsearch ceiling).
+ * <p>When {@code ids} is non-empty, documents are retrieved directly by
ID and the filter,
+ * limit, offset, and {@code filter_query} arguments are not applied.
*
- * <p>The unified {@code filters} DSL parameter is not yet translated to
Elasticsearch's native
- * query DSL — callers needing structured filtering should pass a raw
{@code filter_query} via
- * {@code extraArgs}. TODO: implement equality-DSL translation parallel to
the Python Chroma
- * implementation.
+ * <p>Otherwise, {@code filters} provides equality-only matching against
document metadata. Each
+ * entry is translated to an Elasticsearch {@code term} query on {@code
+ * <metadataField>.<key>.keyword}, and multiple entries are combined with
AND semantics. Because
+ * Elasticsearch dynamic mapping creates {@code .keyword} sub-fields only
for strings, filters
+ * on non-string metadata values do not match; use a raw {@code
filter_query} for those values.
+ * A raw Elasticsearch JSON query may also be supplied as {@code
filter_query} in {@code
+ * extraArgs}; when both forms are present, they are combined with AND
semantics.
*
- * @param ids The ids of the documents.
- * @param collection The name of the collection to be retrieved. If is
null, retrieve the
- * default collection.
- * @param filters Unified filter DSL. Currently ignored — see method
Javadoc.
- * @param limit Maximum number of documents to return; falls back to {@link
- * ElasticsearchVectorStore#MAX_RESULT_WINDOW} when null.
- * @param extraArgs Additional arguments. (offset, filter_query, etc.)
+ * <p>The {@code limit} parameter takes precedence over a {@code limit}
value in {@code
+ * extraArgs}. If neither limit is provided, the request size defaults to
{@link
+ * ElasticsearchVectorStore#MAX_RESULT_WINDOW} (10,000). The effective
Elasticsearch
+ * result-window limit is controlled by the target index's {@code
index.max_result_window}
+ * setting, which defaults to 10,000 but is configurable. Elasticsearch
rejects requests when
+ * the combined {@code offset} and request size exceed that setting rather
than truncating them.
+ * {@code extraArgs} may also contain an {@code offset}.
+ *
+ * @param ids The IDs of documents to retrieve directly.
+ * @param collection The collection name, or null to use the default
collection.
+ * @param filters Equality-only metadata filters combined with AND
semantics.
+ * @param limit Maximum number of documents to return for filtered or
unfiltered searches.
+ * @param extraArgs Additional arguments, including {@code offset} and raw
JSON {@code
+ * filter_query}.
* @return List of documents retrieved.
*/
@Override
@@ -379,22 +387,24 @@ public class ElasticsearchVectorStore extends
BaseVectorStore
}
/**
- * Delete documents in the vector store.
+ * Delete documents from the vector store.
*
- * <p>If ids is not provided, this method will delete documents matched
the {@code filter_query}
- * in additional arguments. If neither {@code filter_query} nor {@code
filters} is provided,
- * this method will delete all the documents.
+ * <p>When {@code ids} is non-empty, documents are deleted directly by ID
and {@code filters}
+ * and {@code filter_query} are not applied.
*
- * <p>The unified {@code filters} DSL parameter is not yet translated to
Elasticsearch's native
- * query DSL — callers needing structured filtering should pass a raw
{@code filter_query} via
- * {@code extraArgs}. TODO: implement equality-DSL translation parallel to
the Python Chroma
- * implementation.
+ * <p>Otherwise, {@code filters} provides equality-only matching against
document metadata. Each
+ * entry is translated to an Elasticsearch {@code term} query on {@code
+ * <metadataField>.<key>.keyword}, and multiple entries are combined with
AND semantics. Because
+ * Elasticsearch dynamic mapping creates {@code .keyword} sub-fields only
for strings, filters
+ * on non-string metadata values do not match; use a raw {@code
filter_query} for those values.
+ * A raw Elasticsearch JSON query may also be supplied as {@code
filter_query} in {@code
+ * extraArgs}; when both forms are present, they are combined with AND
semantics. If neither
+ * form is supplied, all documents in the collection are deleted.
*
- * @param ids The ids of the documents.
- * @param collection The name of the collection the documents belong to.
If is null, use the
- * default collection.
- * @param filters Unified filter DSL. Currently ignored — see method
Javadoc.
- * @param extraArgs Additional arguments. (filter_query, etc.)
+ * @param ids The IDs of documents to delete directly.
+ * @param collection The collection name, or null to use the default
collection.
+ * @param filters Equality-only metadata filters combined with AND
semantics.
+ * @param extraArgs Additional arguments, including raw JSON {@code
filter_query}.
*/
@Override
public void delete(
@@ -571,19 +581,26 @@ public class ElasticsearchVectorStore extends
BaseVectorStore
* Executes a KNN vector search using a pre-computed embedding.
*
* <p>The method prepares a KNN search request using the supplied {@code
embedding} and merges
- * default arguments from the store with the provided {@code args}.
Optional filter queries
- * (JSON DSL) restrict the documents the KNN search may match, so the
nearest neighbours are
+ * default arguments from the store with the provided {@code args}. {@code
filters} provides
+ * equality-only matching against metadata fields. Each entry targets
{@code
+ * <metadataField>.<key>.keyword}; multiple entries are combined with AND
semantics. Because
+ * Elasticsearch dynamic mapping creates {@code .keyword} sub-fields only
for strings, filters
+ * on non-string metadata values do not match; use a raw {@code
filter_query} for those values.
+ *
+ * <p>A raw Elasticsearch JSON query may also be supplied as {@code
filter_query} in {@code
+ * args}. When both filter forms are present, they are combined with AND
semantics. The combined
+ * filter restricts the documents the KNN search may match, so the nearest
neighbours are
* selected from among the matching documents rather than filtered out
afterwards. Up to {@code
* k} matching documents are returned even when the closest vectors
overall do not match.
*
- * @param embedding The embedding vector to search with
- * @param limit Maximum number of items the caller is interested in; used
as a fallback for
- * {@code k} if not explicitly provided
- * @param collection The index to query search. If is null, search the
default index.
- * @param args Additional arguments. Supported keys: {@code k}, {@code
num_candidates}, {@code
- * filter_query}
- * @return A list of matching documents, possibly empty
- * @throws RuntimeException if the search request fails
+ * @param embedding The embedding vector to search with.
+ * @param limit Maximum number of items requested; used as a fallback for
{@code k}.
+ * @param collection The collection name, or null to use the default
collection.
+ * @param filters Equality-only metadata filters combined with AND
semantics.
+ * @param args Additional arguments. Supported keys are {@code k}, {@code
num_candidates}, and
+ * raw JSON {@code filter_query}.
+ * @return A list of matching documents, possibly empty.
+ * @throws RuntimeException if the search request fails.
*/
@SuppressWarnings({"rawtypes", "unchecked"})
@Override
