This is an automated email from the ASF dual-hosted git repository.
Gabriel39 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 96ce54ec58a [docs](lance) Document FE table access cache (#4159)
96ce54ec58a is described below
commit 96ce54ec58a6ca81276cfd770baf96ad4e999d56
Author: Gabriel <[email protected]>
AuthorDate: Mon Sep 21 16:02:33 2026 +0800
[docs](lance) Document FE table access cache (#4159)
Repeated Lance query planning can resolve the same table location on
every read. Document the FE access cache introduced by
https://github.com/apache/doris/pull/68305, including its TTL property,
credential exclusions, refresh behavior, and separation from Dataset
versions and BE caches.
This draft depends on the product PR. Merge after the implementation is
accepted.
Self-review:
- Goal: explain when the cache applies and how to disable or invalidate
it; both English and Chinese describe the same behavior.
- Scope: one property row and one focused section in each existing 4.x
Lance Catalog page.
- Information architecture: existing paths, front matter, navigation,
and anchors remain unchanged. Lance Catalog currently has no
corresponding current-version page; this intentionally updates the
existing 4.x pair only.
- Links: new section links match their headings; the implementation link
identifies the required change without claiming an unreleased feature is
already available in every 4.x build.
- Frontend: no component, styling, configuration, or build-script
changes.
- Validation: git diff whitespace check and standalone MDX compilation
passed for both pages (front matter and existing Docusaurus explicit
heading IDs normalized for the standalone compiler). The i18n sync lint
passed with its existing missing-current-counterpart warning. A full
Docusaurus build was not run.
- Other findings: no additional correctness or usability issues found in
the changed content.
---
.../lakehouse/catalogs/lance-catalog.mdx | 29 ++++++++++++++++++++++
.../lakehouse/catalogs/lance-catalog.mdx | 29 ++++++++++++++++++++++
2 files changed, 58 insertions(+)
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
index 6b74648a323..f2f6ca1bbd4 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
@@ -92,6 +92,7 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
| `lance.namespace.parent` | 否 | 空 | 仅访问指定 Lance Namespace 及其子
Namespace。例如默认分隔符下，`production$analytics` 表示两级 Namespace。 |
| `lance.namespace.delimiter` | 否 | `$` | 用于解析 `lance.namespace.parent`，同时会传递给
REST Namespace 客户端。该配置不改变 Doris 中多级 Namespace 的展示方式。 |
| `lance.namespace.root_database` | 否 | `default` | Lance 根 Namespace 在 Doris
中映射的数据库名。 |
+| `lance.table_access_cache_ttl_seconds` | 否 | `60` | 非负整数，表示 FE
表访问缓存的最长有效期，单位为秒；`0` 表示禁用。适用于 Filesystem 和 REST Catalog。仅缓存不含下发存储配置或携带凭证 URI
的结果，详见 [FE 表访问缓存](#fe-表访问缓存)。 |
### Filesystem Catalog
@@ -871,6 +872,34 @@ ORDER BY _distance ASC, user_id;
`filter` 中引用的列由 Lance 内部读取和计算；如果该列未被 `SELECT` 或其他 Doris 表达式引用，则不需要返回给 Doris。
+## FE 表访问缓存
+
+该配置要求 Doris 构建版本包含[表访问缓存变更](https://github.com/apache/doris/pull/68305)。
+
+每个 FE 可以在 Catalog 内缓存表解析后的 Dataset URI 和访问配置，避免查询规划时重复执行文件系统发现或 REST
`describeTable` 请求。每个 Catalog 客户端最多保留 10,000
个条目。`lance.table_access_cache_ttl_seconds` 默认为 `60` 秒，命中缓存不会延长有效期。此缓存不保存
Dataset 版本：每次读取仍会打开 Dataset 并选择快照。
+
+如果响应包含下发的存储配置，即使提供了 `expires_at_millis`，也会在每次读取时重新解析。包含用户信息、查询参数（包括预签名或 SAS
凭证）或片段的 URI 同样不缓存。固定的凭证到期余量无法保证凭证在整个扫描期间有效，BE 也无法在扫描过程中更新这些凭证。响应不含这些字段的 REST
Catalog 可以使用缓存。
+
+显式刷新表或数据库会清除该 Catalog 的全部表访问缓存；Follower FE
重放刷新时，即使对应的本地数据库或表对象已被淘汰，也会清除缓存。普通的本地数据库对象淘汰不会清除此缓存。启用缓存失效的 Catalog
刷新、Namespace 移除或 Catalog 配置变更也会使访问缓存失效。索引检查和索引任务目标校验始终重新解析访问配置。
+
+远端表 URI 或访问配置变更后，可显式刷新，避免等待 TTL 到期。以已有表 `example_lance.default.items` 为例：
+
+```sql
+REFRESH TABLE example_lance.default.items;
+```
+
+下一次读取会重新解析表访问配置。该操作不会清除 Lance 原生 Session 缓存；在相同 URI 下替换 Dataset 后，应刷新 Catalog
并启用缓存失效。
+
+要禁用已有 Catalog 的表访问缓存：
+
+```sql
+ALTER CATALOG example_lance SET PROPERTIES (
+ "lance.table_access_cache_ttl_seconds" = "0"
+);
+```
+
+后续每次读取都会重新解析表访问配置。下文介绍的 BE 缓存仍独立配置。
+
## Lance Cache
Lance Cache 复用已读取的索引、元数据和数据文件内容，减少重复查询的读取开销。缓存由每个 BE 独立管理，同一 BE 内的 Lance
查询共享缓存；不同 BE 不共享缓存容量或内容。普通扫描、向量检索、全文检索和两阶段读取的回读都可以复用相应缓存，无需在 `CREATE CATALOG`
中增加属性。
diff --git a/versioned_docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
b/versioned_docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
index d78fd014561..6f3c13df07b 100644
--- a/versioned_docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
+++ b/versioned_docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
@@ -92,6 +92,7 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
| `lance.namespace.parent` | No | Empty | Limits access to the specified Lance
Namespace and its child Namespaces. With the default delimiter, for example,
`production$analytics` represents a two-level Namespace. |
| `lance.namespace.delimiter` | No | `$` | Delimiter used to parse
`lance.namespace.parent`. It is also passed to the REST Namespace client. This
property does not change how multilevel Namespaces are displayed in Doris. |
| `lance.namespace.root_database` | No | `default` | Doris database name to
which the root Lance Namespace is mapped. |
+| `lance.table_access_cache_ttl_seconds` | No | `60` | Non-negative integer.
Maximum FE table-access cache lifetime in seconds; `0` disables it. Applies to
Filesystem and REST Catalogs. Only results without vended storage options or
credential-bearing URIs are cached; see [FE Table Access
Cache](#fe-table-access-cache). |
### Filesystem Catalog
@@ -871,6 +872,34 @@ If `user_id > 100` must also participate in candidate
generation, include it in
Lance reads and evaluates columns referenced only by `filter` internally. If
such a column is not referenced by `SELECT` or another Doris expression, it
does not have to be returned to Doris.
+## FE Table Access Cache
+
+This setting requires a Doris build containing [the table access cache
change](https://github.com/apache/doris/pull/68305).
+
+Each FE can cache a table's resolved Dataset URI and access options within a
Catalog, avoiding repeated filesystem discovery or REST `describeTable`
requests during query planning. Each Catalog client holds at most 10,000
entries. `lance.table_access_cache_ttl_seconds` defaults to `60`; cache hits do
not extend the lifetime. This cache does not store Dataset versions: each read
still opens the Dataset and selects its snapshot.
+
+A response containing vended storage options is resolved again on every read,
even if it reports `expires_at_millis`. URIs containing userinfo, query
parameters (including presigned/SAS credentials), or fragments are also not
cached. A fixed credential-expiry margin cannot guarantee that credentials
remain valid throughout a scan, and the BE cannot renew these credentials
during the scan. REST Catalogs without these response fields can use the cache.
+
+An explicit table or database refresh clears all table-access entries in that
Catalog, including on follower FEs replaying the refresh when the corresponding
local database/table object has been evicted. Routine local database-object
eviction does not clear the access cache. Catalog refresh with cache
invalidation, namespace removal, or Catalog configuration changes also
invalidate access entries. Index inspection and index-job target validation
always resolve access afresh.
+
+After a remote table URI or access configuration changes, use an explicit
refresh to avoid waiting for TTL expiry. For an existing table
`example_lance.default.items`:
+
+```sql
+REFRESH TABLE example_lance.default.items;
+```
+
+The next read resolves table access again. This does not clear the native
Lance Session cache; use a Catalog refresh with cache invalidation after
replacing a Dataset at the same URI.
+
+To disable table-access caching for an existing Catalog:
+
+```sql
+ALTER CATALOG example_lance SET PROPERTIES (
+ "lance.table_access_cache_ttl_seconds" = "0"
+);
+```
+
+Subsequent reads resolve table access each time. The BE caches described below
remain independently configured.
+
## Lance Cache
Lance Cache reuses index, metadata, and data-file content to reduce repeated
reads. Each BE manages its own cache, shared by Lance queries on that BE. Cache
capacity and content are not shared across BEs. Ordinary scans, vector search,
full-text search, and second-phase fetches can reuse the corresponding caches
without additional `CREATE CATALOG` properties.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
