This is an automated email from the ASF dual-hosted git repository.
Gabriel39 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new eb0eccce850 [docs] Document Paimon dynamic read options (#4021)
eb0eccce850 is described below
commit eb0eccce850cc01ab260a53c45337c6c786accb8
Author: Gabriel <[email protected]>
AuthorDate: Mon Aug 3 13:18:51 2026 +0800
[docs] Document Paimon dynamic read options (#4021)
## Summary
Document the complete user-visible Paimon read behavior implemented by
apache/doris#66247 and its branch-4.1 backport apache/doris#66297.
- Lists all seven supported bounded batch-read options, their defaults,
accepted values, and runtime effects.
- Documents relation, Catalog, physical-table, and Paimon-default
precedence, including validation of the final effective value.
- Explains relation-local aliases, metadata-neutral projection reuse,
and the local runtime cap for manifest planning.
- Documents atomic Catalog ALTER validation and compatibility behavior
for persisted legacy properties.
- Covers statement-consistent snapshot/schema binding across partitions,
statistics, system tables, and data scans.
- Adds the supported Paimon `@options` time-travel selectors,
combination constraints, view behavior, and explicit failure semantics.
- Lists the system tables that accept `@options` and the unsupported
file-creation-time filter.
- Documents exact quoted top-level and nested identifier preservation on
both Paimon JNI scanner paths.
- Records safe logging, the one-second async-reader profile sampling
interval, and retry-safe cleanup behavior.
The same behavior is documented in current and 4.x English and Chinese
pages.
Related code changes:
- apache/doris#66247 (`master`)
- apache/doris#66297 (`branch-4.1`)
## Version scope
The implementation targets Doris master and branch-4.1, so this change
updates `current` and `4.x`. The behavior is not present in 3.x or 2.1.
## Validation
- `yarn docs:links:changed`
- `yarn docs:features:changed`
- `yarn docs:i18n-sync:changed`
- `yarn docs:lint:changed`
- `yarn build` (all configured English, Chinese, Japanese, and
historical-version pages; exit code 0)
- `git diff --check`
All validation commands pass. The reports retain existing
repository-wide Markdown, sidebar, SEO, external-link, and Japanese
translation-candidate notices; no new error is introduced by these
changes.
## Versions
- [x] dev
- [x] 4.x
- [ ] 3.x (feature not supported)
- [ ] 2.1 or older (feature not supported)
## Languages
- [x] Chinese
- [x] English
- [x] Japanese candidate translation needed
## Docs Checklist
- [x] Checked by AI
- [ ] Test Cases Built
- [x] Updated required version and language counterparts, or explained
why not
- [x] If only one language changed, confirmed whether source/translation
counterparts need sync
---
docs/lakehouse/catalogs/paimon-catalog.mdx | 141 +++++++++++++++++----
.../current/lakehouse/catalogs/paimon-catalog.mdx | 128 +++++++++++++++----
.../lakehouse/catalogs/paimon-catalog.mdx | 123 ++++++++++++++++++
.../lakehouse/catalogs/paimon-catalog.mdx | 134 ++++++++++++++++++++
4 files changed, 473 insertions(+), 53 deletions(-)
diff --git a/docs/lakehouse/catalogs/paimon-catalog.mdx
b/docs/lakehouse/catalogs/paimon-catalog.mdx
index 5b1ec808e7e..dea47409c47 100644
--- a/docs/lakehouse/catalogs/paimon-catalog.mdx
+++ b/docs/lakehouse/catalogs/paimon-catalog.mdx
@@ -86,10 +86,25 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
| "paimon.fs.s3.read.ahead.buffer.size" = "1" |
"fs.s3a.read.ahead.buffer.size" = "1" |
| "paimon.s3a.replication.factor" = "3" | "fs.s3a.replication.factor" = "3" |
- **Passing Through Paimon JNI Reader Options**
+ **Paimon Read Options**
- When reading data with the Paimon JNI Reader, you can configure Paimon table
options in the Catalog using
- `paimon.table-option.<option-key>`. For example:
+ Doris supports the following bounded Paimon batch-read options. Set
Catalog-wide query defaults with
+ `paimon.table-option.<option-key>`, or override one table reference with
+ `table@options('<option-key>'='<value>')`. These settings affect only Doris
reads and never update the physical
+ Paimon table metadata.
+
+ | Paimon option | Paimon default | Values accepted by Doris | Effect |
+ | --- | --- | --- | --- |
+ | `read.batch-size` | `1024` | Integer from `1` to `65536` | Number of rows
requested from each Paimon JNI reader batch. An explicitly configured value is
preserved when Doris opens the reader. |
+ | `file-reader-async-threshold` | `10 MB` | Memory size from `1 MB` to `1
GB` | File size threshold at which the Paimon JNI reader switches to
asynchronous reading. |
+ | `file-index.read.enabled` | `true` | Boolean | Enables Paimon file-index
pruning during a read. |
+ | `source.split.target-size` | `128 MB` | Positive memory size | Target size
used when Paimon combines data files into Doris scan splits. |
+ | `source.split.open-file-cost` | `4 MB` | Non-negative memory size |
Estimated cost of opening a file when Paimon combines files into splits. |
+ | `scan.manifest.parallelism` | Number of available processors | Integer
from `1` to `256` | Requested manifest-read parallelism. At execution time,
Doris caps each planning branch independently to the smaller of this value,
`256`, and the processors available to that FE or BE. |
+ | `scan.plan-sort-partition` | `false` | Boolean | Sorts partitions during
Paimon scan planning. |
+
+ For example, the following Catalog settings override values stored in the
physical Paimon table and become the
+ defaults for Doris queries:
```sql
CREATE CATALOG paimon_hms PROPERTIES (
@@ -98,40 +113,72 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
"hive.metastore.uris" = "thrift://127.0.0.1:9083",
"warehouse" = "s3://bucket/paimon-warehouse",
"paimon.table-option.read.batch-size" = "4096",
- "paimon.table-option.file-reader-async-threshold" = "32 mb"
+ "paimon.table-option.file-reader-async-threshold" = "32 MB",
+ "paimon.table-option.source.split.target-size" = "64 MB",
+ "paimon.table-option.scan.manifest.parallelism" = "1"
);
```
- Doris processes these options in the following order:
+ A query can override these defaults without modifying the Catalog or Paimon
table metadata:
- 1. The FE loads the Paimon table and its existing table options.
- 2. The FE removes the `paimon.table-option.` prefix and uses the Catalog
options to fill in options that are not
- configured on the table.
- 3. The FE serializes the Paimon table with the final options and sends it to
the BE.
- 4. The Paimon JNI Scanner on the BE deserializes the table and uses these
options to create the Paimon reader.
- 5. Options that are not configured on either the table or the Catalog use
the Paimon defaults.
-
- The option precedence is:
-
- ```text
- Paimon table option > Doris Catalog table option > Paimon default
+ ```sql
+ SELECT *
+ FROM paimon_hms.db_name.table_name@options(
+ 'read.batch-size' = '8192',
+ 'source.split.target-size' = '32 MB'
+ );
```
- For example, if the Paimon table already has `read.batch-size=1024`, the
value `4096` configured in the Catalog
- does not override it. If the table does not define this option, the JNI
Reader uses the Catalog value `4096`.
+ Each table reference has an independent option set. For example, two aliases
of the same table can use different
+ batch sizes in one statement:
- Catalog-level options take effect only when Doris reads data through JNI.
They are not written back to or used to
- modify the Paimon table metadata. Non-JNI read paths are not guaranteed to
use these options.
+ ```sql
+ SELECT small.id
+ FROM paimon_hms.db_name.table_name@options('read.batch-size' = '1') small
+ JOIN paimon_hms.db_name.table_name@options('read.batch-size' = '8192') large
+ ON small.id = large.id;
+ ```
- Doris validates option names and values against the Paimon `CoreOptions`
provided by the Paimon version on which
- the current Doris version depends. Map options must use the complete key.
For example:
+ The precedence from highest to lowest is:
```text
- "paimon.table-option.file.compression.per.level" = "0:lz4,1:zstd"
+ relation @options > Doris Catalog property > physical Paimon table option >
Paimon default
```
- Dynamic suffixes such as `paimon.table-option.file.compression.per.level.0`
are not supported. Table structure
- options such as Bucket, Primary Key, Partition, and Merge Engine should be
configured on the Paimon table itself.
+ Doris validates the final value after applying this precedence. Therefore, a
safe Catalog or relation value can
+ replace an invalid physical table value; if the final value is still
invalid, Doris rejects the query before the
+ affected planning or reader stage begins. Manifest limits are also enforced
before partition, row-count/statistics,
+ fallback-branch, and system-table manifest planning.
+
+ The first five options in the table are metadata-neutral and can reuse the
cached latest partition projection.
+ `scan.manifest.parallelism` and `scan.plan-sort-partition` affect metadata
planning, so Doris plans them from the
+ effective relation-specific table handle. Per-query values do not resize
Paimon's JVM-global executor.
+
+ Doris rejects unknown or invalid `paimon.table-option.*` properties during
`CREATE CATALOG` and `ALTER CATALOG`.
+ `ALTER CATALOG` validates the complete candidate configuration before
publishing it, so a failed change leaves the
+ previous Catalog configuration effective. Catalogs persisted by an older
Doris version remain loadable, but
+ unsupported or invalid legacy reader properties are ignored instead of being
applied; their persisted values are
+ retained for image and edit-log compatibility.
+
+ Snapshot and startup-position options are relation context selectors and
must use `@options`, as described in
+ [Time Travel](#time-travel-with-options); they cannot be Catalog defaults.
Doris also excludes
+ `scan.max-splits-per-task`, which belongs to Paimon's Flink source
enumerator, `scan.fallback-branch`, and streaming,
+ layout, write, and compaction options. Configure Bucket, Primary Key,
Partition, Merge Engine, and other physical
+ table behavior in Paimon itself.
+
+ :::info Statement consistency
+ Within one statement, Doris keeps schema binding, partition loading,
row-count/statistics collection, system-table
+ planning, and data scanning on the same Paimon snapshot and table
generation. A snapshot or tag selected by
+ `@options` uses the schema that belongs to that historical version,
including nested `STRUCT`, `MAP`, and `ARRAY`
+ fields. Each execution of a prepared statement obtains fresh statement
state, so commits between executions are
+ visible according to the new execution's selectors and cache settings.
+ :::
+
+ For operational safety, Paimon scanner DEBUG configuration messages report
only the batch size and projected-field
+ count; Doris does not dump the raw scanner parameter map, which can contain
credentials. The
+ `PaimonJniAsyncReaderThreadCount` profile gauge is sampled once per second
across scanners, so it can be up to one
+ second stale; this sampling does not affect query execution or scheduling.
Reader and IOManager cleanup are
+ independent and retry-safe after a partial close failure.
**Paimon JNI IOManager**
@@ -158,8 +205,8 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
```
Enabling IOManager does not force every merge read to spill. Paimon spills
only when the number of merge readers
- exceeds its `sort-spill-threshold`. To provide a Catalog-level default for
tables that do not define this option,
- configure `paimon.table-option.sort-spill-threshold` as described in the
preceding section.
+ exceeds its `sort-spill-threshold`. This is not a Doris dynamic reader
option; configure it on the physical Paimon
+ table when different spill behavior is required.
To use dedicated spill disks, configure local paths that exist or can be
created on every BE:
@@ -311,6 +358,10 @@ The currently dependent Paimon version is 1.0.0.
| row | struct |
|
| other | UNSUPPORTED |
|
+Doris preserves the exact spelling of quoted top-level columns and nested
`STRUCT` fields on both Paimon JNI scanner
+paths. Names containing delimiters such as commas (`region,code`), hash signs
(`nested#value`), colons (`colon:name`),
+spaces, or Unicode characters do not require escaping beyond normal SQL
identifier quoting.
+
:::info Note
Doris currently does not support `Timestamp` types with timezone. All
`timestamp_without_time_zone` and `timestamp_with_local_time_zone` will be
uniformly mapped to `datetime(N)` type. However, during reading, Doris will
correctly handle timezones based on the actual source type. For example, after
specifying a timezone with `SET time_zone=<tz>`, it will affect the return
results of `timestamp_with_local_time_zone` columns.
@@ -1000,6 +1051,29 @@ SELECT * FROM paimon_tbl FOR TIME AS OF 1755381952740;
SELECT * FROM paimon_tbl FOR VERSION AS OF 1;
```
+#### Time Travel with `@options` {#time-travel-with-options}
+
+For Paimon-specific selectors, use `@options` on a query relation. Doris
accepts `scan.snapshot-id`,
+`scan.tag-name`, `scan.version`, `scan.timestamp`, `scan.timestamp-millis`,
`scan.watermark`,
+`scan.file-creation-time-millis`, `scan.creation-time-millis`, and
`scan.mode`. The selected snapshot or tag also
+selects its historical schema; Doris does not bind the latest schema to
historical data.
+
+```sql
+-- Read snapshot 1 with the schema stored for snapshot 1.
+SELECT id, old_name
+FROM paimon_tbl@options('scan.snapshot-id' = '1');
+
+-- A retained tag remains readable even after its ordinary snapshot expires.
+SELECT id, old_name
+FROM paimon_tbl@options('scan.tag-name' = 'tag1');
+```
+
+Only one startup-position option can be specified in a table reference.
`scan.mode` can be combined with a position
+only when Paimon defines the pair as compatible; for example,
`scan.mode='from-creation-timestamp'` requires
+`scan.creation-time-millis`. An unknown or expired snapshot, branch, or tag
fails explicitly instead of silently
+falling back to the latest table. The `@options` syntax is supported on query
relations and can be stored in a view
+definition, but it cannot be applied to a CTE reference or non-query commands
such as `SHOW`.
+
### Branch and Tag
> Since version 3.1.0
@@ -1055,6 +1129,19 @@ To access metadata of a Paimon table, add a `$` symbol
after the table name, fol
SELECT * FROM my_table$system_table_name;
```
+The `audit_log`, `binlog`, `manifests`, `partitions`, `ro`, `row_tracking`,
and `table_indexes` system tables accept
+relation `@options`. The selected snapshot or tag is applied to the
system-table rows and, for system tables that expose
+source columns, the matching historical source-table schema:
+
+```sql
+SELECT rowkind, id, old_name
+FROM paimon_tbl$audit_log@options('scan.tag-name' = 'tag1');
+```
+
+Other system tables reject `@options` because they cannot guarantee that every
row-producing stage observes the
+selected snapshot. Paimon system tables also reject
`scan.file-creation-time-millis`; silently dropping this file
+filter could return rows outside the requested range.
+
:::info Note
Doris does not support reading Paimon global system tables, which are only
supported in Flink.
:::
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/catalogs/paimon-catalog.mdx
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/catalogs/paimon-catalog.mdx
index 86a02d5d81f..9fc09cc461b 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/catalogs/paimon-catalog.mdx
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/catalogs/paimon-catalog.mdx
@@ -86,10 +86,24 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
| "paimon.fs.s3.read.ahead.buffer.size" = "1" |
"fs.s3a.read.ahead.buffer.size" = "1" |
| "paimon.s3a.replication.factor" = "3" | "fs.s3a.replication.factor"
= "3" |
- **Paimon JNI Reader 参数透传**
+ **Paimon 读参数**
- 在使用 Paimon JNI Reader 读取数据时，可以通过
- `paimon.table-option.<option-key>` 在 Catalog 中设置 Paimon Table option。例如：
+ Doris 支持下表中有明确边界的 Paimon 批读参数。可以通过 Catalog 属性
+ `paimon.table-option.<option-key>` 为 Doris 查询设置默认值，也可以通过
+ `table@options('<option-key>'='<value>')` 仅覆盖单个表引用。这些参数只影响 Doris 读取，不会修改物理
+ Paimon 表的元数据。
+
+ | Paimon 参数 | Paimon 默认值 | Doris 接受的值 | 作用 |
+ | --- | --- | --- | --- |
+ | `read.batch-size` | `1024` | `1` 到 `65536` 的整数 | Paimon JNI Reader
每批请求的行数。Doris 打开 Reader 时会保留显式配置的值。 |
+ | `file-reader-async-threshold` | `10 MB` | `1 MB` 到 `1 GB` 的内存大小 | Paimon
JNI Reader 切换为异步文件读取的文件大小阈值。 |
+ | `file-index.read.enabled` | `true` | 布尔值 | 是否在读取时启用 Paimon 文件索引裁剪。 |
+ | `source.split.target-size` | `128 MB` | 正数内存大小 | Paimon 将数据文件组合成 Doris
扫描 split 时使用的目标大小。 |
+ | `source.split.open-file-cost` | `4 MB` | 非负内存大小 | Paimon 组合文件生成 split
时估算的文件打开成本。 |
+ | `scan.manifest.parallelism` | 可用处理器数量 | `1` 到 `256` 的整数 | 请求的 Manifest
读取并行度。执行时，Doris 会分别限制每个规划分支，实际值不超过配置值、`256` 以及执行规划的 FE 或 BE 的可用处理器数量三者中的最小值。 |
+ | `scan.plan-sort-partition` | `false` | 布尔值 | 是否在 Paimon 扫描规划时对分区排序。 |
+
+ 例如，以下 Catalog 配置会覆盖物理 Paimon 表中已有的值，并作为 Doris 查询的默认值：
```sql
CREATE CATALOG paimon_hms PROPERTIES (
@@ -98,40 +112,65 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
"hive.metastore.uris" = "thrift://127.0.0.1:9083",
"warehouse" = "s3://bucket/paimon-warehouse",
"paimon.table-option.read.batch-size" = "4096",
- "paimon.table-option.file-reader-async-threshold" = "32 mb"
+ "paimon.table-option.file-reader-async-threshold" = "32 MB",
+ "paimon.table-option.source.split.target-size" = "64 MB",
+ "paimon.table-option.scan.manifest.parallelism" = "1"
);
```
- Doris 按照以下顺序处理这些参数：
+ 查询可以覆盖这些默认值，而不修改 Catalog 或 Paimon 表元数据：
- 1. FE 加载 Paimon 表及其已有的 Table options。
- 2. FE 移除 `paimon.table-option.` 前缀，并使用 Catalog 中的 option 补充表中未配置的
- option。
- 3. FE 将包含最终 options 的 Paimon Table 序列化并发送给 BE。
- 4. BE 上的 Paimon JNI Scanner 反序列化 Table，并使用这些 options 创建 Paimon Reader。
- 5. 表和 Catalog 都没有配置的 option 使用 Paimon 默认值。
+ ```sql
+ SELECT *
+ FROM paimon_hms.db_name.table_name@options(
+ 'read.batch-size' = '8192',
+ 'source.split.target-size' = '32 MB'
+ );
+ ```
- option 的优先级为：
+ 每个表引用使用独立的参数集合。例如，同一语句中同一张表的两个别名可以使用不同的批大小：
+
+ ```sql
+ SELECT small.id
+ FROM paimon_hms.db_name.table_name@options('read.batch-size' = '1') small
+ JOIN paimon_hms.db_name.table_name@options('read.batch-size' = '8192')
large
+ ON small.id = large.id;
+ ```
+
+ 参数优先级从高到低为：
```text
- Paimon 表自身 option > Doris Catalog 中的 Table option > Paimon 默认值
+ 表引用 @options > Doris Catalog 属性 > 物理 Paimon 表参数 > Paimon 默认值
```
- 例如，如果 Paimon 表自身已经设置 `read.batch-size=1024`，Catalog 中设置的 `4096`
- 不会覆盖它；如果表中没有设置，则 JNI Reader 使用 Catalog 中的 `4096`。
+ Doris 会在应用上述优先级后校验最终值。因此，合法的 Catalog 或表引用参数可以覆盖物理表中的非法值；
+ 如果最终值仍然非法，Doris 会在受影响的规划或 Reader 阶段开始前拒绝查询。Manifest 限制还会在分区、
+ 行数/统计信息、fallback 分支和系统表的 Manifest 规划前生效。
- Catalog 中的配置只在 Doris JNI 读取过程中生效，不会写回或修改 Paimon 表元数据。
- 非 JNI 读取路径不保证使用这些参数。
+ 表中的前五个参数不会改变元数据选择，可以复用已缓存的最新分区投影。
+ `scan.manifest.parallelism` 和 `scan.plan-sort-partition` 会影响元数据规划，因此 Doris
使用当前表引用的
+ 最终 Table Handle 进行规划。查询级参数不会调整 Paimon JVM 全局 Executor 的大小。
- Doris 会根据当前版本依赖的 Paimon `CoreOptions` 检查 option 名称和值。Map 类型的
- option 需要使用完整 key，例如：
+ Doris 会在 `CREATE CATALOG` 和 `ALTER CATALOG` 时拒绝未知或非法的
+ `paimon.table-option.*` 属性。`ALTER CATALOG` 会先校验完整的候选配置，再发布修改，因此失败后旧的
+ Catalog 配置仍然生效。旧 Doris 版本持久化的 Catalog 仍可加载，但不受支持或非法的历史 Reader 参数只会被忽略，
+ 不会被应用；原始值仍会保留，以兼容 image 和 edit log。
- ```text
- "paimon.table-option.file.compression.per.level" = "0:lz4,1:zstd"
- ```
+ 快照和启动位置参数属于表引用上下文选择器，必须使用[通过 `@options` 进行时间旅行](#time-travel-with-options)
+ 中介绍的语法，不能配置为 Catalog 默认值。Doris 也不支持仅供 Paimon Flink Source Enumerator 使用的
+ `scan.max-splits-per-task`，以及 `scan.fallback-branch`、流式 Source、表布局、写入和
Compaction 参数。
+ Bucket、Primary Key、Partition、Merge Engine 等物理表行为应在 Paimon 中配置。
+
+ :::info 语句一致性
+ 在同一条语句内，Doris 会让 Schema 绑定、分区加载、行数/统计信息收集、系统表规划和数据扫描使用同一个
+ Paimon 快照与表版本。通过 `@options` 选择快照或 Tag 时，会使用该历史版本对应的 Schema，包括嵌套的
+ `STRUCT`、`MAP` 和 `ARRAY` 字段。预处理语句每次执行都会获取新的语句状态，因此两次执行之间的新提交会按照
+ 新一次执行的选择器和缓存配置决定是否可见。
+ :::
- 不支持 `paimon.table-option.file.compression.per.level.0` 这类带动态后缀的写法。
- Bucket、Primary Key、Partition 和 Merge Engine 等表结构相关参数应在 Paimon 表中设置。
+ 为保障运维安全，Paimon Scanner 的 DEBUG 配置日志只输出批大小和投影字段数量；Doris 不会输出可能包含凭据的
+ 原始 Scanner 参数 Map。Profile 指标 `PaimonJniAsyncReaderThreadCount` 在所有 Scanner
间按一秒周期采样，
+ 因此可能最多滞后一秒；采样不会影响查询执行或调度。Reader 和 IOManager 会独立清理，部分关闭失败后可以安全重试。
**Paimon JNI IOManager**
@@ -157,8 +196,8 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
```
启用 IOManager 不会强制所有 merge read 都进行 spill。只有 merge reader 数量超过 Paimon 的
- `sort-spill-threshold` 时才会触发 spill。如果需要为未设置该参数的表提供 Catalog 级默认值，
- 可以按照上一节的方式配置 `paimon.table-option.sort-spill-threshold`。
+ `sort-spill-threshold` 时才会触发 spill。该参数不是 Doris 动态 Reader 参数；如需改变 spill 行为，
+ 应在物理 Paimon 表中配置。
如果需要使用专用 spill 磁盘，可以配置每个 BE 上均存在或可以创建的本地路径：
@@ -313,6 +352,10 @@ ORDER BY entry_name;
| row | struct |
|
| other | UNSUPPORTED |
|
+在两个 Paimon JNI Scanner 路径上，Doris 都会保留带引号的顶层列名和嵌套 `STRUCT` 字段名的原始大小写与字符。
+包含逗号（`region,code`）、井号（`nested#value`）、冒号（`colon:name`）、空格或 Unicode 字符的名称，除普通
+SQL 标识符引用外无需额外转义。
+
> 注：
>
> Doris 当前不支持带时区的 `Timestamp` 类型。所有 `timestamp_without_time_zone` 和
> `timestamp_with_local_time_zone` 会统一映射到 `datetime(N)` 类型上。但在读取时，Doris
> 会根据实际源类型正确处理时区。如通过 `SET time_zone=<tz>` 指定时区后，会影响
> `timestamp_with_local_time_zone` 列的返回结果。
@@ -1007,6 +1050,28 @@ SELECT * FROM paimon_tbl FOR TIME AS OF 1755381952740;
SELECT * FROM paimon_tbl FOR VERSION AS OF 1;
```
+#### 通过 `@options` 进行时间旅行 {#time-travel-with-options}
+
+如需使用 Paimon 专用选择器，可以在查询的表引用上使用 `@options`。Doris 支持 `scan.snapshot-id`、
+`scan.tag-name`、`scan.version`、`scan.timestamp`、`scan.timestamp-millis`、`scan.watermark`、
+`scan.file-creation-time-millis`、`scan.creation-time-millis` 和
`scan.mode`。选择快照或 Tag 时也会选择对应的
+历史 Schema；Doris 不会把最新 Schema 错误地绑定到历史数据。
+
+```sql
+-- 使用快照 1 中保存的 Schema 读取快照 1。
+SELECT id, old_name
+FROM paimon_tbl@options('scan.snapshot-id' = '1');
+
+-- 即使普通快照已经过期，保留的 Tag 仍然可以读取。
+SELECT id, old_name
+FROM paimon_tbl@options('scan.tag-name' = 'tag1');
+```
+
+一个表引用只能指定一个启动位置参数。只有 Paimon 定义为兼容的组合才能同时使用 `scan.mode` 和位置参数；例如，
+`scan.mode='from-creation-timestamp'` 必须同时指定
`scan.creation-time-millis`。不存在或已过期的快照、Branch
+或 Tag 会明确报错，不会静默回退到最新表。`@options` 可以用于查询的表引用，也可以保存在 View 定义中，但不能
+应用于 CTE 引用或 `SHOW` 等非查询命令。
+
### Branch 和 Tag
> 该功能自 3.1.0 版本支持
@@ -1062,6 +1127,17 @@ Doris 支持查询 Paimon 系统表，用来查询表的相关元信息。支持
SELECT * FROM my_table$system_table_name;
```
+`audit_log`、`binlog`、`manifests`、`partitions`、`ro`、`row_tracking` 和
`table_indexes` 系统表支持表引用
+`@options`。选择的快照或 Tag 会应用于系统表数据；如果系统表会显示源表列，也会使用匹配的历史源表 Schema：
+
+```sql
+SELECT rowkind, id, old_name
+FROM paimon_tbl$audit_log@options('scan.tag-name' = 'tag1');
+```
+
+其他系统表不支持 `@options`，因为它们无法保证所有生成结果行的阶段都使用选定的快照。Paimon 系统表也不支持
+`scan.file-creation-time-millis`；静默丢弃这个文件过滤条件可能返回请求范围以外的数据。
+
> 注意点：Doris 不支持读取 Paimon 全局系统表，其只在 Flink 中支持。
### schemas
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/paimon-catalog.mdx
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/paimon-catalog.mdx
index af5c2272436..3c513e52079 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/paimon-catalog.mdx
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/paimon-catalog.mdx
@@ -86,6 +86,92 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
| "paimon.fs.s3.read.ahead.buffer.size" = "1" |
"fs.s3a.read.ahead.buffer.size" = "1" |
| "paimon.s3a.replication.factor" = "3" | "fs.s3a.replication.factor"
= "3" |
+ **Paimon 读参数**
+
+ Doris 支持下表中有明确边界的 Paimon 批读参数。可以通过 Catalog 属性
+ `paimon.table-option.<option-key>` 为 Doris 查询设置默认值，也可以通过
+ `table@options('<option-key>'='<value>')` 仅覆盖单个表引用。这些参数只影响 Doris 读取，不会修改物理
+ Paimon 表的元数据。
+
+ | Paimon 参数 | Paimon 默认值 | Doris 接受的值 | 作用 |
+ | --- | --- | --- | --- |
+ | `read.batch-size` | `1024` | `1` 到 `65536` 的整数 | Paimon JNI Reader
每批请求的行数。Doris 打开 Reader 时会保留显式配置的值。 |
+ | `file-reader-async-threshold` | `10 MB` | `1 MB` 到 `1 GB` 的内存大小 | Paimon
JNI Reader 切换为异步文件读取的文件大小阈值。 |
+ | `file-index.read.enabled` | `true` | 布尔值 | 是否在读取时启用 Paimon 文件索引裁剪。 |
+ | `source.split.target-size` | `128 MB` | 正数内存大小 | Paimon 将数据文件组合成 Doris
扫描 split 时使用的目标大小。 |
+ | `source.split.open-file-cost` | `4 MB` | 非负内存大小 | Paimon 组合文件生成 split
时估算的文件打开成本。 |
+ | `scan.manifest.parallelism` | 可用处理器数量 | `1` 到 `256` 的整数 | 请求的 Manifest
读取并行度。执行时，Doris 会分别限制每个规划分支，实际值不超过配置值、`256` 以及执行规划的 FE 或 BE 的可用处理器数量三者中的最小值。 |
+ | `scan.plan-sort-partition` | `false` | 布尔值 | 是否在 Paimon 扫描规划时对分区排序。 |
+
+ 例如，以下 Catalog 配置会覆盖物理 Paimon 表中已有的值，并作为 Doris 查询的默认值：
+
+ ```sql
+ CREATE CATALOG paimon_hms PROPERTIES (
+ "type" = "paimon",
+ "paimon.catalog.type" = "hms",
+ "hive.metastore.uris" = "thrift://127.0.0.1:9083",
+ "warehouse" = "s3://bucket/paimon-warehouse",
+ "paimon.table-option.read.batch-size" = "4096",
+ "paimon.table-option.file-reader-async-threshold" = "32 MB",
+ "paimon.table-option.source.split.target-size" = "64 MB",
+ "paimon.table-option.scan.manifest.parallelism" = "1"
+ );
+ ```
+
+ 查询可以覆盖这些默认值，而不修改 Catalog 或 Paimon 表元数据：
+
+ ```sql
+ SELECT *
+ FROM paimon_hms.db_name.table_name@options(
+ 'read.batch-size' = '8192',
+ 'source.split.target-size' = '32 MB'
+ );
+ ```
+
+ 每个表引用使用独立的参数集合。例如，同一语句中同一张表的两个别名可以使用不同的批大小：
+
+ ```sql
+ SELECT small.id
+ FROM paimon_hms.db_name.table_name@options('read.batch-size' = '1') small
+ JOIN paimon_hms.db_name.table_name@options('read.batch-size' = '8192')
large
+ ON small.id = large.id;
+ ```
+
+ 参数优先级从高到低为：
+
+ ```text
+ 表引用 @options > Doris Catalog 属性 > 物理 Paimon 表参数 > Paimon 默认值
+ ```
+
+ Doris 会在应用上述优先级后校验最终值。因此，合法的 Catalog 或表引用参数可以覆盖物理表中的非法值；
+ 如果最终值仍然非法，Doris 会在受影响的规划或 Reader 阶段开始前拒绝查询。Manifest 限制还会在分区、
+ 行数/统计信息、fallback 分支和系统表的 Manifest 规划前生效。
+
+ 表中的前五个参数不会改变元数据选择，可以复用已缓存的最新分区投影。
+ `scan.manifest.parallelism` 和 `scan.plan-sort-partition` 会影响元数据规划，因此 Doris
使用当前表引用的
+ 最终 Table Handle 进行规划。查询级参数不会调整 Paimon JVM 全局 Executor 的大小。
+
+ Doris 会在 `CREATE CATALOG` 和 `ALTER CATALOG` 时拒绝未知或非法的
+ `paimon.table-option.*` 属性。`ALTER CATALOG` 会先校验完整的候选配置，再发布修改，因此失败后旧的
+ Catalog 配置仍然生效。旧 Doris 版本持久化的 Catalog 仍可加载，但不受支持或非法的历史 Reader 参数只会被忽略，
+ 不会被应用；原始值仍会保留，以兼容 image 和 edit log。
+
+ 快照和启动位置参数属于表引用上下文选择器，必须使用[通过 `@options` 进行时间旅行](#time-travel-with-options)
+ 中介绍的语法，不能配置为 Catalog 默认值。Doris 也不支持仅供 Paimon Flink Source Enumerator 使用的
+ `scan.max-splits-per-task`，以及 `scan.fallback-branch`、流式 Source、表布局、写入和
Compaction 参数。
+ Bucket、Primary Key、Partition、Merge Engine 等物理表行为应在 Paimon 中配置。
+
+ :::info 语句一致性
+ 在同一条语句内，Doris 会让 Schema 绑定、分区加载、行数/统计信息收集、系统表规划和数据扫描使用同一个
+ Paimon 快照与表版本。通过 `@options` 选择快照或 Tag 时，会使用该历史版本对应的 Schema，包括嵌套的
+ `STRUCT`、`MAP` 和 `ARRAY` 字段。预处理语句每次执行都会获取新的语句状态，因此两次执行之间的新提交会按照
+ 新一次执行的选择器和缓存配置决定是否可见。
+ :::
+
+ 为保障运维安全，Paimon Scanner 的 DEBUG 配置日志只输出批大小和投影字段数量；Doris 不会输出可能包含凭据的
+ 原始 Scanner 参数 Map。Profile 指标 `PaimonJniAsyncReaderThreadCount` 在所有 Scanner
间按一秒周期采样，
+ 因此可能最多滞后一秒；采样不会影响查询执行或调度。Reader 和 IOManager 会独立清理，部分关闭失败后可以安全重试。
+
* `{CommonProperties}`
CommonProperties 部分用于填写通用属性。请参阅[数据目录概述](../catalog-overview.md)中【通用属性】部分。
@@ -224,6 +310,10 @@ ORDER BY entry_name;
| row | struct |
|
| other | UNSUPPORTED |
|
+在两个 Paimon JNI Scanner 路径上，Doris 都会保留带引号的顶层列名和嵌套 `STRUCT` 字段名的原始大小写与字符。
+包含逗号（`region,code`）、井号（`nested#value`）、冒号（`colon:name`）、空格或 Unicode 字符的名称，除普通
+SQL 标识符引用外无需额外转义。
+
> 注：
>
> Doris 当前不支持带时区的 `Timestamp` 类型。所有 `timestamp_without_time_zone` 和
> `timestamp_with_local_time_zone` 会统一映射到 `datetime(N)` 类型上。但在读取时，Doris
> 会根据实际源类型正确处理时区。如通过 `SET time_zone=<tz>` 指定时区后，会影响
> `timestamp_with_local_time_zone` 列的返回结果。
@@ -918,6 +1008,28 @@ SELECT * FROM paimon_tbl FOR TIME AS OF 1755381952740;
SELECT * FROM paimon_tbl FOR VERSION AS OF 1;
```
+#### 通过 `@options` 进行时间旅行 {#time-travel-with-options}
+
+如需使用 Paimon 专用选择器，可以在查询的表引用上使用 `@options`。Doris 支持 `scan.snapshot-id`、
+`scan.tag-name`、`scan.version`、`scan.timestamp`、`scan.timestamp-millis`、`scan.watermark`、
+`scan.file-creation-time-millis`、`scan.creation-time-millis` 和
`scan.mode`。选择快照或 Tag 时也会选择对应的
+历史 Schema；Doris 不会把最新 Schema 错误地绑定到历史数据。
+
+```sql
+-- 使用快照 1 中保存的 Schema 读取快照 1。
+SELECT id, old_name
+FROM paimon_tbl@options('scan.snapshot-id' = '1');
+
+-- 即使普通快照已经过期，保留的 Tag 仍然可以读取。
+SELECT id, old_name
+FROM paimon_tbl@options('scan.tag-name' = 'tag1');
+```
+
+一个表引用只能指定一个启动位置参数。只有 Paimon 定义为兼容的组合才能同时使用 `scan.mode` 和位置参数；例如，
+`scan.mode='from-creation-timestamp'` 必须同时指定
`scan.creation-time-millis`。不存在或已过期的快照、Branch
+或 Tag 会明确报错，不会静默回退到最新表。`@options` 可以用于查询的表引用，也可以保存在 View 定义中，但不能
+应用于 CTE 引用或 `SHOW` 等非查询命令。
+
### Branch 和 Tag
> 该功能自 3.1.0 版本支持
@@ -973,6 +1085,17 @@ Doris 支持查询 Paimon 系统表，用来查询表的相关元信息。支持
SELECT * FROM my_table$system_table_name;
```
+`audit_log`、`binlog`、`manifests`、`partitions`、`ro`、`row_tracking` 和
`table_indexes` 系统表支持表引用
+`@options`。选择的快照或 Tag 会应用于系统表数据；如果系统表会显示源表列，也会使用匹配的历史源表 Schema：
+
+```sql
+SELECT rowkind, id, old_name
+FROM paimon_tbl$audit_log@options('scan.tag-name' = 'tag1');
+```
+
+其他系统表不支持 `@options`，因为它们无法保证所有生成结果行的阶段都使用选定的快照。Paimon 系统表也不支持
+`scan.file-creation-time-millis`；静默丢弃这个文件过滤条件可能返回请求范围以外的数据。
+
> 注意点：Doris 不支持读取 Paimon 全局系统表，其只在 Flink 中支持。
### schemas
diff --git a/versioned_docs/version-4.x/lakehouse/catalogs/paimon-catalog.mdx
b/versioned_docs/version-4.x/lakehouse/catalogs/paimon-catalog.mdx
index 52e08f1bf99..62a76632f76 100644
--- a/versioned_docs/version-4.x/lakehouse/catalogs/paimon-catalog.mdx
+++ b/versioned_docs/version-4.x/lakehouse/catalogs/paimon-catalog.mdx
@@ -86,6 +86,100 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
| "paimon.fs.s3.read.ahead.buffer.size" = "1" |
"fs.s3a.read.ahead.buffer.size" = "1" |
| "paimon.s3a.replication.factor" = "3" | "fs.s3a.replication.factor" = "3" |
+ **Paimon Read Options**
+
+ Doris supports the following bounded Paimon batch-read options. Set
Catalog-wide query defaults with
+ `paimon.table-option.<option-key>`, or override one table reference with
+ `table@options('<option-key>'='<value>')`. These settings affect only Doris
reads and never update the physical
+ Paimon table metadata.
+
+ | Paimon option | Paimon default | Values accepted by Doris | Effect |
+ | --- | --- | --- | --- |
+ | `read.batch-size` | `1024` | Integer from `1` to `65536` | Number of rows
requested from each Paimon JNI reader batch. An explicitly configured value is
preserved when Doris opens the reader. |
+ | `file-reader-async-threshold` | `10 MB` | Memory size from `1 MB` to `1
GB` | File size threshold at which the Paimon JNI reader switches to
asynchronous reading. |
+ | `file-index.read.enabled` | `true` | Boolean | Enables Paimon file-index
pruning during a read. |
+ | `source.split.target-size` | `128 MB` | Positive memory size | Target size
used when Paimon combines data files into Doris scan splits. |
+ | `source.split.open-file-cost` | `4 MB` | Non-negative memory size |
Estimated cost of opening a file when Paimon combines files into splits. |
+ | `scan.manifest.parallelism` | Number of available processors | Integer
from `1` to `256` | Requested manifest-read parallelism. At execution time,
Doris caps each planning branch independently to the smaller of this value,
`256`, and the processors available to that FE or BE. |
+ | `scan.plan-sort-partition` | `false` | Boolean | Sorts partitions during
Paimon scan planning. |
+
+ For example, the following Catalog settings override values stored in the
physical Paimon table and become the
+ defaults for Doris queries:
+
+ ```sql
+ CREATE CATALOG paimon_hms PROPERTIES (
+ "type" = "paimon",
+ "paimon.catalog.type" = "hms",
+ "hive.metastore.uris" = "thrift://127.0.0.1:9083",
+ "warehouse" = "s3://bucket/paimon-warehouse",
+ "paimon.table-option.read.batch-size" = "4096",
+ "paimon.table-option.file-reader-async-threshold" = "32 MB",
+ "paimon.table-option.source.split.target-size" = "64 MB",
+ "paimon.table-option.scan.manifest.parallelism" = "1"
+ );
+ ```
+
+ A query can override these defaults without modifying the Catalog or Paimon
table metadata:
+
+ ```sql
+ SELECT *
+ FROM paimon_hms.db_name.table_name@options(
+ 'read.batch-size' = '8192',
+ 'source.split.target-size' = '32 MB'
+ );
+ ```
+
+ Each table reference has an independent option set. For example, two aliases
of the same table can use different
+ batch sizes in one statement:
+
+ ```sql
+ SELECT small.id
+ FROM paimon_hms.db_name.table_name@options('read.batch-size' = '1') small
+ JOIN paimon_hms.db_name.table_name@options('read.batch-size' = '8192') large
+ ON small.id = large.id;
+ ```
+
+ The precedence from highest to lowest is:
+
+ ```text
+ relation @options > Doris Catalog property > physical Paimon table option >
Paimon default
+ ```
+
+ Doris validates the final value after applying this precedence. Therefore, a
safe Catalog or relation value can
+ replace an invalid physical table value; if the final value is still
invalid, Doris rejects the query before the
+ affected planning or reader stage begins. Manifest limits are also enforced
before partition, row-count/statistics,
+ fallback-branch, and system-table manifest planning.
+
+ The first five options in the table are metadata-neutral and can reuse the
cached latest partition projection.
+ `scan.manifest.parallelism` and `scan.plan-sort-partition` affect metadata
planning, so Doris plans them from the
+ effective relation-specific table handle. Per-query values do not resize
Paimon's JVM-global executor.
+
+ Doris rejects unknown or invalid `paimon.table-option.*` properties during
`CREATE CATALOG` and `ALTER CATALOG`.
+ `ALTER CATALOG` validates the complete candidate configuration before
publishing it, so a failed change leaves the
+ previous Catalog configuration effective. Catalogs persisted by an older
Doris version remain loadable, but
+ unsupported or invalid legacy reader properties are ignored instead of being
applied; their persisted values are
+ retained for image and edit-log compatibility.
+
+ Snapshot and startup-position options are relation context selectors and
must use `@options`, as described in
+ [Time Travel](#time-travel-with-options); they cannot be Catalog defaults.
Doris also excludes
+ `scan.max-splits-per-task`, which belongs to Paimon's Flink source
enumerator, `scan.fallback-branch`, and streaming,
+ layout, write, and compaction options. Configure Bucket, Primary Key,
Partition, Merge Engine, and other physical
+ table behavior in Paimon itself.
+
+ :::info Statement consistency
+ Within one statement, Doris keeps schema binding, partition loading,
row-count/statistics collection, system-table
+ planning, and data scanning on the same Paimon snapshot and table
generation. A snapshot or tag selected by
+ `@options` uses the schema that belongs to that historical version,
including nested `STRUCT`, `MAP`, and `ARRAY`
+ fields. Each execution of a prepared statement obtains fresh statement
state, so commits between executions are
+ visible according to the new execution's selectors and cache settings.
+ :::
+
+ For operational safety, Paimon scanner DEBUG configuration messages report
only the batch size and projected-field
+ count; Doris does not dump the raw scanner parameter map, which can contain
credentials. The
+ `PaimonJniAsyncReaderThreadCount` profile gauge is sampled once per second
across scanners, so it can be up to one
+ second stale; this sampling does not affect query execution or scheduling.
Reader and IOManager cleanup are
+ independent and retry-safe after a partial close failure.
+
* `{CommonProperties}`
The CommonProperties section is used to fill in common properties. Please
refer to the [Catalog Overview](../catalog-overview.md) section on [Common
Properties].
@@ -221,6 +315,10 @@ The currently dependent Paimon version is 1.0.0.
| row | struct |
|
| other | UNSUPPORTED |
|
+Doris preserves the exact spelling of quoted top-level columns and nested
`STRUCT` fields on both Paimon JNI scanner
+paths. Names containing delimiters such as commas (`region,code`), hash signs
(`nested#value`), colons (`colon:name`),
+spaces, or Unicode characters do not require escaping beyond normal SQL
identifier quoting.
+
:::info Note
Doris currently does not support `Timestamp` types with timezone. All
`timestamp_without_time_zone` and `timestamp_with_local_time_zone` will be
uniformly mapped to `datetime(N)` type. However, during reading, Doris will
correctly handle timezones based on the actual source type. For example, after
specifying a timezone with `SET time_zone=<tz>`, it will affect the return
results of `timestamp_with_local_time_zone` columns.
@@ -910,6 +1008,29 @@ SELECT * FROM paimon_tbl FOR TIME AS OF 1755381952740;
SELECT * FROM paimon_tbl FOR VERSION AS OF 1;
```
+#### Time Travel with `@options` {#time-travel-with-options}
+
+For Paimon-specific selectors, use `@options` on a query relation. Doris
accepts `scan.snapshot-id`,
+`scan.tag-name`, `scan.version`, `scan.timestamp`, `scan.timestamp-millis`,
`scan.watermark`,
+`scan.file-creation-time-millis`, `scan.creation-time-millis`, and
`scan.mode`. The selected snapshot or tag also
+selects its historical schema; Doris does not bind the latest schema to
historical data.
+
+```sql
+-- Read snapshot 1 with the schema stored for snapshot 1.
+SELECT id, old_name
+FROM paimon_tbl@options('scan.snapshot-id' = '1');
+
+-- A retained tag remains readable even after its ordinary snapshot expires.
+SELECT id, old_name
+FROM paimon_tbl@options('scan.tag-name' = 'tag1');
+```
+
+Only one startup-position option can be specified in a table reference.
`scan.mode` can be combined with a position
+only when Paimon defines the pair as compatible; for example,
`scan.mode='from-creation-timestamp'` requires
+`scan.creation-time-millis`. An unknown or expired snapshot, branch, or tag
fails explicitly instead of silently
+falling back to the latest table. The `@options` syntax is supported on query
relations and can be stored in a view
+definition, but it cannot be applied to a CTE reference or non-query commands
such as `SHOW`.
+
### Branch and Tag
> Since version 3.1.0
@@ -965,6 +1086,19 @@ To access metadata of a Paimon table, add a `$` symbol
after the table name, fol
SELECT * FROM my_table$system_table_name;
```
+The `audit_log`, `binlog`, `manifests`, `partitions`, `ro`, `row_tracking`,
and `table_indexes` system tables accept
+relation `@options`. The selected snapshot or tag is applied to the
system-table rows and, for system tables that expose
+source columns, the matching historical source-table schema:
+
+```sql
+SELECT rowkind, id, old_name
+FROM paimon_tbl$audit_log@options('scan.tag-name' = 'tag1');
+```
+
+Other system tables reject `@options` because they cannot guarantee that every
row-producing stage observes the
+selected snapshot. Paimon system tables also reject
`scan.file-creation-time-millis`; silently dropping this file
+filter could return rows outside the requested range.
+
:::info Note
Doris does not support reading Paimon global system tables, which are only
supported in Flink.
:::
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
