This is an automated email from the ASF dual-hosted git repository. github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/dev/pr-12476-426e44f96ca09618286280176dd7ef931511d6e6 in repository https://gitbox.apache.org/repos/asf/seatunnel.git
commit e2f7ab643fc8a11a3b9140fc19f306ede01cf57a Author: Jast <[email protected]> AuthorDate: Mon Sep 28 11:39:26 2026 +0000 [Docs] Fix transform doc inaccuracies verified against source code (#12476) Co-authored-by: jast <[email protected]> --- docs/en/transforms/copy.md | 12 ++++++++++++ docs/en/transforms/data-validator.md | 2 +- docs/en/transforms/field-rename.md | 2 +- docs/en/transforms/filter.md | 4 ++++ docs/en/transforms/jsonpath.md | 7 ++++--- docs/en/transforms/metadata.md | 2 +- docs/en/transforms/table-filter.md | 2 +- docs/en/transforms/table-merge.md | 1 - docs/en/transforms/table-rename.md | 2 +- docs/zh/transforms/copy.md | 12 ++++++++++++ docs/zh/transforms/data-validator.md | 2 +- docs/zh/transforms/field-rename.md | 2 +- docs/zh/transforms/filter.md | 4 ++++ docs/zh/transforms/jsonpath.md | 7 ++++--- docs/zh/transforms/metadata.md | 2 +- docs/zh/transforms/table-filter.md | 2 +- docs/zh/transforms/table-rename.md | 2 +- 17 files changed, 50 insertions(+), 17 deletions(-) diff --git a/docs/en/transforms/copy.md b/docs/en/transforms/copy.md index 56da863c0f..bae22eb634 100644 --- a/docs/en/transforms/copy.md +++ b/docs/en/transforms/copy.md @@ -11,11 +11,23 @@ Copy a field to a new field. | name | type | required | default value | |--------|--------|----------|---------------| | fields | Object | yes | | +| src_field | String | no | | +| dest_field | String | no | | ### fields [config] Specify the field copy relationship between input and output +### src_field [string] (deprecated) + +The source field you want to copy. This is a deprecated single-field alternative to `fields`; new configurations should use `fields`. + +When `src_field` is used, `dest_field` must also be set, and neither of them can be combined with `fields`. + +### dest_field [string] (deprecated) + +Copy the `src_field` to this destination field. Required when `src_field` is provided. + ### common options [string] Transform plugin common parameters, please refer to [Transform Plugin](common-options/common-options.md) for details diff --git a/docs/en/transforms/data-validator.md b/docs/en/transforms/data-validator.md index 2d78d8e183..fb3349b34f 100644 --- a/docs/en/transforms/data-validator.md +++ b/docs/en/transforms/data-validator.md @@ -25,7 +25,7 @@ Error handling strategy when validation fails: ### row_error_handle_way.error_table [string] -Target table name for routing invalid data when `row_error_handle_way` is set to `ROUTE_TO_TABLE`. This parameter is required when using `ROUTE_TO_TABLE` mode. +Target table name for routing invalid data when `row_error_handle_way` is set to `ROUTE_TO_TABLE`. This option is not validated by the framework, but if it is not configured in `ROUTE_TO_TABLE` mode, DataValidator cannot route invalid rows and will skip them with a warning instead, so it should always be set when using `ROUTE_TO_TABLE`. #### Error Table Schema diff --git a/docs/en/transforms/field-rename.md b/docs/en/transforms/field-rename.md index 82245b060b..7682b03c3d 100644 --- a/docs/en/transforms/field-rename.md +++ b/docs/en/transforms/field-rename.md @@ -10,7 +10,7 @@ FieldRename transform plugin for rename field name. | name | type | required | default value | Description | |:-----------------------:|--------|----------|---------------|-----------------------------------------------------------------------------------------------------------------------| -| convert_case | string | no | | The case conversion type. The options can be `UPPER`, `LOWER` | +| convert_case | enum | no | | The case conversion type. The options can be `UPPER`, `LOWER` | | prefix | string | no | | The prefix to be added to the field name | | suffix | string | no | | The suffix to be added to the field name | | replacements_with_regex | array | no | | The array of replacement rules. Each rule is a map with `replace_from`, `replace_to`, and optional `is_regex` (default `true`). When `is_regex=false`, `replace_from` is treated as an exact field name (full match). | diff --git a/docs/en/transforms/filter.md b/docs/en/transforms/filter.md index 9f8bdeb105..ca9e0d0e3b 100644 --- a/docs/en/transforms/filter.md +++ b/docs/en/transforms/filter.md @@ -19,6 +19,10 @@ Notice, you must set one and only one of `include_fields` and `exclude_fields` p The list of fields that need to be kept. Fields not in the list will be deleted. +:::note +For backward compatibility, the deprecated option name `fields` is still accepted as a fallback for `include_fields`. New configurations should use `include_fields`. +::: + ### exclude_fields [array] The list of fields that need to be deleted. Fields not in the list will be kept. diff --git a/docs/en/transforms/jsonpath.md b/docs/en/transforms/jsonpath.md index 8c866335a1..66de4a84cc 100644 --- a/docs/en/transforms/jsonpath.md +++ b/docs/en/transforms/jsonpath.md @@ -23,6 +23,7 @@ This option is used to specify the processing method when an error occurs in the - FAIL: When `FAIL` is selected, data format error will block and an exception will be thrown. - SKIP: When `SKIP` is selected, data format error will skip this row data. +- ROUTE_TO_TABLE: not implemented by the JsonPath transform yet. The value can be configured, but its actual behavior is identical to `FAIL`: rows that fail to parse make the job fail and are not routed to an error table. ### columns [array] @@ -190,9 +191,9 @@ transform { Then the data result table `fake1` will like this -| data | c1_string | c1_boolean | c1_integer | c1_float | c1_double | c1_decimal | c1_date | c1_datetime | c1_array | -|------------------------------|------------------|------------|------------|----------|-----------|------------|------------|--------------|-----------------------------| -| too much content not to show | this is a string | true | 42 | 3.14 | 3.14 | 10.55 | 2023-10-29 | 16:12:43.459 | ["item1", "item2", "item3"] | +| data | c1_string | c1_boolean | c1_integer | c1_float | c1_double | c1_decimal | c1_date | c1_datetime | c1_array | c1_map_array | +|------------------------------|------------------|------------|------------|----------|-----------|------------|------------|--------------|-----------------------------|------------------------------| +| too much content not to show | this is a string | true | 42 | 3.14 | 3.14 | 10.55 | 2023-10-29 | 16:12:43.459 | ["item1", "item2", "item3"] | [{"key1": "value1", "key2": "value2"}] | ## Read SeatunnelRow Example diff --git a/docs/en/transforms/metadata.md b/docs/en/transforms/metadata.md index 3fe268dfc7..f1ca25eead 100644 --- a/docs/en/transforms/metadata.md +++ b/docs/en/transforms/metadata.md @@ -109,7 +109,7 @@ transform { | name | type | required | default value | description | |:---------------:|------|:--------:|:-------------:|-------------------| -| metadata_fields | map | no | empty map | Mapping relationship between metadata fields and output fields, format: `Metadata Key = output field name` | +| metadata_fields | map | yes | - | Mapping relationship between metadata fields and output fields, format: `Metadata Key = output field name`. Must contain at least one entry. | ### metadata_fields [map] diff --git a/docs/en/transforms/table-filter.md b/docs/en/transforms/table-filter.md index 84bb16e711..31e2da3079 100644 --- a/docs/en/transforms/table-filter.md +++ b/docs/en/transforms/table-filter.md @@ -13,7 +13,7 @@ TableFilter transform plugin for filter tables. | database_pattern | string | no | | Specify database filter pattern, the default value is null, which means no filtering. If you want to filter the database name, please set it to a regular expression. | | schema_pattern | string | no | | Specify schema filter pattern, the default value is null, which means no filtering. If you want to filter the schema name, please set it to a regular expression. | | table_pattern | string | no | | Specify table filter pattern, the default value is null, which means no filtering. If you want to filter the table name, please set it to a regular expression. | -| pattern_mode | string | no | INCLUDE | Specify pattern mode, the default value is INCLUDE, which means include the matched table. If you want to exclude the matched table, please set it to EXCLUDE. | +| pattern_mode | enum | no | INCLUDE | Specify pattern mode, the default value is INCLUDE, which means include the matched table. If you want to exclude the matched table, please set it to EXCLUDE. | ## Examples diff --git a/docs/en/transforms/table-merge.md b/docs/en/transforms/table-merge.md index 108d014951..90c66366d0 100644 --- a/docs/en/transforms/table-merge.md +++ b/docs/en/transforms/table-merge.md @@ -18,7 +18,6 @@ TableMerge transform plugin for merge sharding-tables. ### Merge sharding-tables -` ```hocon env { parallelism = 1 diff --git a/docs/en/transforms/table-rename.md b/docs/en/transforms/table-rename.md index 2ee1b6832f..636408a196 100644 --- a/docs/en/transforms/table-rename.md +++ b/docs/en/transforms/table-rename.md @@ -10,7 +10,7 @@ TableRename transform plugin for rename table name. | name | type | required | default value | Description | |:-----------------------:|--------|----------|---------------|-----------------------------------------------------------------------------------------------------------------------| -| convert_case | string | no | | The case conversion type. The options can be `UPPER`, `LOWER` | +| convert_case | enum | no | | The case conversion type. The options can be `UPPER`, `LOWER` | | prefix | string | no | | The prefix to be added to the table name | | suffix | string | no | | The suffix to be added to the table name | | replacements_with_regex | array | no | | The array of replacement rules with regex. The replacement rule is a map with `replace_from` and `replace_to` fields. | diff --git a/docs/zh/transforms/copy.md b/docs/zh/transforms/copy.md index 1139474678..8b59a6be8b 100644 --- a/docs/zh/transforms/copy.md +++ b/docs/zh/transforms/copy.md @@ -11,11 +11,23 @@ | 名称 | 类型 | 是否必须 | 默认值 | |--------|--------|------|-----| | fields | Object | yes | | +| src_field | String | no | | +| dest_field | String | no | | ### fields [config] 指定输入和输出之间的字段复制关系 +### src_field [string]（已废弃） + +想要复制的源字段。这是 `fields` 的废弃单字段替代写法，新配置请使用 `fields`。 + +使用 `src_field` 时必须同时设置 `dest_field`，且两者不能与 `fields` 同时使用。 + +### dest_field [string]（已废弃） + +将 `src_field` 复制到的目标字段。当配置了 `src_field` 时必须设置。 + ### 常见选项 [string] 转换插件的常见参数, 请参考 [Transform Plugin](common-options/common-options.md) 了解详情。 diff --git a/docs/zh/transforms/data-validator.md b/docs/zh/transforms/data-validator.md index 28ee4bd8af..937de608f0 100644 --- a/docs/zh/transforms/data-validator.md +++ b/docs/zh/transforms/data-validator.md @@ -25,7 +25,7 @@ DataValidator 转换插件会根据配置规则校验字段值，并按照指定 ### row_error_handle_way.error_table [string] -当 `row_error_handle_way` 设置为 `ROUTE_TO_TABLE` 时，用于路由无效数据的目标表名。使用 `ROUTE_TO_TABLE` 模式时此参数为必需。 +当 `row_error_handle_way` 设置为 `ROUTE_TO_TABLE` 时，用于路由无效数据的目标表名。框架不会强制校验该参数，但如果在 `ROUTE_TO_TABLE` 模式下未配置，DataValidator 无法路由无效行，将输出警告日志并跳过这些行，因此使用 `ROUTE_TO_TABLE` 时应始终配置该参数。 #### 错误表Schema diff --git a/docs/zh/transforms/field-rename.md b/docs/zh/transforms/field-rename.md index b705b26ccb..c4b166ebb9 100644 --- a/docs/zh/transforms/field-rename.md +++ b/docs/zh/transforms/field-rename.md @@ -10,7 +10,7 @@ FieldRename 转换插件用于批量重命名字段名。 | 参数 | 类型 | 必选 | 默认值 | 说明 | |:-----------------------:|--------|------|--------|---------------------------------------------------------------------------------------------------------| -| convert_case | string | 否 | | 字母大小写转换类型，可选 `UPPER`、`LOWER` | +| convert_case | enum | 否 | | 字母大小写转换类型，可选 `UPPER`、`LOWER` | | prefix | string | 否 | | 追加到字段名前的前缀 | | suffix | string | 否 | | 追加到字段名后的后缀 | | replacements_with_regex | array | 否 | | 替换规则数组，元素为包含 `replace_from`、`replace_to` 以及可选 `is_regex`（默认 `true`）的映射；当 `is_regex=false` 时，`replace_from` 按字段名精确匹配（全匹配） | diff --git a/docs/zh/transforms/filter.md b/docs/zh/transforms/filter.md index 3a6cbdb255..62c89153ae 100644 --- a/docs/zh/transforms/filter.md +++ b/docs/zh/transforms/filter.md @@ -17,6 +17,10 @@ 需要保留的字段列表。不在列表中的字段将被删除。 +:::note +为了向后兼容，已废弃的选项名 `fields` 仍然可以作为 `include_fields` 的替代被接受。新配置请使用 `include_fields`。 +::: + ### exclude_fields [array] 需要删除的字段列表。不在列表中的字段将被保留。 diff --git a/docs/zh/transforms/jsonpath.md b/docs/zh/transforms/jsonpath.md index e01c473f66..05e335616c 100644 --- a/docs/zh/transforms/jsonpath.md +++ b/docs/zh/transforms/jsonpath.md @@ -23,6 +23,7 @@ JsonPath 转换插件支持使用 JSONPath 选择数据。 - FAIL：选择`FAIL`时，数据格式错误会阻塞并抛出异常。 - SKIP：选择`SKIP`时，数据格式错误会跳过该行数据。 +- ROUTE_TO_TABLE：JsonPath 转换尚未实现该处理方式。该值目前可以配置，但实际行为与 `FAIL` 完全相同：解析失败的行会直接使作业失败，不会被路由到错误表。 ### columns [array] @@ -189,9 +190,9 @@ transform { 那么数据结果表 `fake1` 将会像这样 -| data | c1_string | c1_boolean | c1_integer | c1_float | c1_double | c1_decimal | c1_date | c1_datetime | c1_array | -|------------------------------|------------------|------------|------------|----------|-----------|------------|------------|--------------|-----------------------------| -| too much content not to show | this is a string | true | 42 | 3.14 | 3.14 | 10.55 | 2023-10-29 | 16:12:43.459 | ["item1", "item2", "item3"] | +| data | c1_string | c1_boolean | c1_integer | c1_float | c1_double | c1_decimal | c1_date | c1_datetime | c1_array | c1_map_array | +|------------------------------|------------------|------------|------------|----------|-----------|------------|------------|--------------|-----------------------------|------------------------------| +| too much content not to show | this is a string | true | 42 | 3.14 | 3.14 | 10.55 | 2023-10-29 | 16:12:43.459 | ["item1", "item2", "item3"] | [{"key1": "value1", "key2": "value2"}] | ## 读取 SeatunnelRow 示例 diff --git a/docs/zh/transforms/metadata.md b/docs/zh/transforms/metadata.md index 0b46b05966..ee3520998e 100644 --- a/docs/zh/transforms/metadata.md +++ b/docs/zh/transforms/metadata.md @@ -109,7 +109,7 @@ transform { | 参数名 | 类型 | 是否必填 | 默认值 | 说明 | |:---------------:|------|:--------:|:-------------:|-------------------| -| metadata_fields | map | 否 | 空映射 | 元数据字段与输出字段的映射关系，格式为 `元数据Key = 输出字段名` | +| metadata_fields | map | 是 | - | 元数据字段与输出字段的映射关系，格式为 `元数据Key = 输出字段名`。至少需要配置一个映射项。 | ### metadata_fields [map] diff --git a/docs/zh/transforms/table-filter.md b/docs/zh/transforms/table-filter.md index ab77eb6726..f6a8358f51 100644 --- a/docs/zh/transforms/table-filter.md +++ b/docs/zh/transforms/table-filter.md @@ -13,7 +13,7 @@ TableFilter 转换插件用于按表名、库名或 schema 规则，正向或反 | database_pattern | string | 否 | | 数据库过滤规则。默认不过滤；如需过滤数据库名称，请填写正则表达式。 | | schema_pattern | string | 否 | | schema 过滤规则。默认不过滤；如需过滤 schema 名称，请填写正则表达式。 | | table_pattern | string | 否 | | 表过滤规则。默认不过滤；如需过滤表名称，请填写正则表达式。 | -| pattern_mode | string | 否 | INCLUDE | 过滤模式。`INCLUDE` 表示保留匹配的表，`EXCLUDE` 表示排除匹配的表。 | +| pattern_mode | enum | 否 | INCLUDE | 过滤模式。`INCLUDE` 表示保留匹配的表，`EXCLUDE` 表示排除匹配的表。 | ## 示例 diff --git a/docs/zh/transforms/table-rename.md b/docs/zh/transforms/table-rename.md index 5402061d11..9f0ff9bf0f 100644 --- a/docs/zh/transforms/table-rename.md +++ b/docs/zh/transforms/table-rename.md @@ -10,7 +10,7 @@ TableRename 转换插件用于重命名表名。 | 参数 | 类型 | 必选 | 默认值 | 说明 | |:-----------------------:|--------|------|--------|---------------------------------------------------------------------------------------------------------| -| convert_case | string | 否 | | 字母大小写转换类型，可选 `UPPER`、`LOWER` | +| convert_case | enum | 否 | | 字母大小写转换类型，可选 `UPPER`、`LOWER` | | prefix | string | 否 | | 追加到表名前的前缀 | | suffix | string | 否 | | 追加到表名后的后缀 | | replacements_with_regex | array | 否 | | 正则替换规则数组，元素为包含 `replace_from`、`replace_to` 的映射，用于批量替换表名 |
