zhangshenghang opened a new pull request, #13327:
URL: https://github.com/apache/seatunnel/pull/13327
Yes, I am contributing to Apache SeaTunnel.
## What documentation issues were found
While auditing connector/transform documentation against the actual source
code on `dev`, the following verified discrepancies were found:
1. **MongoDB-CDC metadata field table listed wrong key names (en + zh).**
The table documented `database`, `table`, `rowKind`, `ts_ms`, `delay`, but the
keys the connector actually exposes are `Database`, `Table`, `RowKind`,
`EventTime`, `Delay` (`CommonOptions` in `seatunnel-api`, matched
case-sensitively by `MetadataTransform`). A user following the old table (e.g.
`ts_ms`) gets a "cannot find metadata field" error at transform initialization.
The old description of `ts_ms` ("Source event timestamp") also conflated
`EventTime` (connector fetch time) with `SourceTimestamp`, which MongoDB-CDC
never populates.
2. **PostgreSQL-CDC metadata field table had the same wrong names and missed
real fields (en + zh).** Fixed the key names, corrected the semantics, and
added `SourceTimestamp` (populated by the shared Debezium deserializer from
`source.ts_ms`). Added a note that `BinlogFile`/`BinlogPos`/`BinlogRow`/`Gtid`
are schema-declared but only populated by MySQL-CDC (always `null` for
PostgreSQL), verified in `SourceRecordUtils`.
3. **Opengauss-CDC (en + zh):** `server-time-zone` default was shown as `-`
although the code defaults to `ZoneId.systemDefault().getId()`
(`JdbcSourceOptions`); now documented as "JVM system time zone", consistent
with the PostgreSQL-CDC doc. Added the missing functional `schema-name` option
(read by the shared `PostgresSourceConfigFactory` used by
`OpengaussIncrementalSource extends PostgresIncrementalSource`).
4. **Http sink documented `retry` / `retry_backoff_multiplier_ms` /
`retry_backoff_max_ms` as functional (en + zh),** but `HttpSink` never reads
them, so no retries ever happen. Updated the descriptions to the same honest
wording already used by the sibling Feishu / Enterprise-WeChat sink docs. The
HTTP **source** doc was verified correct (the source does apply retries) and
was not touched.
5. **Kafka source missed `key_converter_schema_enabled` (en + zh).** The
option exists in `KafkaConnectJsonFormatOptions` (Boolean, default `true`) and
is consumed for `format = compatible_kafka_connect_json`, symmetric with the
already documented `value_converter_schema_enabled`.
6. **Klaviyo source missed two functional inherited options (en + zh):**
`keep_params_as_form` and `keep_page_param_as_http_param` (defined in
`HttpSourceOptions`, honored by `KlaviyoSource extends HttpSource`).
7. **Six transform docs missed the multi-table options their factories
declare (en + zh):** `copy`, `sql`, `jsonpath`, `encrypt`, `python`,
`metadata`. Each factory's `optionRule()` declares `table_match_regex` (String,
default `.*`), `table_transform` and `rule_match_mode` (String, default
`FIRST_MATCH`) via `TransformCommonOptions`. Added the rows (plus short section
descriptions where the doc style has them), mirroring the wording already used
by `filter.md` / `field-mapper.md`.
8. **BigQuery connector changelog was empty (en + zh)** even though both the
sink (#10485) and the new source (#13088) features are merged. Filled the table
with the two rows exactly in the format produced by
`tools/documents/update_connector_change_log.py` (verified titles, 10-char
commit links, `dev` version column).
## What was changed
- Corrected metadata field name tables with a case-sensitivity warning
- Corrected a false default value and added a missing option row
- Corrected false option-behavior claims to match actual sink behavior
- Added missing option rows and short option descriptions
- Filled an empty changelog table
## Which areas/files were updated
26 files (13 en + 13 zh), no code changes:
- `docs/{en,zh}/connectors/source/MongoDB-CDC.md`
- `docs/{en,zh}/connectors/source/PostgreSQL-CDC.md`
- `docs/{en,zh}/connectors/source/Opengauss-CDC.md`
- `docs/{en,zh}/connectors/source/Kafka.md`
- `docs/{en,zh}/connectors/source/Klaviyo.md`
- `docs/{en,zh}/connectors/sink/Http.md`
- `docs/{en,zh}/transforms/{copy,sql,jsonpath,encrypt,python,metadata}.md`
(12 files)
- `docs/{en,zh}/connectors/changelog/connector-bigquery.md`
## Whether English and Chinese docs were both checked
Yes. Every fix was applied to both the English and Chinese page in the same
way; option names, types and defaults are now identical on both sides.
## How duplicate PRs from the last 7 days were checked
Listed all PRs by this author in the last 7 days via `gh pr list --author
zhangshenghang` and inspected the changed-file list of each open docs PR
(#13234, #13258, #13274, #13282, #13284, #13289, #13302). None of them touched
the CDC metadata-field sections of MongoDB-CDC/PostgreSQL-CDC, the
Opengauss-CDC option table, the Http sink retry rows, Kafka
`key_converter_schema_enabled`, Klaviyo, the six transform multi-table option
tables, or the BigQuery changelog. This PR deliberately avoids all areas
covered by those PRs (JDBC/file connectors/Zeta
deployment/Doris/StarRocks/Elasticsearch/Redis/Clickhouse/MySQL-CDC option
tables, etc.).
## How the change was verified
- Every documented name, type, default and behavior claim above was checked
against the actual source files (factory `optionRule()` definitions,
`CommonOptions`, `MetadataTransform`, `SeaTunnelRowDebeziumDeserializeSchema`,
`SourceRecordUtils`, `HttpSink`, `KafkaConnectJsonFormatOptions`,
`HttpSourceOptions`, `TransformCommonOptions`).
- Changelog rows were verified against the output format of
`tools/documents/update_connector_change_log.py` and `git log --abbrev=10`.
- A script validated all 26 modified files: every relative markdown link
resolves and every touched table has consistent column counts.
- No Markdown lint/build tooling exists in this repository's `docs/`
directory (it is rendered by the separate seatunnel-site infrastructure), so a
full docs-site build was not run; the link/table validation above is the
docs-relevant check available here.
## Scope
26 files changed, 144 insertions, 34 deletions - satisfies the "more than 10
documentation files" requirement with real content fixes only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]