zhangshenghang opened a new pull request, #12602:
URL: https://github.com/apache/seatunnel/pull/12602
## Purpose of this PR
This PR fixes a batch of real documentation issues found by systematically
cross-checking connector/format doc option tables against the actual `Option`
definitions and factory `OptionRule`s in the Java source code on `dev`. Every
option name, type, required flag, and default value written in this PR was
verified against the code before being documented.
## What issues were found and changed
### 1. Sink options defined in code but missing from the docs (option rows
added, EN + ZH)
| Connector | Missing options | Source evidence |
|---|---|---|
| Doris sink | `default-database` (String, default `information_schema`) |
`DorisSinkOptions.java`, listed in `DorisSinkFactory.optionRule()` |
| SelectDB Cloud sink | `sink.label-prefix` (String, default random UUID),
`sink.flush.queue-size` (int, default 1) | `SelectDBSinkOptions.java`, exposed
by `SelectDBSinkFactory.optionRule()` |
| Typesense sink | `protocol` (String, default `http`) |
`TypesenseBaseOptions.PROTOCOL`; already documented on the source page but
missing on the sink page |
| Sls sink | `log_group_size` (int, default 100) |
`SlsSinkOptions.LOG_GROUP_SIZE`, read by `SlsSinkWriter` |
| Kafka source | `key_converter_schema_enabled` (Boolean, default `true`) |
`KafkaConnectJsonFormatOptions`, read in `KafkaSourceConfig` for
`compatible_kafka_connect_json`; the sibling `value_converter_schema_enabled`
was already documented |
### 2. Clickhouse sink: the `ClickhouseFile` plugin was entirely undocumented
The connector ships a second sink plugin with factory identifier
`ClickhouseFile` (`ClickhouseFileSinkFactory`), but the sink doc never
mentioned it. Added (EN + ZH):
- A new "ClickhouseFile Sink" section explaining how the plugin works
(generate part files locally via `clickhouse-local`, copy via `scp`/`rsync`,
attach to the target table)
- An option table for the 8 plugin-specific options
(`clickhouse_local_path`, `copy_method`, `compatible_mode`,
`node_free_password`, `node_pass`, `key_path`, `file_fields_delimiter`,
`file_temp_path`) with the exact required/optional flags and defaults from
`ClickhouseFileSinkFactory.optionRule()` and `ClickhouseFileSinkOptions.java`
- A working task example
### 3. Format docs documented options that are never read from job
configuration
A repo-wide search showed that the following documented options have no
consumer anywhere in the codebase (the helper methods that define them are dead
code, and every connector hardcodes its own behavior):
- `docs/{en,zh}/connectors/formats/canal-json.md`:
`canal_json.ignore-parse-errors`, `canal_json.database.include`,
`canal_json.table.include`. Actual behavior: Kafka and Pulsar always skip parse
errors; Amazon SQS never skips; no connector applies the include filtering.
- `docs/{en,zh}/connectors/formats/ogg-json.md` and `maxwell-json.md`: same
problem; only the Kafka source supports these formats and it always skips parse
errors.
- `docs/{en,zh}/connectors/formats/debezium-json.md`:
`debezium-json.ignore-parse-errors` is not read; schema inclusion is controlled
by connector-level options such as the Kafka source's
`debezium_record_include_schema`.
Each of these pages now carries a warning block documenting the actual
behavior, so users no longer configure options that silently do nothing. This
is documentation-only; no code behavior was changed in this PR.
### 4. `kafka-compatible-kafkaconnect-json.md` had no option table at all
Added the missing Format Options table (`format`,
`key_converter_schema_enabled`, `value_converter_schema_enabled`) in EN and ZH,
matching `KafkaConnectJsonFormatOptions.java`.
## Areas / files updated
22 files (11 EN + 11 ZH), +189 lines:
-
`docs/{en,zh}/connectors/sink/{Doris,SelectDB-Cloud,Typesense,Sls,Clickhouse}.md`
- `docs/{en,zh}/connectors/source/Kafka.md`
-
`docs/{en,zh}/connectors/formats/{canal-json,debezium-json,maxwell-json,ogg-json,kafka-compatible-kafkaconnect-json}.md`
## Were both English and Chinese docs checked?
Yes. The change was driven by a scripted comparison of all connector option
tables against code, plus a full EN/ZH parity diff of the connector, transform,
and format docs. Every fix above was applied to both language versions. (The
parity sweep found no additional EN/ZH desync beyond the files recently fixed
by other PRs.)
## Duplicate PR check (last 7 days)
Checked PRs authored in the last 7 days before submitting:
- #12570 (connector option fixes: Paimon, MongoDB-CDC, Redis, SqlServer-CDC,
Persistiq) — no file overlap
- #12553 (EN/ZH table sync: file sinks, Greenplum, StarRocks, PostgreSQL,
Splunk, ...) — no file overlap
- #12499 (merged, file connectors + InfluxDB/Socket/Milvus/MongoDB) — no
file overlap
- #12487 (merged, Doris/SelectDB-Cloud option-key bug +
incompatible-changes) — touches Doris/SelectDB-Cloud docs but did not add
`default-database`, `sink.label-prefix` or `sink.flush.queue-size` (verified
against current `dev`); no content overlap
- #12476 (merged, transform docs) — no file overlap
## Verification
- Scripted doc-vs-code comparison of all connector option tables (option
keys, sides, required/optional) and a scripted EN/ZH option-table parity diff;
all findings manually verified against the Java sources listed above
- Markdown table column-consistency check across all 22 changed files: 0
issues
- Repo-wide internal relative-link check: no broken links introduced
- No Java code was modified, so no build/test run is required;
spotless/verify were not run for this reason
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
