dybyte opened a new pull request, #11922:
URL: https://github.com/apache/seatunnel/pull/11922
Related to: #10777
### Purpose of this pull request
This PR partially addresses #10777 by supporting PostgreSQL CDC `ADD COLUMN`
schema evolution.
PostgreSQL `pgoutput` does not provide the original DDL statement to
Debezium. Instead of parsing DDL, this PR:
- Detects updated table metadata from `pgoutput` RELATION messages.
- Compares it with SeaTunnel's authoritative `CatalogTable`, including
checkpoint-restored schemas.
- Emits an `ADD COLUMN` schema event before the following DML record.
- Reuses the existing schema evolution and checkpoint coordination path to
update downstream sinks.
### Does this PR introduce _any_ user-facing change?
Yes.
Users can enable PostgreSQL CDC `ADD COLUMN` synchronization with:
```hocon
decoding.plugin.name = "pgoutput"
schema-changes.enabled = true
```
Currently only appended ADD COLUMN changes are supported. Other row-schema
changes fail fast before processing DML with the unsupported schema.
The option is disabled by default, so existing jobs are unaffected.
### How was this patch tested?
- Unit tests: verify multiple `ADD COLUMN` detection, unchanged-schema
skipping, and fail-fast behavior for unsupported changes.
- E2E test: verifies that savepoint recovery detects a source-only `BIGINT`
column addition, evolves the sink schema, and replicates the new row.
### Check list
* [ ] If any new Jar binary package adding in your PR, please add License
Notice according
[New License
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
* [x] If necessary, please update the documentation to describe the new
feature. https://github.com/apache/seatunnel/tree/dev/docs
* [ ] If necessary, please update `incompatible-changes.md` to describe the
incompatibility caused by this PR.
* [ ] If you are contributing the connector code, please check that the
following files are updated:
1. Update
[plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties)
and add new connector information in it
2. Update the pom file of
[seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml)
3. Add ci label in
[label-scope-conf](https://github.com/apache/seatunnel/blob/dev/.github/workflows/labeler/label-scope-conf.yml)
4. Add e2e testcase in
[seatunnel-e2e](https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/)
5. Update connector
[plugin_config](https://github.com/apache/seatunnel/blob/dev/config/plugin_config)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
