goutamadwant opened a new pull request, #12547:
URL: https://github.com/apache/seatunnel/pull/12547
### Purpose of this pull request
Fix #12546.
pgjdbc 42.7.5+ returns the database name as `TABLE_CAT`, so Debezium tracks
tables loaded from JDBC metadata as `db.schema.table`, while pgoutput RELATION
ids have no catalog. `RelationAwarePostgresSchema#applySchemaChangesForTable`
then misses the tracked table on the first RELATION after the job starts and
skips the schema-change listener, so an `ADD COLUMN` in that transaction is
lost.
This falls back to the configured database name when the catalog-less lookup
misses, the same approach as `PostgresSnapshotSplitReadTask#resolveTable`
(#10843).
### Does this PR introduce _any_ user-facing change?
Yes. With pgjdbc 42.7.5+ and `schema-changes.enabled = true`, `ADD COLUMN`
is applied on the first RELATION after start instead of being dropped. No
change with older drivers.
| ADD COLUMN, then first change since job start | new column values in sink |
|---|---|
| pgjdbc 42.7.13, before | column missing, values lost |
| pgjdbc 42.7.13, after | all values present |
| pgjdbc 42.4.3, before and after | all values present |
### How was this patch tested?
- Added `RelationAwarePostgresSchemaTest`.
`testRelationNotifiesListenerWhenSchemaWasLoadedWithCatalog` fails without the
fix and passes with it on JDK 8 and JDK 11.
- `./mvnw -pl seatunnel-connectors-v2/connector-cdc/connector-cdc-postgres
test` (32 tests) on JDK 8 and JDK 11, `spotless:check`.
- `PostgresCDCIT#testAddColumnSchemaEvolutionWithRestore` and
`#testMPostgresCdcCheckDataE2e` pass locally.
- Manual run on PostgreSQL 18.6 and 17.9 with pgjdbc 42.4.3 and 42.7.13: ADD
COLUMN before the first change, after a prior change, and across
savepoint/restore all reach the sink.
### Check list
* [ ] If any new Jar binary package adding in your PR, please add License
Notice according
[New License
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
* [ ] If necessary, please update the documentation to describe the new
feature. https://github.com/apache/seatunnel/tree/dev/docs
* [ ] If necessary, please update `incompatible-changes.md` to describe the
incompatibility caused by this PR.
* [ ] If you are contributing the connector code, please check that the
following files are updated:
1. Update
[plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties)
and add new connector information in it
2. Update the pom file of
[seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml)
3. Add ci label in
[label-scope-conf](https://github.com/apache/seatunnel/blob/dev/.github/workflows/labeler/label-scope-conf.yml)
4. Add e2e testcase in
[seatunnel-e2e](https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/)
5. Update connector
[plugin_config](https://github.com/apache/seatunnel/blob/dev/config/plugin_config)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
