goutamadwant opened a new pull request, #12588:
URL: https://github.com/apache/seatunnel/pull/12588
### Purpose of this pull request
Fix #12586.
Tables with enum columns failed with COMMON-21 in Postgres-CDC and the JDBC
source. #11232 maps unknown types to STRING only for `Types.OTHER`, but the
catalog path never sets a `sqlType`, and pgjdbc and Debezium report enums as
`Types.VARCHAR`.
- `PostgresCatalog` reads `pg_type.typtype` and maps enum columns to STRING,
with `format_type()` as the source type (schema-qualified and quoted when
needed, e.g. `inv.mood`, `inv."JobStatus"`). This also covers enums named like
built-in types.
- `PostgresTypeConverter` treats `Types.VARCHAR` like `Types.OTHER` for
unknown type names (JDBC query mode, CDC schema changes). The reported name is
kept as `sourceType` only if it is a plain or quoted identifier, otherwise
`text`, because a PostgreSQL sink copies it into DDL.
- Kingbase and Redshift still map only `Types.OTHER`.
### Does this PR introduce _any_ user-facing change?
Yes. Enum columns are read as STRING instead of failing the job.
| Case (PostgreSQL 18.6 and 17.9) | before | after |
|---|---|---|
| Postgres-CDC snapshot + INSERT/UPDATE/DELETE | COMMON-21 | source == sink |
| CDC with `schema-changes.enabled = true` | COMMON-21 | source == sink |
| JDBC `table_path` | COMMON-21 | source == sink |
| JDBC `query` | COMMON-21 | source == sink |
| 27 common column types | – | mapping unchanged |
Not changed: enum arrays and domains still fail as before. A PostgreSQL sink
writing into enum columns needs `stringtype=unspecified`, as for json.
### How was this patch tested?
- Added `PostgresCatalogBuildColumnTest` and new cases in
`PostgresTypeConverterTest`, `PostgresRelationSchemaChangeResolverTest`,
`KingbaseTypeConverterTest` and `RedshiftTypeConverterTest`. `connector-jdbc`
and `connector-cdc-postgres` unit tests pass on JDK 8 and JDK 11.
- `JdbcPostgresIT` and `PostgresCDCIT` pass locally.
- Manual runs on PostgreSQL 18.6 and 17.9 for the cases in the table, plus a
type name containing SQL, which now becomes `text` in the sink DDL.
### Check list
* [ ] If any new Jar binary package adding in your PR, please add License
Notice according
[New License
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
* [ ] If necessary, please update the documentation to describe the new
feature. https://github.com/apache/seatunnel/tree/dev/docs
* [ ] If necessary, please update `incompatible-changes.md` to describe the
incompatibility caused by this PR.
* [ ] If you are contributing the connector code, please check that the
following files are updated:
1. Update
[plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties)
and add new connector information in it
2. Update the pom file of
[seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml)
3. Add ci label in
[label-scope-conf](https://github.com/apache/seatunnel/blob/dev/.github/workflows/labeler/label-scope-conf.yml)
4. Add e2e testcase in
[seatunnel-e2e](https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/)
5. Update connector
[plugin_config](https://github.com/apache/seatunnel/blob/dev/config/plugin_config)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
