SEPURI-SAI-KRISHNA opened a new pull request, #13308:
URL: https://github.com/apache/seatunnel/pull/13308
Yes, glad to contribute to Apache SeaTunnel.
### Purpose of this pull request
Closes #13307.
With `schema-changes.enabled = true`, MySQL-CDC ignored `ALTER TABLE ...
RENAME COLUMN old TO new`, although the MySQL-CDC page lists `rename column` as
supported. `CustomAlterTableParserListener` had no handler for `RENAME COLUMN`,
so the statement gave no event (`Unsupported resolve schemaChangeEvent ...,
just skip.`). Debezium applies the rename to its own table model, so the
following change records carry the new name, while the SeaTunnel schema keeps
the old one and the row converter sets the missing field to NULL. In
`MysqlCDCWithSchemaChangeIT` this fails the JDBC sink with `Column 'add_column'
cannot be null`; a nullable column gets NULL instead of its values without an
error.
Changes:
- `CustomAlterTableParserListener.enterAlterByRenameColumn` emits
`AlterTableChangeColumnEvent.change(table, old, column)`, where the column
carries only the new name, as the Oracle-CDC listener does for its `RENAME
COLUMN` (#9314). `RENAME COLUMN` cannot change the definition, and
`AbstractSchemaChangeResolver.completeSchemaChangeEvents` already fills in the
definition of the old column from the current catalog table, so
`connector-cdc-base` needs no change.
- A rename that changes only the case (`name TO NAME`) gives no event:
Debezium keeps the existing name in that case, and its records keep carrying it.
- Docs: the `change.column` row of the event filter table on the MySQL-CDC
page (EN and ZH) names both `CHANGE COLUMN` and `RENAME COLUMN`.
Not covered here:
- A rename that names the old column in another case (`RENAME COLUMN NAME TO
full_name` for `name`) fails while completing the event, like `CHANGE COLUMN
NAME ...` on dev. #13297 resolves such names before completion and covers both.
- Swapping or cycling names in one statement (`RENAME COLUMN a TO b, RENAME
COLUMN b TO a`): MySQL resolves every clause against the original columns,
while Debezium 1.9.8 applies them in order and loses a column in its own model.
- Renaming a primary key column keeps the old name in the schema's primary
key, because `AlterTableSchemaEventHandler` copies the primary key unchanged.
`CHANGE COLUMN` has the same limit on dev; it is a separate fix.
I searched open and closed PRs and issues for `RENAME COLUMN` and checked
the diffs of the open PRs that touch MySQL-CDC parsing or DDL files (#12014,
#11162, #12284, #12258, #13297): none handles `RENAME COLUMN` (#11162 only
mentions it in the schema evolution guide). This PR and #13297 change different
lines of the MySQL-CDC page and no common file otherwise.
### Does this PR introduce _any_ user-facing change?
Yes. With `schema-changes.enabled = true`, `RENAME COLUMN` is applied
downstream as a column rename, like `CHANGE COLUMN`, and the renamed column
keeps its values. It can be filtered as `change.column`. No option, default or
API changes.
### How was this patch tested?
- `CustomMySqlAntlrDdlParserTest#testParseAlterTableRenameColumn`: `RENAME
COLUMN c_name TO c_full_name, ADD COLUMN c_note ...` gives a change event with
the old and new names followed by the add event.
`#testParseAlterTableRenameColumnOnlyInCaseHasNoEvent`: `c_name TO C_NAME` and
`c_name TO c_name` give no event. With the listener of dev the first test fails
(1 event instead of 2), and with a case-sensitive comparison the second one
fails.
- Locally, against Debezium's `MySqlAntlrDdlParser` table model, the
resulting schema matched for a plain rename (type and length kept), a case-only
rename, a rename with `ADD ... AFTER` the new name, a rename with `MODIFY` of
the new name, and backquoted names with spaces.
- `connector-cdc-base` 143 and `connector-cdc-mysql` 61 unit tests pass.
- E2E: `ddl/change_columns.sql` gains a `RENAME COLUMN add_column TO
add_column_renamed` step followed by rows. The column keeps the new name: the
sink may apply schema changes later than the source, and a name used only in
between would be missing from the sink while the source no longer has it.
`comment_changes.sql` documents the new table state.
- `MysqlCDCWithSchemaChangeIT#testMysqlCdcWithSchemaEvolutionCase` and
`#testMysqlCdcWithSchemaEvolutionCaseExactlyOnce` (Zeta) pass; each sink ran
`CHANGE COLUMN add_column add_column_renamed INT NOT NULL DEFAULT 1`. With the
listener of dev both fail: the sink keeps `add_column` and fails with `Column
'add_column' cannot be null`.
- `MysqlCDCWithFlinkSchemaChangeIT` passes on Flink 1.13.6, 1.15.3, 1.18.0
and 1.20.1 (8 runs).
- `spotless:check`, `MarkdownTest`, `git diff --check`.
### Check list
* [x] If any new Jar binary package adding in your PR, please add License
Notice according
[New License
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
(no new jar)
* [x] If necessary, please update the documentation to describe the new
feature. https://github.com/apache/seatunnel/tree/dev/docs
* [x] If necessary, please update `incompatible-changes.md` to describe the
incompatibility caused by this PR. (not needed: no incompatible change)
* [ ] If you are contributing the connector code, please check that the
following files are updated: (not a new connector)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]