SEPURI-SAI-KRISHNA opened a new issue, #13296: URL: https://github.com/apache/seatunnel/issues/13296
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Bug%22) and found no similar issues. ### What happened MySQL column names are case-insensitive, so MySQL accepts DDL that names an existing column in another case than the table defines it, for example `ALTER TABLE orders ADD COLUMN email VARCHAR(64) AFTER NAME` for the column `name`. With `schema-changes.enabled = true`, MySQL-CDC passes such names on as written: `CustomAlterTableParserListener` takes them from the statement with `parseName`, and `MySqlSchemaChangeResolver` parses with an empty Debezium `Tables`, so nothing maps `NAME` to `name`. `AlterTableSchemaEventHandler` then compares names exactly. On dev `c812d79ce1`, with a table `orders (id, name, amount)`, the resolver plus `AlterTableSchemaEventHandler` give: | DDL | Result | |---|---| | `ADD COLUMN email VARCHAR(64) AFTER NAME` | `SchemaEvolutionException`: `Cannot add column 'email' after 'NAME'` (since #13267; before it, the column was added first) | | `DROP COLUMN Name` | not applied, the schema keeps `name` | | `MODIFY COLUMN NAME VARCHAR(128)` | not applied, `name` keeps its old definition | | `MODIFY COLUMN name VARCHAR(64) AFTER AMOUNT` | `NoSuchElementException` | | `CHANGE COLUMN NAME full_name VARCHAR(64)` | `IndexOutOfBoundsException` | | `CHANGE COLUMN name NAME VARCHAR(128)` | the schema renames the column to `NAME`, while Debezium keeps `name` and its change records carry `name` | In a running job (config below, Zeta, MySQL 8.0 in Testcontainers), `alter table products_column_case add column note varchar(64) null after NAME` fails the source task, and the job ends `FAILED`. Debezium itself resolves these names ignoring case and keeps the existing names: after `MODIFY COLUMN NOTE ...` its column is still `note`, and the change records use that name. ### SeaTunnel Version dev (`c812d79ce1`) ### SeaTunnel Config ```conf env { parallelism = 1 job.mode = "STREAMING" checkpoint.interval = 5000 } source { MySQL-CDC { url = "jdbc:mysql://mysql:3306/shop" username = "st_user_source" password = "mysqlpw" table-names = ["shop.products_column_case"] schema-changes.enabled = true } } sink { jdbc { url = "jdbc:mysql://mysql:3306/shop" driver = "com.mysql.cj.jdbc.Driver" user = "st_user_sink" password = "mysqlpw" generate_sink_sql = true database = shop table = mysql_cdc_e2e_sink_table_column_case primary_keys = ["id"] } } ``` Then, while the job runs: `alter table products_column_case add column note varchar(64) null after NAME;` ### Running Command ```shell Zeta, MysqlCDCWithSchemaChangeIT (Testcontainers, MySQL 8.0) ``` ### Error Exception ```log org.apache.seatunnel.api.table.schema.exception.SchemaEvolutionException: ErrorCode:[SE-10], ErrorDescription:[Failed to process schema change event] - Cannot add column 'note' after 'NAME': table .null.products_column_case has no such column, existing columns: [id, name, description, weight] at org.apache.seatunnel.api.table.schema.handler.AlterTableSchemaEventHandler.applyAddColumn(AlterTableSchemaEventHandler.java:129) at org.apache.seatunnel.api.table.schema.handler.AlterTableSchemaEventHandler.apply(AlterTableSchemaEventHandler.java:82) at org.apache.seatunnel.api.table.schema.handler.AlterTableSchemaEventHandler.apply(AlterTableSchemaEventHandler.java:62) at org.apache.seatunnel.api.table.schema.handler.TableSchemaChangeEventDispatcher.apply(TableSchemaChangeEventDispatcher.java:69) at org.apache.seatunnel.connectors.cdc.debezium.row.SeaTunnelRowDebeziumDeserializeSchema.deserializeSchemaChangeRecord(SeaTunnelRowDebeziumDeserializeSchema.java:188) at org.apache.seatunnel.connectors.cdc.debezium.row.SeaTunnelRowDebeziumDeserializeSchema.deserialize(SeaTunnelRowDebeziumDeserializeSchema.java:123) at org.apache.seatunnel.connectors.cdc.base.source.reader.IncrementalSourceRecordEmitter.emitElement(IncrementalSourceRecordEmitter.java:201) ``` ### Zeta or Flink or Spark Version Zeta (the MySQL-CDC schema evolution ITs run on Zeta only). ### Java or Scala Version JDK 17 locally. ### Screenshots _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
