SEPURI-SAI-KRISHNA opened a new issue, #13296:
URL: https://github.com/apache/seatunnel/issues/13296

   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Bug%22)
 and found no similar issues.
   
   ### What happened
   
   MySQL column names are case-insensitive, so MySQL accepts DDL that names an 
existing column in another case than the table defines it, for example `ALTER 
TABLE orders ADD COLUMN email VARCHAR(64) AFTER NAME` for the column `name`. 
With `schema-changes.enabled = true`, MySQL-CDC passes such names on as 
written: `CustomAlterTableParserListener` takes them from the statement with 
`parseName`, and `MySqlSchemaChangeResolver` parses with an empty Debezium 
`Tables`, so nothing maps `NAME` to `name`. `AlterTableSchemaEventHandler` then 
compares names exactly.
   
   On dev `c812d79ce1`, with a table `orders (id, name, amount)`, the resolver 
plus `AlterTableSchemaEventHandler` give:
   
   | DDL | Result |
   |---|---|
   | `ADD COLUMN email VARCHAR(64) AFTER NAME` | `SchemaEvolutionException`: 
`Cannot add column 'email' after 'NAME'` (since #13267; before it, the column 
was added first) |
   | `DROP COLUMN Name` | not applied, the schema keeps `name` |
   | `MODIFY COLUMN NAME VARCHAR(128)` | not applied, `name` keeps its old 
definition |
   | `MODIFY COLUMN name VARCHAR(64) AFTER AMOUNT` | `NoSuchElementException` |
   | `CHANGE COLUMN NAME full_name VARCHAR(64)` | `IndexOutOfBoundsException` |
   | `CHANGE COLUMN name NAME VARCHAR(128)` | the schema renames the column to 
`NAME`, while Debezium keeps `name` and its change records carry `name` |
   
   In a running job (config below, Zeta, MySQL 8.0 in Testcontainers), `alter 
table products_column_case add column note varchar(64) null after NAME` fails 
the source task, and the job ends `FAILED`. Debezium itself resolves these 
names ignoring case and keeps the existing names: after `MODIFY COLUMN NOTE 
...` its column is still `note`, and the change records use that name.
   
   ### SeaTunnel Version
   
   dev (`c812d79ce1`)
   
   ### SeaTunnel Config
   
   ```conf
   env {
     parallelism = 1
     job.mode = "STREAMING"
     checkpoint.interval = 5000
   }
   source {
     MySQL-CDC {
       url = "jdbc:mysql://mysql:3306/shop"
       username = "st_user_source"
       password = "mysqlpw"
       table-names = ["shop.products_column_case"]
       schema-changes.enabled = true
     }
   }
   sink {
     jdbc {
       url = "jdbc:mysql://mysql:3306/shop"
       driver = "com.mysql.cj.jdbc.Driver"
       user = "st_user_sink"
       password = "mysqlpw"
       generate_sink_sql = true
       database = shop
       table = mysql_cdc_e2e_sink_table_column_case
       primary_keys = ["id"]
     }
   }
   ```
   
   Then, while the job runs: `alter table products_column_case add column note 
varchar(64) null after NAME;`
   
   ### Running Command
   
   ```shell
   Zeta, MysqlCDCWithSchemaChangeIT (Testcontainers, MySQL 8.0)
   ```
   
   ### Error Exception
   
   ```log
   org.apache.seatunnel.api.table.schema.exception.SchemaEvolutionException: 
ErrorCode:[SE-10], ErrorDescription:[Failed to process schema change event] - 
Cannot add column 'note' after 'NAME': table .null.products_column_case has no 
such column, existing columns: [id, name, description, weight]
        at 
org.apache.seatunnel.api.table.schema.handler.AlterTableSchemaEventHandler.applyAddColumn(AlterTableSchemaEventHandler.java:129)
        at 
org.apache.seatunnel.api.table.schema.handler.AlterTableSchemaEventHandler.apply(AlterTableSchemaEventHandler.java:82)
        at 
org.apache.seatunnel.api.table.schema.handler.AlterTableSchemaEventHandler.apply(AlterTableSchemaEventHandler.java:62)
        at 
org.apache.seatunnel.api.table.schema.handler.TableSchemaChangeEventDispatcher.apply(TableSchemaChangeEventDispatcher.java:69)
        at 
org.apache.seatunnel.connectors.cdc.debezium.row.SeaTunnelRowDebeziumDeserializeSchema.deserializeSchemaChangeRecord(SeaTunnelRowDebeziumDeserializeSchema.java:188)
        at 
org.apache.seatunnel.connectors.cdc.debezium.row.SeaTunnelRowDebeziumDeserializeSchema.deserialize(SeaTunnelRowDebeziumDeserializeSchema.java:123)
        at 
org.apache.seatunnel.connectors.cdc.base.source.reader.IncrementalSourceRecordEmitter.emitElement(IncrementalSourceRecordEmitter.java:201)
   ```
   
   ### Zeta or Flink or Spark Version
   
   Zeta (the MySQL-CDC schema evolution ITs run on Zeta only).
   
   ### Java or Scala Version
   
   JDK 17 locally.
   
   ### Screenshots
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to