Zhuojun Jiang created FLINK-40340:
-------------------------------------
Summary: UnsupportedOperationException for MySQL ZEROFILL types
without UNSIGNED
Key: FLINK-40340
URL: https://issues.apache.org/jira/browse/FLINK-40340
Project: Flink
Issue Type: Bug
Components: Flink CDC
Affects Versions: cdc-3.5.0, cdc-3.4.0, cdc-3.6.0
Reporter: Zhuojun Jiang
Fix For: cdc-3.7.0
h3. Problem
When a MySQL table declares a numeric column with the ZEROFILL modifier alone
(e.g. {{{}INT ZEROFILL{}}}) without explicitly writing {{{}UNSIGNED{}}}, Flink
CDC's
{{MySqlTypeUtils}} fails to recognize the type and throws:
{noformat}
java.lang.UnsupportedOperationException: Unsupported type: INT ZEROFILL
at
org.apache.flink.cdc.connectors.mysql.schema.MySqlTypeUtils.fromDbzColumn(...)
{noformat}
This breaks schema conversion and prevents the connector from reading such
tables.
h3. Root Cause
MySQL treats {{ZEROFILL}} as an independent column modifier that *implicitly
implies* {{UNSIGNED}} (see MySQL docs: "When you use ZEROFILL, MySQL also adds
the UNSIGNED attribute automatically"). Therefore {{INT ZEROFILL}} and
{{INT UNSIGNED ZEROFILL}} are semantically equivalent.
However, {{MySqlTypeUtils}} only registers constants and mapping branches for
{{{}<TYPE> UNSIGNED ZEROFILL{}}}. The bare {{<TYPE> ZEROFILL}} variants are
missing,
so they fall through to the unsupported-type branch.
h3. Affected Types
All numeric types that already have an {{UNSIGNED ZEROFILL}} mapping:
{{{}TINYINT{}}}, {{{}SMALLINT{}}}, {{{}MEDIUMINT{}}}, {{{}INT{}}},
{{{}INTEGER{}}}, {{{}BIGINT{}}},
{{{}REAL{}}}, {{{}FLOAT{}}}, {{{}DOUBLE{}}}, {{{}DOUBLE PRECISION{}}},
{{{}NUMERIC{}}},
{{{}FIXED{}}}, {{{}DECIMAL{}}}.
h3. Reproduction
{code:sql}
CREATE TABLE t (
id INT ZEROFILL,
val DECIMAL(10,2) ZEROFILL
);
{code}
Start a Flink CDC MySQL source reading from this table; Then execute
{code:sql}
ALTER TABLE t ADD COLUMN new_col INT ZEROFILL;
{code}
The pipeline will failover with {{{}UnsupportedOperationException{}}}.
h3. Proposed Fix
Add explicit {{<TYPE> ZEROFILL}} constants and mapping branches that mirror the
existing {{<TYPE> UNSIGNED ZEROFILL}} mappings, in both:
* {{flink-connector-mysql-cdc}} ({{{}MySqlTypeUtils{}}})
* {{flink-cdc-pipeline-connector-mysql}} ({{{}MySqlTypeUtils{}}})
Add unit tests covering ZEROFILL-only mappings, equivalence with
UNSIGNED ZEROFILL, FLOAT length-dependent behavior, and NOT NULL column
handling.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
