Ramin Gharib created FLINK-39495:
------------------------------------
Summary: FROM_CHANGELOG silently drops rows with unmapped
operation codes
Key: FLINK-39495
URL: https://issues.apache.org/jira/browse/FLINK-39495
Project: Flink
Issue Type: Bug
Components: Table SQL / Planner
Reporter: Ramin Gharib
FROM_CHANGELOG silently drops input rows when the operation code column
contains a value not present in the op_mapping (or not matching the default
mapping). This makes data loss invisible and very hard to debug — users have no
indication that rows are being discarded.
For example, if a CDC source produces an unexpected operation code like
"SNAPSHOT" or a typo like "INSRT", those rows are silently skipped with no
error or warning.
The fix changes the default behavior to throw a RuntimeException when an
unmapped operation code is encountered, making data issues visible immediately
rather than causing silent data loss.
Before
(broken):
{code:java}
-- Row with op='UNKNOWN' is silently dropped — no error, no log
SELECT * FROM FROM_CHANGELOG(input => TABLE cdc_stream) {code}
After
(fixed):
{code:java}
-- Row with op='UNKNOWN' throws:
-- "Received invalid op code 'UNKNOWN'. Defined op codes are: [INSERT,
UPDATE_BEFORE, UPDATE_AFTER, DELETE]. Failing as configured."
SELECT * FROM FROM_CHANGELOG(input => TABLE cdc_stream)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
