[
https://issues.apache.org/jira/browse/FLINK-40148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-40148:
-----------------------------------
Labels: pull-request-available (was: )
> TableIdRouter resolves wrong sink table when source regex has multi-digit
> suffix and sink-table has no back-reference
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-40148
> URL: https://issues.apache.org/jira/browse/FLINK-40148
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
> Affects Versions: cdc-3.6.0
> Environment: flink 1.20.3
> flinkcdc 3.6.0
> Reporter: peiyu
> Priority: Major
> Labels: pull-request-available
> Fix For: cdc-3.7.0
>
>
> ## Problem
> TableIdRouter.resolveReplacement can produce incorrect sink table names when
> the source-table
> regex contains a capturing group whose value may span multiple characters
> (e.g. multi-digit
> suffixes like `13`, `14`, `16`) and the sink-table does not use a `$N`
> back-reference.
> In this case the unmatched characters after the captured prefix are appended
> to the configured
> sink-table, producing wrong sink names. For example, given:
> route:
> - source-table: db_6.table_([1-9]|1[0-6])
> sink-table: new_db_6.table_merged
> `db_6.table_13` is wrongly routed to `new_db_6.table_merged3` (instead of the
> configured
> `new_db_6.table_merged`), `db_6.table_14` to `new_db_6.table_merged4`,
> `db_6.table_16` to
> `new_db_6.table_merged6`, etc. Tables 1-9 are not affected because the regex
> consumes the
> entire single-digit suffix.
> ## Reproduction
> 1. Configure a pipeline with the route above.
> 2. Run the job against a source that has tables `db_6.table_1` …
> `db_6.table_16`.
> 3. Observe in the logs / sink database that tables 10-16 are written to sink
> tables named
> `new_db_6.table_merged0`, `new_db_6.table_merged1`,
> `new_db_6.table_merged3`, … instead of
> the single configured sink table.
> A unit test that reproduces the bug (without the fix) is added in this
> ticket; it fails before
> the patch and passes after.
> ## Root Cause
> `TableIdRouter.resolveReplacement` builds a `Matcher` against the source
> table id and calls
> `Matcher.find()` followed by `Matcher.replaceAll(sinkTable)`:
> Matcher matcher = route.f0.matcher(originalTable.toString());
> if (matcher.find()) {
> return TableId.parse(matcher.replaceAll(route.f1));
> }
> Two issues combine to produce the bug:
> 1. `find()` only matches a *prefix* of the source table id. With the regex
> `([1-9]|1[0-6])`, the alternation `[1-9]` wins first, matching only `1`
> from input `13`.
> 2. `Matcher.replaceAll` internally calls `reset()` and iterates with
> `find()`, so it only
> replaces the prefix matched in step 1 and leaves the trailing characters
> of the source
> table id (`0`, `3`, `4`, `5`, `6`) appended to the sink-table. When the
> sink-table has no
> `$N` reference, those trailing characters end up in the result.
> The same root cause also makes `find()`-based capturing groups return only
> the first matched
> digit when the sink-table *does* use `$1`, so the bug is not strictly limited
> to back-reference
> free sinks.
> ## Proposed Fix
> Switch to `Matcher.matches()` to ensure the entire source table id is
> consumed, and use
> `Matcher.appendReplacement` / `Matcher.appendTail` to perform the `$N`
> substitution on the
> matched region only — this avoids the trailing-characters-left-over problem
> of `replaceAll`.
> ## Tests
> - Add `TableIdRouterTest#testRouteWithoutBackReferenceWithMultiDigitSuffix`
> to lock down the
> reported scenario (sink-table has no back-reference).
> - Add `TableIdRouterTest#testRouteWithBackReferenceAndMultiDigitCapture` to
> cover the related
> case where sink-table uses `$1` and the captured group must be the full
> multi-digit value.
> - All existing tests in `TableIdRouterTest`, `TableIdRouterMatchModeTest`, and
> `SchemaDerivatorTest` continue to pass (27 tests total).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
