yuminn-k opened a new pull request, #13279:
URL: https://github.com/apache/seatunnel/pull/13279
### Purpose of this pull request
Fixes #13207
This is a contribution to Apache SeaTunnel. It lets SqlServer-CDC read
tables from a database whose name contains `.`, such as `LIS6.0`.
With `database-names = ["LIS6.0"]` and `table-names =
["LIS6.0.dbo.LIS_ITEM"]`, source creation fails today with `Cannot get split
'LIS6.0.dbo.LIS_ITEM' to get databaseName and tableName`, because
`TablePath.of` accepts at most three dot-separated parts.
Following the maintainer guidance in the issue, the global `TablePath`
syntax is not changed. The connector resolves the name itself, using
`database-names` as the source of truth.
Changes:
1. `SqlServerIncrementalSourceFactory#toTablePath` (connector): a name with
more than three parts is split as `<database>.<schema>.<table>`, where schema
and table are the last two parts and the database is everything before them.
The database must be listed in `database-names`. Otherwise it throws
`IllegalArgumentException` instead of guessing, so ambiguous input (for example
a dot inside a schema or table name, or a database not listed in
`database-names`) is rejected with a clear message. Names with three or fewer
parts still go through `TablePath.of` unchanged. The same function is used for
`table-names-config`.
2. `SqlServerIncrementalSourceFactory#getCatalogTables` (connector):
`CatalogTableUtil.getCatalogTables` / `Catalog#getTables` parse `table-names`
with `TablePath.of`, so when a name has more than three parts the factory
builds the `TablePath`s with `toTablePath` and looks them up in the SQL Server
catalog directly. When every name has three or fewer parts, the existing
`CatalogTableUtil.getCatalogTables` path is used as before.
3. `SqlServerSourceConfigFactory` (connector): `table.include.list` for
Debezium used everything after the first `.`, which turned
`LIS6.0.dbo.LIS_ITEM` into `0.dbo.LIS_ITEM`. It now keeps the last two parts
(`dbo.LIS_ITEM`). For ordinary `db.schema.table` names the result is the same
as before.
4. `ConnectorMetricsCalcContext#normalizeTableName` (Zeta engine): with only
the connector fixed, the job still fails at runtime in the metrics path.
`updateMetrics` calls `normalizeTableName`, which calls `TablePath.of(tableId)`
on the row's table id and throws the same `Cannot get split
'LIS6.0.dbo.LIS_ITEM'` error. This code came in with #10100 and is part of
3.0.0. It now falls back to the original table id when `TablePath.of` throws
`IllegalArgumentException`. For such an id, `TablePath.of("LIS6.0", "dbo",
"LIS_ITEM").getFullName()` is the same string, so per-table metrics keep the
key they were registered with. This is why the PR touches the engine as well as
the connector.
`SqlServerSnapshotFetchTask` is not changed: in the local run below the
snapshot phase worked without changes there.
### Does this PR introduce _any_ user-facing change?
Yes. A SqlServer-CDC job whose database name contains `.` previously failed
during source creation. It now starts when the database is listed in
`database-names` and the table is written as `<database>.<schema>.<table>`. A
name with more than three parts whose database prefix is not in
`database-names` fails with a message explaining the requirement. Behavior for
names with three or fewer parts is unchanged.
### How was this patch tested?
Unit tests:
- `SqlServerIncrementalSourceFactoryTest` (new cases): dotted database name
resolves to `TablePath("LIS6.0", "dbo", "LIS_ITEM")`; unresolvable dotted names
are rejected with `IllegalArgumentException`; `table.include.list` is
`dbo.LIS_ITEM` for a dotted database name.
`./mvnw -pl seatunnel-connectors-v2/connector-cdc/connector-cdc-sqlserver
test -Dtest=SqlServerIncrementalSourceFactoryTest`: `Tests run: 29, Failures:
0, Errors: 0, Skipped: 0`
- `ConnectorMetricsCalcContextTest#testUpdateMetricsWithDotInDatabaseName`:
`updateMetrics` with table id `LIS6.0.dbo.LIS_ITEM` counts the row in the total
and per-table counters. It fails without the engine change
(`IllegalArgumentException: Cannot get split 'LIS6.0.dbo.LIS_ITEM' to get
databaseName and tableName`) and passes with it.
`./mvnw -pl seatunnel-engine/seatunnel-engine-server test
-Dtest=ConnectorMetricsCalcContextTest`: `Tests run: 3, Failures: 0, Errors: 0,
Skipped: 0`
Local run against Azure SQL Edge (arm64) with the source settings from the
issue (`database-names = ["LIS6.0"]`, `table-names = ["LIS6.0.dbo.LIS_ITEM",
"LIS6.0.dbo.LIS_HISITEM"]`, `startup.mode = "initial"`):
- Before this change the job failed with `Cannot get split
'LIS6.0.dbo.LIS_ITEM'`, first in the source factory and, with only the
connector changes applied, in the engine metrics path.
- Snapshot: the Zeta job read the 5 snapshot rows (3 from
`LIS6.0.dbo.LIS_ITEM`, 2 from `LIS6.0.dbo.LIS_HISITEM`).
- Incremental: Azure SQL Edge has no CLR, so the SQL Server CDC LSN
functions the connector relies on are not available there. The incremental
phase was checked only with a test-only LSN shim: 5 change rows were read with
the expected table ids (`LIS6.0.dbo.LIS_ITEM`: INSERT, UPDATE_BEFORE,
UPDATE_AFTER; `LIS6.0.dbo.LIS_HISITEM`: DELETE, INSERT). The shim is not part
of this PR.
### Check list
* [ ] If any new Jar binary package adding in your PR, please add License
Notice according
[New License
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
* [ ] If necessary, please update the documentation to describe the new
feature. https://github.com/apache/seatunnel/tree/dev/docs
* [ ] If necessary, please update `incompatible-changes.md` to describe the
incompatibility caused by this PR.
* [ ] If you are contributing the connector code, please check that the
following files are updated:
1. Update
[plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties)
and add new connector information in it
2. Update the pom file of
[seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml)
3. Add ci label in
[label-scope-conf](https://github.com/apache/seatunnel/blob/dev/.github/workflows/labeler/label-scope-conf.yml)
4. Add e2e testcase in
[seatunnel-e2e](https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/)
5. Update connector
[plugin_config](https://github.com/apache/seatunnel/blob/dev/config/plugin_config)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]