goutamadwant commented on code in PR #12308:
URL: https://github.com/apache/seatunnel/pull/12308#discussion_r4003789002
##########
seatunnel-connectors-v2/connector-cdc/connector-cdc-tidb/src/main/java/org/apache/seatunnel/connectors/seatunnel/cdc/tidb/source/enumerator/TiDBSourceSplitEnumerator.java:
##########
@@ -102,19 +152,38 @@ public void open() {
@Override
public void run() throws Exception {
Set<Integer> readers = context.registeredReaders();
+ List<TiDBSourceSplit> sourceSplits = new ArrayList<>();
if (shouldEnumerate) {
- List<TiDBSourceSplit> sourceSplits = getTiDBSourceSplit();
+ sourceSplits = getTiDBSourceSplit(tableIds.keySet());
log.info(
- "{} Enumerated TiDB CDC splits, database={}, table={},
splitCount={}.",
+ "{} Enumerated TiDB CDC splits, tables={}, splitCount={}.",
CDC_DIAG_PREFIX,
- sourceConfig.getDatabaseName(),
- sourceConfig.getTableName(),
+ tableIds.keySet(),
sourceSplits.size());
synchronized (stateLock) {
+ enumeratedTables.addAll(tableIds.keySet());
addPendingSplit(sourceSplits);
shouldEnumerate = false;
- assignSplit(readers);
}
+ } else if (enumeratedTables != null) {
Review Comment:
A pre-change checkpoint restores `enumeratedTables` as `null`. Because
`shouldEnumerate` is already false, this branch is skipped and the null is
saved again in every later checkpoint.
I reproduced restoring a legacy checkpoint containing `db.table_one` with
the configuration expanded to `[db.table_one, db.table_two]`: only `table_one`
was assigned, while `table_two` never received a split or initial snapshot.
That silently violates the documented add-table restore behavior.
Please initialize the legacy state from the restored split tables and
enumerate any configured tables that are missing, or fail restore clearly if
this upgrade path cannot be supported.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
