SEPURI-SAI-KRISHNA commented on PR #13243:
URL: https://github.com/apache/seatunnel/pull/13243#issuecomment-6909933440

   @DanielLeens Thanks, all four are addressed in `276d0d01d0`.
   
   **Issue 1.** I chose a clear rejection, not a migration. Before it creates 
any writer, `MultiTableSink.restoreWriter` now checks the states: if non-empty 
state of a table the job still writes is stored under an index no writer of 
this subtask gets (`subtask * replicaNum + replica`), the restore fails, names 
those writers and says what to do. That covers 2 replicas restored with 1, and 
the state of another subtask after the parallelism shrinks, which was also 
dropped silently. Empty state and tables the job no longer writes or skips are 
ignored, so a Jdbc job (no writer state) can still change its replica count. 
Tests: `testRestoreWithFewerReplicasFailsInsteadOfDroppingState` (2 to 1, no 
writer created), `testRestoreFailsForStateOfAnotherSubtask`, 
`testRestoreIgnoresEmptyStateAndTablesTheJobNoLongerWrites`. The EN and ZH 
migration guide now says such a checkpoint cannot be restored: let a batch job 
finish or stop a streaming job after its last checkpoint, submit it with the 
new val
 ue without restoring, and start the source early enough, for example with the 
CDC `timestamp` or `specific` startup mode; an append-only sink then writes the 
replayed changes twice. The option page says the same.
   
   **Issue 2.** `MultiTableSinkWriter.random` is a package-private seam; the 
test installs a round robin, so each of 3 writers gets exactly 100 of 300 rows. 
The new tests close their writers in `finally`.
   
   **Issue 3.** `sinkPrimaryKeys` and `sinkReplicaModes` now say what they 
hold, that they are filled once per table after `validateReplicaRouting`, 
removed with an isolated table, and not re-read; `replicaModeOf` and 
`primaryKeyOf` document their `null` handling.
   
   **Issue 4.** 
`FakeSourceToAssertIT#testSinkWithoutReplicaModeIsRejectedWithReplicas` runs a 
job with `multi_table_sink_replica = 2` on a sink without a mode and expects 
the rejection (Spark skipped for #13155). It passes locally on Zeta and on 
Flink 1.13.6, 1.15.3, 1.18.0 and 1.20.1. The accepted KEYED path already runs 
as a real job: `MysqlCDCWithSchemaChangeIT` writes Jdbc with replica 2, with 
updates, deletes and savepoint restores, and compares the tables. No connector 
here combines a mode with non-empty state, so the restore rejection is covered 
by the unit tests.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to