1don-ljy opened a new pull request, #13281:
URL: https://github.com/apache/seatunnel/pull/13281

   ### Purpose of this pull request
   
   Yes, I am contributing a focused SQL Server CDC fix to Apache SeaTunnel.
   
   Fixes #13280.
   
   An INITIAL snapshot can remain in bounded backfill when another captured 
table advances the database-wide HIGH watermark but the selected table produces 
no events in LOW–HIGH. The previous completion check used the selected table's 
last row-event offset, which cannot reach HIGH in that case. The job can keep 
reporting RUNNING and completing checkpoints while the snapshot rows remain 
buffered.
   
   This change:
   
   - Caps each bounded CDC query at the original snapshot HIGH.
   - Validates that the capture instances cover the requested table and 
interval; a missing capture is not treated as an empty successful result.
   - Completes backfill at the original HIGH only after all query results and 
DDL have been consumed and the connection rollback succeeds.
   - Leaves row-event offsets untouched and does not mark the task finished if 
END dispatch is interrupted.
   - Keeps ordinary unbounded streaming behavior and checkpoint serialization 
unchanged.
   
   ### Does this PR introduce _any_ user-facing change?
   
   Yes. Idle selected tables can complete their initial snapshot when other 
tables advance the CDC watermark. No new configuration option, default change, 
dependency, output format, or checkpoint format is introduced. The English and 
Chinese SQL Server CDC documentation describes the bounded completion behavior.
   
   ### How was this patch tested?
   
   - Scoped Spotless formatting passed for `connector-cdc-sqlserver` and 
`connector-cdc-sqlserver-e2e`; `git diff --check` passed.
   - **23/23 targeted JUnit cases passed** with JDK 11. The current modified 
production sources and new test class were compiled with `--release 8` against 
the stock 3.0.0 runtime dependencies plus JUnit 5.9.0 / Mockito 4.11.0. This is 
a supplemental isolated run, **not a successful current-dev Maven reactor run**.
   - Coverage includes empty-result completion without modifying row offsets, 
HIGH clamping, missing/gapped capture coverage, interrupted END dispatch, 
unchanged unbounded behavior, and three real `executeIteration` control-flow 
tests: complete result consumption followed by rollback before END, query 
failure without END, and rollback failure without END.
   - Added `testSnapshotBackfillWithNoSelectedTableChanges` to the existing SQL 
Server CDC E2E suite. It holds the selected table's snapshot query, waits for 
LOW, advances CDC only through another table, releases the lock, and checks 
snapshot delivery while the selected table's change table stays empty. **This 
container E2E has not been run locally.**
   - The affected-reactor `test` attempt stopped before the target connector in 
`seatunnel-config-shade`, with missing shaded `ConfigOrigin` / 
`AbstractConfigObject` types. An affected-reactor `verify -DskipTests` attempt 
has also been started; a complete successful dev reactor build/test run remains 
outstanding at submission.
   
   This PR is submitted as a draft while the full reactor and container 
validation remain outstanding.
   
   Earlier targeted validation of the equivalent two production classes against 
SeaTunnel 3.0.0 and disposable SQL Server 2022 databases reproduced the stock 
failure, restored a checkpoint from the stalled stock connector, completed the 
idle snapshot, and restored a subsequent checkpoint followed by 
insert/update/delete convergence. Direct bounded-reader checks with 
`max.iteration.transactions` set to both `0` and `1` included the HIGH 
transaction and excluded later transactions. These previous real-database 
checks are distinct from the current development-branch tests and are not a 
claim of exhaustive DDL/capture-transition or deployment coverage.
   
   The reviewed 3.0.0 patch sources, portable build instructions, and isolated 
boundary tests are also available in [the companion source 
repository](https://github.com/1don-ljy/seatunnel-sqlserver-patches/tree/main/cdc-backfill).
 This PR contains only the CDC fix; the separate identity-write extension is 
not part of it.
   
   ### Check list
   
   - [x] No new binary JAR or third-party dependency is added; existing Apache 
headers are retained and new files have the project header.
   - [x] English and Chinese connector documentation is updated.
   - [x] No incompatible API, option, or checkpoint-format change; no 
incompatible-changes entry is required.
   - [x] Regression coverage is added to the SQL Server CDC connector and its 
existing E2E test area.
   - [x] No new connector/module: plugin mappings, distribution packaging, 
connector registration, and CI scope mapping need no change.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to