JNSimba opened a new pull request, #720:
URL: https://github.com/apache/doris-flink-connector/pull/720
# Proposed changes
Issue Number: #715
## Problem Summary:
In initial mode, snapshot tablets are planned before the incremental start
timestamp is captured. A partition created and populated between these steps
can be absent from the snapshot, while its rows have timestamps below the
incremental start and are skipped by incremental reads.
Capture the incremental start before planning snapshot tablets by deferring
construction of the snapshot assigner. Extend the existing startup regression
test to advance the server timestamp during tablet planning and verify that the
checkpoint retains the earlier start timestamp.
This addresses the startup tablet-discovery gap in #715.
Snapshot/incremental overlap and delayed transaction visibility remain outside
this change.
## Checklist(Required)
1. Does it affect the original behavior: Yes
2. Has unit tests been added: Yes, extended the existing startup regression
test
3. Has document been added or modified: No Need
4. Does it need to update dependencies: No
5. Are there any changes that cannot be rolled back: No
## Further comments
Validation:
- The regression failed before the fix: expected the pre-planning timestamp
`13:00:00`, but captured `13:00:10`.
- Flink 1 Base module suite: 479 tests, 0 failures, 0 errors, 1 existing
skipped test.
`mvn -o -f flink-doris-connector/pom.xml -Pflink1 -pl
flink-doris-connector-base -am test`
- Flink 2 affected Source tests: 39 tests, 0 failures, 0 errors, 0 skipped.
`mvn -o -f flink-doris-connector/pom.xml -Pflink2 -pl
flink-doris-connector-base -am
-Dtest=DorisSourceOffsetRestoreTest,DorisSnapshotSplitAssignerTest,DorisHybridSourceEnumeratorTest
-Dsurefire.failIfNoSpecifiedTests=false test`
- Spotless, Checkstyle and `git diff --check` passed. No live-cluster
reproduction was performed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
