leaves12138 opened a new pull request, #10780:
URL: https://github.com/apache/paimon/pull/10780
### Purpose
Row ID reassignment currently aborts as soon as a concurrent COMPACT
snapshot is committed. Allow it to advance through compaction by rebinding the
original mappings to the current manifests when logical row ranges are
preserved. If compaction changes those ranges, including deletion-vector
materialization or a file that only partially overlaps the plan, replan the
originally selected partitions against the latest snapshot. Skip the operation
when no selected partition still requires reassignment.
Recover from expired planning manifest lists and input metadata removed
during a rewrite when a newer snapshot is available. Retain the existing retry
budget and rejection of concurrent OVERWRITE snapshots or missing required
history. Keep historical ADD/DELETE pairs and current data, BLOB,
deletion-vector and global-index metadata consistent.
### Tests
- Passed 150 related tests on JDK 8 with normal Maven checks, and on JDK 11
in an isolated Linux checkout:
```bash
mvn -pl paimon-core -DwildcardSuites=none \
-Dtest=DataEvolutionRowIdReassignerTest,DataEvolutionRowIdReassignerCompactTest,CompactReassignReuseTest,DataEvolutionDeletionVectorTest
test
```
- Added coverage for manifest/data/BLOB/index compaction, partial overlap
and repeated replanning, original partition scope, null partition filters, DV
materialization with BLOB and partial-column updates, expired metadata during
serial/parallel rewriting, and the rejection/retry-limit paths.
- Additional isolated metadata-only replay at manifest scan parallelism 1
and 6 passed entry-by-entry metadata comparison and an independent
interval-union audit, including compaction that removes the original rewrite
inputs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
