wangyong9999 opened a new pull request, #300: URL: https://github.com/apache/paimon-cpp/pull/300
### Purpose Linked issue: close #299 `FileStoreScan::ReadManifestEntriesWithCache` caches the merged live manifest entries of a bucket per snapshot and takes `cached->snapshot_id == snapshot.Id()` as a hit. A snapshot id does not identify a snapshot, though: `rollback_to` deletes the newest snapshots together with the data files only they referenced, and the next commits reuse the same ids for different content (dropping and recreating a table at the same path does the same). A process that keeps the cache across such a rollback then plans the rewritten snapshot from the entries of the deleted one: reads of the deleted files fail with `Not exist`, rows committed after the rollback are missing, and this lasts until the ids move past the cached ones. The cached snapshot now records the delta manifest list it was built from. Its name carries a UUID, so a rewritten snapshot never matches; a mismatch falls through to the existing rebuild, which replaces the stale entries in place. - `SnapshotLiveManifestEntries` keeps the delta manifest list per cached snapshot, `Put()` takes it, and the new `Find(snapshot_id, delta_manifest_list)` reports a hit only when both match. `LatestBeforeOrEqual()` is unchanged apart from carrying the name. - `FileStoreScan::ReadManifestEntriesWithCache` looks the snapshot up with `Find()` and stores the name with the rebuilt entries. - The serialized form gains the name per snapshot; the magic changes from `SMEC` to `SMED`, so bytes written by the old layout fail deserialization and are rebuilt instead of being misread. ### Tests - `FileStoreScanTest.TestSnapshotLiveManifestEntries` (extended): `Find()` misses on another delta manifest list or another id, and putting the same id again replaces the stale entries in place. - `FileStoreScanTest.TestSnapshotLiveManifestEntriesSerialization` (extended): the name round-trips. - `TableScanTest.TestManifestEntryCacheIgnoresRewrittenSnapshot` (new): three commits are planned through an `LruCache` with `scan.manifest-entry-cache.max-snapshots = 2`, the table is rolled back to snapshot 1 the way `rollback_to` does it (snapshot files removed, `LATEST` moved), two more commits create a new snapshot 2 and 3, and the next plan through the same cache reads `k1=v1c, k2=v2, k4=v4` rather than the rows of the deleted snapshot; the plan after that is a cache hit. ### API and Format No change under `include/`. The in-memory cache value format of `SnapshotLiveManifestEntries` changes (new magic); it is never written to storage. ### Documentation No. ### Generative AI tooling Generated-by: Claude Code (Claude Fable 5.1) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
