wangyong9999 commented on PR #284:
URL: https://github.com/apache/paimon-cpp/pull/284#issuecomment-5586008521
One small follow-up on the inferred cache rebuild in
`ReadAndMergeBucketFileEntries`. Entries are retained when
`entry.File()->schema_id != table_schema_->Id()`, so after a schema change that
keeps the bucket keys (an added column, say) every bucket's cached set carries
all pre-change entries of the whole table until compaction rewrites them, and
each lookup filters them again. `HasCompatibleBucketKeys()` already tells
whether an older schema hashes the key the same way, and a compatible entry
with the current bucket count sits in exactly `Bucket(count)`, so it only needs
retaining when `entry.Bucket() == bucket`:
```cpp
bool retain = entry.Bucket() == bucket;
if (!retain && inferred_bucket) {
if (entry.TotalBuckets() != core_options_.GetBucket()) {
retain = true;
} else {
PAIMON_ASSIGN_OR_RAISE(bool compatible,
HasCompatibleBucketKeys(entry.File()->schema_id));
retain = !compatible;
}
}
```
Measured on a two-bucket primary key table with partition `p1` written
before an ADD COLUMN and `p2` after it: each bucket's cached set goes from 3
entries (its own file plus both `p1` files) to 2 (its own two files), all 80
lookups still return their row, and the scan suites pass. Entries written with
another bucket count are unchanged, `TotalBuckets() != count` keeps them. Happy
to send it as a follow-up if you would rather keep this PR as it is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
