924060929 commented on code in PR #69983:
URL: https://github.com/apache/doris/pull/69983#discussion_r4442813456
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergSnapshotCacheValue.java:
##########
@@ -137,9 +139,18 @@ private IcebergSnapshotCacheValue(IcebergPartitionInfo
partitionInfo, IcebergSna
}
public IcebergPartitionInfo getPartitionInfo() {
+ return partitionInfoLoader == null ? partitionInfo :
partitionInfoLoader.get();
Review Comment:
Fixed in 350e47d018c349a8b0d8dd839a90763aad2c99d0. Native and HMS Iceberg
now load the partition view once per bulk MV operation and derive all requested
versions from it. The view remains local to that operation; disabled or
weight-rejected entries are not retained in the shared snapshot. Added
disabled/rejected-admission tests (three requested partitions, one full load),
HMS dispatch coverage, and fallback/per-partition failure tests. All 197
related FE unit tests and the full FE build including Checkstyle pass locally.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergExternalMetaCache.java:
##########
@@ -990,6 +999,11 @@ private void
invalidateRelatedEntriesOnTableRemoval(NameMapping nameMapping) {
if (snapshots != null) {
snapshots.invalidateIf(key ->
key.getNameMapping().equals(nameMapping));
}
+ MetaCacheEntry<IcebergSnapshotEntryKey, IcebergPartitionInfo>
partitions =
+ partitionEntry.getIfInitialized(nameMapping.getCtlId());
+ if (partitions != null) {
+ partitions.invalidateIf(key ->
key.getNameMapping().equals(nameMapping));
Review Comment:
Fixed in 350e47d018c349a8b0d8dd839a90763aad2c99d0. Table-generation
replacement now invalidates partition entries from obsolete physical
generations. Partition lookups also check the published table generation before
and after loading, so a pinned old projection can read its own immutable view
without republishing it after retirement. Same-generation and unrelated-table
entries remain reusable. Added cached-replacement, replacement-during-load, and
late-access tests. All 197 related FE unit tests and the full FE build
including Checkstyle pass locally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
