github-actions[bot] commented on code in PR #65334:
URL: https://github.com/apache/doris/pull/65334#discussion_r3536346200
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HiveExternalMetaCache.java:
##########
@@ -641,6 +644,13 @@ private void invalidatePartitionCache(ExternalTable
dorisTable, String partition
PartitionCacheKey partKey = new PartitionCacheKey(nameMapping,
values);
HivePartition partition = partitionEntry.getIfPresent(partKey);
if (partition == null) {
+ // Partition metadata cache miss: the exact FileCacheKey
cannot be rebuilt here because it
+ // needs the partition path and input format carried by
HivePartition. A stale file listing
+ // for this partition may still exist though, because the file
cache and partition cache are
+ // independent and evict separately. Fall back to a
predicate-based invalidation keyed by
Review Comment:
This fallback still does not clear every file-cache entry the refreshed
partition can reuse.
First, this branch is only reached after
`partitionValuesEntry.getIfPresent(key)` succeeds. If the table-level
partition-values entry has been evicted while the file cache is still
populated, line 635 returns before any file-cache invalidation, so the same
stale listing survives. That state is reachable because
`ENTRY_PARTITION_VALUES` and `ENTRY_FILE` are separate caches with independent
capacities, and the partition values can be derived from `partitionName` with
`HiveUtil.toPartitionValues(partitionName)` instead of depending on the cache
entry.
Second, the `k.isSameTable(tableId)` predicate is narrower than normal
`FileCacheKey` reuse. `getFilesByPartitions` stores a table id in the key, but
non-dummy `FileCacheKey.equals`/`hashCode` ignore that id and compare only
catalog, location, input format, and partition values. If another table with
the same partition location/input format/values populated the shared cache
entry first, this predicate skips it even though a later scan of the refreshed
table can still hit it. Please make the fallback consistent with the actual
cache identity, or conservatively invalidate all entries the target partition
could reuse, and add tests for the missing partition-values entry and
shared-key cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
