924060929 commented on code in PR #68196:
URL: https://github.com/apache/doris/pull/68196#discussion_r4057424435
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalMetaCacheMgr.java:
##########
@@ -502,9 +544,36 @@ public void invalidateTableByEngine(long catalogId, String
engine, String dbName
public void invalidatePartitions(long catalogId,
String dbName, String tableName, List<String> partitions) {
- routeCatalogEngines(catalogId, cache -> safeInvalidate(
- cache, catalogId, "invalidatePartitions",
- () -> cache.invalidatePartitions(catalogId, dbName, tableName,
partitions)));
+ Optional<ExternalDatabase<? extends ExternalTable>> db =
getCachedDb(catalogId, dbName);
+ try {
+ routeCatalogEngines(catalogId, cache -> safeInvalidate(
+ cache, catalogId, "invalidatePartitions",
+ () -> cache.invalidatePartitions(catalogId, dbName,
tableName, partitions)));
Review Comment:
Fixed in 8b4749c00f5. invalidateTableByEngine now captures the cached
DB/table identity and fences rowCountCache in a finally block, sharing the same
conservative table/DB/catalog fallback as the normal invalidation path. Added
testEngineSpecificTableInvalidationAlsoFencesRowCount.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/CatalogMgr.java:
##########
@@ -1050,6 +1050,7 @@ public void addExternalPartitions(String catalogName,
String dbName, String tabl
}
HiveExternalMetaCache cache =
Env.getCurrentEnv().getExtMetaCacheMgr().hive(catalog.getId());
cache.addPartitionsCache(hmsTable.getOrBuildNameMapping(),
partitionNames, partitionColumnTypes);
+
Env.getCurrentEnv().getExtMetaCacheMgr().invalidateRowCountCache(hmsTable);
hmsTable.setUpdateTime(updateTime);
Review Comment:
Fixed in 8b4749c00f5. ADD_PARTITION now fences the independent row-count
cache before partition-column decoding, so the NotSupportedException early
return cannot preserve a stale count. Added
testUnsupportedAddPartitionEventStillInvalidatesRowCount.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalCatalog.java:
##########
@@ -1163,10 +1171,12 @@ public void unregisterDatabase(String dbName) {
if (LOG.isDebugEnabled()) {
LOG.debug("unregister database [{}]", dbName);
}
+ // Resolve the canonical database object before removing it from the
local metadata cache.
Review Comment:
Fixed in 8b4749c00f5. unregisterDatabase captures the canonical database
name and numeric ID first, then performs local MetaCache removal in finally so
an engine invalidation exception cannot leave the dropped DB visible. The
manager-level invalidateDb finally block still fences row counts. Added
testUnregisterDatabaseRemovesLocalEntryWhenEngineInvalidationFails.
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/RefreshManager.java:
##########
@@ -168,6 +168,7 @@ public void replayRefreshTable(ExternalObjectLog log) {
}
// See comment in refreshDbInternal for why db and table may be null.
if (!db.isPresent()) {
+
Env.getCurrentEnv().getExtMetaCacheMgr().invalidateRowCountCache(catalog.getId());
LOG.warn("failed to find db when replaying refresh table: {}",
log.debugForRefreshTable());
Review Comment:
Fixed in 8b4749c00f5. When replayRefreshDb cannot recover a cached database
object, it now retires the catalog row-count scope without loading remote
metadata. Added testColdDatabaseReplayInvalidatesCatalogRowCount.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
