924060929 commented on code in PR #69806:
URL: https://github.com/apache/doris/pull/69806#discussion_r4439687248
##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergExceptionUtils.java:
##########
@@ -31,12 +33,25 @@ private IcebergExceptionUtils() {
static RuntimeException wrapTableLoadFailure(
IcebergTableHandle handle, Exception failure, String
fallbackPrefix) {
+ if (isTableAbsent(failure)) {
Review Comment:
Fixed in 12e40ddc17c6. SHOW PARTITIONS (both names and partition-statistics
branches) and PARTITIONS now capture the table's MetadataRead before resolving
its handle through resolveConnectorTableHandle, then retain the observation
through the provider list call. Iceberg listPartitionNames preserves native
table absence for the existing typed normalizer instead of returning empty.
Successful cached reads add no existence probe. PARTITIONS retains its existing
empty-OK behavior on an absent handle, after guarded cleanup; its
operation-local statement scope still closes in finally. Tests cover both
absent-handle consumers and all three listing consumers, including confirmed
cleanup and delayed same-name recreation. The native adapter test and five
positive FE cleanup scenarios fail before this repair; five recreation controls
already pass.
Validation: 973 focused unit cases passed across 50 classes; FE build and
Checkstyle passed. Controlled providers, not a live external-service
integration run.
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/RefreshManager.java:
##########
@@ -212,47 +212,58 @@ private void applyTableRefresh(ExternalCatalog catalog,
String localDbName, Stri
boolean invalidateDatabase = invalidateTableLookup &&
catalog.getLowerCaseTableNames() != 0;
ExternalMetaCacheMgr cacheMgr =
Env.getCurrentEnv().getExtMetaCacheMgr();
Optional<ExternalDatabase<? extends ExternalTable>> db =
catalog.getDbForReplay(localDbName);
- if (!db.isPresent()) {
- try {
- invalidateAllConnectorCachesIfPresent(catalog);
- } finally {
- if (invalidateDatabase) {
- cacheMgr.invalidateDb(catalogId, dbId, localDbName);
- } else {
- cacheMgr.invalidateTable(catalogId, dbId, localDbName,
tableId, localTableName);
- }
- }
- LOG.info("database object cache is cold when refreshing table; "
- + "invalidated caches by local names {}.{} in
catalog {}",
- localDbName, localTableName, catalogId);
- return;
+ Optional<? extends ExternalTable> table = db.flatMap(database ->
database.getTableForReplay(localTableName));
+ // Cold paths invalidate connector database/catalog caches without a
retained table object.
+ boolean fenceLookup = invalidateTableLookup || !table.isPresent();
Review Comment:
Fixed in 12e40ddc17c6. The common refreshTableInternal implementation now
brackets metadata-only refreshes with the existing catalog-owned epoch/active
count as well as the original per-table guard. applyTableRefresh leaves
hot-table fencing to that common method and retains its own interval only for
cold branches. Normal TTL/capacity eviction cannot produce an unprotected
replacement while mutation/event/hot replay refresh is active. The change
governs cleanup authorization only: existing connector/engine invalidation
scope and table-name cache behavior are unchanged, and provider IO stays
outside the monitor. Deterministic tests pause connector invalidation, evict
only the table object, publish a replacement, delay its confirmed absence,
finish refresh and warm recreated metadata, then verify its numeric
route/schema/connector caches survive without a names reload. Mutation, event,
metadata replay and hot/cold truncate replay variants all fail before the
repair; existing overlap
/failure and three casing-mode ordinary-write controls are retained.
Validation: 973 focused unit cases passed across 50 classes; FE build and
Checkstyle passed. Controlled providers, not a live external-service
integration run.
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/RefreshManager.java:
##########
@@ -227,32 +227,41 @@ private void applyTableRefresh(ExternalCatalog catalog,
String localDbName, Stri
localDbName, localTableName, catalogId);
return;
}
- Optional<? extends ExternalTable> table =
db.get().getTableForReplay(localTableName);
- if (!table.isPresent()) {
- try {
- if (catalog instanceof PluginDrivenExternalCatalog) {
- // A cold table object cannot provide the mapped remote
table name.
- ((PluginDrivenExternalCatalog)
catalog).getConnector().invalidateDb(db.get().getRemoteName());
- }
- } finally {
+ if (invalidateTableLookup) {
Review Comment:
Fixed the remaining cold replay variants carried by the latest review in
12e40ddc17c6. replayRefreshDb's cold-database branch and replayTruncateTable's
cold table/database branches now retain the existing catalog-owned interval
through both connector and engine invalidation. The common hot database refresh
also uses this interval. No replay journal or remote-loading contract changes,
and every interval releases in finally. New cold DB refresh/truncate and
cold-table truncate tests publish replacement objects during connector
invalidation, delay absence until after refresh and recreated metadata warming,
and verify numeric ID, schema and connector-cache survival. All three tests
fail before the repair.
Validation: 973 focused unit cases passed across 50 classes; FE build and
Checkstyle passed. Controlled providers, not a live external-service
integration run.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
