github-actions[bot] commented on code in PR #67695:
URL: https://github.com/apache/doris/pull/67695#discussion_r3964720106
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ExecuteActionCommand.java:
##########
@@ -104,7 +104,7 @@ public void run(ConnectContext ctx, StmtExecutor executor)
throws Exception {
}
action.validate(tableNameInfo, ctx.getCurrentUserIdentity());
- ResultSet resultSet = action.execute(table);
+ ResultSet resultSet = executeAuthenticated(action, (ExternalTable)
table);
Review Comment:
[P1] cherrypick_snapshot and publish_changes still escape this scope on
non-fast-forward commits. Iceberg 1.10.1 routes those cases through
SnapshotProducer/MergingSnapshotProducer, whose manifest metadata and merge
work defaults to the process-global worker pool; the public ManageSnapshots
path used by Doris never supplies the catalog's authenticated executor. On a
Kerberized Hadoop catalog, shared workers have no per-task binding to this UGI,
so a WAP publish after the main branch advances (or another non-fast-forward
cherry-pick) can fail during manifest reads/writes. Please route
snapshot-producer workers through the same generation-fenced auth context and
add a default-HadoopFileIO Kerberos case that forces non-fast-forward manifest
merging.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ExecuteActionCommand.java:
##########
@@ -144,6 +144,22 @@ public Optional<Expression> getWhereCondition() {
return whereCondition;
}
+ private ResultSet executeAuthenticated(ExecuteAction action, ExternalTable
table) throws Exception {
+ try {
+ // Iceberg tables retain filesystem configuration, not the
caller's UGI, so table loading and
+ // metadata commits must stay within one catalog authentication
scope.
+ return table.getCatalog().getExecutionAuthenticator().execute(()
-> {
+ try {
+ return action.execute(table);
Review Comment:
[P1] Authenticate expiration's cleanup-planning executor too. This doAs
scope covers the caller thread, but Iceberg 1.10.1 defaults
ExpireSnapshots.planExecutorService() to its process-global worker pool when
planWith(...) is omitted, and IncrementalFileCleanup performs manifest reads
there with failures suppressed after the metadata commit. With normal
HadoopFileIO those workers have no per-task binding to this catalog's UGI, so
ALTER can succeed and remove the snapshot while silently leaving old files
behind. Please supply a generation-fenced pre-authenticated planning executor
(and similarly authenticate any delete pool), then exercise default
HadoopFileIO and assert physical cleanup.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ExecuteActionCommand.java:
##########
@@ -144,6 +144,22 @@ public Optional<Expression> getWhereCondition() {
return whereCondition;
}
+ private ResultSet executeAuthenticated(ExecuteAction action, ExternalTable
table) throws Exception {
+ try {
+ // Iceberg tables retain filesystem configuration, not the
caller's UGI, so table loading and
+ // metadata commits must stay within one catalog authentication
scope.
+ return table.getCatalog().getExecutionAuthenticator().execute(()
-> {
Review Comment:
[P1] Bind this scope to the writable table's catalog generation. A
concurrent catalog property/credential update clears executionAuthenticator; if
it lands before this getter, actions now fail before their existing
writable-table loader can lazily reinitialize. If it lands just after capture,
that loader can acquire a generation-B table under authenticator B, then return
here and commit it under the still-active generation-A UGI. This defeats
IcebergExternalMetaCache's explicit (authenticator, ops, table) generation
fence. Please acquire and execute through a generation-aware context that
rejects/retries movement; calling makeSureInitialized() alone does not close
the A/B splice.
##########
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/plans/commands/ExecuteActionCommandTest.java:
##########
@@ -0,0 +1,112 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.trees.plans.commands;
+
+import org.apache.doris.catalog.DatabaseIf;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.common.security.authentication.ExecutionAuthenticator;
+import org.apache.doris.datasource.CatalogIf;
+import org.apache.doris.datasource.CatalogMgr;
+import org.apache.doris.datasource.iceberg.IcebergExternalCatalog;
+import org.apache.doris.datasource.iceberg.IcebergExternalTable;
+import org.apache.doris.info.TableNameInfo;
+import org.apache.doris.nereids.trees.plans.commands.execute.ExecuteAction;
+import
org.apache.doris.nereids.trees.plans.commands.execute.ExecuteActionFactory;
+import org.apache.doris.persist.EditLog;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.StmtExecutor;
+
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import org.mockito.MockedStatic;
+import org.mockito.Mockito;
+
+import java.util.Collections;
+import java.util.Optional;
+import java.util.concurrent.Callable;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+class ExecuteActionCommandTest {
+
+ @Test
+ void executesActionWithinCatalogAuthenticationScope() throws Exception {
+ Env env = Mockito.mock(Env.class);
+ CatalogMgr catalogMgr = Mockito.mock(CatalogMgr.class);
+ CatalogIf<?> catalog = Mockito.mock(CatalogIf.class);
+ DatabaseIf<?> database = Mockito.mock(DatabaseIf.class);
+ IcebergExternalCatalog externalCatalog =
Mockito.mock(IcebergExternalCatalog.class);
+ IcebergExternalTable table = Mockito.mock(IcebergExternalTable.class);
+ ExecuteAction action = Mockito.mock(ExecuteAction.class);
+ TableNameInfo tableName = Mockito.mock(TableNameInfo.class);
+ ConnectContext context = Mockito.mock(ConnectContext.class);
+ StmtExecutor executor = Mockito.mock(StmtExecutor.class);
+ EditLog editLog = Mockito.mock(EditLog.class);
+ RecordingAuthenticator authenticator = new RecordingAuthenticator();
+ AtomicBoolean actionExecutedInScope = new AtomicBoolean();
+
+ Mockito.when(env.getCatalogMgr()).thenReturn(catalogMgr);
+ Mockito.when(env.getEditLog()).thenReturn(editLog);
+ Mockito.when(tableName.getCtl()).thenReturn("iceberg");
+ Mockito.when(tableName.getDb()).thenReturn("db");
+ Mockito.when(tableName.getTbl()).thenReturn("tbl");
+ Mockito.when(catalogMgr.getCatalog("iceberg")).thenReturn(catalog);
+ Mockito.when(catalog.getDbNullable("db")).thenReturn(database);
Review Comment:
[P1] Make these wildcard-return stubs type-safe. Calling getDbNullable() on
CatalogIf<?> produces an unknown captured return type, so thenReturn(database)
cannot compile; the next line has the same problem for DatabaseIf<?> and
IcebergExternalTable. Please use doReturn(...).when(...) for both calls (or
concrete generic mock types). As written, FE test compilation stops before the
new authentication assertion can run.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ExecuteActionCommand.java:
##########
@@ -144,6 +144,22 @@ public Optional<Expression> getWhereCondition() {
return whereCondition;
}
+ private ResultSet executeAuthenticated(ExecuteAction action, ExternalTable
table) throws Exception {
+ try {
+ // Iceberg tables retain filesystem configuration, not the
caller's UGI, so table loading and
+ // metadata commits must stay within one catalog authentication
scope.
+ return table.getCatalog().getExecutionAuthenticator().execute(()
-> {
+ try {
+ return action.execute(table);
+ } catch (UserException e) {
+ throw new AuthenticatedActionException(e);
+ }
+ });
Review Comment:
[P1] This scope also misses rewrite_data_files' initial manifest scan.
RewriteDataFilePlanner calls TableScan.planFiles() without planWith(...); in
Iceberg 1.10.1, scans use the process-global worker pool by default and
multi-manifest tables read manifests there. Those pooled threads are not bound
per task to this catalog's UGI, so a Kerberized rewrite can fail before any
rewrite task is submitted despite the caller doAs. Please pass a
generation-fenced catalog pre-auth pool into that planner and add a
default-HadoopFileIO Kerberos case with multiple manifests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
