This is an automated email from the ASF dual-hosted git repository.
zy-kkk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 8fc58e929b2 [fix](iceberg) Reject server planning before snapshot
conversion (#68095)
8fc58e929b2 is described below
commit 8fc58e929b2151c9ff4ae71d07375a7f8e944697
Author: zy-kkk <[email protected]>
AuthorDate: Sat Sep 19 20:32:12 2026 +0800
[fix](iceberg) Reject server planning before snapshot conversion (#68095)
Issue Number: close #68035
Related feature: #68036
Doris reconstructs loaded Iceberg tables as BaseTable snapshot views.
BaseTable implements SupportsDistributedScanPlanning with a default true
result, so wrapping RESTTable loses its false override. The existing scan-entry
guard then allows unsupported local planning. With restricted metadata
credentials this surfaces as an S3 error; a separately documented simulated
server residual demonstrates why ignoring the server plan can also change
results.
Check the original table before statement snapshot conversion,
tracked-table publication, and cached TableOwner metadata serialization/weight
estimation. Reject unsupported tables before local manifest access and release
acquired table/catalog ownership on failure. Preserve the existing scan-entry
guard and normal client-planned snapshot semantics.
**Rejection includes metadata acquisition, not only SELECT scans.** DESC,
SHOW CREATE TABLE, and INSERT target-column binding also fail for
server-planned tables. Row-count and partition metadata paths using the same
read acquisition are subject to this guard. The sharedWritableTable method
itself is unchanged; this does not imply SQL INSERT remains supported.
Add a dedicated native Iceberg REST fixture, pinned by multi-platform
digest, to the existing Iceberg Compose stack. Keep the existing Spark/Postgres
REST baseline. MinIO initialization creates a private test bucket and data-only
credentials. The enabled regression requires a working /plan endpoint: an old
or missing fixture fails instead of skipping.
The fixture digest fixes the contents of an upstream `main` snapshot, built
on 2026-04-29 from `b0df3ca01d61b2f7ae7143ac660c6b16e33b6e46` (recorded in the
JAR build properties). This commit precedes the `apache-iceberg-1.11.0` tag by
105 commits; the image is not a release build. Moving `latest` cannot change
the pinned contents. The retention risk is that the old manifest can become
untagged and later be removed upstream, breaking fresh pulls.
**Request to CI maintainers:** build a linux/amd64 + linux/arm64 fixture
from the upstream `apache-iceberg-1.11.0` tag using the upstream shadowJar task
and Dockerfile, and publish it as
`doristhirdpartydocker/iceberg-rest-fixture:1.11.0`. Keep the current digest
until that artifact exists; verify source/architectures and rerun this
regression, then switch to the versioned image with its verified digest.
Mirroring the current image into the Doris namespace is a retention fallback,
but [...]
The regression invokes the real /plan endpoint, verifies restricted
data-file reads succeed and manifest access returns 403, and checks explicit
Doris rejection for SELECT, metadata statements, INSERT, empty tables, and
$files with cache enabled/disabled. Client-planned reads remain four rows
before and after. No policy proxy is part of this Docker regression.
Server-forced planning mode is additionally covered by HTTP RESTCatalog unit
tests.
---
.../thirdparties/docker-compose/iceberg/README.md | 88 ++++++
.../docker-compose/iceberg/iceberg.env | 2 +
.../docker-compose/iceberg/iceberg.yaml.tpl | 36 +++
.../scripts/scan-planning/data-only-policy.json | 32 +++
.../scan-planning/init-storage.sh} | 16 +-
.../connector/iceberg/IcebergScanPlanProvider.java | 10 +-
.../connector/iceberg/IcebergScanPlanning.java | 38 +++
.../connector/iceberg/IcebergStatementScope.java | 36 ++-
.../doris/connector/iceberg/IcebergTableCache.java | 16 +-
.../iceberg/IcebergRestScanPlanningTest.java | 303 +++++++++++++++++++++
.../iceberg/IcebergStatementScopeTest.java | 2 +-
regression-test/conf/regression-conf.groovy | 1 +
.../test_iceberg_rest_server_planning_guard.out | 13 +
.../pipeline/external/conf/regression-conf.groovy | 1 +
.../nonConcurrent/conf/regression-conf.groovy | 1 +
.../pipeline/p0/conf/regression-conf.groovy | 1 +
.../test_iceberg_rest_server_planning_guard.groovy | 177 ++++++++++++
17 files changed, 747 insertions(+), 26 deletions(-)
diff --git a/docker/thirdparties/docker-compose/iceberg/README.md
b/docker/thirdparties/docker-compose/iceberg/README.md
index fdac87f128f..41d82b2affd 100644
--- a/docker/thirdparties/docker-compose/iceberg/README.md
+++ b/docker/thirdparties/docker-compose/iceberg/README.md
@@ -55,3 +55,91 @@ amd64 emulation:
CONTAINER_UID=dorismac- /opt/homebrew/bin/bash \
docker/thirdparties/run-thirdparties-docker.sh -c iceberg-rest
```
+
+## Server-side scan planning regression
+
+The `iceberg` stack also starts `rest-scan-planning` and `scan-planning-init`.
+These services are required by `test_iceberg_rest_server_planning_guard`
whenever
+`enableIcebergTest=true`. The existing `rest` service remains the
Spark/Postgres
+baseline; the new REST service has its own disposable SQLite catalog and uses
+the private `scan-planning` bucket in the same MinIO service.
+
+The new fixture is pinned to the multi-platform image digest used by
+[Iceberg Go's integration
tests](https://github.com/apache/iceberg-go/issues/1880).
+There is currently no `apache/iceberg-rest-fixture:1.11.0` tag. Do not replace
the
+pin with a floating `latest` tag or an older released image: the regression
+requires the real `/plan` implementation. The pinned index includes both
+`linux/amd64` and `linux/arm64`.
+
+The image was built on 2026-04-29 from upstream `main` commit
+[`b0df3ca01d61b2f7ae7143ac660c6b16e33b6e46`](https://github.com/apache/iceberg/commit/b0df3ca01d61b2f7ae7143ac660c6b16e33b6e46),
+as recorded in its JAR's `iceberg-build.properties`. This commit is an ancestor
+of `apache-iceberg-1.11.0`, with 105 intervening commits; it is a development
+snapshot, not a 1.11.0 release build. The digest fixes the image contents;
+`latest` moving does not change what is pulled. If the old manifest becomes
+untagged and upstream removes it, fresh pulls will fail.
+
+The preferred replacement is a multi-platform image built from the upstream
+`apache-iceberg-1.11.0` tag and published by Doris CI maintainers as
+`doristhirdpartydocker/iceberg-rest-fixture:1.11.0`. That image has not yet
been
+provisioned for this change. Keep the current digest until the replacement is
+available, then verify its source and architectures, pin its digest, and rerun
+the regression before switching. Mirroring the current image into the Doris
+namespace is a fallback for retention, but would still be a snapshot build.
+Upstream versioned-image availability is tracked by Iceberg Go issue #1880;
+this change does not depend on an upstream publishing fix arriving first.
+
+The standard `run-thirdparties-docker.sh -c iceberg` command starts the new
+services automatically. Docker initialization creates the private bucket and a
+test-only `scan_data_reader` identity. This identity can read table data but is
+explicitly denied access to metadata under `wh/rest_guard_db/`. Initialization
+is repeatable and does not change the existing `warehouse` bucket's policy.
+
+Ports and regression settings must match:
+
+| Compose environment variable | Default | Regression setting |
+| --- | --- | --- |
+| `ICEBERG_SCAN_PLANNING_REST_PORT` | 18182 |
`iceberg_scan_planning_rest_uri_port` |
+| `MINIO_API_PORT` | 19001 | `iceberg_minio_port` |
+
+After starting the stack and FE/BE, run from the repository root:
+
+```bash
+bash run-regression-test.sh --run -d external_table_p0/iceberg \
+ -s test_iceberg_rest_server_planning_guard
+```
+
+The regression creates an empty table and a four-row table through a normal
+client-planned catalog. It submits a real REST plan, reads a byte from every
+planned data file with restricted credentials, and checks that a manifest HEAD
+fails with HTTP 403. Doris server-planned reads must then report the explicit
+unsupported-mode error, including with restricted storage credentials. `DESC`,
+`SHOW CREATE TABLE`, and `INSERT` are also rejected during metadata
acquisition;
+client-planned reads remain unchanged. Both enabled and disabled table caches
+are covered.
+
+An absent service, missing `/plan`, failed storage initialization, or an
invalid
+storage-policy control **fails** the enabled suite. There is no
capability-based
+skip. The normal top-level `enableIcebergTest` gate still applies to jobs that
do
+not run external Iceberg tests.
+
+For a lightweight local run without the Spark data archive, render the same
+Compose template with a unique prefix, then start only the new fixture and its
+MinIO dependencies (from this directory):
+
+```bash
+sed 's/doris--/scan-check-/g' iceberg.yaml.tpl > iceberg.yaml
+# Override these ports if another local test stack already uses the defaults.
+docker compose -p scan-check --env-file iceberg.env -f iceberg.yaml \
+ up -d --wait rest-scan-planning
+```
+
+This uses the same repository services and initialization as the full stack.
+Do not share the generated YAML between simultaneously running stacks. Set the
+matching host/ports and `enableIcebergTest=true` in your local regression
config.
+The new catalog is ephemeral across container recreation; the suite recreates
+its own tables. The bucket remains in MinIO's existing data volume.
+
+This bug regression does not simulate an additional server row policy. A
+client-supplied filter and a server-imposed policy are different contracts;
+server-forced planning mode is covered separately by real RESTCatalog unit
tests.
diff --git a/docker/thirdparties/docker-compose/iceberg/iceberg.env
b/docker/thirdparties/docker-compose/iceberg/iceberg.env
index 0950783075c..dfaa5f962a0 100644
--- a/docker/thirdparties/docker-compose/iceberg/iceberg.env
+++ b/docker/thirdparties/docker-compose/iceberg/iceberg.env
@@ -23,3 +23,5 @@ SPARK_THRIFT_PORT=11000
REST_CATALOG_PORT=18181
MINIO_UI_PORT=9000
MINIO_API_PORT=19001
+
+ICEBERG_SCAN_PLANNING_REST_PORT=18182
diff --git a/docker/thirdparties/docker-compose/iceberg/iceberg.yaml.tpl
b/docker/thirdparties/docker-compose/iceberg/iceberg.yaml.tpl
index a9fecf4a936..de392befe8a 100644
--- a/docker/thirdparties/docker-compose/iceberg/iceberg.yaml.tpl
+++ b/docker/thirdparties/docker-compose/iceberg/iceberg.yaml.tpl
@@ -101,6 +101,42 @@ services:
- /usr/lib/iceberg-rest/iceberg-rest-adapter.jar:/opt/jdbc/postgresql.jar
- org.apache.iceberg.rest.RESTCatalogServer
+ # Dedicated, mandatory fixture for the server-planning regression suite.
Keep the
+ # existing Spark/Postgres baseline on its released REST fixture.
+ rest-scan-planning:
+ # Content pinned by digest: upstream main
b0df3ca01d61b2f7ae7143ac660c6b16e33b6e46,
+ # built 2026-04-29 before the 1.11.0 release; same pin as
apache/iceberg-go.
+ # Replace after CI maintainers publish a verified 1.11.0 build to
doristhirdpartydocker.
+ image:
apache/iceberg-rest-fixture:latest@sha256:db8de90b5b7693d4ac334c336f91d9bbe320d7b19f4f514d26de84cdfbcbfe8d
+ container_name: doris--iceberg-rest-scan-planning
+ ports:
+ - ${ICEBERG_SCAN_PLANNING_REST_PORT:-18182}:8181
+ depends_on:
+ scan-planning-init:
+ condition: service_completed_successfully
+ environment:
+ AWS_ACCESS_KEY_ID: admin
+ AWS_SECRET_ACCESS_KEY: password
+ AWS_REGION: us-east-1
+ CATALOG_WAREHOUSE: s3://scan-planning/wh/
+ CATALOG_IO__IMPL: org.apache.iceberg.aws.s3.S3FileIO
+ CATALOG_S3_ENDPOINT: http://minio:9000
+ CATALOG_S3_PATH__STYLE__ACCESS: "true"
+ networks:
+ - doris--iceberg
+
+ scan-planning-init:
+ image: doristhirdpartydocker/mc:RELEASE.2025-01-17T23-25-50Z
+ container_name: doris--iceberg-scan-planning-init
+ depends_on:
+ minio:
+ condition: service_healthy
+ volumes:
+ - ./scripts/scan-planning:/fixtures:ro
+ entrypoint: ["/bin/sh", "/fixtures/init-storage.sh"]
+ networks:
+ - doris--iceberg
+
trino:
image: trinodb/trino:482
container_name: doris--iceberg-trino
diff --git
a/docker/thirdparties/docker-compose/iceberg/scripts/scan-planning/data-only-policy.json
b/docker/thirdparties/docker-compose/iceberg/scripts/scan-planning/data-only-policy.json
new file mode 100644
index 00000000000..1ee22e00fa0
--- /dev/null
+++
b/docker/thirdparties/docker-compose/iceberg/scripts/scan-planning/data-only-policy.json
@@ -0,0 +1,32 @@
+{
+ "Version": "2012-10-17",
+ "Statement": [
+ {
+ "Effect": "Allow",
+ "Action": [
+ "s3:GetBucketLocation"
+ ],
+ "Resource": [
+ "arn:aws:s3:::scan-planning"
+ ]
+ },
+ {
+ "Effect": "Allow",
+ "Action": [
+ "s3:GetObject"
+ ],
+ "Resource": [
+ "arn:aws:s3:::scan-planning/wh/rest_guard_db/*/data/*"
+ ]
+ },
+ {
+ "Effect": "Deny",
+ "Action": [
+ "s3:GetObject"
+ ],
+ "Resource": [
+ "arn:aws:s3:::scan-planning/wh/rest_guard_db/*/metadata/*"
+ ]
+ }
+ ]
+}
diff --git a/docker/thirdparties/docker-compose/iceberg/iceberg.env
b/docker/thirdparties/docker-compose/iceberg/scripts/scan-planning/init-storage.sh
similarity index 68%
copy from docker/thirdparties/docker-compose/iceberg/iceberg.env
copy to
docker/thirdparties/docker-compose/iceberg/scripts/scan-planning/init-storage.sh
index 0950783075c..df796c3919d 100644
--- a/docker/thirdparties/docker-compose/iceberg/iceberg.env
+++
b/docker/thirdparties/docker-compose/iceberg/scripts/scan-planning/init-storage.sh
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/bin/sh
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
@@ -16,10 +16,10 @@
# specific language governing permissions and limitations
# under the License.
-NOTEBOOK_SERVER_PORT=8888
-SPARK_DRIVER_UI_PORT=8080
-SPARK_HISTORY_UI_PORT=10000
-SPARK_THRIFT_PORT=11000
-REST_CATALOG_PORT=18181
-MINIO_UI_PORT=9000
-MINIO_API_PORT=19001
+set -eu
+mc alias set local http://minio:9000 admin password
+mc mb --ignore-existing local/scan-planning
+mc anonymous set none local/scan-planning
+mc admin user add local scan_data_reader ScanDataOnly2026
+mc admin policy create local scan-data-only /fixtures/data-only-policy.json
+mc admin policy attach local scan-data-only --user scan_data_reader
diff --git
a/fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergScanPlanProvider.java
b/fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergScanPlanProvider.java
index 3fb1472114a..5d1f12a57fb 100644
---
a/fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergScanPlanProvider.java
+++
b/fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergScanPlanProvider.java
@@ -70,7 +70,6 @@ import org.apache.iceberg.SchemaAwareDataTableScan;
import org.apache.iceberg.SchemaParser;
import org.apache.iceberg.Snapshot;
import org.apache.iceberg.SplittableScanTask;
-import org.apache.iceberg.SupportsDistributedScanPlanning;
import org.apache.iceberg.Table;
import org.apache.iceberg.TableOperations;
import org.apache.iceberg.TableProperties;
@@ -3112,14 +3111,7 @@ public class IcebergScanPlanProvider implements
ConnectorScanPlanProvider {
}
private static void rejectServerSideScanPlanning(Table table,
IcebergTableHandle handle) {
- if (table instanceof SupportsDistributedScanPlanning
- && !((SupportsDistributedScanPlanning)
table).allowDistributedPlanning()) {
- // Iceberg 1.11 marks REST server-planned tables this way. Doris
reads manifests and table.io()
- // before planFiles(), when REST scan-scoped credentials do not
exist, so fail before any local I/O.
- throw new DorisConnectorException("Iceberg server-side scan
planning is not supported for table "
- + handle.getDbName() + "." + handle.getTableName()
- + "; configure the REST catalog to use client-side scan
planning");
- }
+ IcebergScanPlanning.rejectServerSideScanPlanning(table,
handle.getDbName() + "." + handle.getTableName());
}
/**
diff --git
a/fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergScanPlanning.java
b/fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergScanPlanning.java
new file mode 100644
index 00000000000..bf48244ff84
--- /dev/null
+++
b/fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergScanPlanning.java
@@ -0,0 +1,38 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.connector.iceberg;
+
+import org.apache.doris.connector.spi.DorisConnectorException;
+
+import org.apache.iceberg.SupportsDistributedScanPlanning;
+import org.apache.iceberg.Table;
+
+/** Checks native table capabilities before snapshot wrappers replace them
with BaseTable defaults. */
+final class IcebergScanPlanning {
+ private IcebergScanPlanning() {}
+
+ static void rejectServerSideScanPlanning(Table table, String tableName) {
+ if (table instanceof SupportsDistributedScanPlanning
+ && !((SupportsDistributedScanPlanning)
table).allowDistributedPlanning()) {
+ // RESTTable overrides this to false; plain BaseTable inherits
true. Doris cannot consume
+ // server tasks/residuals or scan-scoped credentials yet, so
reject before local manifest I/O.
+ throw new DorisConnectorException("Iceberg server-side scan
planning is not supported for table "
+ + tableName + "; configure the REST catalog to use
client-side scan planning");
+ }
+ }
+}
diff --git
a/fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergStatementScope.java
b/fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergStatementScope.java
index d64ded6f039..06919691b1b 100644
---
a/fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergStatementScope.java
+++
b/fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergStatementScope.java
@@ -83,7 +83,7 @@ final class IcebergStatementScope {
// historical "iceberg.table:" key prefix byte-for-byte, so the funnel
keeps identical hits / misses / NONE
// fall-through (proved by
IcebergStatementScopeTest#sharedTableKeyReproducesLegacyPrefixByteForByte).
return ConnectorStatementScopes.resolveInStatement(session,
TABLE_NAMESPACE, dbName, tableName,
- () -> snapshotReadTable(loader.get()));
+ () -> snapshotReadTable(loader.get(), dbName, tableName));
}
/**
@@ -97,7 +97,7 @@ final class IcebergStatementScope {
if (session == null || session.getStatementScope() ==
ConnectorStatementScope.NONE) {
// NONE has no statement-end callback, so it cannot safely own a
lease. Preserve its original direct
// load-every-time behavior; creating a lease here would drop its
only close handle and leak forever.
- return snapshotReadTable(unscopedLoader.get());
+ return snapshotReadTable(unscopedLoader.get(), dbName, tableName);
}
ScopedBorrow borrowed = ConnectorStatementScopes.resolveInStatement(
session, TABLE_NAMESPACE, dbName, tableName, () -> new
ScopedBorrow(loader.get()));
@@ -121,11 +121,34 @@ final class IcebergStatementScope {
IcebergCatalogResourceTracker resourceTracker, Supplier<Table>
loader,
Function<Table, Runnable> cleanupFactory) {
if (session == null || session.getStatementScope() ==
ConnectorStatementScope.NONE) {
- return snapshotReadTable(loader.get());
+ Table table = loader.get();
+ try {
+ return snapshotReadTable(table, dbName, tableName);
+ } catch (RuntimeException | Error failure) {
+ try {
+ cleanupFactory.apply(table).run();
+ } catch (RuntimeException | Error cleanupFailure) {
+ failure.addSuppressed(cleanupFailure);
+ }
+ throw failure;
+ }
}
TrackedTable tracked = ConnectorStatementScopes.resolveInStatement(
session, TABLE_NAMESPACE, dbName, tableName,
- () -> trackedTable(resourceTracker, loader, cleanupFactory,
true));
+ () -> {
+ TrackedTable loaded = trackedTable(resourceTracker,
loader, cleanupFactory, true);
+ try {
+
IcebergScanPlanning.rejectServerSideScanPlanning(loaded.table(), dbName + "." +
tableName);
+ return loaded;
+ } catch (RuntimeException | Error failure) {
+ try {
+ loaded.close();
+ } catch (RuntimeException | Error cleanupFailure) {
+ failure.addSuppressed(cleanupFailure);
+ }
+ throw failure;
+ }
+ });
return tracked.table();
}
@@ -135,7 +158,7 @@ final class IcebergStatementScope {
Function<Table, Runnable> cleanupFactory, Function<Table, T>
action) {
if (session == null || session.getStatementScope() ==
ConnectorStatementScope.NONE) {
try (TrackedTable tracked = trackedTable(resourceTracker, loader,
cleanupFactory, false)) {
- return action.apply(snapshotReadTable(tracked.table()));
+ return action.apply(snapshotReadTable(tracked.table(), dbName,
tableName));
}
}
return action.apply(sharedTrackedTable(
@@ -254,7 +277,8 @@ final class IcebergStatementScope {
}
}
- private static Table snapshotReadTable(Table table) {
+ private static Table snapshotReadTable(Table table, String dbName, String
tableName) {
+ IcebergScanPlanning.rejectServerSideScanPlanning(table, dbName + "." +
tableName);
if (!(table instanceof BaseTable)) {
return table;
}
diff --git
a/fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergTableCache.java
b/fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergTableCache.java
index eb9bf97256f..43be412bd04 100644
---
a/fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergTableCache.java
+++
b/fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergTableCache.java
@@ -143,8 +143,20 @@ final class IcebergTableCache {
}
}
};
- TableOwner loaded = new TableOwner(table, cleanup, true,
- entry.isEnabled() && entry.isWeightBounded());
+ TableOwner loaded;
+ try {
+ // Reject before weight estimation/metadata
serialization as well as snapshot conversion.
+
IcebergScanPlanning.rejectServerSideScanPlanning(table, identifier.toString());
+ loaded = new TableOwner(table, cleanup, true,
+ entry.isEnabled() && entry.isWeightBounded());
+ } catch (RuntimeException | Error failure) {
+ try {
+ cleanup.run();
+ } catch (RuntimeException | Error cleanupFailure) {
+ failure.addSuppressed(cleanupFailure);
+ }
+ throw failure;
+ }
loadedHere[0] = loaded;
return loaded;
} finally {
diff --git
a/fe/fe-connector/fe-connector-iceberg/src/test/java/org/apache/doris/connector/iceberg/IcebergRestScanPlanningTest.java
b/fe/fe-connector/fe-connector-iceberg/src/test/java/org/apache/doris/connector/iceberg/IcebergRestScanPlanningTest.java
new file mode 100644
index 00000000000..bb48030e317
--- /dev/null
+++
b/fe/fe-connector/fe-connector-iceberg/src/test/java/org/apache/doris/connector/iceberg/IcebergRestScanPlanningTest.java
@@ -0,0 +1,303 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.connector.iceberg;
+
+import org.apache.doris.connector.cache.CacheSpec;
+import org.apache.doris.connector.cache.CatalogMetaCache;
+import
org.apache.doris.connector.iceberg.IcebergStatementScopeTest.ScopeSession;
+import org.apache.doris.connector.spi.ConnectorSession;
+import org.apache.doris.connector.spi.ConnectorStatementScope;
+import org.apache.doris.connector.spi.DorisConnectorException;
+import org.apache.doris.connector.spi.scan.ConnectorScanRequest;
+
+import com.sun.net.httpserver.HttpServer;
+import org.apache.iceberg.BaseTable;
+import org.apache.iceberg.DataFiles;
+import org.apache.iceberg.PartitionSpec;
+import org.apache.iceberg.Schema;
+import org.apache.iceberg.SupportsDistributedScanPlanning;
+import org.apache.iceberg.Table;
+import org.apache.iceberg.catalog.Namespace;
+import org.apache.iceberg.catalog.TableIdentifier;
+import org.apache.iceberg.inmemory.InMemoryCatalog;
+import org.apache.iceberg.io.FileIO;
+import org.apache.iceberg.io.InputFile;
+import org.apache.iceberg.io.OutputFile;
+import org.apache.iceberg.rest.Endpoint;
+import org.apache.iceberg.rest.RESTCatalog;
+import org.apache.iceberg.rest.responses.ConfigResponse;
+import org.apache.iceberg.rest.responses.ConfigResponseParser;
+import org.apache.iceberg.rest.responses.LoadTableResponse;
+import org.apache.iceberg.rest.responses.LoadTableResponseParser;
+import org.apache.iceberg.types.Types;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.function.Executable;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.nio.charset.StandardCharsets;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicInteger;
+
+/** Exercises actual SDK RESTTable capabilities across Doris read ownership
boundaries. */
+public class IcebergRestScanPlanningTest {
+ private static final TableIdentifier ID = TableIdentifier.of("db", "t");
+
+ @Test
+ public void directReadRejectsBeforeSnapshotConversion() throws Exception {
+ for (boolean serverOverride : new boolean[] {false, true}) {
+ for (boolean populated : new boolean[] {false, true}) {
+ try (RestFixture fixture = new RestFixture(serverOverride,
populated, true)) {
+ Table table = fixture.load();
+ Assertions.assertInstanceOf(BaseTable.class, table);
+ Assertions.assertFalse(((SupportsDistributedScanPlanning)
table).allowDistributedPlanning());
+ for (ConnectorStatementScope scope :
List.of(ConnectorStatementScope.NONE,
+ new TestStatementScope())) {
+ ConnectorSession session = new ScopeSession(1L, "q",
scope);
+ reject(() ->
IcebergStatementScope.sharedTable(session, "db", "t", () -> table));
+ if (scope == ConnectorStatementScope.NONE) {
+ reject(() ->
IcebergStatementScope.sharedBorrowedTable(session, "db", "t", () -> {
+ throw new AssertionError("NONE must use direct
loader");
+ }, () -> table));
+ }
+ scope.closeAll();
+ }
+ Assertions.assertEquals(0, fixture.io(table).reads.get());
+ Assertions.assertEquals(0, fixture.planRequests.get());
+ }
+ }
+ }
+ }
+
+ @Test
+ public void cacheRejectsBeforeSizingAndReleasesOwnership() throws
Exception {
+ for (boolean serverOverride : new boolean[] {false, true}) {
+ for (boolean populated : new boolean[] {false, true}) {
+ try (RestFixture fixture = new RestFixture(serverOverride,
populated, true)) {
+ for (CacheSpec spec : List.of(CacheSpec.ofConnectorTtl(0,
10),
+ CacheSpec.ofConnectorTtl(100, 10),
CacheSpec.ofWeight(true, 100, 1000, 1000000))) {
+ AtomicInteger tableCleanups = new AtomicInteger();
+ AtomicInteger catalogCleanups = new AtomicInteger();
+ IcebergCatalogResourceTracker tracker = new
IcebergCatalogResourceTracker();
+ try (CatalogMetaCache owner =
CatalogMetaCache.unmanaged()) {
+ IcebergTableCache cache = new
IcebergTableCache(owner, spec,
+ ignored -> tableCleanups::incrementAndGet,
tracker);
+ Table table = fixture.load();
+ for (int attempt = 0; attempt < 2; attempt++) {
+ reject(() -> cache.borrow(ID, () -> table));
+ }
+ Assertions.assertEquals(0, cache.size());
+ Assertions.assertEquals(2, tableCleanups.get());
+ Assertions.assertEquals(0,
fixture.io(table).reads.get());
+ cache.close();
+ tracker.close(catalogCleanups::incrementAndGet);
+ Assertions.assertEquals(1, catalogCleanups.get(),
"rejected loads must release leases");
+ Assertions.assertEquals(0,
tracker.retainedCleanupCount());
+ }
+ }
+ }
+ }
+ }
+ }
+
+ @Test
+ public void
trackedReadsReleaseRejectedTablesButWritableTablesRemainAvailable() throws
Exception {
+ try (RestFixture fixture = new RestFixture(true, true, true)) {
+ for (ConnectorStatementScope scope :
List.of(ConnectorStatementScope.NONE, new TestStatementScope())) {
+ ConnectorSession session = new ScopeSession(1L, "q", scope);
+ IcebergCatalogResourceTracker tracker = new
IcebergCatalogResourceTracker();
+ AtomicInteger tableCleanups = new AtomicInteger();
+ AtomicInteger catalogCleanups = new AtomicInteger();
+ Table table = fixture.load();
+ reject(() -> IcebergStatementScope.withTrackedTable(session,
"db", "t", tracker,
+ () -> table, ignored ->
tableCleanups::incrementAndGet, value -> value));
+ Assertions.assertEquals(1, tableCleanups.get());
+ Assertions.assertSame(table,
+ IcebergStatementScope.sharedWritableTable(session,
"db", "t", () -> table));
+ scope.closeAll();
+ tracker.close(catalogCleanups::incrementAndGet);
+ Assertions.assertEquals(1, tableCleanups.get(), "failed scope
load must not retain a close callback");
+ Assertions.assertEquals(1, catalogCleanups.get());
+ Assertions.assertEquals(0, fixture.io(table).reads.get());
+ }
+ }
+ }
+
+ @Test
+ public void unscopedDirectRejectionReleasesTableOwnedResources() throws
Exception {
+ try (RestFixture fixture = new RestFixture(true, true, true)) {
+ Table table = fixture.load();
+ AtomicInteger cleanups = new AtomicInteger();
+ reject(() -> IcebergStatementScope.sharedTrackedTable(null, "db",
"t",
+ new IcebergCatalogResourceTracker(), () -> table, ignored
-> cleanups::incrementAndGet));
+ Assertions.assertEquals(1, cleanups.get());
+ Assertions.assertEquals(0, fixture.io(table).reads.get());
+ }
+ }
+
+ @Test
+ public void clientPlanningStillCreatesPinnedReadViews() throws Exception {
+ try (RestFixture fixture = new RestFixture(false, true, false);
+ CatalogMetaCache owner = CatalogMetaCache.unmanaged()) {
+ Table table = fixture.load();
+ Assertions.assertTrue(((SupportsDistributedScanPlanning)
table).allowDistributedPlanning());
+ Table direct = IcebergStatementScope.sharedTable(null, "db", "t",
() -> table);
+ Assertions.assertNotSame(table, direct);
+ Assertions.assertEquals(table.currentSnapshot().snapshotId(),
direct.currentSnapshot().snapshotId());
+ IcebergTableCache cache = new IcebergTableCache(owner, 100, 10,
ignored -> () -> { }, null);
+ try (IcebergTableCache.TableLease lease = cache.borrow(ID, () ->
table)) {
+ Table cached = lease.snapshotReadTable();
+ Assertions.assertNotSame(table, cached);
+ Assertions.assertEquals(direct.currentSnapshot().snapshotId(),
cached.currentSnapshot().snapshotId());
+ }
+ cache.close();
+ Assertions.assertEquals(0, fixture.io(table).reads.get());
+ }
+ }
+
+ @Test
+ public void scanAndMetadataTableRejectThroughBothAcquisitionPaths() throws
Exception {
+ try (RestFixture fixture = new RestFixture(true, true, true)) {
+ for (boolean cached : new boolean[] {false, true}) {
+ for (ConnectorStatementScope scope :
List.of(ConnectorStatementScope.NONE, new TestStatementScope())) {
+ ConnectorSession session = new ScopeSession(1L, "q",
scope);
+ Table table = fixture.load();
+ RecordingIcebergCatalogOps ops = new
RecordingIcebergCatalogOps();
+ ops.table = table;
+ IcebergTableCache cache = new IcebergTableCache(100, 10);
+ try {
+ IcebergScanPlanProvider provider = new
IcebergScanPlanProvider(
+ IcebergCatalogProperties.of(Map.of()), ignored
-> ops, null, null,
+ cached ? cache : null);
+ reject(() -> provider.planScan(session,
ConnectorScanRequest.builder(
+ new IcebergTableHandle("db", "t"),
List.of()).build()));
+ reject(() -> provider.planScan(session,
ConnectorScanRequest.builder(
+ IcebergTableHandle.forSystemTable("db", "t",
"files", -1L, null, -1L),
+ List.of()).build()));
+ Assertions.assertEquals(0,
fixture.io(table).reads.get());
+ } finally {
+ scope.closeAll();
+ cache.close();
+ }
+ }
+ }
+ Assertions.assertEquals(0, fixture.planRequests.get());
+ }
+ }
+
+ private static void reject(Executable action) {
+ DorisConnectorException failure =
Assertions.assertThrows(DorisConnectorException.class, action);
+ Assertions.assertEquals("Iceberg server-side scan planning is not
supported for table db.t; "
+ + "configure the REST catalog to use client-side scan
planning", failure.getMessage());
+ }
+
+ /** A minimal real HTTP catalog; any attempt to read local manifests fails
immediately. */
+ private static final class RestFixture implements AutoCloseable {
+ private final HttpServer server;
+ private final RESTCatalog catalog = new RESTCatalog();
+ private final AtomicInteger planRequests = new AtomicInteger();
+
+ private RestFixture(boolean serverOverride, boolean populated, boolean
serverPlanning) throws IOException {
+ byte[] loadResponse;
+ try (InMemoryCatalog storage = new InMemoryCatalog()) {
+ storage.initialize("storage", Map.of());
+ storage.createNamespace(Namespace.of("db"));
+ Table table = storage.createTable(ID,
+ new Schema(Types.NestedField.required(1, "id",
Types.IntegerType.get())));
+ if (populated) {
+
table.newAppend().appendFile(DataFiles.builder(PartitionSpec.unpartitioned())
+
.withPath("memory://data.parquet").withRecordCount(4).withFileSizeInBytes(100).build())
+ .commit();
+ }
+ LoadTableResponse.Builder response =
LoadTableResponse.builder()
+ .withTableMetadata(((BaseTable)
table).operations().current());
+ if (serverOverride) {
+ response.addConfig("scan-planning-mode", "server");
+ }
+ loadResponse =
LoadTableResponseParser.toJson(response.build()).getBytes(StandardCharsets.UTF_8);
+ }
+ byte[] config =
ConfigResponseParser.toJson(ConfigResponse.builder()
+ .withEndpoints(List.of(Endpoint.V1_LOAD_TABLE,
Endpoint.V1_SUBMIT_TABLE_SCAN_PLAN)).build())
+ .getBytes(StandardCharsets.UTF_8);
+ server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0),
0);
+ server.createContext("/v1/", exchange -> {
+ String path = exchange.getRequestURI().getPath();
+ byte[] response;
+ int status = 200;
+ if (path.equals("/v1/config")) {
+ response = config;
+ } else if (path.equals("/v1/namespaces/db/tables/t")) {
+ response = loadResponse;
+ } else {
+ planRequests.incrementAndGet();
+ response = new byte[0];
+ status = 500;
+ }
+ exchange.getResponseHeaders().set("Content-Type",
"application/json");
+ exchange.sendResponseHeaders(status, response.length);
+ try (var out = exchange.getResponseBody()) {
+ out.write(response);
+ }
+ });
+ server.start();
+ catalog.initialize("rest", Map.of("uri", "http://127.0.0.1:"; +
server.getAddress().getPort(),
+ "scan-planning-mode", serverPlanning && !serverOverride ?
"server" : "client",
+ "rest.auth.type", "none", "io-impl",
RecordingFileIO.class.getName()));
+ }
+
+ private Table load() {
+ return catalog.loadTable(ID);
+ }
+
+ private RecordingFileIO io(Table table) {
+ return (RecordingFileIO) table.io();
+ }
+
+ @Override
+ public void close() throws IOException {
+ try {
+ catalog.close();
+ } finally {
+ server.stop(0);
+ }
+ }
+ }
+
+ public static class RecordingFileIO implements FileIO {
+ private final AtomicInteger reads = new AtomicInteger();
+
+ @Override
+ public InputFile newInputFile(String path) {
+ reads.incrementAndGet();
+ throw new AssertionError("Unexpected local metadata read: " +
path);
+ }
+
+ @Override
+ public OutputFile newOutputFile(String path) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public void deleteFile(String path) {
+ throw new UnsupportedOperationException();
+ }
+ }
+
+}
diff --git
a/fe/fe-connector/fe-connector-iceberg/src/test/java/org/apache/doris/connector/iceberg/IcebergStatementScopeTest.java
b/fe/fe-connector/fe-connector-iceberg/src/test/java/org/apache/doris/connector/iceberg/IcebergStatementScopeTest.java
index bab105775e1..2df7d37c00b 100644
---
a/fe/fe-connector/fe-connector-iceberg/src/test/java/org/apache/doris/connector/iceberg/IcebergStatementScopeTest.java
+++
b/fe/fe-connector/fe-connector-iceberg/src/test/java/org/apache/doris/connector/iceberg/IcebergStatementScopeTest.java
@@ -314,7 +314,7 @@ public class IcebergStatementScopeTest {
}
/** Minimal {@link ConnectorSession} carrying a catalog id, queryId and
scope for the key + memo assertions. */
- private static final class ScopeSession implements ConnectorSession {
+ static final class ScopeSession implements ConnectorSession {
private final long catalogId;
private final String queryId;
private final ConnectorStatementScope scope;
diff --git a/regression-test/conf/regression-conf.groovy
b/regression-test/conf/regression-conf.groovy
index afbe0b1a5c3..8240d3f082e 100644
--- a/regression-test/conf/regression-conf.groovy
+++ b/regression-test/conf/regression-conf.groovy
@@ -239,6 +239,7 @@ extArrowFlightHttpPort= 8030
// iceberg rest catalog config
iceberg_rest_uri_port=18181
+iceberg_scan_planning_rest_uri_port=18182
iceberg_minio_port=19001
iceberg_rest_uri_port_s3=19181
iceberg_rest_uri_port_oss=19182
diff --git
a/regression-test/data/external_table_p0/iceberg/test_iceberg_rest_server_planning_guard.out
b/regression-test/data/external_table_p0/iceberg/test_iceberg_rest_server_planning_guard.out
new file mode 100644
index 00000000000..f95ccb035c7
--- /dev/null
+++
b/regression-test/data/external_table_p0/iceberg/test_iceberg_rest_server_planning_guard.out
@@ -0,0 +1,13 @@
+-- This file is automatically generated. You should know what you did if you
want to edit this
+-- !client --
+1
+2
+3
+4
+
+-- !client_after --
+1
+2
+3
+4
+
diff --git a/regression-test/pipeline/external/conf/regression-conf.groovy
b/regression-test/pipeline/external/conf/regression-conf.groovy
index d6679434ecc..4195d59f214 100644
--- a/regression-test/pipeline/external/conf/regression-conf.groovy
+++ b/regression-test/pipeline/external/conf/regression-conf.groovy
@@ -144,6 +144,7 @@ enableRefactorParamsTest=true
// iceberg test config
iceberg_rest_uri_port=18181
+iceberg_scan_planning_rest_uri_port=18182
iceberg_rest_uri_port_s3=19181
iceberg_rest_uri_port_oss=19182
iceberg_rest_uri_port_cos=19183
diff --git a/regression-test/pipeline/nonConcurrent/conf/regression-conf.groovy
b/regression-test/pipeline/nonConcurrent/conf/regression-conf.groovy
index b6f43050860..2d6ac80d06b 100644
--- a/regression-test/pipeline/nonConcurrent/conf/regression-conf.groovy
+++ b/regression-test/pipeline/nonConcurrent/conf/regression-conf.groovy
@@ -122,6 +122,7 @@ kafka_port=19193
// iceberg test config
iceberg_rest_uri_port=18181
+iceberg_scan_planning_rest_uri_port=18182
iceberg_minio_port=19001
iceberg_rest_uri_port_s3=19181
iceberg_rest_uri_port_oss=19182
diff --git a/regression-test/pipeline/p0/conf/regression-conf.groovy
b/regression-test/pipeline/p0/conf/regression-conf.groovy
index 6c8b89be058..a44485ec44d 100644
--- a/regression-test/pipeline/p0/conf/regression-conf.groovy
+++ b/regression-test/pipeline/p0/conf/regression-conf.groovy
@@ -129,6 +129,7 @@ kafka_port=19193
// iceberg test config
iceberg_rest_uri_port=18181
+iceberg_scan_planning_rest_uri_port=18182
iceberg_minio_port=19001
iceberg_rest_uri_port_s3=19181
iceberg_rest_uri_port_oss=19182
diff --git
a/regression-test/suites/external_table_p0/iceberg/test_iceberg_rest_server_planning_guard.groovy
b/regression-test/suites/external_table_p0/iceberg/test_iceberg_rest_server_planning_guard.groovy
new file mode 100644
index 00000000000..c455270d098
--- /dev/null
+++
b/regression-test/suites/external_table_p0/iceberg/test_iceberg_rest_server_planning_guard.groovy
@@ -0,0 +1,177 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+import com.amazonaws.auth.AWSStaticCredentialsProvider
+import com.amazonaws.auth.BasicAWSCredentials
+import com.amazonaws.client.builder.AwsClientBuilder
+import com.amazonaws.services.s3.AmazonS3ClientBuilder
+import com.amazonaws.services.s3.model.AmazonS3Exception
+import com.amazonaws.services.s3.model.GetObjectRequest
+import groovy.json.JsonSlurper
+import org.junit.jupiter.api.Assertions
+
+suite("test_iceberg_rest_server_planning_guard", "p0,external") {
+ if
(!"true".equalsIgnoreCase(context.config.otherConfigs.get("enableIcebergTest")))
{
+ return
+ }
+ String host = context.config.otherConfigs.get("externalEnvIp")
+ String restPort =
context.config.otherConfigs.get("iceberg_scan_planning_rest_uri_port")
+ String minioPort = context.config.otherConfigs.get("iceberg_minio_port")
+ String uri = "http://${host}:${restPort}";
+ // The dedicated fixture is mandatory whenever Iceberg tests are enabled.
Missing capabilities
+ // must fail the suite, not silently turn this regression into a passing
no-op.
+ def config = new JsonSlurper().parseText(new
URL("${uri}/v1/config").getText(
+ connectTimeout: 10000, readTimeout: 10000))
+ assertTrue(config.endpoints?.any { it.startsWith("POST ") &&
it.endsWith("/tables/{table}/plan") },
+ "Dedicated Iceberg REST fixture must advertise scan planning")
+
+ sql "DROP CATALOG IF EXISTS rest_guard_client"
+ sql """CREATE CATALOG rest_guard_client PROPERTIES (
+ 'type' = 'iceberg', 'iceberg.catalog.type' = 'rest', 'uri' = '${uri}',
+ 's3.access_key' = 'admin', 's3.secret_key' = 'password',
+ 's3.endpoint' = 'http://${host}:${minioPort}', 's3.region' =
'us-east-1',
+ 'use_path_style' = 'true', 'scan-planning-mode' = 'client'
+ )"""
+ sql "CREATE DATABASE IF NOT EXISTS rest_guard_client.rest_guard_db"
+ sql "DROP TABLE IF EXISTS rest_guard_client.rest_guard_db.populated"
+ sql "DROP TABLE IF EXISTS rest_guard_client.rest_guard_db.empty_table"
+ sql "CREATE TABLE rest_guard_client.rest_guard_db.populated (id INT)"
+ sql "CREATE TABLE rest_guard_client.rest_guard_db.empty_table (id INT)"
+ sql "INSERT INTO rest_guard_client.rest_guard_db.populated VALUES (1),
(2), (3), (4)"
+ order_qt_client "SELECT * FROM rest_guard_client.rest_guard_db.populated
ORDER BY id"
+
+ // Exercise the real /plan endpoint before testing rejection. A capability
advertisement alone
+ // would not prove the selected image can plan this table using its own
storage credentials.
+ String tablePath = "/v1/namespaces/rest_guard_db/tables/populated"
+ def connection = (HttpURLConnection) new
URL("${uri}${tablePath}/plan").openConnection()
+ connection.setConnectTimeout(10000)
+ connection.setReadTimeout(30000)
+ connection.setRequestMethod("POST")
+ connection.setRequestProperty("Content-Type", "application/json")
+ connection.setDoOutput(true)
+ def plan
+ try {
+ connection.outputStream.withCloseable {
it.write("{}".getBytes("UTF-8")) }
+ assertEquals(200, connection.responseCode, "Real server-side planning
must succeed")
+ connection.inputStream.withCloseable { plan = new
JsonSlurper().parse(it) }
+ } finally {
+ connection.disconnect()
+ }
+ assertEquals("completed", plan.status)
+ assertTrue(!plan["file-scan-tasks"].isEmpty(), "The real fixture must
return data-file tasks")
+
+ def loaded = new JsonSlurper().parseText(new
URL("${uri}${tablePath}").getText(
+ connectTimeout: 10000, readTimeout: 10000))
+ def snapshot = loaded.metadata.snapshots.find {
+ it["snapshot-id"] == loaded.metadata["current-snapshot-id"]
+ }
+ def restrictedStorage = AmazonS3ClientBuilder.standard()
+ .withEndpointConfiguration(new
AwsClientBuilder.EndpointConfiguration(
+ "http://${host}:${minioPort}";, "us-east-1"))
+ .withCredentials(new AWSStaticCredentialsProvider(
+ new BasicAWSCredentials("scan_data_reader",
"ScanDataOnly2026")))
+ .withPathStyleAccessEnabled(true)
+ .build()
+ try {
+ // Verify both halves of the actual policy: data GET succeeds,
metadata HEAD is forbidden.
+ for (def task : plan["file-scan-tasks"]) {
+ URI file = new URI(task["data-file"]["file-path"])
+ def request = new GetObjectRequest(file.host,
file.path.substring(1)).withRange(0, 0)
+ restrictedStorage.getObject(request).withCloseable { obj ->
+ assertTrue(obj.objectContent.read() >= 0, "Restricted client
must be able to read data")
+ }
+ }
+ URI manifest = new URI(snapshot["manifest-list"])
+ try {
+ restrictedStorage.getObjectMetadata(manifest.host,
manifest.path.substring(1))
+ Assertions.fail("Restricted client unexpectedly read manifest
metadata")
+ } catch (AmazonS3Exception denied) {
+ assertEquals(403, denied.statusCode, "The failure must be a real
storage permission denial")
+ }
+ } finally {
+ restrictedStorage.shutdown()
+ }
+ // Cancel only after the storage checks succeed, so cancellation cannot
mask their failure.
+ // Completed plans may omit plan-id; only cancel when the server provides
one.
+ if (plan["plan-id"] != null) {
+ def cancel = (HttpURLConnection) new
URL("${uri}${tablePath}/plan/${plan['plan-id']}").openConnection()
+ try {
+ cancel.setRequestMethod("DELETE")
+ cancel.setConnectTimeout(10000)
+ cancel.setReadTimeout(10000)
+ assertEquals(204, cancel.responseCode)
+ } finally {
+ cancel.disconnect()
+ }
+ }
+
+ for (String cacheEnabled : ["true", "false"]) {
+ String catalog = "rest_guard_server_${cacheEnabled}"
+ sql "DROP CATALOG IF EXISTS ${catalog}"
+ sql """CREATE CATALOG ${catalog} PROPERTIES (
+ 'type' = 'iceberg', 'iceberg.catalog.type' = 'rest', 'uri' =
'${uri}',
+ 's3.access_key' = 'admin', 's3.secret_key' = 'password',
+ 's3.endpoint' = 'http://${host}:${minioPort}', 's3.region' =
'us-east-1',
+ 'use_path_style' = 'true', 'scan-planning-mode' = 'server',
+ 'meta.cache.iceberg.table.enable' = '${cacheEnabled}'
+ )"""
+ test {
+ sql "SELECT * FROM ${catalog}.rest_guard_db.populated ORDER BY id"
+ exception "Iceberg server-side scan planning is not supported"
+ }
+ test {
+ sql "DESC ${catalog}.rest_guard_db.populated"
+ exception "Iceberg server-side scan planning is not supported"
+ }
+ test {
+ sql "SHOW CREATE TABLE ${catalog}.rest_guard_db.populated"
+ exception "Iceberg server-side scan planning is not supported"
+ }
+ test {
+ sql "INSERT INTO ${catalog}.rest_guard_db.populated VALUES (99)"
+ exception "Iceberg server-side scan planning is not supported"
+ }
+ test {
+ sql "SELECT * FROM ${catalog}.rest_guard_db.empty_table"
+ exception "Iceberg server-side scan planning is not supported"
+ }
+ test {
+ sql "SELECT * FROM ${catalog}.rest_guard_db.`populated\$files`"
+ exception "Iceberg server-side scan planning is not supported"
+ }
+ }
+ sql "DROP CATALOG IF EXISTS rest_guard_restricted"
+ sql """CREATE CATALOG rest_guard_restricted PROPERTIES (
+ 'type' = 'iceberg', 'iceberg.catalog.type' = 'rest', 'uri' = '${uri}',
+ 's3.access_key' = 'scan_data_reader', 's3.secret_key' =
'ScanDataOnly2026',
+ 's3.endpoint' = 'http://${host}:${minioPort}', 's3.region' =
'us-east-1',
+ 'use_path_style' = 'true', 'scan-planning-mode' = 'server'
+ )"""
+ test {
+ sql "SELECT * FROM rest_guard_restricted.rest_guard_db.populated ORDER
BY id"
+ exception "Iceberg server-side scan planning is not supported"
+ }
+ test {
+ sql "SELECT * FROM
rest_guard_restricted.rest_guard_db.`populated\$files`"
+ exception "Iceberg server-side scan planning is not supported"
+ }
+ order_qt_client_after "SELECT * FROM
rest_guard_client.rest_guard_db.populated ORDER BY id"
+ sql "DROP CATALOG rest_guard_restricted"
+ sql "DROP CATALOG rest_guard_server_true"
+ sql "DROP CATALOG rest_guard_server_false"
+ sql "DROP CATALOG rest_guard_client"
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
