924060929 commented on code in PR #63582:
URL: https://github.com/apache/doris/pull/63582#discussion_r3542798744
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/PluginDrivenExternalCatalog.java:
##########
@@ -232,6 +238,44 @@ public Connector getConnector() {
return connector;
}
+ /**
+ * Routes {@code CREATE TABLE} through the SPI's
+ * {@code ConnectorTableOps.createTable(session, request)} instead of the
+ * legacy {@code metadataOps} path used by other {@link ExternalCatalog}
+ * subclasses.
+ *
+ * <p>Connectors that have not overridden the new SPI default fall through
+ * to the SPI's "CREATE TABLE not supported" exception, which is wrapped
+ * here as a {@link DdlException} to match the existing caller
contract.</p>
+ *
+ * <p>The SPI signature is {@code void}: it does not distinguish
+ * "newly created" from "already existed (IF NOT EXISTS)". This override
+ * conservatively assumes creation happened and writes the edit log,
matching
+ * the more common branch of the legacy path. Refining this when a
connector
+ * actually needs the distinction is left to P5/P6/P7 connector
migrations.</p>
+ */
+ @Override
+ public boolean createTable(CreateTableInfo createTableInfo) throws
UserException {
+ makeSureInitialized();
+ ConnectorSession session = buildConnectorSession();
+ ConnectorCreateTableRequest request =
CreateTableInfoToConnectorRequestConverter
+ .convert(createTableInfo, createTableInfo.getDbName());
+ try {
+ connector.getMetadata(session).createTable(session, request);
+ } catch (DorisConnectorException e) {
Review Comment:
Error-message parity (tracking issue invariant #3): when a connector does
not support CREATE TABLE, users now see the SPI default message `CREATE TABLE
not supported` (no catalog name), while the legacy path raises `Create table is
not supported for catalog: <name>` (ExternalCatalog.createTable). The same
drift applies to every unsupported op routed through ConnectorTableOps defaults
(DROP/RENAME/ADD COLUMN ...).
Suggestion: wrap here as `throw new DdlException(e.getMessage() + " for
catalog: " + getName(), e)` (or normalize messages centrally) so client-side
error assertions keep working across the cutover. Still present at tip
(PluginDrivenExternalCatalog.java:406-407).
##########
fe/fe-connector/fe-connector-api/src/main/java/org/apache/doris/connector/api/ConnectorTableOps.java:
##########
@@ -65,6 +67,27 @@ default void createTable(ConnectorSession session,
"CREATE TABLE not supported");
}
+ /**
+ * Creates a table with full DDL semantics (partition, bucket, external,
+ * {@code IF NOT EXISTS}).
+ *
+ * <p>Connectors should override this when they support advanced
+ * {@code CREATE TABLE} options. The default degrades to the legacy
+ * {@link #createTable(ConnectorSession, ConnectorTableSchema, Map)},
+ * dropping partition / bucket / external / {@code ifNotExists} info.</p>
+ *
+ * @throws DorisConnectorException if the connector cannot honor the
request
+ */
+ default void createTable(ConnectorSession session,
+ ConnectorCreateTableRequest request) {
+ ConnectorTableSchema schema = new ConnectorTableSchema(
+ request.getTableName(),
+ request.getColumns(),
+ null,
+ request.getProperties());
+ createTable(session, schema, request.getProperties());
Review Comment:
The default `createTable(session, request)` silently discards
`partitionSpec`, `bucketSpec`, `comment`, `external` and `ifNotExists` when
degrading to the legacy 3-arg overload. A connector that only implements the
legacy signature will accept `CREATE TABLE ... PARTITION BY ...` and create an
unpartitioned table, reporting success.
Suggestion: throw when `request.getPartitionSpec() != null ||
request.getBucketSpec() != null` before degrading. In-tree connectors override
this today (iceberg/maxcompute), but the trap remains for every
future/third-party connector. Still present at tip.
##########
fe/fe-core/src/main/java/org/apache/doris/connector/ddl/CreateTableInfoToConnectorRequestConverter.java:
##########
@@ -0,0 +1,209 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.connector.ddl;
+
+import org.apache.doris.catalog.PartitionType;
+import org.apache.doris.connector.api.ConnectorColumn;
+import org.apache.doris.connector.api.ConnectorType;
+import org.apache.doris.connector.api.ddl.ConnectorBucketSpec;
+import org.apache.doris.connector.api.ddl.ConnectorCreateTableRequest;
+import org.apache.doris.connector.api.ddl.ConnectorPartitionField;
+import org.apache.doris.connector.api.ddl.ConnectorPartitionSpec;
+import org.apache.doris.datasource.ConnectorColumnConverter;
+import org.apache.doris.nereids.analyzer.UnboundFunction;
+import org.apache.doris.nereids.analyzer.UnboundSlot;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.literal.IntegerLikeLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.trees.plans.commands.info.ColumnDefinition;
+import org.apache.doris.nereids.trees.plans.commands.info.CreateTableInfo;
+import
org.apache.doris.nereids.trees.plans.commands.info.DistributionDescriptor;
+import org.apache.doris.nereids.trees.plans.commands.info.PartitionTableInfo;
+import org.apache.doris.nereids.types.DataType;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+/**
+ * Converts a nereids {@link CreateTableInfo} into a connector-SPI
+ * {@link ConnectorCreateTableRequest}.
+ *
+ * <p>Covers Hive-style {@code IDENTITY}, Iceberg-style {@code TRANSFORM}, and
+ * Doris {@code LIST} / {@code RANGE} partitioning, plus hash / random
+ * distribution.</p>
+ */
+public final class CreateTableInfoToConnectorRequestConverter {
+
+ private CreateTableInfoToConnectorRequestConverter() {
+ }
+
+ /**
+ * @param info the nereids CREATE TABLE info (must be analyzed)
+ * @param dbName target database name (caller may normalize case)
+ */
+ public static ConnectorCreateTableRequest convert(CreateTableInfo info,
+ String dbName) {
+ return ConnectorCreateTableRequest.builder()
+ .dbName(dbName)
+ .tableName(info.getTableName())
+ .columns(convertColumns(info.getColumnDefinitions()))
+ .partitionSpec(convertPartition(info.getPartitionTableInfo()))
+ .bucketSpec(convertBucket(info.getDistribution()))
+ .comment(info.getComment())
+ .properties(info.getProperties())
+ .ifNotExists(info.isIfNotExists())
+ .external(info.isExternal())
+ .build();
+ }
+
+ // -------- columns --------
+
+ private static List<ConnectorColumn> convertColumns(
+ List<ColumnDefinition> defs) {
+ if (defs == null || defs.isEmpty()) {
+ return Collections.emptyList();
+ }
+ List<ConnectorColumn> out = new ArrayList<>(defs.size());
+ for (ColumnDefinition d : defs) {
+ DataType nereidsType = d.getType();
+ ConnectorType type = ConnectorColumnConverter.toConnectorType(
+ nereidsType.toCatalogDataType());
+ // Default value is not exposed via a public getter on
ColumnDefinition
+ // (private Optional<DefaultValue>); pass null until the SPI gains
a
+ // typed default-value carrier. See HANDOFF open issues.
+ out.add(new ConnectorColumn(
+ d.getName(), type, d.getComment(),
+ d.isNullable(), null, d.isKey()));
+ }
+ return out;
+ }
+
+ // -------- partition --------
+
+ private static ConnectorPartitionSpec convertPartition(
+ PartitionTableInfo info) {
+ if (info == null) {
+ return null;
+ }
+ String pType = info.getPartitionType();
+ List<Expression> exprs = info.getPartitionList();
+ boolean isList = PartitionType.LIST.name().equalsIgnoreCase(pType);
+ boolean isRange = PartitionType.RANGE.name().equalsIgnoreCase(pType);
+ boolean hasExprs = exprs != null && !exprs.isEmpty();
+ if (!isList && !isRange && !hasExprs) {
+ return null;
+ }
+
+ ConnectorPartitionSpec.Style style;
+ if (isList) {
+ style = ConnectorPartitionSpec.Style.LIST;
+ } else if (isRange) {
+ style = ConnectorPartitionSpec.Style.RANGE;
+ } else if (hasAnyTransform(exprs)) {
+ style = ConnectorPartitionSpec.Style.TRANSFORM;
+ } else {
+ style = ConnectorPartitionSpec.Style.IDENTITY;
+ }
+
+ List<ConnectorPartitionField> fields = hasExprs
+ ? convertFields(exprs)
+ : Collections.emptyList();
+ // LIST/RANGE PartitionDefinition values are not lowered here: each
+ // PartitionDefinition is a sealed family
(InPartition/LessThanPartition/
+ // FixedRangePartition/StepPartition) carrying nereids Expressions that
+ // require full analysis to flatten into List<List<String>>. Connectors
+ // that need the initial values today read the Doris PartitionDesc
+ // directly; this converter passes an empty list and leaves richer
+ // lowering for a follow-up.
+ return new ConnectorPartitionSpec(style, fields,
Collections.emptyList());
+ }
+
+ private static boolean hasAnyTransform(List<Expression> exprs) {
+ for (Expression e : exprs) {
+ if (e instanceof UnboundFunction) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ private static List<ConnectorPartitionField> convertFields(
+ List<Expression> exprs) {
+ List<ConnectorPartitionField> out = new ArrayList<>(exprs.size());
+ for (Expression e : exprs) {
+ if (e instanceof UnboundSlot) {
+ out.add(new ConnectorPartitionField(
+ ((UnboundSlot) e).getName(), "identity",
+ Collections.emptyList()));
+ } else if (e instanceof UnboundFunction) {
+ out.add(convertTransformField((UnboundFunction) e));
+ }
+ // Unknown expression shapes are dropped; the connector can still
Review Comment:
`convertFields` drops unrecognized partition expression shapes with only a
comment. For DDL, silent degradation is dangerous: a CREATE TABLE with an
unsupported partition transform yields a mis-partitioned (or unpartitioned)
table that reports success, and the connector cannot detect the loss because
the drop happens here in fe-core before the request is built.
Suggestion: fail fast on unconvertible shapes; a connector that wants
leniency can opt in explicitly. Still present at tip (the same applies to
`convertTransformField`'s silent arg drops).
##########
tools/check-connector-imports.sh:
##########
@@ -0,0 +1,64 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Forbidden-import gate for fe-connector modules.
+# See plan-doc/01-spi-extensions-rfc.md §15.4.
+#
+# Connector modules MUST NOT import fe-core internals (catalog / common /
+# datasource / qe / analysis / nereids / planner). Anything they need from
+# fe-core has to be exposed through the SPI in
+# org.apache.doris.connector.{api,spi,extension,...}
+# or shared types in org.apache.doris.thrift / org.apache.doris.filesystem.
+#
+# Usage:
+# tools/check-connector-imports.sh # search default root
+# tools/check-connector-imports.sh <fe-connector> # search supplied root
+#
+# Exit code:
+# 0 — no forbidden imports
+# 1 — at least one forbidden import found (offending lines printed)
+
+set -e
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+DEFAULT_ROOT="${SCRIPT_DIR}/../fe/fe-connector"
+ROOT="${1:-${DEFAULT_ROOT}}"
+
+if [ ! -d "${ROOT}" ]; then
+ echo "check-connector-imports: search root not found: ${ROOT}" >&2
+ exit 2
+fi
+
+FORBIDDEN='org\.apache\.doris\.(catalog|common|datasource|qe|analysis|nereids|planner)'
Review Comment:
Three latent holes in this gate (all still present at tip; no live
violations today — verified by grep):
1. `^import org\.apache\.doris\.(...)` does not match `import static
org.apache.doris.catalog...` — static imports bypass the gate;
2. the blacklist omits fe-core packages such as
`org.apache.doris.{persist,transaction,fs,statistics,mysql,service}` —
importing those passes;
3. only `src/main/java` is scanned, so test code can re-couple freely.
Since this script is the mechanical enforcement of the series' invariant #1,
consider inverting to an allowlist (only permit
`org.apache.doris.{connector,thrift,filesystem,extension}`), matching `^import
(static )?`, and scanning test sources too.
##########
fe/fe-core/src/main/java/org/apache/doris/connector/ddl/CreateTableInfoToConnectorRequestConverter.java:
##########
@@ -0,0 +1,209 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.connector.ddl;
+
+import org.apache.doris.catalog.PartitionType;
+import org.apache.doris.connector.api.ConnectorColumn;
+import org.apache.doris.connector.api.ConnectorType;
+import org.apache.doris.connector.api.ddl.ConnectorBucketSpec;
+import org.apache.doris.connector.api.ddl.ConnectorCreateTableRequest;
+import org.apache.doris.connector.api.ddl.ConnectorPartitionField;
+import org.apache.doris.connector.api.ddl.ConnectorPartitionSpec;
+import org.apache.doris.datasource.ConnectorColumnConverter;
+import org.apache.doris.nereids.analyzer.UnboundFunction;
+import org.apache.doris.nereids.analyzer.UnboundSlot;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.literal.IntegerLikeLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.trees.plans.commands.info.ColumnDefinition;
+import org.apache.doris.nereids.trees.plans.commands.info.CreateTableInfo;
+import
org.apache.doris.nereids.trees.plans.commands.info.DistributionDescriptor;
+import org.apache.doris.nereids.trees.plans.commands.info.PartitionTableInfo;
+import org.apache.doris.nereids.types.DataType;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+/**
+ * Converts a nereids {@link CreateTableInfo} into a connector-SPI
+ * {@link ConnectorCreateTableRequest}.
+ *
+ * <p>Covers Hive-style {@code IDENTITY}, Iceberg-style {@code TRANSFORM}, and
+ * Doris {@code LIST} / {@code RANGE} partitioning, plus hash / random
+ * distribution.</p>
+ */
+public final class CreateTableInfoToConnectorRequestConverter {
+
+ private CreateTableInfoToConnectorRequestConverter() {
+ }
+
+ /**
+ * @param info the nereids CREATE TABLE info (must be analyzed)
+ * @param dbName target database name (caller may normalize case)
+ */
+ public static ConnectorCreateTableRequest convert(CreateTableInfo info,
+ String dbName) {
+ return ConnectorCreateTableRequest.builder()
+ .dbName(dbName)
+ .tableName(info.getTableName())
+ .columns(convertColumns(info.getColumnDefinitions()))
+ .partitionSpec(convertPartition(info.getPartitionTableInfo()))
+ .bucketSpec(convertBucket(info.getDistribution()))
+ .comment(info.getComment())
+ .properties(info.getProperties())
+ .ifNotExists(info.isIfNotExists())
+ .external(info.isExternal())
+ .build();
+ }
+
+ // -------- columns --------
+
+ private static List<ConnectorColumn> convertColumns(
+ List<ColumnDefinition> defs) {
+ if (defs == null || defs.isEmpty()) {
+ return Collections.emptyList();
+ }
+ List<ConnectorColumn> out = new ArrayList<>(defs.size());
+ for (ColumnDefinition d : defs) {
+ DataType nereidsType = d.getType();
+ ConnectorType type = ConnectorColumnConverter.toConnectorType(
+ nereidsType.toCatalogDataType());
+ // Default value is not exposed via a public getter on
ColumnDefinition
+ // (private Optional<DefaultValue>); pass null until the SPI gains
a
+ // typed default-value carrier. See HANDOFF open issues.
+ out.add(new ConnectorColumn(
+ d.getName(), type, d.getComment(),
+ d.isNullable(), null, d.isKey()));
+ }
+ return out;
+ }
+
+ // -------- partition --------
+
+ private static ConnectorPartitionSpec convertPartition(
+ PartitionTableInfo info) {
+ if (info == null) {
+ return null;
+ }
+ String pType = info.getPartitionType();
+ List<Expression> exprs = info.getPartitionList();
+ boolean isList = PartitionType.LIST.name().equalsIgnoreCase(pType);
+ boolean isRange = PartitionType.RANGE.name().equalsIgnoreCase(pType);
+ boolean hasExprs = exprs != null && !exprs.isEmpty();
+ if (!isList && !isRange && !hasExprs) {
+ return null;
+ }
+
+ ConnectorPartitionSpec.Style style;
+ if (isList) {
+ style = ConnectorPartitionSpec.Style.LIST;
+ } else if (isRange) {
+ style = ConnectorPartitionSpec.Style.RANGE;
+ } else if (hasAnyTransform(exprs)) {
+ style = ConnectorPartitionSpec.Style.TRANSFORM;
+ } else {
+ style = ConnectorPartitionSpec.Style.IDENTITY;
+ }
+
+ List<ConnectorPartitionField> fields = hasExprs
+ ? convertFields(exprs)
+ : Collections.emptyList();
+ // LIST/RANGE PartitionDefinition values are not lowered here: each
+ // PartitionDefinition is a sealed family
(InPartition/LessThanPartition/
+ // FixedRangePartition/StepPartition) carrying nereids Expressions that
+ // require full analysis to flatten into List<List<String>>. Connectors
+ // that need the initial values today read the Doris PartitionDesc
+ // directly; this converter passes an empty list and leaves richer
+ // lowering for a follow-up.
+ return new ConnectorPartitionSpec(style, fields,
Collections.emptyList());
+ }
+
+ private static boolean hasAnyTransform(List<Expression> exprs) {
+ for (Expression e : exprs) {
+ if (e instanceof UnboundFunction) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ private static List<ConnectorPartitionField> convertFields(
+ List<Expression> exprs) {
+ List<ConnectorPartitionField> out = new ArrayList<>(exprs.size());
+ for (Expression e : exprs) {
+ if (e instanceof UnboundSlot) {
+ out.add(new ConnectorPartitionField(
+ ((UnboundSlot) e).getName(), "identity",
+ Collections.emptyList()));
+ } else if (e instanceof UnboundFunction) {
+ out.add(convertTransformField((UnboundFunction) e));
+ }
+ // Unknown expression shapes are dropped; the connector can still
+ // honor the spec via its own analysis if richer info is required.
+ }
+ return out;
+ }
+
+ private static ConnectorPartitionField convertTransformField(
+ UnboundFunction fn) {
+ String transform = fn.getName().toLowerCase();
+ String columnName = null;
+ List<Integer> args = new ArrayList<>();
+ for (Expression child : fn.children()) {
+ if (child instanceof UnboundSlot && columnName == null) {
+ columnName = ((UnboundSlot) child).getName();
+ } else if (child instanceof IntegerLikeLiteral) {
+ args.add(((IntegerLikeLiteral) child).getIntValue());
+ } else if (child instanceof Literal) {
+ Object v = ((Literal) child).getValue();
+ if (v instanceof Number) {
+ args.add(((Number) v).intValue());
+ }
+ }
+ }
+ if (columnName == null) {
+ columnName = fn.toString();
+ }
+ return new ConnectorPartitionField(columnName, transform, args);
+ }
+
+ // -------- bucket --------
+
+ private static ConnectorBucketSpec convertBucket(DistributionDescriptor d)
{
+ if (d == null) {
+ return null;
+ }
+ List<String> cols = d.getCols() == null
+ ? Collections.emptyList()
+ : d.getCols();
+ // bucketNum is private; read it off the translated catalog desc so we
+ // do not depend on private internals.
+ int numBuckets = readBucketNum(d);
+ String algorithm = d.isHash() ? "doris_default" : "doris_random";
+ return new ConnectorBucketSpec(cols, numBuckets, algorithm);
+ }
+
+ private static int readBucketNum(DistributionDescriptor d) {
+ try {
+ return d.translateToCatalogStyle().getBuckets();
+ } catch (Exception ignored) {
Review Comment:
`readBucketNum` swallows all exceptions and silently returns 0 buckets. If
`translateToCatalogStyle()` throws (e.g. a distribution form it cannot
translate), the connector receives `numBuckets=0` and creates a table with a
distribution the user never asked for — the CREATE TABLE should fail instead.
This also conflicts with the repo AGENTS.md coding standard ("upon discovering
errors or unexpected situations, report errors or crash—never allow the process
to continue"). Still present at tip.
##########
fe/fe-core/src/main/java/org/apache/doris/connector/ExternalMetaCacheInvalidator.java:
##########
@@ -0,0 +1,82 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.connector;
+
+import org.apache.doris.catalog.Env;
+import org.apache.doris.connector.spi.ConnectorMetaInvalidator;
+import org.apache.doris.datasource.ExternalMetaCacheMgr;
+
+import java.util.List;
+import java.util.Objects;
+
+/**
+ * fe-core side bridge from the connector SPI {@link ConnectorMetaInvalidator}
to the
+ * engine's {@link ExternalMetaCacheMgr}. Returned by
+ * {@link DefaultConnectorContext#getMetaInvalidator()} so connectors that
receive
+ * external change notifications (e.g. HMS notification events) can drop the
right
+ * cache entries without depending on fe-core internals directly.
+ */
+public final class ExternalMetaCacheInvalidator implements
ConnectorMetaInvalidator {
+
+ private final long catalogId;
+
+ public ExternalMetaCacheInvalidator(long catalogId) {
+ this.catalogId = catalogId;
+ }
+
+ @Override
+ public void invalidateAll() {
+ mgr().invalidateCatalog(catalogId);
+ }
+
+ @Override
+ public void invalidateDatabase(String dbName) {
+ mgr().invalidateDb(catalogId, Objects.requireNonNull(dbName,
"dbName"));
+ }
+
+ @Override
+ public void invalidateTable(String dbName, String tableName) {
+ mgr().invalidateTable(catalogId,
+ Objects.requireNonNull(dbName, "dbName"),
+ Objects.requireNonNull(tableName, "tableName"));
+ }
+
+ @Override
+ public void invalidatePartition(String dbName, String tableName,
List<String> partitionValues) {
+ // The SPI carries partition column VALUES (e.g. ["2024", "01"]) but
the engine's
+ // partition cache is keyed by partition NAMES (e.g.
"year=2024/month=01").
+ // Reconstructing the name requires partition column names which are
not carried by
+ // the SPI today. Until the SPI grows that metadata, fall back to
table-level
+ // invalidation — correct but over-broad.
+ mgr().invalidateTable(catalogId,
+ Objects.requireNonNull(dbName, "dbName"),
+ Objects.requireNonNull(tableName, "tableName"));
+ }
+
+ @Override
+ public void invalidateStatistics(String dbName, String tableName) {
Review Comment:
`invalidateStatistics` is a silent no-op (and `invalidatePartition` falls
back to whole-table invalidation). The caveat lives only in this fe-core impl
comment — invisible to a plugin author compiling against fe-connector-spi,
whose javadoc promises "Invalidates cached statistics for one table".
Until a stats-only invalidation entry point exists, suggest documenting the
limitation on the SPI interface itself (and logging here), so the
HMS-notification use case this SPI was designed for does not silently keep
planning on stale stats. Behavior is pinned by ExternalMetaCacheInvalidatorTest
at tip.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
