github-actions[bot] commented on code in PR #66246:
URL: https://github.com/apache/doris/pull/66246#discussion_r3681896031
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/BindRelation.java:
##########
@@ -425,6 +427,14 @@ private Optional<LogicalPlan> handleMetaTable(TableIf
table, UnboundRelation unb
validatePaimonSystemTableScanParams(
(PaimonSysExternalTable) sysExternalTable,
unboundRelation.getScanParams());
}
+ TableIf snapshotTable = sysExternalTable instanceof
IcebergSysExternalTable
+ ? ((IcebergSysExternalTable)
sysExternalTable).getSourceTable()
+ : sysExternalTable;
+ // A metadata-table scan reads base-table snapshots, so bind its
fence from the base
+ // relation even though the synthetic system-table wrapper is not
itself MVCC-aware.
+ Optional<MvccSnapshot> relationSnapshot =
cascadesContext.getStatementContext().loadSnapshots(
Review Comment:
[P2] Skip snapshot resolution for metadata tables that ignore it
`supportsSnapshotSelection()` deliberately makes static and `ALL_*` metadata
tables ignore a supplied snapshot/ref, and `IcebergScanNode` consequently never
calls `useSnapshot`. This binding path still passes the original qualifier to
`loadSnapshots` on the base table, so an expired numeric snapshot or removed
branch/tag fails in `getQuerySpecSnapshot` before the scan reaches that ignore
logic, even though it cannot affect the returned metadata rows. For
non-selectable system tables, load only a latest-generation fence with empty
snapshot/scan parameters, and cover an expired/ref qualifier through binding.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalFileScan.java:
##########
@@ -91,24 +107,78 @@ protected LogicalFileScan(RelationId id, ExternalTable
table, List<String> quali
Optional<TableSnapshot> tableSnapshot, Optional<TableScanParams>
scanParams,
Optional<GroupExpression> groupExpression,
Optional<LogicalProperties> logicalProperties,
Optional<List<Slot>> cachedSlots) {
+ this(id, table, qualifier, selectedPartitions, operativeSlots,
virtualColumns, tableSample, tableSnapshot,
+ scanParams, groupExpression, logicalProperties, cachedSlots,
Optional.empty());
+ }
+
+ /**
+ * Constructor for LogicalFileScan.
+ */
+ protected LogicalFileScan(RelationId id, ExternalTable table, List<String>
qualifier,
+ SelectedPartitions selectedPartitions, Collection<Slot>
operativeSlots,
+ List<NamedExpression> virtualColumns, Optional<TableSample>
tableSample,
+ Optional<TableSnapshot> tableSnapshot, Optional<TableScanParams>
scanParams,
+ Optional<GroupExpression> groupExpression,
Optional<LogicalProperties> logicalProperties,
+ Optional<List<Slot>> cachedSlots, Optional<List<Column>>
relationSchema) {
+ this(id, table, qualifier, selectedPartitions, operativeSlots,
virtualColumns, tableSample, tableSnapshot,
+ scanParams, groupExpression, logicalProperties, cachedSlots,
relationSchema,
+ MvccUtil.getSnapshotFromContext(table));
+ }
+
+ protected LogicalFileScan(RelationId id, ExternalTable table, List<String>
qualifier,
+ SelectedPartitions selectedPartitions, Collection<Slot>
operativeSlots,
+ List<NamedExpression> virtualColumns, Optional<TableSample>
tableSample,
+ Optional<TableSnapshot> tableSnapshot, Optional<TableScanParams>
scanParams,
+ Optional<GroupExpression> groupExpression,
Optional<LogicalProperties> logicalProperties,
+ Optional<List<Slot>> cachedSlots, Optional<List<Column>>
relationSchema,
+ Optional<MvccSnapshot> relationSnapshot) {
super(id, PlanType.LOGICAL_FILE_SCAN, table, qualifier,
operativeSlots, virtualColumns,
groupExpression, logicalProperties);
this.selectedPartitions = selectedPartitions;
this.tableSample = tableSample;
this.tableSnapshot = tableSnapshot;
this.scanParams = scanParams;
this.cachedOutputs = cachedSlots;
+ this.relationSchema = relationSchema;
+ this.relationSnapshot = relationSnapshot;
}
private static SelectedPartitions initialSelectedPartitions(
- ExternalTable table, Optional<TableScanParams> scanParams) {
+ ExternalTable table, Optional<TableScanParams> scanParams,
+ Optional<MvccSnapshot> relationSnapshot) {
if ((table instanceof PaimonExternalTable || table instanceof
PaimonSysExternalTable)
&& scanParams.isPresent() && scanParams.get().isOptions()) {
// A relation-scoped historical snapshot cannot reuse partitions
cached for the
// statement-level latest snapshot; Paimon will prune its selected
snapshot instead.
return SelectedPartitions.NOT_PRUNED;
}
- return
table.initSelectedPartitions(MvccUtil.getSnapshotFromContext(table));
+ return table.initSelectedPartitions(relationSnapshot);
+ }
+
+ private static Optional<List<Column>> captureRelationSchema(
+ ExternalTable table, Optional<TableScanParams> scanParams,
+ Optional<MvccSnapshot> relationSnapshot) {
+ if (scanParams.isPresent() && scanParams.get().isOptions()) {
+ if (table instanceof PaimonExternalTable) {
+ return Optional.of(ImmutableList.copyOf(
+ ((PaimonExternalTable)
table).getFullSchema(scanParams.get())));
+ }
+ if (table instanceof PaimonSysExternalTable) {
+ return Optional.of(ImmutableList.copyOf(
+ ((PaimonSysExternalTable)
table).getFullSchema(scanParams.get())));
+ }
+ }
+ return captureRelationSchema(table, relationSnapshot);
+ }
+
+ protected static Optional<List<Column>> captureRelationSchema(
+ ExternalTable table, Optional<MvccSnapshot> relationSnapshot) {
+ if (!(table instanceof MvccTable)) {
Review Comment:
[P2] Pin the metadata-table output schema with its frozen generation
The snapshot attached in `BindRelation` belongs to the Iceberg base table,
but the synthetic `IcebergSysExternalTable` is not an `MvccTable`, so this
returns no captured schema. `computeOutput` later initializes the wrapper
schema from `sourceTable.getIcebergTable()`, while
`IcebergScanNode.useFrozenTableGeneration()` reconstructs snapshot-selectable
tables such as `$files` and `$entries` from the retained base generation. A
concurrent schema/spec refresh between those steps can therefore bind
generation-B partition/metrics fields or types while the scan still exposes
generation A. Capture the system-table schema from the retained base
generation, ensure scan equivalence distinguishes retained generations, and
cover a refresh-between-bind-and-scan case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
