github-actions[bot] commented on code in PR #66717:
URL: https://github.com/apache/doris/pull/66717#discussion_r3825845032
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergSysExternalTable.java:
##########
@@ -100,24 +99,43 @@ public boolean supportsSnapshotSelection() {
}
public Table getSysIcebergTable() {
- if (sysIcebergTable == null) {
- synchronized (this) {
- if (sysIcebergTable == null) {
- Table baseTable = sourceTable.getIcebergTable();
- MetadataTableType tableType =
MetadataTableType.from(sysTableType);
- if (tableType == null) {
- throw new IllegalArgumentException("Unknown iceberg
system table type: " + sysTableType);
- }
- sysIcebergTable =
MetadataTableUtils.createMetadataTableInstance(baseTable, tableType);
- }
+ MetadataTableType tableType = MetadataTableType.from(sysTableType);
+ if (tableType == null) {
+ throw new IllegalArgumentException("Unknown iceberg system table
type: " + sysTableType);
+ }
+ // Metadata tables capture their base operations. Keep them
statement-local so exact
+ // previousFiles/history state and stale-generation retry never leak
into this table object.
+ return
MetadataTableUtils.createMetadataTableInstance(resolveBaseTable(), tableType);
+ }
+
+ /**
+ * The base generation this statement binds the metadata table to.
Snapshot-selectable
+ * metadata tables derive both their scan and their schema from the source
relation's frozen
+ * snapshot (the same generation IcebergScanNode scans), so analysis and
execution cannot see
+ * different partition specs or schemas when the table entry refreshes
mid-statement. Static
+ * metadata tables and statements without a bound snapshot read the latest
generation.
+ *
+ * <p>The statement snapshot is looked up by source table (like the scan
node's fallback);
+ * a statement that time-travels the same table under several relations
resolves the default
+ * or, if ambiguous, the latest generation for the schema.
+ */
+ @VisibleForTesting
+ Table resolveBaseTable() {
+ if (supportsSnapshotSelection()) {
Review Comment:
[P1] Bind ALL_* schema and scan to one base generation
`ALL_DATA_FILES`, `ALL_DELETE_FILES`, `ALL_FILES`, and `ALL_ENTRIES`
correctly ignore a selected snapshot, but this predicate also makes
`resolveBaseTable()` and `IcebergScanNode` skip the relation's statement-local
base generation. `getFullSchema()` and the later `IcebergApiSource` call
therefore create metadata tables from separate latest generations; Iceberg
derives these file/entry schemas from the current source schema and unified
partition type, so a concurrent schema/spec refresh can pair T0 analyzed slots
with a T1 scan table. Please separate snapshot-selection semantics from
base-generation binding, reuse one latest generation for both phases, and add
an ALL_FILES or ALL_ENTRIES evolution regression.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
