github-actions[bot] commented on code in PR #66297:
URL: https://github.com/apache/doris/pull/66297#discussion_r3694727320
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ExecuteCommand.java:
##########
@@ -72,6 +72,7 @@ public void run(ConnectContext ctx, StmtExecutor executor)
throws Exception {
StatementContext statementContext = ctx.getStatementContext();
statementContext.setPrepareStage(false);
statementContext.setIsInsert(false);
+ statementContext.resetMvccSnapshots();
Review Comment:
[P1] Reset relation-local scan state for every prepared command that retains
a query, not only insert/overwrite/update. DeleteFromUsingCommand is classified
by the later blanket Command branch, so its retained Paimon OPTIONS source
keeps scan.snapshot-id=S after the first EXECUTE. This line clears the
statement fence; after latest advances, the second EXECUTE tries to pin S+1 on
that same TableScanParams and throws Conflicting resolved table scan
parameters. MergeIntoCommand retains a source/CTE with the same omission.
Traverse these command roots before replanning and test two executions
separated by a Paimon commit.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/StatementContext.java:
##########
@@ -943,20 +941,65 @@ public Optional<MvccSnapshot> loadSnapshots(TableIf
specificTable, Optional<Tabl
if (!(specificTable instanceof MvccTable)) {
return Optional.empty();
}
- MvccTableInfo mvccTableInfo = new MvccTableInfo(specificTable);
+ MvccTableInfo mvccTableInfo = new MvccTableInfo(specificTable,
+ versionKeyOf(tableSnapshot, scanParams));
MvccSnapshot snapshot;
- if (tableSnapshot.isPresent() || scanParams.isPresent()) {
+ if (scanParams != null && scanParams.isPresent() &&
scanParams.get().isOptions()) {
+ // OPTIONS defines a relation-scoped projection, so aliases with
the same selector
+ // reuse one handle while different selectors never overwrite each
other.
+ snapshot = snapshots.get(mvccTableInfo);
+ if (snapshot == null) {
+ MvccTable mvccTable = (MvccTable) specificTable;
+ if (mvccTable.requiresLatestSnapshotFence(tableSnapshot,
scanParams)) {
+ MvccTableInfo latestKey = new MvccTableInfo(specificTable);
+ MvccSnapshot latestFence =
latestSnapshotFences.computeIfAbsent(latestKey,
+ key -> latestSnapshots.containsKey(key)
+ ? latestSnapshots.get(key) :
mvccTable.loadLatestSnapshotFence());
+ // Planning options still need separate projections, but
their version selector
+ // must come from one statement fence rather than separate
live latest reads.
+ snapshot = mvccTable.loadSnapshot(tableSnapshot,
scanParams, Optional.of(latestFence));
+ } else {
+ snapshot = mvccTable.loadSnapshot(tableSnapshot,
scanParams);
+ }
+ snapshots.put(mvccTableInfo, snapshot);
+ scanParams.flatMap(TableScanParams::getResolvedMapParams)
+ .ifPresent(params ->
resolvedSnapshotScanParams.put(mvccTableInfo, params));
+ } else if (resolvedSnapshotScanParams.containsKey(mvccTableInfo)) {
+ // Snapshot de-duplication also de-duplicates dynamic option
resolution. Seed later
+ // aliases so their scan phase consumes the selector used by
the cached snapshot.
+
scanParams.get().reuseResolvedMapParams(resolvedSnapshotScanParams.get(mvccTableInfo));
+ }
+ } else if (tableSnapshot.isPresent() || scanParams.isPresent()) {
snapshot = ((MvccTable) specificTable).loadSnapshot(tableSnapshot,
scanParams);
} else {
// Keep latest metadata separate: a historical relation may
temporarily become the
// table-scoped snapshot, but it must not redefine what a later
latest relation sees.
snapshot = latestSnapshots.computeIfAbsent(mvccTableInfo,
- key -> ((MvccTable)
specificTable).loadSnapshot(tableSnapshot, scanParams));
+ key -> latestSnapshotFences.containsKey(key)
+ ? ((MvccTable)
specificTable).loadSnapshot(tableSnapshot, scanParams,
+ Optional.of(latestSnapshotFences.get(key)))
+ : ((MvccTable)
specificTable).loadSnapshot(tableSnapshot, scanParams));
+ // A full latest projection is also a valid version fence.
Recording it makes
+ // plain-first and options-first aliases pin the same statement
version.
+ latestSnapshotFences.putIfAbsent(mvccTableInfo, snapshot);
}
snapshots.put(mvccTableInfo, snapshot);
+ tableMetadataSnapshots.putIfAbsent(new MvccTableInfo(specificTable),
snapshot);
return Optional.of(snapshot);
}
+ /**
+ * Clear MVCC state retained by a prepared statement between executions. A
snapshot fence belongs
+ * to one EXECUTE only; carrying it forward would make a later commit
permanently invisible.
+ */
+ public void resetMvccSnapshots() {
Review Comment:
[P1] Reset the preload completion gate together with these snapshot maps.
PREPARE's analyzed-plan pass already runs collect/preload on this same
StatementContext; the first EXECUTE then discards that Paimon snapshot here but
leaves externalMetadataPreloadResult present. When collection re-enters,
PreloadExternalMetadata treats the stale result as completion, so binding
reloads latest metadata only after internal table locks and every execution
reuses the old timing. Clear the execution-scoped result here (while retaining
the collected candidates), and cover first/repeated EXECUTE with preload
enabled and an internal lock table.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/BindRelation.java:
##########
@@ -427,9 +427,16 @@ private Optional<LogicalPlan> handleMetaTable(TableIf
table, UnboundRelation unb
validatePaimonSystemTableScanParams(
(PaimonSysExternalTable) sysExternalTable,
unboundRelation.getScanParams());
}
- TableIf snapshotTable = sysExternalTable instanceof
IcebergSysExternalTable
- ? ((IcebergSysExternalTable)
sysExternalTable).getSourceTable()
- : sysExternalTable;
+ TableIf snapshotTable;
+ if (sysExternalTable instanceof IcebergSysExternalTable) {
+ snapshotTable = ((IcebergSysExternalTable)
sysExternalTable).getSourceTable();
+ } else if (sysExternalTable instanceof PaimonSysExternalTable) {
+ // Paimon system OPTIONS resolve against the data table too;
using the synthetic
+ // wrapper here bypasses the statement's common latest fence
and reopens live latest.
+ snapshotTable = ((PaimonSysExternalTable)
sysExternalTable).getSourceTable();
Review Comment:
[P1] Do not apply the source table's empty-data sentinel directly to every
native Paimon system scan. With this new branch, an empty data table gives the
LogicalFileScan a PaimonMvccSnapshot with INVALID_SNAPSHOT_ID; PaimonScanNode
then returns no splits before it loads the system wrapper. That silently
empties metadata tables such as $schemas and $options even though they have
rows before the first data commit. Make the system path consume an appropriate
fenced source without taking the ordinary data-scan early return, and cover an
empty table's schema/options system scans.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
