924060929 commented on code in PR #67904:
URL: https://github.com/apache/doris/pull/67904#discussion_r4001906642
##########
fe/fe-connector/fe-connector-paimon/src/main/java/org/apache/doris/connector/paimon/PaimonConnectorMetadata.java:
##########
@@ -577,7 +581,38 @@ public Optional<ConnectorMvccSnapshot> beginQuerySnapshot(
Identifier identifier =
Identifier.create(paimonHandle.getDatabaseName(), paimonHandle.getTableName());
long id = latestSnapshotCache.getOrLoad(identifier,
() ->
catalogOps.latestSnapshotId(resolveTable(paimonHandle)).orElse(-1L));
- return
Optional.of(ConnectorMvccSnapshot.builder().snapshotId(id).build());
+ return Optional.of(ConnectorMvccSnapshot.builder().snapshotId(id)
+ .schemaId(statementSchemaId(paimonHandle,
resolveTable(paimonHandle))).build());
+ }
+
+ private <T> T readSchemaAuthenticated(Supplier<T> read) {
+ // Cached tables do not cache schema files: exact/latest schema reads
still need plugin UGI and TCCL.
+ try {
+ return context.executeAuthenticated(read::get);
+ } catch (RuntimeException e) {
+ throw e;
+ } catch (Exception e) {
+ throw new RuntimeException("Failed to read Paimon schema", e);
+ }
+ }
+
+ private long statementSchemaId(PaimonTableHandle handle, Table table) {
+ return statementSchemas.computeIfAbsent(handle,
+ ignored -> readSchemaAuthenticated(() ->
catalogOps.latestSchema(table)))
+ .map(PaimonCatalogOps.PaimonSchemaSnapshot::schemaId)
+ .orElse(-1L);
+ }
+
+ private PaimonCatalogOps.PaimonSchemaSnapshot
schemaForPin(PaimonTableHandle handle, Table table, long schemaId) {
+ // External recreation can reuse a schema ID. Latest pins must retain
the actual statement
+ // schema instead of consulting the name/ID-keyed historical memo from
an earlier table.
+ Optional<PaimonCatalogOps.PaimonSchemaSnapshot> captured =
+ statementSchemas.getOrDefault(handle, Optional.empty());
+ if (captured.isPresent() && captured.get().schemaId() == schemaId) {
+ return captured.get();
+ }
+ return schemaAtMemo.getOrLoad(handle, schemaId,
Review Comment:
Explicit snapshot, timestamp, and tag reads do not call beginQuerySnapshot,
so statementSchemas is empty when they reach this fallback. After an external
DROP/CREATE of the same name, the replacement table can reuse both snapshotId
and schemaId; schemaAtMemo is still keyed only by db/table/branch/schemaId and
can therefore return the previous table schema while the scan reads the
replacement table snapshot. Matching field positions can silently expose
replacement-table data under the old schema, while incompatible shapes can fail
during column mapping or decoding. Please key the cross-query memo by a
reliable table-generation identity, or re-read schemaAt once per statement for
explicit time travel when no such identity exists. Add a regression that warms
the old memo, recreates the table with reused IDs, and then resolves FOR
VERSION AS OF or tag/time without calling beginQuerySnapshot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
