github-actions[bot] commented on code in PR #66530:
URL: https://github.com/apache/doris/pull/66530#discussion_r4004686665
##########
fe/fe-connector/fe-connector-hudi/src/main/java/org/apache/doris/connector/hudi/HudiScanPlanProvider.java:
##########
@@ -159,6 +162,24 @@ public boolean supportsFileCache() {
@Override
public List<ConnectorScanRange> planScan(ConnectorSession session,
ConnectorScanRequest request) {
+ // Statement-scoped reuse: within one statement the identical scan
(same table, same
+ // instant/incremental pin, same partition set) plans once and every
duplicated relation
+ // shares the result. The scope is NONE for offline planning and
tests, in which case the
+ // loader runs on every call. Session variables are constant within a
statement and
+ // deliberately absent from the key.
+ if (session == null || !session.isExternalScanTaskReuseEnabled()) {
+ return doPlanScan(session, request);
+ }
+ String memoKey = SCAN_REUSE_NAMESPACE + ":" + session.getCatalogId() +
":" + session.getQueryId();
+ Map<HudiScanReuseKey, List<ConnectorScanRange>> scanReuse =
session.getStatementScope().computeIfAbsent(
Review Comment:
[P1] Fence Hudi reuse to the physical table generation. For a plain read,
`applySnapshot` leaves `queryInstant` null, so two aliases at the same
db/table/basePath still have an equal key if the Hudi table is recreated
between them. Alias A can memoize native ranges stamped with generation-A
per-file `schema_id` values, while each node independently rebuilds
`history_schema_info` from a fresh meta client; alias B can therefore attach
generation B's dictionary to A's ranges. If the recreated table reuses an id
(non-schema-on-read tables use id 0), BE can map an A file through B's field
identities and return wrong/null columns. If B lacks A's id, FileScannerV2
falls back to `BY_NAME` and loses rename/reorder correctness, while V1 errors
with `miss table/file schema info`. Reuse-disabled planning would pair B's
ranges and dictionary. Please resolve one statement-scoped Hudi generation for
both paths (or add a stable generation token) and cover two equal logical
handles backed by diffe
rent same-path generations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
