Gabriel39 commented on PR #66529: URL: https://github.com/apache/doris/pull/66529#issuecomment-5212411502
@924060929 Thanks for the detailed review. I addressed every actionable point in commit 5662a6f513: 1. **Repeated delete-manifest walks (P1):** added a compact, capacity-bounded cache of equality-delete field IDs keyed by table location and immutable snapshot ID. It is used even when the optional full manifest cache is disabled. Load failures are not cached, catalog refresh clears the projection, and the first load still fails closed on any unreadable delete manifest. I deliberately did not early-exit after seeing the current IDs or ignore an unreadable manifest, because a later/failed manifest can contain a dropped equality key and either shortcut could introduce a correctness false negative. 2. **Negative/unstamped IDs (P2):** an unstamped root column now fails closed instead of being treated as an exact top-level source with an empty path. 3. **Nested source lookup (P2):** production resolution now looks up the Nereids `ExprId` using the top-level source column stable ID, while using the nested Iceberg source ID only to derive `sourceFieldPath`. The id-only test seam retains its existing top-level behavior. 4. **Requiredness ancestry trade-off:** the conservative fence is intentional and is now documented in the PR description. Snapshot schema IDs are optional, so proving ancestry requires an O(snapshot-count) walk. Once schema history exposes a projected requiredness hazard, fencing every non-empty selected snapshot can reduce rolling-upgrade availability, but it cannot create a correctness false negative. Added coverage for same-snapshot reuse, snapshot rollover, retry after cache-load failure, catalog invalidation, provider integration with full manifest caching disabled, production nested-source resolution, and fail-closed unstamped IDs. The combined focused suites passed 158 tests, and FE Checkstyle passed with zero violations. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
