924060929 commented on PR #64965: URL: https://github.com/apache/doris/pull/64965#issuecomment-4999655896
Thanks @hutiefang76 for digging into this — the root-cause analysis here (the sorted-range / selected-partition **generation mismatch** under concurrent partition add/drop) is spot on. Cross-linking for whoever triages this: **#65659** now fixes the same bug (#64800) with the *capture-at-t1* approach discussed in this thread. It freezes `sortedPartitionRanges` **together with** the partition map inside `SelectedPartitions` at `initSelectedPartitions` time, and removes the re-read of the cached sorted ranges in `PruneFileScanPartition`. Because the mixed `(G1.map, G2.ranges)` pair can no longer be constructed, it covers **both** directions of the divergence — including the "map has a live partition the snapshot lacks" case that consumer-side filtering here cannot re-add (the concern I raised earlier in this thread). Given that, I'd suggest **closing this PR in favor of #65659**. The one piece worth carrying over is the observability idea — a cheap `partitionCount` precheck plus a `LOG.warn` when a divergence is actually detected — which would keep the underlying cache-coherency issue diagnosable in production even after the freeze fix. Could be folded into #65659 or a small follow-up. Either way, thanks for surfacing and analyzing this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
