princejain756 opened a new pull request, #26699: URL: https://github.com/apache/datafusion/pull/26699
## Which issue does this PR close? Closes #26692. ## Rationale for this change `PullUpCorrelatedExpr` tracked per-node pulled-up state (`correlated_subquery_cols_map` and `collected_count_expr_map`) in `HashMap<LogicalPlan, ...>` keyed by the rewritten `LogicalPlan` value. When a subquery contained two subtrees that became structurally equal after `Filter` removal (for example, one branch had its correlated filter stripped by `PullUpCorrelatedExpr::f_up` and the other branch was an identical unfiltered scan or alias), the second branch visited in bottom-up order overwrote the map entry of the first branch with an empty `BTreeSet` (or `None`). Additionally, `decorrelate_lateral_join` read only the top entry of `correlated_subquery_cols_map.values().last()` rather than the union of correlated columns across the subquery. ## What changes are included in this PR? 1. Replace `correlated_subquery_cols_map: HashMap<LogicalPlan, BTreeSet<Column>>` and `collected_count_expr_map: HashMap<LogicalPlan, ExprResultMap>` on `PullUpCorrelatedExpr` with a tree-frame stack (`pulled_up_frames: Vec<PullUpFrame>`) pushed in `f_down` and popped in `f_up`, alongside accumulated `correlated_subquery_cols: BTreeSet<Column>` and root `collected_count_expr_map: Option<ExprResultMap>`. 2. Recompute the schema of pass-through multi-input or unary operators (`Join`, `AsOfJoin`, `Distinct`, `Repartition`) in `f_up` when correlated columns are added to their children. 3. Update `scalar_subquery_to_join`, `decorrelate_predicate_subquery`, and `decorrelate_lateral_join` to read `pull_up.correlated_subquery_cols` and `pull_up.collected_count_expr_map` directly. ## Are these changes tested? Yes, added sqllogictest regression coverage in `subquery.slt` covering `EXISTS`, scalar `count(*)` subqueries, and `CROSS JOIN LATERAL` with duplicate subplans. ## Are there any user-facing changes? Bug fix only; correlated subqueries containing equal subplans now produce the right results instead of dropping the correlation predicate or failing with a schema error. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
