siddubakka commented on code in PR #26723:
URL: https://github.com/apache/datafusion/pull/26723#discussion_r4385350432
##########
datafusion/optimizer/src/scalar_subquery_to_join.rs:
##########
@@ -399,6 +399,33 @@ fn build_join(
// introduced is the untyped `Null` of the HAVING arm. The expressions
// themselves come from a plan `TypeCoercion` has already coerced.
let mut expr_rewrite = TypeCoercionRewriter::new(new_plan.schema());
+ // Evaluate the HAVING-like conditions (recorded during decorrelation
+ // from filters sitting above the aggregate) against the empty-input
+ // aggregate defaults: `count(*)` becomes `0`, and so on.
+ let having_on_default =
+ match conjunction(pull_up.having_like_join_filters.clone()) {
+ Some(combined) => {
+ let requalified = replace_qualified_name(
+ combined,
+ &all_correlated_cols,
+ subquery_alias,
+ )?;
+ let substituted = requalified
+ .transform_up(|e| {
+ if let Expr::Column(Column { name, .. }) = &e
+ && let Some(default_val) = expr_map.get(name)
Review Comment:
Ah, good catch - I only had the aggregate output in mind there. Switched the
match to require `relation: Some(relation)` with `relation.table() ==
subquery_alias` like you suggested, so an outer `x.n` is left alone. Added your
query to `subquery.slt` - (5, 9) gives 0 now, (0, 9) stays NULL.
##########
datafusion/optimizer/src/scalar_subquery_to_join.rs:
##########
@@ -417,7 +444,17 @@ fn build_join(
// outer table has a column with the same name as the aggregate.
let value_col = Column::new(Some(subquery_alias), name);
- let mut builder = when(Expr::Column(indicator_col).is_null(),
result);
+ // On an unmatched LEFT JOIN row, a HAVING-like condition means
+ // the subquery produced no row. Return the empty-input default
+ // only when that condition passes on the defaults; otherwise
+ // return NULL.
+ let unmatched_value = match &having_on_default {
Review Comment:
Done - moved them out of the ON and into the compensation CASE the way you
sketched. The unmatched arm checks the condition against the defaults, and a
new `WHEN ... IS NOT TRUE THEN NULL` arm handles a matched group it rejects, so
unmatched now only means "no group". For your caveat I gated the move on the
defaults: if any empty-input default is NULL there is no CASE to check the
condition in, so it keeps filtering in the ON exactly as before. Your `<` query
gives NULL, 1, 0 now and is in `subquery.slt` too.
##########
datafusion/optimizer/src/scalar_subquery_to_join.rs:
##########
@@ -399,6 +399,33 @@ fn build_join(
// introduced is the untyped `Null` of the HAVING arm. The expressions
// themselves come from a plan `TypeCoercion` has already coerced.
let mut expr_rewrite = TypeCoercionRewriter::new(new_plan.schema());
+ // Evaluate the HAVING-like conditions (recorded during decorrelation
+ // from filters sitting above the aggregate) against the empty-input
+ // aggregate defaults: `count(*)` becomes `0`, and so on.
+ let having_on_default =
Review Comment:
Rebased - `all_correlated_cols` was gone by the time I got to it. Both the
default arm and the new actual-output arm now go through `build_join_filter`
with `correlated_subquery_cols_map`, and the default substitution runs on top
of the requalified expression, with the alias-qualification check from your
other comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
