yashaggarwal85d opened a new pull request, #26558: URL: https://github.com/apache/datafusion/pull/26558
## Which issue does this PR close? - Closes #26285. ## Rationale for this change A correlated subquery that compares against an outer column whose type is only decided by type coercion fails at execution with `Can't compare arrays of different types` (and panics with `join key data types differ` in debug builds). For example, an `EXISTS` / `NOT EXISTS` / `IN` / scalar subquery referencing a column of a `UNION ALL` of `INT` and `BIGINT` columns, or of a `CASE` with `INT` and `BIGINT` branches. PostgreSQL returns the expected rows. The subquery is type-coerced before its outer plan, against the types its outer reference columns had when the plan was built. When coercing the outer plan widens such a column (here `Int32` to `Int64`), the subquery's `u.c0 = outer_ref(s.k)` gets no cast, so the decorrelated join compares `Int32` with `Int64`. ## What changes are included in this PR? - `TypeCoercionRewriter::coerce_subquery`: when coercing a subquery expression (`EXISTS`, `IN`, scalar subquery, `ANY`/`ALL`), give its outer reference columns the types of the columns they reference in the already-coerced outer schema. If any changed, coerce the subquery plan again. Subqueries whose outer references are unchanged are coerced exactly as before. - The four subquery arms of the rewriter now share this helper. - `coerce_plan_and_subqueries` takes the session time zone instead of `ConfigOptions` (the only option it used), so the helper can call it. ## What is the testing strategy for this PR? New sqllogictest cases at the end of `subquery.slt` cover the two reproducers from the issue (`CASE` and `UNION ALL`), plus `NOT EXISTS`, correlated `IN` (which also errored before this change) and a scalar subquery. An `EXPLAIN` case shows the cast added to the join. The new cases fail on `main` and pass with this change. Known limitation (not changed by this PR): outer references from a subquery nested two levels deep to the outermost query are not re-typed. Such queries already fail on `main` with `No field named ...` even when all types match, which seems to be a separate decorrelation limitation. ## Are there any user-facing changes? Queries that previously failed now return correct results. No API changes. --- AI-assisted: I used an AI coding assistant while preparing this PR. I have reviewed and understand all changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
