yadavay-amzn opened a new pull request, #56077:
URL: https://github.com/apache/spark/pull/56077
### What changes were proposed in this pull request?
Guard `joinCond.get` with `isDefined` check in `rewriteDomainJoinsIfPresent`.
### Why are the changes needed?
When constant folding eliminates all correlated predicates in a subquery
(e.g., `WHERE FALSE AND correlated_pred`), `rewriteExistentialExpr` returns
`None` for `joinCond`. The subsequent call to `rewriteDomainJoinsIfPresent`
then crashes with `NoSuchElementException: None.get`.
Repro:
```sql
SELECT t0.value FROM VALUES (CAST(0 AS TINYINT)) AS t0(value)
WHERE NOT EXISTS (
SELECT t1.value FROM VALUES (CAST(0 AS TINYINT)) AS t1(value)
WHERE FALSE AND (t0.value = t1.value)
)
```
### Does this PR introduce _any_ user-facing change?
Yes -- queries with EXISTS/NOT EXISTS subqueries where all correlated
predicates are eliminated by constant folding no longer crash.
### How was this patch tested?
Added test in `SubquerySuite` covering both `NOT EXISTS` (returns all rows)
and `EXISTS` (returns no rows) with `WHERE FALSE AND (correlated_pred)`. Test
verifies correct query semantics, not just absence of crash.
- Without fix: `NoSuchElementException: None.get` at subquery.scala:76
- With fix: correct results returned
- Full SubquerySuite (142 tests) passes with no regressions.
### Was this patch authored or co-authored using generative AI tooling?
Yes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
