hemanthboyina opened a new pull request, #59605: URL: https://github.com/apache/spark/pull/59605
**What changes were proposed in this pull request?**
Add left semi and left anti joins to the DSv2 join pushdown API.The JDBC
implementation generates EXISTS or NOT EXISTS queries and returns only
left-side columns. Simple correlated existence subqueries are rewritten early
enough for join pushdown. Complex subqueries retain the existing rewrite path.
Null-aware NOT IN remains outside this change.
**Why are the changes needed?**
Spark currently reads both inputs and executes these joins itself. Pushing
them down lets the database evaluate the existence condition and return only
the qualifying left rows, reducing data transfer to Spark.
**Does this PR introduce any user facing change?**
Yes. Eligible left semi/anti joins and correlated EXISTS/NOT EXISTS queries
can run in the database when DSv2 join pushdown is enabled.
Query results and configuration defaults remain unchanged.
**How was this patch tested?**
Added Catalyst regression tests and shared JDBC tests covering duplicates,
nulls, predicates, projections, and fallback behavior.
**Was this patch authored or co-authored using generative AI tooling?**
Yes, used gpt 6.1 sol
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
