jayzhan211 commented on code in PR #26677:
URL: https://github.com/apache/datafusion/pull/26677#discussion_r4366208642
##########
datafusion/optimizer/src/decorrelate_predicate_subquery.rs:
##########
@@ -854,6 +854,35 @@ fn build_join(
if !pull_up.can_pull_up {
return Ok(None);
}
+ // Stopgap until a follow-up adds count-bug compensation for `IN`/`NOT IN`.
+ // The subquery stays correlated unless an outer row with no match gets the
+ // right answer anyway.
+ if !exists_sub_query {
Review Comment:
nit: the extra `rewrite(true)` only finds something when a groupless
aggregate was pulled up, and the first rewrite already records that. Gating on
it skips the second pass for every other `IN` subquery. Optimizer tests and
`subquery.slt` still pass with this:
```diff
- if !exists_sub_query {
+ if !exists_sub_query && pull_up.pulled_up_scalar_agg {
```
##########
datafusion/sqllogictest/test_files/subquery.slt:
##########
@@ -2124,14 +2124,35 @@ WHERE t1_int + 2 IN (SELECT max(t2_int) FROM t2 WHERE
t2.t2_id = t1.t1_id)
----
11
Review Comment:
nit: the `LeftMark` branch that is allowed through (an `IN` under `OR` in a
`WHERE`) has no test. Something like this would pin it down:
```sql
SELECT t1_id FROM t1
WHERE t1_int + 2 IN (SELECT max(t2_int) FROM t2 WHERE t2.t2_id = t1.t1_id)
OR t1_id = 33
ORDER BY t1_id;
-- 11
-- 33
```
##########
datafusion/optimizer/src/decorrelate_predicate_subquery.rs:
##########
@@ -854,6 +854,35 @@ fn build_join(
if !pull_up.can_pull_up {
return Ok(None);
}
+ // Stopgap until a follow-up adds count-bug compensation for `IN`/`NOT IN`.
Review Comment:
nit: could this point at the tracking issue, so whoever adds the `IN`
count-bug compensation knows to remove the block?
```diff
- // Stopgap until a follow-up adds count-bug compensation for `IN`/`NOT
IN`.
+ // TODO(#24960): stopgap until count-bug compensation covers `IN`/`NOT
IN`.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
