mbutrovich commented on PR #21352: URL: https://github.com/apache/datafusion/pull/21352#issuecomment-4226299367
I'm going to review this further, but one quick suggestion for a regression test: DuckDB recently shipped a bug ([duckdb/duckdb#21609](https://github.com/duckdb/duckdb/issues/21609)) where a `WHERE l.k IS NOT NULL` filter failed to eliminate NULL rows produced by LEFT JOIN LATERAL. It was fixed in 1.5.1, but it shows this is an easy edge case to get wrong. None of the current tests exercise interaction between a post-join WHERE filter and left-preserved NULL rows. Something like: ```sql -- Outer WHERE filter must eliminate NULL-producing LEFT lateral rows SELECT r.k AS rk, l.k AS lk FROM src_r AS r LEFT JOIN LATERAL ( SELECT l.k FROM src_l AS l WHERE l.k > r.k ) AS l ON TRUE WHERE l.k IS NOT NULL; ``` where `src_r` and `src_l` both contain `'2001-01-01'` -- the lateral produces no match, LEFT JOIN fills NULLs, and the WHERE should return 0 rows. I verified this returns the correct result (0 rows) on this branch. What I see so far looks great, but I'll keep digging. Thanks @neilconway! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
