xudong963 opened a new pull request, #26132: URL: https://github.com/apache/datafusion/pull/26132
## Which issue does this PR close? - Closes #26131. ## Rationale for this change An outer range can make every branch of an OR impossible, but the logical optimizer currently considers the direct comparisons and the disjunction independently. For example, `x >= 10 AND ((x >= 1 AND x < 3) OR (x >= 5 AND x < 7))` retains a scan despite never matching a row. This matters for a `UNION ALL` containing historical backfill branches: a request outside their time windows should eliminate those branches before physical planning calls their table providers, including when the query has an outer `ORDER BY ... LIMIT`. ## What changes are included in this PR? - Carry direct column/literal comparisons from enclosing conjunctions into positive AND/OR descendants. Reuse the existing per-column comparison simplifier to prove contradictions and remove impossible disjuncts. - Retain the original assumption predicates. Do not infer through NOT, casts, arbitrary functions, or across sibling OR branches. Bound contextual analysis by depth and work limits without expanding expressions into DNF. - Preserve predicate rewrites even when the number of conjuncts is unchanged, and report their transformation status when pushdown itself is unavailable. Preserve existing predicate order when comparison grouping only permutes the conjuncts. - Convert an impossible filter directly into an `EmptyRelation` before provider pushdown; an `Exact` provider must not absorb FALSE into its scan and still receive a `scan()` call. ## What is the testing strategy for this PR? - `simplify_predicate_disjunctions.slt`: empty intersections, gaps, partial overlap, retained outer assumptions, equality/inequality predicates, NULL, NOT, casts, arithmetic, timestamp precision/time zones, and union branch removal with an outer sort/limit. - `simplify_predicates.slt`: existing regression coverage and an updated plan expectation for independently simplified OR arms. - A public API test with a provider whose `scan()` returns an error verifies that the eliminated union branch is never scanned under Exact, Inexact, or Unsupported pushdown. - Unit tests cover conservative fallback when contextual analysis reaches its work/depth limits. Targeted validation: ```text cargo test --profile ci -p datafusion-optimizer simplify_predicates --lib cargo test --profile ci -p datafusion-optimizer push_down_filter --lib cargo test --profile ci --test sqllogictests -- simplify_predicate_disjunctions.slt simplify_predicates.slt cargo test --profile ci -p datafusion --test core_integration contradictory_disjunction_eliminates_provider_scan cargo fmt --all -- --check cargo clippy --all-targets --all-features -- -D warnings uv run ./dev/rust_lint.sh ``` All checks above passed. The full lint script used an isolated current `cargo-audit` and a temporary workspace boundary for `dev/depcheck` because this checkout is nested inside another Cargo workspace; no tooling or manifest changes are included in the patch. Only the targeted tests above were run, not the full workspace test suite. ## Are there any user-facing changes? Queries with contradictory ranges across AND/OR can avoid planning and scanning impossible inputs. Partial simplification retains all matching rows. No public API or configuration changes. This reasoning is specific to filter truth sets; it does not change nullable Boolean projection results or derive cross-column date/timestamp equivalences. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
