avantgardnerio opened a new pull request, #2262: URL: https://github.com/apache/datafusion-ballista/pull/2262
## Summary **Purely additive.** Adds `RangeFilterExec` alongside `PerPartitionFilterExec`. No in-tree callers, no mutations. Follow-up PRs wire it into the parallel-window path. `RangeFilterExec` is the sorted-input specialisation of `PerPartitionFilterExec`: - Applies a half-open `[lo, hi)` predicate per input partition, widened by a lower/upper halo (for `RANGE`-frame boundary context in bounded window ops). - Fast path when the input advertises ordering on `routing_expr` ascending: batch min/max bound the range, so batches entirely inside/outside the window skip `filter_record_batch` — `RecordBatch::slice` (zero-copy) or `Arc`-clone pass-through. - Falls back to `filter_record_batch` when the input isn't sorted or has nulls in the routing column. ## Design notes - Bounds are late-bound via `resolve_bounds()`. Wire encode refuses before resolution — over-the-wire plans always carry resolved bounds. Two constructors: `try_new_pending` (rule path — bounds resolved later by the scheduler) and `try_new_resolved` (wire decode + task-restriction path). - **Primitive-generic APIs**: halos and bounds use `ScalarValue` at the API + proto surface so future KLL migration to non-Float64 numeric primitives doesn't break callers. Internal impl is Float64-only today; errors loudly on other numeric types. - Halos are functional on RFE (widen `raw_bounds → widened_bounds` once at resolve time, cached). Not decoration. - Scope: RFE is a pure per-partition filter. Range-partitioning concerns (cuts → per-partition ranges, task-slice mapping) belong to the scheduler in follow-up work; not in this PR. ## Test plan - [x] `cargo test -p ballista-core --lib -- range_filter` — 22 tests (fast/slow paths, halo widening, sorted-key detection, null handling, pending/resolved constructors, resolve validation) - [x] Serde roundtrip + pending-refuses-to-serialize tests - [x] `cargo test --workspace` — 1200 pass / 0 fail / 8 ignored - [x] `cargo clippy --all-targets` clean - [x] `cargo fmt --all --check` clean - [x] `RUSTDOCFLAGS='-D warnings' cargo doc --workspace --no-deps` clean 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
