Kevin-Li-2025 commented on PR #23226: URL: https://github.com/apache/datafusion/pull/23226#issuecomment-4916351091
Thanks, agreed — prefix pruning needs to stay conservative here. I pushed `562846c31` to address this. `evaluate_partition_prefix` now stops before a partition value whose percent-encoded spelling differs from the raw literal, rather than listing only the encoded spelling. This means cases like `category = 'John Doe'` no longer prune to only `category=John%20Doe`, so valid raw directories such as `category=John Doe/` can still be discovered and filtered by the normal partition-value comparison. If a previous partition is still safe, we keep that earlier prefix, e.g. `a = 'foo' AND b = 'John Doe'` still prunes to `a=foo` and then stops before `b`. I updated `test_evaluate_partition_prefix` for both first-partition encoded values and the later-partition stop case. Local checks run: - `cargo fmt --all --check` - `cargo test -p datafusion-catalog-listing test_evaluate_partition_prefix` - `cargo test -p datafusion-catalog-listing` - `git diff --check` CI has started on head `562846c31`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
