gabotechs opened a new issue, #25614: URL: https://github.com/apache/datafusion/issues/25614
## Describe the bug String equality and pattern predicates all receive the same 20% fallback. LIKE and NOT LIKE also get identical estimates even though they partition this non-null input. ## To Reproduce From the repository root, with the CLI fix from [PR #25570](https://github.com/apache/datafusion/pull/25570) applied: ```sh cargo build --profile ci --locked -p datafusion-benchmarks --bin dfbench cargo install tpchgen-cli --version 1.1.1 --locked # if not already installed repro_dir=$(mktemp -d) tpchgen-cli --scale-factor 1 --format parquet \ --parquet-compression 'ZSTD(1)' --parts 1 --output-dir "$repro_dir/data" cat > "$repro_dir/repro.sql" <<'SQL' SET datafusion.execution.target_partitions = 1; SET datafusion.optimizer.enable_dynamic_filter_pushdown = false; SELECT o_orderkey FROM orders WHERE o_comment LIKE '%special%requests%'; SELECT o_orderkey FROM orders WHERE o_comment NOT LIKE '%special%requests%'; SELECT n_nationkey FROM nation WHERE n_name = 'GERMANY'; SELECT s_suppkey FROM supplier WHERE s_comment LIKE '%Customer%Complaints%'; SQL target/ci/dfbench statistics \ --path "$repro_dir/data" --query_path "$repro_dir/repro.sql" ``` Observed with `tpchgen-cli` 1.1.1 at [6c320561b5](https://github.com/apache/datafusion/commit/6c320561b5b1aef7a235a12435c3c96b62956c67). Inspect the SELECT reports; ignore the empty `SET` reports. | SELECT | FilterExec node | Estimated rows | Actual rows | | --------------- | --------------- | -------------: | ----------: | | Orders LIKE | `0` | 300,000 | 16,082 | | Orders NOT LIKE | `0` | 300,000 | 1,483,918 | | Nation equality | `0` | 5 | 1 | | Supplier LIKE | `0` | 2,000 | 4 | ## Expected behavior Use NDVs/frequencies for equality where available, distinguish pattern forms, and make LIKE/NOT LIKE estimates complementary over non-null rows. ## Additional context The predicates come from TPC-H Q11/Q13/Q16. Related: [filter selectivity #14237](https://github.com/apache/datafusion/issues/14237). Part of #25610. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
