mzabaluev commented on PR #11999:
URL: https://github.com/apache/arrow-rs/pull/11999#issuecomment-6596703883
### Benchmarks with the list cases from #11872
`main` has no list filter benchmarks with runs of selected rows, so I
temporarily
applied the `arrow/benches/filter_kernels.rs` changes from #11872 on top of
this
branch and ran `cargo bench -p arrow --features test_utils --bench
filter_kernels -- list`
for three versions of `arrow-select/src/filter.rs`, all built in the same
target
directory and compared with criterion baselines:
- `main`
- the first commit of this PR ("keep the filter mask only where kernels read
it")
- the second commit ("scan the kept mask in the MutableArrayData filter
fallback")
On `main`, list arrays are filtered through the `MutableArrayData` fallback.
With the
first commit alone, the fallback found the runs of selected rows by merging
consecutive
materialized indices, while `main` scanned the mask. That made optimized
predicates
(the `context` cases) with run-shaped selections up to a third slower. The
second commit
scans the mask whenever the materialized `Indices` keep it, which brings
these cases back
to `main`'s level:
| `filter context …` | `main` → 1st commit | 1st → 2nd commit | `main` → 2nd
commit |
|---|---:|---:|---:|
| `list<i32> w NULLs` (runs of 64, kept 1/2) | +10.6% | −10.3% | −0.8% |
| `list<i32> w NULLs` (prefix, kept 1/2) | +19.7% | −18.4% | −2.3% |
| `list<i32> w NULLs` (one run, kept 1/8) | +14.4% | −13.9% | −1.5% |
| `short list<i32> w NULLs` (runs of 64, kept 1/2) | +23.7% | −19.8% | −0.8%
|
| `short list<i32> w NULLs` (prefix, kept 1/2) | +32.9% | −26.6% | −2.4% |
| `short list<i32> w NULLs` (one run, kept 1/8) | +25.9% | −23.0% | −3.0% |
| `empty rows of list<i32>` (runs of 64, kept 1/2) | +26.0% | −21.8% | −1.5%
|
Sparse optimized predicates don't keep the mask, so they still filter lists
faster
than on `main`:
| `filter context …` (kept 1/1024) | `main` → 2nd commit |
|---|---:|
| `list<i32> w NULLs` | −21.8% |
| `short list<i32> w NULLs` | −24.2% |
| `list<struct<i64, utf8>> w NULLs` | −12.8% |
| `fixed_size_list<i32> w NULLs` | −33.1% |
| `fixed_size_list<struct<i64, utf8>>` | −17.5% |
All the other list cases are within ±3% of `main`. One case,
`filter context short list<i32> w NULLs high selectivity (kept 1023/1024)`,
first
showed +11.8%. It uses the `Slices` strategy, which neither commit changes,
and two
more runs of that case put both versions within 43–45 µs, so it was noise.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
