david-mollitor-db opened a new pull request, #58484:
URL: https://github.com/apache/spark/pull/58484
### What changes were proposed in this pull request?
`LikeSimplification` rewrites simple `LIKE` patterns into cheaper predicates
(`'A%'` ->
`StartsWith`, `'%B'` -> `EndsWith`, `'A%B'` -> length guard + `StartsWith` +
`EndsWith`,
`'%B%'` -> `Contains`, exact string -> `EqualTo`). Multi-wildcard patterns
that have a
*leading literal* but match none of those shapes -- e.g. `'A%B%'`,
`'AB%CD%EF'`, `'A_B%'` --
fall through unchanged and remain a full regex `Like`, so the data source
receives no
predicate and the per-row regex runs on every row.
This PR makes `LikeSimplification` additionally derive the leading literal
`A` as the
necessary condition `StartsWith(col, A)`, keeping the original `LIKE` as the
exact residual:
```
col LIKE 'A%B%' ==> StartsWith(col, A) && (col LIKE 'A%B%')
```
`StartsWith` is placed first so the cheap check short-circuits the regex.
The derivation is
restricted to binary-equality collations
(`StringType.supportsBinaryEquality`, i.e.
`UTF8_BINARY`) and a `TreeNodeTag` on the residual `Like` keeps the rule
idempotent under the
fixed-point optimizer batch. Only `StartsWith` is derived from the leading
literal; the
`LikeAll`/`LikeAny` paths are unchanged.
### Why are the changes needed?
- **Pushdown / pruning.** On `UTF8_BINARY`, `StartsWith` translates to
`sources.StringStartsWith`, which prunes Parquet row groups via min/max
statistics. Readers
cannot prune on the raw `Like`, so today a leading-literal multi-wildcard
`LIKE` reads every
row group.
- **Cheaper per-row evaluation.** The `StartsWith` short-circuits the more
expensive regex on
rows that fail the prefix.
- **Results are unchanged.** `StartsWith(A)` is implied by `LIKE 'A%...'`,
and the exact `LIKE`
is retained as the residual, so the conjunction accepts exactly the same
rows. This mirrors
how PostgreSQL, SQL Server and SQLite turn a leading-literal `LIKE` into a
sargable prefix
predicate plus a residual recheck.
The derivation is gated on binary equality for correctness as well as
benefit: under a
collation-aware collation (e.g. `UTF8_LCASE`) the `Like` regex match (Java
regex case flags)
and `StartsWith` (`CollationSupport`) can disagree, so `StartsWith(A)` would
not be a sound
necessary condition; and `StringStartsWith` is pushed down only for
`UTF8_BINARY` (non-binary
is wrapped as `CollatedStringStartsWith`, which readers ignore).
`EndsWith`/`Contains` are not
derived from trailing/inner literals because Parquet's
`StringEndsWith`/`StringContains` have
`canDrop = false` (no pruning).
### Does this PR introduce _any_ user-facing change?
No. Query results are identical; this is a performance improvement (added
pushdown and a
short-circuit conjunct on an otherwise unsimplified `LIKE`).
### How was this patch tested?
- New unit tests in `LikeSimplificationSuite` covering: derivation for
`'a%b%'`, a multi-char
prefix with multiple wildcards, `'_'` patterns, no derivation when there
is no leading literal,
no derivation when the pattern contains the escape char, no derivation for
a non-binary
(`UTF8_LCASE`) collation, and idempotency.
- A new end-to-end test in `ParquetFilterSuite` verifying that `LIKE
'ab%cd%'` / `'ab%cd%ef'` /
`'a_b%'` push a `StringStartsWith` and prune Parquet row groups.
- `build/sbt 'catalyst/testOnly *LikeSimplificationSuite'` and
`build/sbt 'sql/testOnly *ParquetV1FilterSuite -- -z "leading-literal"'`
pass; scalastyle clean.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 4.8
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
