david-mollitor-db opened a new pull request, #58644:
URL: https://github.com/apache/spark/pull/58644
### What changes were proposed in this pull request?
`LikeSimplification` gives up on a pattern the moment it contains the escape
character
(`if (pattern.contains(escapeChar)) None`), so escaped-literal patterns stay
a full per-row
regex even when they are trivially sargable once the escapes are decoded:
```
col LIKE 'ma\%ca' ==> col = 'ma%ca'
col LIKE 'abc\%def%' ==> StartsWith(col, 'abc%def')
col LIKE '%mn\%' ==> EndsWith(col, 'mn%')
col LIKE 'abbc' ESCAPE 'b' ==> col = 'abc'
```
This PR decodes valid escape sequences and applies the rule's existing shape
simplifications
(`EqualTo` / `StartsWith` / `EndsWith` / `Contains` /
`StartsWith`+`EndsWith`) to the decoded
literals.
How it works:
- **Fast path (unchanged):** a quick `pattern.contains(escapeChar)` scan —
when the escape
character is absent, the existing five regexes run exactly as before, so
the common case pays
nothing for the decode logic.
- **Guard:** otherwise, if the escape character is itself `%` or `_` (a
pathological
`ESCAPE '%'` / `ESCAPE '_'`), skip — unchanged behavior. This guard
follows the fast-path scan
on purpose: an `ESCAPE '%'` pattern that contains no `%` is still
simplifiable via the fast
path.
- **Decode:** otherwise, walk the pattern turning `\%` / `\_` / `\\` into
literal characters,
treating unescaped `%`/`_` as wildcards, and classify the decoded shape.
An unescaped `_` wildcard and `LikeAll`/`LikeAny` shapes beyond what
`simplifyLike` already
covers remain out of scope.
### Why are the changes needed?
Escaped-literal patterns become sargable: the resulting
`EqualTo`/`StartsWith`/etc. push down to
data sources (e.g. Parquet prunes on `StringStartsWith`) and short-circuit
the per-row regex,
instead of always running the regex and pushing nothing.
### Does this PR introduce _any_ user-facing change?
No. Query results are identical, and error cases still error; this is a
performance improvement.
A valid-escape pattern's decoded literal is exactly the string the `LIKE`
matches, so the rewrite
is behavior-preserving. An escape character followed by anything other than
`%`, `_`, or itself,
or a trailing escape character, makes `LIKE` throw at runtime
(`INVALID_FORMAT.ESC_IN_THE_MIDDLE`
/ `ESC_AT_THE_END`); the rule returns `None` for these, so the `Like` is
kept and throws exactly
as today. Every shape fully replaces the `Like` (no residual), so the
rewrite is idempotent.
### How was this patch tested?
`LikeSimplificationSuite`: updated the existing escape tests (they
previously asserted the
pattern was skipped and now assert the simplified form, including the
`LikeAll`/`LikeAny`
multi-pattern tests whose escaped entries now fold in), and added tests for
valid-escape
simplification across all shapes and for the still-skipped cases (invalid
escape, trailing
escape, `ESCAPE '%'`/`ESCAPE '_'`, and an unescaped `_`).
`build/sbt 'catalyst/testOnly *LikeSimplificationSuite'` passes; scalastyle
clean.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 4.8
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
