[
https://issues.apache.org/jira/browse/SPARK-59355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-59355:
-----------------------------------
Labels: pull-request-available (was: )
> Simplify LIKE patterns containing escaped wildcards
> ---------------------------------------------------
>
> Key: SPARK-59355
> URL: https://issues.apache.org/jira/browse/SPARK-59355
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.1.0
> Reporter: David Mollitor
> Priority: Minor
> Labels: pull-request-available
>
> h3. What
> {{LikeSimplification}} currently gives up on a pattern the moment it contains
> the escape character (`if (pattern.contains(escapeChar)) None`), so
> escaped-literal patterns stay a full per-row regex even when they are
> trivially sargable once the escapes are decoded:
> {code:java}
> col LIKE 'ma\%ca' ==> col = 'ma%ca'
> col LIKE 'abc\%def%' ==> StartsWith(col, 'abc%def')
> col LIKE '%mn\%' ==> EndsWith(col, 'mn%')
> col LIKE 'abbc' ESCAPE 'b' ==> col = 'abc'
> {code}
> This decodes valid escape sequences and applies the rule's existing shape
> simplifications (EqualTo / StartsWith / EndsWith / Contains /
> StartsWith+EndsWith) to the decoded literals.
> h3. Why are the changes needed?
> Escaped-literal patterns become sargable: the resulting
> {{{}EqualTo{}}}/{{{}StartsWith{}}}/etc. push down to data sources (e.g.
> Parquet prunes on {{{}StringStartsWith{}}}) and short-circuit the per-row
> regex, instead of always running the regex and pushing nothing.
> h3. How it works
> * *Fast path (unchanged).* A quick {{pattern.contains(escapeChar)}} scan:
> when the escape character is absent, the existing five regexes run exactly as
> before – the common case pays nothing for the decode logic.
> * *Guard.* Otherwise, if the escape character is itself {{%}} or {{_}} (a
> pathological
> {{ESCAPE '%'}} / {{{}ESCAPE '_'{}}}), skip – unchanged behavior. (This guard
> must follow the fast-path scan: an {{ESCAPE '%'}} pattern that contains no
> {{%}} is still simplifiable via the fast path.)
> * *Decode.* Otherwise, walk the pattern turning {{%}} / {{_}} / \{{}} into
> literal characters, treating unescaped {{{}%{}}}/{{{}_{}}} as wildcards, and
> classify the decoded shape.
> h3. Correctness
> * *Behavior-preserving.* A valid-escape pattern's decoded literal is exactly
> the string the {{LIKE}} matches, so the resulting predicate accepts the same
> rows.
> * *Invalid and trailing escapes are left alone.* An escape character
> followed by anything other than {{{}%{}}}, {{{}_{}}}, or itself, or a
> trailing escape character, makes {{LIKE}} throw at runtime
> ({{{}INVALID_FORMAT.ESC_IN_THE_MIDDLE{}}} / {{{}ESC_AT_THE_END{}}}); the rule
> returns None for these, so the {{Like}} is kept and throws exactly as today.
> * Decoded literals may contain {{{}%{}}}/{{{}_{}}}/the escape character;
> that is fine, since the
> predicates compare literal strings. Collation handling is unchanged. Every
> shape fully replaces the {{Like}} (no residual), so no idempotency tag is
> needed.
> * Out of scope: an unescaped {{_}} single-character wildcard, and
> {{{}LikeAll{}}}/{{{}LikeAny{}}} shapes beyond what the shared
> {{simplifyLike}} already covers.
> h3. Does this PR introduce any user-facing change?
> No. Query results are identical (and error cases still error); this is a
> performance improvement.
> h3. How was this patch tested?
> {{{}LikeSimplificationSuite{}}}: updated the existing escape tests (they
> previously asserted the pattern was skipped and now assert the simplified
> form, including the {{{}LikeAll{}}}/{{{}LikeAny{}}} multi-pattern tests whose
> escaped entries now fold in), and added tests for valid-escape simplification
> across all shapes and for the still-skipped cases (invalid escape, trailing
> escape, {{{}ESCAPE '%'{}}}/{{{}ESCAPE '{_}'{_}{}}}{_}, and an unescaped
> {{_}}). Scalastyle clean.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
