david-mollitor-db opened a new pull request, #58641: URL: https://github.com/apache/spark/pull/58641
### What changes were proposed in this pull request? A `LIKE` pattern containing the `_` wildcard (which matches exactly one code point) is not simplified today — `LikeSimplification` leaves it as a full per-row regex. Since `_` constrains length, this PR derives a code-point **length guard**: a pattern with no `%` fixes the length (`Length(col) = N`), and one with `%` gives a lower bound (`Length(col) >= N`), where N is the number of non-`%` code points. When the pattern has no literals (only `_`/`%`) the guard is exactly equivalent, so it replaces the `LIKE`: ``` col LIKE '___' ==> Length(col) = 3 col LIKE '_%' ==> Length(col) >= 1 ``` When the pattern also has literals, the guard is only a necessary condition, so the exact `LIKE` is kept as the residual: ``` col LIKE 'a_c' ==> Length(col) = 3 && (col LIKE 'a_c') col LIKE 'a_b%' ==> Length(col) >= 3 && (col LIKE 'a_b%') ``` Patterns with escape characters are skipped (as elsewhere in the rule). The anchored-exact rewrite (`'a__c'` -> `Length && StartsWith && EndsWith`), positional `substring` rewrites, and `LikeAll`/`LikeAny` are out of scope. ### Why are the changes needed? `Length(col)` is a cheap check that fails fast before the regex, so short strings are rejected (and, for the literal-free cases, the regex is eliminated entirely). This is a CPU/short-circuit improvement; `Length(col)` is a function of the column rather than a pushable column reference, so it does not push down to the data source or prune I/O. **Correctness.** `Length` is a *code-point* count — the right measure for `_`, which matches one code point regardless of its UTF-8 byte width (a byte length would be wrong here). The rewrite is valid in every context (not just predicates) and needs no collation gate: `Length(col) = N` agrees with `col LIKE '...'` even on `null` (both are null-intolerant), and for the literal case `And(guard, LIKE)` is just the `LIKE` conjoined with one of its necessary conditions. It assumes each pattern token consumes exactly one input code point, which holds for Spark's `LIKE` (Java-regex simple, 1:1 case folding). Idempotency under the fixed-point optimizer batch is maintained via a `TreeNodeTag` on the residual `Like`. ### Does this PR introduce _any_ user-facing change? No. Query results are identical; this is a performance improvement. ### How was this patch tested? New tests in `LikeSimplificationSuite`: - Exact replacement for `_`-only patterns: `'_'`, `'___'` -> `Length =`. - Minimum-length replacement for `_` with `%`: `'_%'`, `'%_%'`, `'_%_'` -> `Length >=`. - Additive guard for `_` with literals: `'a_c'` -> `Length = 3 && LIKE`; `'a_b%'` -> `Length >= 3 && LIKE`. - Not rewritten: escaped `_` (`'a\_b'`) and a no-`_` pattern (`'abc%'` still -> `StartsWith`). `build/sbt 'catalyst/testOnly *LikeSimplificationSuite'` passes (24/24); scalastyle clean. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 4.8 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
