LuciferYang opened a new pull request, #59428:
URL: https://github.com/apache/spark/pull/59428
### What changes were proposed in this pull request?
This backports #59300 to branch-4.3. The change to `LikeSimplification` is
the same; the length guard on this branch is `Length` rather than the
`OctetLength` of SPARK-59063, which this branch does not have, so the test
expects `Length(substr) >= 6`.
`LikeSimplification` rewrites `input LIKE 'prefix%suffix'` into a length
check, a `StartsWith` and an `EndsWith`, each over its own copy of `input`.
This PR makes that rewrite only when the input is deterministic. A
nondeterministic input keeps the `LIKE`. The other single-pattern rewrites read
the input once and are unchanged.
### Why are the changes needed?
A nondeterministic input is evaluated once per copy, so the three checks can
each see a different value and pass together for a row where no single value
matches the pattern:
```scala
val pick = udf(() => if (scala.util.Random.nextBoolean()) "ay" else
"bz").asNondeterministic()
spark.udf.register("pick", pick)
spark.sql(
"SELECT count_if(r) FROM (SELECT pick() LIKE 'a%z' AS r FROM range(0,
1000, 1, 1))").show()
// a few hundred without this change (239 in one run on master); 0 with it
```
The optimized plan without this change computes `r` as `((length(pick()) >=
2) AND (StartsWith(pick(), a) AND EndsWith(pick(), z)))`. `pick() ILIKE 'A%Z'`
becomes `lower(pick()) LIKE 'a%z'` and fails the same way. The `LIKE ALL` and
`LIKE ANY` rewrites already skip an input that is not cheap.
### Does this PR introduce _any_ user-facing change?
Yes. `LIKE` or `ILIKE` with a `'prefix%suffix'` pattern over a
nondeterministic input now evaluates the input once per row, so the result is
that of matching that one value.
### How was this patch tested?
`LikeSimplificationSuite`: `uuid() LIKE 'abc%def'` is left unchanged,
`uuid() LIKE 'abc%'` is still rewritten to `StartsWith`, and `substring(a, 1,
5) LIKE 'abc%def'`, deterministic but not cheap, is still split into the length
check, `StartsWith` and `EndsWith`. The first assertion fails without the
change.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
