stevomitric commented on code in PR #58637:
URL: https://github.com/apache/spark/pull/58637#discussion_r3987493000
##########
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/ReplaceNullWithFalseInPredicateSuite.scala:
##########
@@ -452,6 +452,49 @@ class ReplaceNullWithFalseInPredicateSuite extends
PlanTest {
}
Review Comment:
Not blocking on this PR, just noting for a potential followup:
Since ILIKE is replaced with: `override lazy val replacement: Expression =
Like(Lower(left), Lower(right), escapeChar)`, this optimization would rewrite:
`col ILIKE '%' to IsNotNull(Lower(col))` rather than `IsNotNull(col)`.
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ReplaceNullWithFalseInPredicate.scala:
##########
@@ -120,6 +121,14 @@ object ReplaceNullWithFalseInPredicate extends
Rule[LogicalPlan] {
case Not(InSet(value, list)) if isNullLiteral(value) ||
list.contains(null) =>
FalseLiteral
+ // `x LIKE '%'` (a pattern of only unescaped `%`) is true for any non-null
value and null for
+ // null. In a predicate, null is treated as false, so it is equivalent to
`IsNotNull(x)`,
+ // which is cheaper and can be pushed down. This case is reached only
through null-preserving
+ // operators, so it never fires under `Not`, where the rewrite would be
unsafe.
+ case Like(child, Literal(pattern, _: StringType), escapeChar)
Review Comment:
Consider `LIKE '%'` inside a CASE in the SELECT list, not in a WHERE clause,
i.e.:
```sql
SELECT CASE WHEN col LIKE '%' THEN 1 ELSE 2 END FROM t
```
we could maybe optimize here as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
