Holden Karau created SPARK-60903:
------------------------------------
Summary: Extract constraints from case/when to push down to
optimizer
Key: SPARK-60903
URL: https://issues.apache.org/jira/browse/SPARK-60903
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 4.4.0
Reporter: Holden Karau
In SPARK-58456 it was brought up in
[https://github.com/apache/spark/pull/59216#discussion_r4260389154] that
chained comparisions can't be simple broken up and pushed safely. Long term we
should handle this (even for non-UDF cases) where a case/when block has a core
restriction that we can safely push (e.g. a < b < c -> we can push a < b as a
read restriction since it always evals).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
