[ https://issues.apache.org/jira/browse/SPARK-33847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yuming Wang updated SPARK-33847: -------------------------------- Description: {code:scala} spark.sql("create table t1 using parquet as select id from range(10)") spark.sql("select id from t1 where (CASE WHEN id = 1 THEN 'a' WHEN id = 3 THEN 'b' end) = 'c' ").explain() {code} Before: {noformat} == Physical Plan == *(1) Filter CASE WHEN (id#1L = 1) THEN false WHEN (id#1L = 3) THEN false END +- *(1) ColumnarToRow +- FileScan parquet default.t1[id#1L] Batched: true, DataFilters: [CASE WHEN (id#1L = 1) THEN false WHEN (id#1L = 3) THEN false END], Format: Parquet, Location: InMemoryFileIndex[file:/Users/yumwang/opensource/spark/spark-warehouse/org.apache.spark.sql.DataF..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<id:bigint> {noformat} After: {noformat} == Physical Plan == LocalTableScan <empty>, [id#1L] {noformat} was: {code:scala} spark.sql("create table t1 using parquet as select id from range(10)") spark.sql("select id from t1 where (CASE WHEN id = 1 THEN 'a' WHEN id = 3 THEN 'b' end) = 'c' ").explain() {code} Before: {noformat} == Physical Plan == *(1) Filter CASE WHEN (id#1L = 1) THEN false WHEN (id#1L = 3) THEN false END +- *(1) ColumnarToRow +- FileScan parquet default.t1[id#1L] Batched: true, DataFilters: [CASE WHEN (id#1L = 1) THEN false WHEN (id#1L = 3) THEN false END], Format: Parquet, Location: InMemoryFileIndex[file:/Users/yumwang/opensource/spark/spark-warehouse/org.apache.spark.sql.DataF..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<id:bigint> {noformat} After: {noformat} == Physical Plan == LocalTableScan <empty>, [id#1L] {noformat} > Replace None of elseValue inside CaseWhen if all branches are FalseLiteral > -------------------------------------------------------------------------- > > Key: SPARK-33847 > URL: https://issues.apache.org/jira/browse/SPARK-33847 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 3.2.0 > Reporter: Yuming Wang > Priority: Major > > {code:scala} > spark.sql("create table t1 using parquet as select id from range(10)") > spark.sql("select id from t1 where (CASE WHEN id = 1 THEN 'a' WHEN id = 3 > THEN 'b' end) = 'c' ").explain() > {code} > Before: > {noformat} > == Physical Plan == > *(1) Filter CASE WHEN (id#1L = 1) THEN false WHEN (id#1L = 3) THEN false END > +- *(1) ColumnarToRow > +- FileScan parquet default.t1[id#1L] Batched: true, DataFilters: [CASE > WHEN (id#1L = 1) THEN false WHEN (id#1L = 3) THEN false END], Format: > Parquet, Location: > InMemoryFileIndex[file:/Users/yumwang/opensource/spark/spark-warehouse/org.apache.spark.sql.DataF..., > PartitionFilters: [], PushedFilters: [], ReadSchema: struct<id:bigint> > {noformat} > After: > {noformat} > == Physical Plan == > LocalTableScan <empty>, [id#1L] > {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org