[
https://issues.apache.org/jira/browse/SPARK-33847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuming Wang updated SPARK-33847:
--------------------------------
Description:
{code:scala}
spark.sql("create table t1 using parquet as select id from range(10)")
spark.sql("select id from t1 where (CASE WHEN id = 1 THEN 'a' WHEN id = 3 THEN
'b' end) = 'c' ").explain()
{code}
Before:
{noformat}
== Physical Plan ==
*(1) Filter CASE WHEN (id#1L = 1) THEN false WHEN (id#1L = 3) THEN false END
+- *(1) ColumnarToRow
+- FileScan parquet default.t1[id#1L] Batched: true, DataFilters: [CASE WHEN
(id#1L = 1) THEN false WHEN (id#1L = 3) THEN false END], Format: Parquet,
Location:
InMemoryFileIndex[file:/Users/yumwang/opensource/spark/spark-warehouse/org.apache.spark.sql.DataF...,
PartitionFilters: [], PushedFilters: [], ReadSchema: struct<id:bigint>
{noformat}
After:
{noformat}
== Physical Plan ==
LocalTableScan <empty>, [id#1L]
{noformat}
was:
{code:scala}
spark.sql("create table t1 using parquet as select id from range(10)")
spark.sql("select id from t1 where (CASE WHEN id = 1 THEN 'a' WHEN id = 3
THEN 'b' end) = 'c' ").explain()
{code}
Before:
{noformat}
== Physical Plan ==
*(1) Filter CASE WHEN (id#1L = 1) THEN false WHEN (id#1L = 3) THEN false END
+- *(1) ColumnarToRow
+- FileScan parquet default.t1[id#1L] Batched: true, DataFilters: [CASE WHEN
(id#1L = 1) THEN false WHEN (id#1L = 3) THEN false END], Format: Parquet,
Location:
InMemoryFileIndex[file:/Users/yumwang/opensource/spark/spark-warehouse/org.apache.spark.sql.DataF...,
PartitionFilters: [], PushedFilters: [], ReadSchema: struct<id:bigint>
{noformat}
After:
{noformat}
== Physical Plan ==
LocalTableScan <empty>, [id#1L]
{noformat}
> Replace None of elseValue inside CaseWhen if all branches are FalseLiteral
> --------------------------------------------------------------------------
>
> Key: SPARK-33847
> URL: https://issues.apache.org/jira/browse/SPARK-33847
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.2.0
> Reporter: Yuming Wang
> Priority: Major
>
> {code:scala}
> spark.sql("create table t1 using parquet as select id from range(10)")
> spark.sql("select id from t1 where (CASE WHEN id = 1 THEN 'a' WHEN id = 3
> THEN 'b' end) = 'c' ").explain()
> {code}
> Before:
> {noformat}
> == Physical Plan ==
> *(1) Filter CASE WHEN (id#1L = 1) THEN false WHEN (id#1L = 3) THEN false END
> +- *(1) ColumnarToRow
> +- FileScan parquet default.t1[id#1L] Batched: true, DataFilters: [CASE
> WHEN (id#1L = 1) THEN false WHEN (id#1L = 3) THEN false END], Format:
> Parquet, Location:
> InMemoryFileIndex[file:/Users/yumwang/opensource/spark/spark-warehouse/org.apache.spark.sql.DataF...,
> PartitionFilters: [], PushedFilters: [], ReadSchema: struct<id:bigint>
> {noformat}
> After:
> {noformat}
> == Physical Plan ==
> LocalTableScan <empty>, [id#1L]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
