[ https://issues.apache.org/jira/browse/SPARK-35449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
L. C. Hsieh updated SPARK-35449: -------------------------------- Affects Version/s: 3.1.1 > Should not extract common expressions from value expressions when elseValue > is empty in CaseWhen > ------------------------------------------------------------------------------------------------ > > Key: SPARK-35449 > URL: https://issues.apache.org/jira/browse/SPARK-35449 > Project: Spark > Issue Type: Sub-task > Components: SQL > Affects Versions: 3.1.1, 3.2.0 > Reporter: L. C. Hsieh > Priority: Major > > [https://github.com/apache/spark/pull/30245] added support for creating > subexpressions that are present in all branches of conditional statements. > However, for a statement to be in "all branches" of a CaseWhen statement, it > must also be in the elseValue. This can lead to a subexpression to be created > and run for branches of a conditional that don't pass. This can cause issues > especially with a UDF in a branch that gets executed assuming the condition > is true. For example: > {code:java} > val col = when($"id" < 0, myUdf($"id")) > spark.range(1).select(when(col > 0, col)).show() > {code} > myUdf($"id") gets extracted as a subexpression and executed even though both > conditions don't pass and it should never be executed. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org