[ https://issues.apache.org/jira/browse/SPARK-39093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17533408#comment-17533408 ]
Apache Spark commented on SPARK-39093: -------------------------------------- User 'bersprockets' has created a pull request for this issue: https://github.com/apache/spark/pull/36481 > Dividing interval by integral can result in codegen compilation error > --------------------------------------------------------------------- > > Key: SPARK-39093 > URL: https://issues.apache.org/jira/browse/SPARK-39093 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.2.1, 3.3.0, 3.4.0 > Reporter: Bruce Robbins > Assignee: Bruce Robbins > Priority: Minor > Fix For: 3.3.1 > > > Assume this data: > {noformat} > create or replace temp view v1 as > select * FROM VALUES > (interval '10' months, interval '10' day, 2) > as v1(period, duration, num); > cache table v1; > {noformat} > These two queries work: > {noformat} > spark-sql> select period/num from v1; > 0-5 > Time taken: 0.143 seconds, Fetched 1 row(s) > {noformat} > {noformat} > spark-sql> select duration/num from v1; > 5 00:00:00.000000000 > Time taken: 0.094 seconds, Fetched 1 row(s) > {noformat} > However, these two queries get a codegen compilation error: > {noformat} > spark-sql> select period/(num + 3) from v1; > 22/05/03 08:56:37 ERROR CodeGenerator: failed to compile: > org.codehaus.commons.compiler.CompileException: File 'generated.java', Line > 40, Column 44: Expression "project_value_2" is not an rvalue > ... > 22/05/03 08:56:37 WARN UnsafeProjection: Expr codegen error and falling back > to interpreter mode > ... > 0-2 > Time taken: 0.149 seconds, Fetched 1 row(s) > {noformat} > {noformat} > spark-sql> select duration/(num + 3) from v1; > 22/05/03 08:57:29 ERROR CodeGenerator: failed to compile: > org.codehaus.commons.compiler.CompileException: File 'generated.java', Line > 40, Column 54: Expression "project_value_2" is not an rvalue > ... > 22/05/03 08:57:29 WARN UnsafeProjection: Expr codegen error and falling back > to interpreter mode > ... > 2 00:00:00.000000000 > Time taken: 0.089 seconds, Fetched 1 row(s) > {noformat} > Even the first two queries will get a compilation error if you turn off > whole-stage codegen: > {noformat} > spark-sql> set spark.sql.codegen.wholeStage=false; > spark.sql.codegen.wholeStage false > Time taken: 0.055 seconds, Fetched 1 row(s) > spark-sql> select period/num from v1; > 22/05/03 09:16:42 ERROR CodeGenerator: failed to compile: > org.codehaus.commons.compiler.CompileException: File 'generated.java', Line > 37, Column 5: Expression "value_1" is not an rvalue > .... > 0-5 > Time taken: 0.175 seconds, Fetched 1 row(s) > spark-sql> select duration/num from v1; > 22/05/03 09:17:41 ERROR CodeGenerator: failed to compile: > org.codehaus.commons.compiler.CompileException: File 'generated.java', Line > 37, Column 5: Expression "value_1" is not an rvalue > ... > 5 00:00:00.000000000 > Time taken: 0.104 seconds, Fetched 1 row(s) > {noformat} > Note that in the error cases, the queries still return a result because Spark > falls back on interpreting the divide expression (so I marked this as > "minor"). -- This message was sent by Atlassian Jira (v8.20.7#820007) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org