[
https://issues.apache.org/jira/browse/SPARK-13731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15184034#comment-15184034
]
Ian edited comment on SPARK-13731 at 3/8/16 2:35 AM:
-----------------------------------------------------
The test case we provided is using simple arithmetic expression like divisions
for double,
but in fact, many other math functions are having the same behaviors that
returns null for NaN/Infinity.
For instance, log() and corr().
{code}
SELECT log(a/b), corr(a, b) FROM testNan order by a, b
{code}
was (Author: ianlcsd):
The test case we provided is using simple arithmetic expression like divisions
for double,
but in fact, many other math functions are having the same behaviors that
returns null for NaN/Infinity.
For instance, log() and corr().
{code}
SELECT log(a/b), corn(a, b) FROM testNan order by a, b
{code}
> expression evaluation for NaN in select statement
> -------------------------------------------------
>
> Key: SPARK-13731
> URL: https://issues.apache.org/jira/browse/SPARK-13731
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.6.0
> Reporter: Ian
>
> We are expecting that arithmetic expression a/b should be:
> 1. returning NaN if a=0 and b=0
> 2. returning Infinity if a=1 and b=0
> Is the expectation reasonable?
> The following is a simple test case snippet that reads from storage and
> evaluates arithmetic expressions in select.
> It is assuming org.apache.spark.sql.hive.execution.SQLQuerySuite:
> {code}
> test("Expression should be evaluated to Nan/Infinity in Select") {
> withTable("testNan") {
> withTempTable("src") {
> Seq((1d, 0d), (0d, 0d)).toDF().registerTempTable("src")
> sql("CREATE TABLE testNan(a double, b double) STORED AS PARQUET AS
> SELECT * FROM src")
> }
> checkAnswer(sql(
> """
> |SELECT a/b FROM testNan
> """.stripMargin),
> Seq(
> Row(Double.PositiveInfinity),
> Row(Double.NaN)
> )
> )
> }
> }
> == Physical Plan ==
> Project [(a#28 / b#29) AS _c0#30]
> +- Scan ParquetRelation: default.testnan[a#28,b#29] InputPaths:
> file:/private/var/folders/dy/19y6pfm92pj9s40mbs8xd9hm0000gp/T/warehouse--5b617080-e909-4812-90e8-63d2dd0aef5a/testnan
> == Results ==
> !== Correct Answer - 2 == == Spark Answer - 2 ==
> ![Infinity] [null]
> ![NaN] [null]
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
