[jira] [Updated] (SPARK-32640) Spark 3.1 log(NaN) returns null instead of NaN
[ https://issues.apache.org/jira/browse/SPARK-32640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-32640: - Labels: correctness (was: ) > Spark 3.1 log(NaN) returns null instead of NaN > -- > > Key: SPARK-32640 > URL: https://issues.apache.org/jira/browse/SPARK-32640 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.1.0 >Reporter: Thomas Graves >Assignee: Wenchen Fan >Priority: Major > Labels: correctness > Fix For: 3.1.0 > > > I was testing Spark 3.1.0 and I noticed that if you take the log(NaN) it now > returns a null whereas in Spark 3.0 it returned a NaN. I'm not an expert in > this but I thought NaN was correct. > Spark 3.1.0 Example: > >>> df.selectExpr(["value", "log1p(value)"]).show() > +--+-+ > | value| LOG1P(value)| > +--+-+ > |-3.4028235E38| null| > |3.4028235E38|88.72283906194683| > | 0.0| 0.0| > | -0.0| -0.0| > | 1.0|0.6931471805599453| > | -1.0| null| > | NaN| null| > +--+-+ > > Spark 3.0.0 example: > > +-+--+ > | value| LOG1P(value)| > +-+--+ > |-3.4028235E38| null| > | 3.4028235E38| 88.72283906194683| > | 0.0| 0.0| > | -0.0| -0.0| > | 1.0|0.6931471805599453| > | -1.0| null| > | NaN| NaN| > +-+--+ > > Note it also does the same for log1p, log2, log10 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-32640) Spark 3.1 log(NaN) returns null instead of NaN
[ https://issues.apache.org/jira/browse/SPARK-32640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-32640: - Labels: correction (was: ) > Spark 3.1 log(NaN) returns null instead of NaN > -- > > Key: SPARK-32640 > URL: https://issues.apache.org/jira/browse/SPARK-32640 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.1.0 >Reporter: Thomas Graves >Assignee: Wenchen Fan >Priority: Major > Labels: correction > Fix For: 3.1.0 > > > I was testing Spark 3.1.0 and I noticed that if you take the log(NaN) it now > returns a null whereas in Spark 3.0 it returned a NaN. I'm not an expert in > this but I thought NaN was correct. > Spark 3.1.0 Example: > >>> df.selectExpr(["value", "log1p(value)"]).show() > +--+-+ > | value| LOG1P(value)| > +--+-+ > |-3.4028235E38| null| > |3.4028235E38|88.72283906194683| > | 0.0| 0.0| > | -0.0| -0.0| > | 1.0|0.6931471805599453| > | -1.0| null| > | NaN| null| > +--+-+ > > Spark 3.0.0 example: > > +-+--+ > | value| LOG1P(value)| > +-+--+ > |-3.4028235E38| null| > | 3.4028235E38| 88.72283906194683| > | 0.0| 0.0| > | -0.0| -0.0| > | 1.0|0.6931471805599453| > | -1.0| null| > | NaN| NaN| > +-+--+ > > Note it also does the same for log1p, log2, log10 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-32640) Spark 3.1 log(NaN) returns null instead of NaN
[ https://issues.apache.org/jira/browse/SPARK-32640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-32640: - Labels: (was: correction) > Spark 3.1 log(NaN) returns null instead of NaN > -- > > Key: SPARK-32640 > URL: https://issues.apache.org/jira/browse/SPARK-32640 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.1.0 >Reporter: Thomas Graves >Assignee: Wenchen Fan >Priority: Major > Fix For: 3.1.0 > > > I was testing Spark 3.1.0 and I noticed that if you take the log(NaN) it now > returns a null whereas in Spark 3.0 it returned a NaN. I'm not an expert in > this but I thought NaN was correct. > Spark 3.1.0 Example: > >>> df.selectExpr(["value", "log1p(value)"]).show() > +--+-+ > | value| LOG1P(value)| > +--+-+ > |-3.4028235E38| null| > |3.4028235E38|88.72283906194683| > | 0.0| 0.0| > | -0.0| -0.0| > | 1.0|0.6931471805599453| > | -1.0| null| > | NaN| null| > +--+-+ > > Spark 3.0.0 example: > > +-+--+ > | value| LOG1P(value)| > +-+--+ > |-3.4028235E38| null| > | 3.4028235E38| 88.72283906194683| > | 0.0| 0.0| > | -0.0| -0.0| > | 1.0|0.6931471805599453| > | -1.0| null| > | NaN| NaN| > +-+--+ > > Note it also does the same for log1p, log2, log10 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-32640) Spark 3.1 log(NaN) returns null instead of NaN
[ https://issues.apache.org/jira/browse/SPARK-32640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-32640: -- Description: I was testing Spark 3.1.0 and I noticed that if you take the log(NaN) it now returns a null whereas in Spark 3.0 it returned a NaN. I'm not an expert in this but I thought NaN was correct. Spark 3.1.0 Example: >>> df.selectExpr(["value", "log1p(value)"]).show() +--+-+ | value| LOG1P(value)| +--+-+ |-3.4028235E38| null| |3.4028235E38|88.72283906194683| | 0.0| 0.0| | -0.0| -0.0| | 1.0|0.6931471805599453| | -1.0| null| | NaN| null| +--+-+ Spark 3.0.0 example: +-+--+ | value| LOG1P(value)| +-+--+ |-3.4028235E38| null| | 3.4028235E38| 88.72283906194683| | 0.0| 0.0| | -0.0| -0.0| | 1.0|0.6931471805599453| | -1.0| null| | NaN| NaN| +-+--+ Note it also does the same for log1p, log2, log10 was: I was testing Spark 3.1.0 and I noticed that if you take the log(NaN) it now returns a null whereas in Spark 3.0 it returned a NaN. I'm not an expert in this but I thought NaN was correct. Spark 3.1.0 Example: >>> df.selectExpr(["value", "log1p(value)"]).show() +-+--+ | value| LOG1P(value)| +-+--+ |-3.4028235E38| null| | 3.4028235E38| 88.72283906194683| | 0.0| 0.0| | -0.0| -0.0| | 1.0|0.6931471805599453| | -1.0| null| | NaN| null| +-+--+ Spark 3.0.0 example: +-+-+ | value| LOG(E(), value)| +-+-+ |-3.4028235E38| null| | 3.4028235E38|88.72283906194683| | 0.0| null| | -0.0| null| | 1.0| 0.0| | -1.0| null| | NaN| NaN| +-+-+ Note it also does the same for log1p, log2, log10 > Spark 3.1 log(NaN) returns null instead of NaN > -- > > Key: SPARK-32640 > URL: https://issues.apache.org/jira/browse/SPARK-32640 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.1.0 >Reporter: Thomas Graves >Priority: Major > > I was testing Spark 3.1.0 and I noticed that if you take the log(NaN) it now > returns a null whereas in Spark 3.0 it returned a NaN. I'm not an expert in > this but I thought NaN was correct. > Spark 3.1.0 Example: > >>> df.selectExpr(["value", "log1p(value)"]).show() > +--+-+ > | value| LOG1P(value)| > +--+-+ > |-3.4028235E38| null| > |3.4028235E38|88.72283906194683| > | 0.0| 0.0| > | -0.0| -0.0| > | 1.0|0.6931471805599453| > | -1.0| null| > | NaN| null| > +--+-+ > > Spark 3.0.0 example: > > +-+--+ > | value| LOG1P(value)| > +-+--+ > |-3.4028235E38| null| > | 3.4028235E38| 88.72283906194683| > | 0.0| 0.0| > | -0.0| -0.0| > | 1.0|0.6931471805599453| > | -1.0| null| > | NaN| NaN| > +-+--+ > > Note it also does the same for log1p, log2, log10 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
