[jira] [Commented] (SPARK-36453) Improve consistency processing floating point special literals

2021-08-15 Thread Apache Spark (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-36453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17399417#comment-17399417
 ] 

Apache Spark commented on SPARK-36453:
--

User 'planga82' has created a pull request for this issue:
https://github.com/apache/spark/pull/33747

> Improve consistency processing floating point special literals
> --
>
> Key: SPARK-36453
> URL: https://issues.apache.org/jira/browse/SPARK-36453
> Project: Spark
>  Issue Type: Improvement
>  Components: SQL
>Affects Versions: 3.3.0
>Reporter: Pablo Langa Blanco
>Priority: Minor
>
> Special literal in floating point are not consistent between cast and json 
> expressions
>  
> {code:java}
> scala> spark.sql("SELECT CAST('+Inf' as Double)").show
> ++
> |CAST(+Inf AS DOUBLE)|
> ++
> |            Infinity|
> ++
> {code}
>  
> {code:java}
> scala> val schema =  StructType(StructField("a", DoubleType) :: Nil)
> scala> Seq("""{"a" : 
> "+Inf"}""").toDF("col1").select(from_json(col("col1"),schema)).show
> +---+
> |from_json(col1)|
> +---+
> |         {null}|
> +---+
> scala> Seq("""{"a" : "+Inf"}""").toDF("col").withColumn("col", 
> from_json(col("col"), StructType.fromDDL("a 
> DOUBLE"))).write.json("/tmp/jsontests12345")
> scala> 
> spark.read.schema(StructType(Seq(StructField("col",schema.json("/tmp/jsontests12345").show
> +--+
> |   col|
> +--+
> |{null}|
> +--+
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-36453) Improve consistency processing floating point special literals

2021-08-08 Thread Pablo Langa Blanco (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-36453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17395658#comment-17395658
 ] 

Pablo Langa Blanco commented on SPARK-36453:


I'm working on it

> Improve consistency processing floating point special literals
> --
>
> Key: SPARK-36453
> URL: https://issues.apache.org/jira/browse/SPARK-36453
> Project: Spark
>  Issue Type: Improvement
>  Components: SQL
>Affects Versions: 3.3.0
>Reporter: Pablo Langa Blanco
>Priority: Minor
>
> Special literal in floating point are not consistent between cast and json 
> expressions
>  
> {code:java}
> scala> spark.sql("SELECT CAST('+Inf' as Double)").show
> ++
> |CAST(+Inf AS DOUBLE)|
> ++
> |            Infinity|
> ++
> {code}
>  
> {code:java}
> scala> val schema =  StructType(StructField("a", DoubleType) :: Nil)
> scala> Seq("""{"a" : 
> "+Inf"}""").toDF("col1").select(from_json(col("col1"),schema)).show
> +---+
> |from_json(col1)|
> +---+
> |         {null}|
> +---+
> scala> Seq("""{"a" : "+Inf"}""").toDF("col").withColumn("col", 
> from_json(col("col"), StructType.fromDDL("a 
> DOUBLE"))).write.json("/tmp/jsontests12345")
> scala> 
> spark.read.schema(StructType(Seq(StructField("col",schema.json("/tmp/jsontests12345").show
> +--+
> |   col|
> +--+
> |{null}|
> +--+
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org