Pablo Langa Blanco created SPARK-36453: ------------------------------------------
Summary: Improve consistency processing floating point special literals Key: SPARK-36453 URL: https://issues.apache.org/jira/browse/SPARK-36453 Project: Spark Issue Type: Improvement Components: SQL Affects Versions: 3.3.0 Reporter: Pablo Langa Blanco Special literal in floating point are not consistent between cast and json expressions {code:java} scala> spark.sql("SELECT CAST('+Inf' as Double)").show +--------------------+ |CAST(+Inf AS DOUBLE)| +--------------------+ | Infinity| +--------------------+ {code} {code:java} scala> val schema = StructType(StructField("a", DoubleType) :: Nil) scala> Seq("""{"a" : "+Inf"}""").toDF("col1").select(from_json(col("col1"),schema)).show +---------------+ |from_json(col1)| +---------------+ | {null}| +---------------+ scala> Seq("""{"a" : "+Inf"}""").toDF("col").withColumn("col", from_json(col("col"), StructType.fromDDL("a DOUBLE"))).write.json("/tmp/jsontests12345") scala> spark.read.schema(StructType(Seq(StructField("col",schema)))).json("/tmp/jsontests12345").show +------+ | col| +------+ |{null}| +------+ {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org