Kousuke Saruta created SPARK-35374:
--------------------------------------
Summary: Add string-to-number conversion support to JacksonParser
Key: SPARK-35374
URL: https://issues.apache.org/jira/browse/SPARK-35374
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.1.1, 2.4.7, 3.2.0
Reporter: Kousuke Saruta
Assignee: Kousuke Saruta
In the current implementation, spark.read.json doesn't convert numbers
represented as string even though types are specified by schema.
Here is anexample.
{code}
$ cat test.json
{ "value": "foo" }
{ "value": "100" }
{ "value": "257" }
{ "value": "32768" }
{ "value": "2147483648" }
{ "value": "2.71f" }
{ "value": "1.0E100" }
$ bin/spark-shell
import org.apache.spark.sql.types._
val df = spark.read.
schema(StructType(StructField("value", DoubleType)::Nil)).json("test.json")
df.show
+-----+
|value|
+-----+
| null|
| null|
| null|
| null|
| null|
| null|
| null|
+-----+
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
