[ https://issues.apache.org/jira/browse/SPARK-40646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Sadikov updated SPARK-40646: --------------------------------- Description: I recently found an issue when parsing the following JSON file: {code:java} {"a": {"x": 1, "y": true}, "b": {"x": 1}} {"a": {"x": 2}, "b": {"x": 2}}{code} Trying to read such table with fixed schema where {{y}} is a struct column and not a boolean: {code:java} val df = spark.read .schema("a struct<x: int, y: struct<x: int>>, b struct<x: int>") .json("path") {code} results in the following answer: {code:java} a b null null {"x":2,"y":null} {"x":2} {code} > Fix returning partial results in JSON data source and JSON functions > -------------------------------------------------------------------- > > Key: SPARK-40646 > URL: https://issues.apache.org/jira/browse/SPARK-40646 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 3.4.0 > Reporter: Ivan Sadikov > Priority: Major > > I recently found an issue when parsing the following JSON file: > {code:java} > {"a": {"x": 1, "y": true}, "b": {"x": 1}} > {"a": {"x": 2}, "b": {"x": 2}}{code} > Trying to read such table with fixed schema where {{y}} is a struct column > and not a boolean: > {code:java} > val df = spark.read > .schema("a struct<x: int, y: struct<x: int>>, b struct<x: int>") > .json("path") {code} > results in the following answer: > {code:java} > a b > null null > {"x":2,"y":null} {"x":2} {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org