[
https://issues.apache.org/jira/browse/SPARK-40820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anthony Wainer Cachay Guivin updated SPARK-40820:
-------------------------------------------------
Attachment: bug .png
> creating StructType from Json
> -----------------------------
>
> Key: SPARK-40820
> URL: https://issues.apache.org/jira/browse/SPARK-40820
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 3.3.0
> Reporter: Anthony Wainer Cachay Guivin
> Priority: Minor
> Attachments: bug .png
>
>
> When create a StructType from a Python dictionary you utilize the
> [StructType.fromJson|https://github.com/apache/spark/blob/master/python/pyspark/sql/types.py#L569-L571]
> method.
> To create a schema can be created as follows from the code below, but it
> requires to put inside the json: Nullable and Metadata, this is inconsistent
> because within the DataType class this by default.
> {code:python}
> json = {
> "name": "name",
> "type": "string"
> }
> StructField.fromJson(json)
> {code}
> Error:
> !image-2022-10-17-15-52-21-787.png!
>
> Proposed coding solution:
> Instead use indexes for getting from a dictionary, it would be better to use
> .get
> {code:python}
> def fromJson(cls, json: Dict[str, Any]) -> "StructField":
> return StructField(
> json["name"],
> _parse_datatype_json_value(json["type"]),
> json.get("nullable"),
> json.get("metadata"),
> )
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
