[
https://issues.apache.org/jira/browse/SPARK-38408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17501008#comment-17501008
]
Erik Krogen commented on SPARK-38408:
-------------------------------------
Sounds very similar to what was done in SPARK-34378 for the Avro datasource
specifically. Is what you're discussing a general feature, or specific to a
datasource? I don't quite follow the use-case.
> Add option for partial nested field writes with nullable fields.
> ----------------------------------------------------------------
>
> Key: SPARK-38408
> URL: https://issues.apache.org/jira/browse/SPARK-38408
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.3.0
> Reporter: Holden Karau
> Priority: Major
>
> If a nested field has nullable entries a user shouldn't have to specify all
> of the fields, instead the nullable fields can have the default value of
> null. This will allow for more effective schema migration. We might want to
> put this behind a feature flag though.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
