Jackey Lee created SPARK-29444:
----------------------------------
Summary: Add configuration to support JacksonGenrator to keep
fields with null values
Key: SPARK-29444
URL: https://issues.apache.org/jira/browse/SPARK-29444
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.0.0
Reporter: Jackey Lee
DataSet.toJSON will lost some column when field data is null. Maybe it is
better to keep null data in some scenarios.
Such as sparkmagic, which is widely used in jupyter with livy, we use toJSON to
toJSON is used to get sql results. with toJSON sparkmagic may return empty
results, which confused users.
Maybe adding a config is the best choice. This configuration retains the
current semantics and will remain fields with null until the configuration is
modified.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
