gengliangwang commented on a change in pull request #23639: [SPARK-26716][SQL]
FileFormat: the supported types of read/write should be consistent
URL: https://github.com/apache/spark/pull/23639#discussion_r250886350
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonGenerator.scala
##########
@@ -178,7 +178,7 @@ private[sql] class JacksonGenerator(
var i = 0
while (i < row.numFields) {
val field = schema(i)
- if (!row.isNullAt(i)) {
+ if (!row.isNullAt(i) || field.dataType == NullType) {
Review comment:
Overall this is trivial. I think we should choose either way:
1. support output NullType column and read it back. When one field is all
nulls, we will have to provide schema to specify that it is NullType. The main
point is about outputting the null column.
2. Keep the current behavior.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
