Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21732#discussion_r228782536
--- Diff: docs/sql-migration-guide-upgrade.md ---
@@ -17,6 +17,8 @@ displayTitle: Spark SQL Upgrading Guide
- The `ADD JAR` command previously returned a result set with the single
value 0. It now returns an empty result set.
+ - In Spark version 2.4 and earlier, `Dataset` doesn't support to encode
`Option[Product]` at top-level row, because in Spark SQL entire top-level row
can't be null. Since Spark 3.0, `Option[Product]` at top-level is encoded as a
row with single struct column. Then with this support, `Aggregator` can also
use use `Option[Product]` as buffer and output column types.
--- End diff --
Usually we only add migration guide if something is broken and users must
be aware of it when upgrading.
I think this one is not the case? It's just a new feature.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
