Github user viirya commented on the issue: https://github.com/apache/spark/pull/21611 This affects the dataset with `Aggregator` of output type `Option`. For normal dataset schema, it does affect the following case: ```scala val ds = Seq(Some(1), None, null).toDS() ds.show ds.printSchema ``` Before: ``` [info] If the schema is inferred from a Scala tuple/case class, or a Java bean, please try to use scala.Option[_] or other nullable types ( e.g. java.lang.Integer instead of int/scala.Int). ``` After: ```scala +-----+ |value| +-----+ | 1| | null| | null| +-----+ root |-- value: integer (nullable = true) ``` This dataset's schema is a row with a nullable integer. I feel that it should be allowed to have null.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org