On Mon, Jul 4, 2016 at 6:14 AM, <wenc...@apache.org> wrote:
> Repository: spark
> Updated Branches:
> refs/heads/master 88134e736 -> 8cdb81fa8
>
>
> [SPARK-15204][SQL] improve nullability inference for Aggregator
>
> ## What changes were proposed in this pull request?
>
> TypedAggregateExpression sets nullable based on the schema of the
> outputEncoder
>
> ## How was this patch tested?
>
> Add test in DatasetAggregatorSuite
>
> Author: Koert Kuipers <ko...@tresata.com>
...
> + assert(ds1.select(typed.sum((i: Int) => i)).schema.head.nullable ===
> false)
> + val ds2 = Seq(AggData(1, "a"), AggData(2, "a")).toDS()
> + assert(ds2.select(SeqAgg.toColumn).schema.head.nullable === true)
> + val ds3 = sql("SELECT 'Some String' AS b, 1279869254 AS a").as[AggData]
> + assert(ds3.select(NameAgg.toColumn).schema.head.nullable === true)
Why do we assert predicates? If it's true, it's true already (no need
to compare whether it's true or not). I'd vote to "fix" it.
Jacek
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
