Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21882#discussion_r205518092 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala --- @@ -183,6 +183,13 @@ case class InMemoryTableScanExec( private val stats = relation.partitionStatistics private def statsFor(a: Attribute) = stats.forAttribute(a) + // For some ColumnStats, for instance, ObjectColumnStats always has nulls for lower and upper + // bounds. + private def nullSafeEval( + attr: AttributeReference)(func: AttributeReference => Expression): Expression = { + attr.isNull || func(attr) --- End diff -- hmmmmm .. if this can whitelist the cases we support, I thought it's okay to use the suggestion above. BTW, looks we should exclude binary type too. It will still supports itnull or isnotnull though.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org