Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19494#discussion_r145637774 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala --- @@ -102,6 +102,7 @@ case class InMemoryTableScanExec( case IsNull(a: Attribute) => statsFor(a).nullCount > 0 case IsNotNull(a: Attribute) => statsFor(a).count - statsFor(a).nullCount > 0 + case In(_: AttributeReference, list: Seq[Expression]) if list.isEmpty => Literal.FalseLiteral --- End diff -- Under this special context of filtering partitions, this `In` with empty list will result in a false literal in the end, no matter the attribute is null or not. We don't possibly have some expressions like `IsNull(In(a, Nil))` as the filter predicate for now.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org