Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19494#discussion_r144621674 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala --- @@ -104,7 +104,8 @@ case class InMemoryTableScanExec( case In(a: AttributeReference, list: Seq[Expression]) if list.forall(_.isInstanceOf[Literal]) => list.map(l => statsFor(a).lowerBound <= l.asInstanceOf[Literal] && - l.asInstanceOf[Literal] <= statsFor(a).upperBound).reduce(_ || _) + l.asInstanceOf[Literal] <= statsFor(a).upperBound) --- End diff -- This still looks more complex and less efficient than ``` list.exists(l => statsFor(a).lowerBound <= l.asInstanceOf[Literal] && l.asInstanceOf[Literal] <= statsFor(a).upperBound) ``` or better ``` val stats = statsFor(a) list.exists { l => val literal = l.asInstanceOf[Literal] stats.lowerBound <= literal && literal <= stats.upperBound } ``` The point being that you should be able to short-circuit evaluation here. Or have I missed something basic like that these aren't Booleans?
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org