Patrick Woody created SPARK-17170: ------------------------------------- Summary: Enable whole partition pruning for InMemoryTableScanExec Key: SPARK-17170 URL: https://issues.apache.org/jira/browse/SPARK-17170 Project: Spark Issue Type: Improvement Components: SQL Affects Versions: 2.0.0 Reporter: Patrick Woody
Currently InMemoryTableScanExec will prune cached batches executor side, possibly pruning the entire partition in the process. We should be able to leverage the same stats to determine if an entire partition can be removed up front and avoid launching a task. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org