Adrian Ionescu created SPARK-20194: -------------------------------------- Summary: Support partition pruning for InMemoryCatalog Key: SPARK-20194 URL: https://issues.apache.org/jira/browse/SPARK-20194 Project: Spark Issue Type: Improvement Components: Optimizer Affects Versions: 2.1.0 Reporter: Adrian Ionescu
{{listPartitionsByFilter()}} is not yet implemented for {{InMemoryCatalog}}: {quote} // TODO: Provide an implementation throw new UnsupportedOperationException( "listPartitionsByFilter is not implemented for InMemoryCatalog") {quote} Because of this, there is a hack in {{FindDataSourceTable}} that avoids passing along the {{CatalogTable}} to the {{DataSource}} it creates when the catalog implementation is not "hive", so that, when the latter is resolved, an {{InMemoryFileIndex}} is created instead of a {{CatalogFileIndex}} which the {{PruneFileSourcePartitions}} rule matches for. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org