[
https://issues.apache.org/jira/browse/IMPALA-7141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16774451#comment-16774451
]
ASF subversion and git services commented on IMPALA-7141:
---------------------------------------------------------
Commit e83dd19fc91b316466869d439770b1c8686842ed in impala's branch
refs/heads/2.x from Todd Lipcon
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=e83dd19 ]
IMPALA-7141 (part 2). Extract interfaces for partition pruning without loading
This adds a new PrunablePartition interface which HdfsPartition
implements. The interface includes only the partition ID and the
partition key/values.
For the case of the current catalog implementation, this provides no
benefit. However, for LocalCatalog, we want to defer loading partition
information until after pruning. With this interface, we can construct
PrunablePartition objects using just the partition names, and then load
the partitions once pruning is complete.
Change-Id: Ifdcfd6cffcd298bbf44531e1ec2f47c3a5b7d1fa
Reviewed-on: http://gerrit.cloudera.org:8080/10631
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Reviewed-by: Todd Lipcon <t...@apache.org>
> Extract interfaces for partition pruning prior to fetching partitions
> ---------------------------------------------------------------------
>
> Key: IMPALA-7141
> URL: https://issues.apache.org/jira/browse/IMPALA-7141
> Project: IMPALA
> Issue Type: Sub-task
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Major
> Fix For: Impala 3.1.0
>
>
> In the LocalCatalog, we want to only fetch the partitions that are referenced
> by a query -- i.e. we must prune partitions based only on the partition names
> and not the entire partition objects. However, the PartitionPruner
> implementation currently expect to be able to fetch the full map of
> HdfsPartition objects from the table and work on them as is.
> This JIRA is to do some refactorings such that the PartitionPruner interacts
> with a slightly more restricted interface that only exposes the minimal
> interaction points with the table and the partition map. Once it has computed
> a list of remaining partitions, it can then instruct the table to fully load
> them to yield the resulting full Partition objects.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
