[
https://issues.apache.org/jira/browse/HAWQ-583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15397931#comment-15397931
]
Kavinder Dhaliwal commented on HAWQ-583:
----------------------------------------
This is being implemented for ORC files via HAWQ-886
> Extend PXF to allow plugins to support returning partial content of
> SELECT(column projection)
> ---------------------------------------------------------------------------------------------
>
> Key: HAWQ-583
> URL: https://issues.apache.org/jira/browse/HAWQ-583
> Project: Apache HAWQ
> Issue Type: New Feature
> Components: PXF
> Reporter: Michael Andre Pearce (IG)
> Assignee: Kavinder Dhaliwal
> Fix For: backlog
>
>
> Currently PXF supports being able to push down the predicate WHERE logic to
> the external system to reduce the amount data needed to be retrieved.
> SELECT a, b FROM external_pxf_source WHERE z < 3 AND x > 6
> As such we can filter the rows returned, but currently still would have to
> return all the fields / complete row.
> This proposal is so that we can return only the columns in SELECT part.
> For data sources where it is columnar storage or selectable such as remote
> database that PXF can read or connect to this has advantages in the data that
> needs to be accessed or even transferred.
> As like with the push down Filter it should be optional so that plugins that
> provide support can use it but others that do not, continue to work as they
> do.
> The proposal would be for
> 1) create an interface created for plugins to optionally implement, where the
> columns needed to be returned are given to the plugin.
> 2) update pxf api for hawq to send columns defined in SELECT, for pxf to
> invoke the plugin interface and pass this information onto if provided
> 3) update pxf integration within hawq itself so that hawq passes this
> additonal information to pxf.
> This Ticket is off the back of discussion on HAWQ-492.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
