Yu Xu created PIG-2749:
--------------------------
Summary: Keep predicates in PIG after they have been pushed down
to give data sources more flexibility and enable more optimization
Key: PIG-2749
URL: https://issues.apache.org/jira/browse/PIG-2749
Project: Pig
Issue Type: Improvement
Reporter: Yu Xu
Priority: Minor
Currently PIG removes any predicate it has pushed down to data sources (via
LoadMetaData).
This JIRA ticket is created to request PIG optimizer to keep all predicates it
has pushed down so that:
1) the underlying data sources have more flexibility/opportunities to do
projections/indexed lookups. The only requirement for the underlying data
sources is that what they return should be be superset of the correct results.
As the result of the new flexibility, the underlying data sources can return
more results than the exact predicates pushed down would return (for example
using approximate predicate filtering), but still more efficient than full
table/file scan.
2) it's easier for programming correction verification since PIG would still
apply all predicates.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
