[jira] Updated: (HIVE-279) Implement predicate push down for hive queries
[ https://issues.apache.org/jira/browse/HIVE-279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Thusoo updated HIVE-279: --- Resolution: Fixed Status: Resolved (was: Patch Available) committed. Thanks Prasad!! Implement predicate push down for hive queries -- Key: HIVE-279 URL: https://issues.apache.org/jira/browse/HIVE-279 Project: Hadoop Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.2.0 Reporter: Prasad Chakka Assignee: Prasad Chakka Fix For: 0.4.0 Attachments: hive-279.2.patch, hive-279.3.patch, hive-279.4.patch, hive-279.5.patch, hive-279.6.patch, hive-279.patch Push predicates that are expressed in outer queries into inner queries where possible so that rows will get filtered out sooner. eg. select a.*, b.* from a join b on (a.uid = b.uid) where a.age = 20 and a.gender = 'm' current compiler generates the filter predicate in the reducer after the join so all the rows have to be passed from mapper to reducer. by pushing the filter predicate to the mapper, query performance should improve. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-279) Implement predicate push down for hive queries
[ https://issues.apache.org/jira/browse/HIVE-279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Chakka updated HIVE-279: --- Fix Version/s: 0.4.0 Hadoop Flags: [Reviewed] Status: Patch Available (was: Open) Implement predicate push down for hive queries -- Key: HIVE-279 URL: https://issues.apache.org/jira/browse/HIVE-279 Project: Hadoop Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.2.0 Reporter: Prasad Chakka Assignee: Prasad Chakka Fix For: 0.4.0 Attachments: hive-279.2.patch, hive-279.3.patch, hive-279.4.patch, hive-279.5.patch, hive-279.6.patch, hive-279.patch Push predicates that are expressed in outer queries into inner queries where possible so that rows will get filtered out sooner. eg. select a.*, b.* from a join b on (a.uid = b.uid) where a.age = 20 and a.gender = 'm' current compiler generates the filter predicate in the reducer after the join so all the rows have to be passed from mapper to reducer. by pushing the filter predicate to the mapper, query performance should improve. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-279) Implement predicate push down for hive queries
[ https://issues.apache.org/jira/browse/HIVE-279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Chakka updated HIVE-279: --- Attachment: hive-279.6.patch added following tests 1) multi insert (which doesn't push predicates) 2) query containing left and right outer joins This patch is intended to be committed. Implement predicate push down for hive queries -- Key: HIVE-279 URL: https://issues.apache.org/jira/browse/HIVE-279 Project: Hadoop Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.2.0 Reporter: Prasad Chakka Assignee: Prasad Chakka Fix For: 0.4.0 Attachments: hive-279.2.patch, hive-279.3.patch, hive-279.4.patch, hive-279.5.patch, hive-279.6.patch, hive-279.patch Push predicates that are expressed in outer queries into inner queries where possible so that rows will get filtered out sooner. eg. select a.*, b.* from a join b on (a.uid = b.uid) where a.age = 20 and a.gender = 'm' current compiler generates the filter predicate in the reducer after the join so all the rows have to be passed from mapper to reducer. by pushing the filter predicate to the mapper, query performance should improve. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-279) Implement predicate push down for hive queries
[ https://issues.apache.org/jira/browse/HIVE-279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Chakka updated HIVE-279: --- Attachment: hive-279.5.patch updated patch incorporating Namit's comments Implement predicate push down for hive queries -- Key: HIVE-279 URL: https://issues.apache.org/jira/browse/HIVE-279 Project: Hadoop Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.2.0 Reporter: Prasad Chakka Assignee: Prasad Chakka Attachments: hive-279.2.patch, hive-279.3.patch, hive-279.4.patch, hive-279.5.patch, hive-279.patch Push predicates that are expressed in outer queries into inner queries where possible so that rows will get filtered out sooner. eg. select a.*, b.* from a join b on (a.uid = b.uid) where a.age = 20 and a.gender = 'm' current compiler generates the filter predicate in the reducer after the join so all the rows have to be passed from mapper to reducer. by pushing the filter predicate to the mapper, query performance should improve. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-279) Implement predicate push down for hive queries
[ https://issues.apache.org/jira/browse/HIVE-279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Chakka updated HIVE-279: --- Attachment: hive-279.4.patch updated the patch to the latest trunk Implement predicate push down for hive queries -- Key: HIVE-279 URL: https://issues.apache.org/jira/browse/HIVE-279 Project: Hadoop Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.2.0 Reporter: Prasad Chakka Assignee: Prasad Chakka Attachments: hive-279.2.patch, hive-279.3.patch, hive-279.4.patch, hive-279.patch Push predicates that are expressed in outer queries into inner queries where possible so that rows will get filtered out sooner. eg. select a.*, b.* from a join b on (a.uid = b.uid) where a.age = 20 and a.gender = 'm' current compiler generates the filter predicate in the reducer after the join so all the rows have to be passed from mapper to reducer. by pushing the filter predicate to the mapper, query performance should improve. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-279) Implement predicate push down for hive queries
[ https://issues.apache.org/jira/browse/HIVE-279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Chakka updated HIVE-279: --- Attachment: hive-279.3.patch incorporated comments from Namit and added a bunch of new testcases. due to changes to exprNodeDesc, the outputs of all parse unit tests have changed. I will upload them in couple of days. otherwise there are no pending code changes for this JIRA. will open jiras for the following further optimizatons 1) mult-insert 2) intermediate filter operators for partial pushdown 3) remove pushed preds from original operator to eliminate repeat evaluation. 4) merge successive filter ops into single op Implement predicate push down for hive queries -- Key: HIVE-279 URL: https://issues.apache.org/jira/browse/HIVE-279 Project: Hadoop Hive Issue Type: New Feature Affects Versions: 0.2.0 Reporter: Prasad Chakka Assignee: Prasad Chakka Attachments: hive-279.2.patch, hive-279.3.patch, hive-279.patch Push predicates that are expressed in outer queries into inner queries where possible so that rows will get filtered out sooner. eg. select a.*, b.* from a join b on (a.uid = b.uid) where a.age = 20 and a.gender = 'm' current compiler generates the filter predicate in the reducer after the join so all the rows have to be passed from mapper to reducer. by pushing the filter predicate to the mapper, query performance should improve. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-279) Implement predicate push down for hive queries
[ https://issues.apache.org/jira/browse/HIVE-279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Chakka updated HIVE-279: --- Attachment: hive-279.2.patch Here is an updated patch that simplifies some of the column reference conversions. Now every operator converts the column references in the predicate expressions so that they will be understood by the parent operator(s) Implement predicate push down for hive queries -- Key: HIVE-279 URL: https://issues.apache.org/jira/browse/HIVE-279 Project: Hadoop Hive Issue Type: New Feature Affects Versions: 0.2.0 Reporter: Prasad Chakka Assignee: Prasad Chakka Attachments: hive-279.2.patch, hive-279.patch Push predicates that are expressed in outer queries into inner queries where possible so that rows will get filtered out sooner. eg. select a.*, b.* from a join b on (a.uid = b.uid) where a.age = 20 and a.gender = 'm' current compiler generates the filter predicate in the reducer after the join so all the rows have to be passed from mapper to reducer. by pushing the filter predicate to the mapper, query performance should improve. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-279) Implement predicate push down for hive queries
[ https://issues.apache.org/jira/browse/HIVE-279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Chakka updated HIVE-279: --- Attachment: hive-279.patch this is a drop for initial review since i suspect there will be lot of comments :). it should work for all cases except for multi-insert queries. i have not enabled this by default but added a new config param called hive.optimize.ppd to enable this feature. i have not modified existing testcases but added couple of new testcases. will add more while uploading final patch. Implement predicate push down for hive queries -- Key: HIVE-279 URL: https://issues.apache.org/jira/browse/HIVE-279 Project: Hadoop Hive Issue Type: New Feature Affects Versions: 0.2.0 Reporter: Prasad Chakka Assignee: Prasad Chakka Attachments: hive-279.patch Push predicates that are expressed in outer queries into inner queries where possible so that rows will get filtered out sooner. eg. select a.*, b.* from a join b on (a.uid = b.uid) where a.age = 20 and a.gender = 'm' current compiler generates the filter predicate in the reducer after the join so all the rows have to be passed from mapper to reducer. by pushing the filter predicate to the mapper, query performance should improve. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
