[jira] [Updated] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
[ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-3996: --- Fix Version/s: 0.11.0 > Correctly enforce the memory limit on the multi-table map-join > -- > > Key: HIVE-3996 > URL: https://issues.apache.org/jira/browse/HIVE-3996 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Fix For: 0.11.0 > > Attachments: HIVE-3996_2.patch, HIVE-3996_3.patch, HIVE-3996_4.patch, > HIVE-3996_5.patch, HIVE-3996_6.patch, HIVE-3996_7.patch, HIVE-3996_8.patch, > HIVE-3996_9.patch, hive.3996.9.patch-nohcat, HIVE-3996.patch > > > Currently with HIVE-3784, the joins are converted to map-joins based on > checks of the table size against the config variable: > hive.auto.convert.join.noconditionaltask.size. > However, the current implementation will also merge multiple mapjoin > operators into a single task regardless of whether the sum of the table sizes > will exceed the configured value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
[ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3996: - Fix Version/s: (was: 0.11.0) > Correctly enforce the memory limit on the multi-table map-join > -- > > Key: HIVE-3996 > URL: https://issues.apache.org/jira/browse/HIVE-3996 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-3996_2.patch, HIVE-3996_3.patch, HIVE-3996_4.patch, > HIVE-3996_5.patch, HIVE-3996_6.patch, HIVE-3996_7.patch, HIVE-3996_8.patch, > HIVE-3996_9.patch, hive.3996.9.patch-nohcat, HIVE-3996.patch > > > Currently with HIVE-3784, the joins are converted to map-joins based on > checks of the table size against the config variable: > hive.auto.convert.join.noconditionaltask.size. > However, the current implementation will also merge multiple mapjoin > operators into a single task regardless of whether the sum of the table sizes > will exceed the configured value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
[ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3996: - Resolution: Fixed Fix Version/s: 0.11.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed. Thanks Vikram > Correctly enforce the memory limit on the multi-table map-join > -- > > Key: HIVE-3996 > URL: https://issues.apache.org/jira/browse/HIVE-3996 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Fix For: 0.11.0 > > Attachments: HIVE-3996_2.patch, HIVE-3996_3.patch, HIVE-3996_4.patch, > HIVE-3996_5.patch, HIVE-3996_6.patch, HIVE-3996_7.patch, HIVE-3996_8.patch, > HIVE-3996_9.patch, hive.3996.9.patch-nohcat, HIVE-3996.patch > > > Currently with HIVE-3784, the joins are converted to map-joins based on > checks of the table size against the config variable: > hive.auto.convert.join.noconditionaltask.size. > However, the current implementation will also merge multiple mapjoin > operators into a single task regardless of whether the sum of the table sizes > will exceed the configured value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
[ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3996: - Attachment: hive.3996.9.patch-nohcat > Correctly enforce the memory limit on the multi-table map-join > -- > > Key: HIVE-3996 > URL: https://issues.apache.org/jira/browse/HIVE-3996 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-3996_2.patch, HIVE-3996_3.patch, HIVE-3996_4.patch, > HIVE-3996_5.patch, HIVE-3996_6.patch, HIVE-3996_7.patch, HIVE-3996_8.patch, > HIVE-3996_9.patch, hive.3996.9.patch-nohcat, HIVE-3996.patch > > > Currently with HIVE-3784, the joins are converted to map-joins based on > checks of the table size against the config variable: > hive.auto.convert.join.noconditionaltask.size. > However, the current implementation will also merge multiple mapjoin > operators into a single task regardless of whether the sum of the table sizes > will exceed the configured value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
[ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-3996: - Status: Patch Available (was: Open) > Correctly enforce the memory limit on the multi-table map-join > -- > > Key: HIVE-3996 > URL: https://issues.apache.org/jira/browse/HIVE-3996 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-3996_2.patch, HIVE-3996_3.patch, HIVE-3996_4.patch, > HIVE-3996_5.patch, HIVE-3996_6.patch, HIVE-3996_7.patch, HIVE-3996_8.patch, > HIVE-3996_9.patch, HIVE-3996.patch > > > Currently with HIVE-3784, the joins are converted to map-joins based on > checks of the table size against the config variable: > hive.auto.convert.join.noconditionaltask.size. > However, the current implementation will also merge multiple mapjoin > operators into a single task regardless of whether the sum of the table sizes > will exceed the configured value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
[ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-3996: - Attachment: HIVE-3996_9.patch Addressed Namit's comments. > Correctly enforce the memory limit on the multi-table map-join > -- > > Key: HIVE-3996 > URL: https://issues.apache.org/jira/browse/HIVE-3996 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-3996_2.patch, HIVE-3996_3.patch, HIVE-3996_4.patch, > HIVE-3996_5.patch, HIVE-3996_6.patch, HIVE-3996_7.patch, HIVE-3996_8.patch, > HIVE-3996_9.patch, HIVE-3996.patch > > > Currently with HIVE-3784, the joins are converted to map-joins based on > checks of the table size against the config variable: > hive.auto.convert.join.noconditionaltask.size. > However, the current implementation will also merge multiple mapjoin > operators into a single task regardless of whether the sum of the table sizes > will exceed the configured value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
[ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3996: - Status: Open (was: Patch Available) minor comments -- looks good mostly > Correctly enforce the memory limit on the multi-table map-join > -- > > Key: HIVE-3996 > URL: https://issues.apache.org/jira/browse/HIVE-3996 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-3996_2.patch, HIVE-3996_3.patch, HIVE-3996_4.patch, > HIVE-3996_5.patch, HIVE-3996_6.patch, HIVE-3996_7.patch, HIVE-3996_8.patch, > HIVE-3996.patch > > > Currently with HIVE-3784, the joins are converted to map-joins based on > checks of the table size against the config variable: > hive.auto.convert.join.noconditionaltask.size. > However, the current implementation will also merge multiple mapjoin > operators into a single task regardless of whether the sum of the table sizes > will exceed the configured value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
[ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-3996: - Attachment: HIVE-3996_8.patch Refreshed on top of trunk again. [~namit] could you take a look. Thanks Vikram. > Correctly enforce the memory limit on the multi-table map-join > -- > > Key: HIVE-3996 > URL: https://issues.apache.org/jira/browse/HIVE-3996 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-3996_2.patch, HIVE-3996_3.patch, HIVE-3996_4.patch, > HIVE-3996_5.patch, HIVE-3996_6.patch, HIVE-3996_7.patch, HIVE-3996_8.patch, > HIVE-3996.patch > > > Currently with HIVE-3784, the joins are converted to map-joins based on > checks of the table size against the config variable: > hive.auto.convert.join.noconditionaltask.size. > However, the current implementation will also merge multiple mapjoin > operators into a single task regardless of whether the sum of the table sizes > will exceed the configured value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
[ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-3996: - Attachment: HIVE-3996_7.patch > Correctly enforce the memory limit on the multi-table map-join > -- > > Key: HIVE-3996 > URL: https://issues.apache.org/jira/browse/HIVE-3996 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-3996_2.patch, HIVE-3996_3.patch, HIVE-3996_4.patch, > HIVE-3996_5.patch, HIVE-3996_6.patch, HIVE-3996_7.patch, HIVE-3996.patch > > > Currently with HIVE-3784, the joins are converted to map-joins based on > checks of the table size against the config variable: > hive.auto.convert.join.noconditionaltask.size. > However, the current implementation will also merge multiple mapjoin > operators into a single task regardless of whether the sum of the table sizes > will exceed the configured value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
[ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-3996: - Status: Patch Available (was: Open) > Correctly enforce the memory limit on the multi-table map-join > -- > > Key: HIVE-3996 > URL: https://issues.apache.org/jira/browse/HIVE-3996 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-3996_2.patch, HIVE-3996_3.patch, HIVE-3996_4.patch, > HIVE-3996_5.patch, HIVE-3996_6.patch, HIVE-3996_7.patch, HIVE-3996.patch > > > Currently with HIVE-3784, the joins are converted to map-joins based on > checks of the table size against the config variable: > hive.auto.convert.join.noconditionaltask.size. > However, the current implementation will also merge multiple mapjoin > operators into a single task regardless of whether the sum of the table sizes > will exceed the configured value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
[ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3996: - Status: Open (was: Patch Available) Comments > Correctly enforce the memory limit on the multi-table map-join > -- > > Key: HIVE-3996 > URL: https://issues.apache.org/jira/browse/HIVE-3996 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-3996_2.patch, HIVE-3996_3.patch, HIVE-3996_4.patch, > HIVE-3996_5.patch, HIVE-3996_6.patch, HIVE-3996.patch > > > Currently with HIVE-3784, the joins are converted to map-joins based on > checks of the table size against the config variable: > hive.auto.convert.join.noconditionaltask.size. > However, the current implementation will also merge multiple mapjoin > operators into a single task regardless of whether the sum of the table sizes > will exceed the configured value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
[ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-3996: - Status: Patch Available (was: Open) > Correctly enforce the memory limit on the multi-table map-join > -- > > Key: HIVE-3996 > URL: https://issues.apache.org/jira/browse/HIVE-3996 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-3996_2.patch, HIVE-3996_3.patch, HIVE-3996_4.patch, > HIVE-3996_5.patch, HIVE-3996_6.patch, HIVE-3996.patch > > > Currently with HIVE-3784, the joins are converted to map-joins based on > checks of the table size against the config variable: > hive.auto.convert.join.noconditionaltask.size. > However, the current implementation will also merge multiple mapjoin > operators into a single task regardless of whether the sum of the table sizes > will exceed the configured value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
[ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-3996: - Attachment: HIVE-3996_6.patch Patch rebased on latest trunk. > Correctly enforce the memory limit on the multi-table map-join > -- > > Key: HIVE-3996 > URL: https://issues.apache.org/jira/browse/HIVE-3996 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-3996_2.patch, HIVE-3996_3.patch, HIVE-3996_4.patch, > HIVE-3996_5.patch, HIVE-3996_6.patch, HIVE-3996.patch > > > Currently with HIVE-3784, the joins are converted to map-joins based on > checks of the table size against the config variable: > hive.auto.convert.join.noconditionaltask.size. > However, the current implementation will also merge multiple mapjoin > operators into a single task regardless of whether the sum of the table sizes > will exceed the configured value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
[ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-3996: - Attachment: HIVE-3996_5.patch Added tests for the sub-query as pointed by Namit. This update should be applied after applying HIVE-3891. ~Namit, The commonJoinDispatcher is created new for every query. The only case where we can see an issue with the global presence of aliasToSize would be if we update a table and then join in a single query. I am not able to think of any such query. Let me know if there is a way. I will generate a review request once HIVE-3891 has been committed because currently this will not apply directly on trunk. Thanks Vikram. > Correctly enforce the memory limit on the multi-table map-join > -- > > Key: HIVE-3996 > URL: https://issues.apache.org/jira/browse/HIVE-3996 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-3996_2.patch, HIVE-3996_3.patch, HIVE-3996_4.patch, > HIVE-3996_5.patch, HIVE-3996.patch > > > Currently with HIVE-3784, the joins are converted to map-joins based on > checks of the table size against the config variable: > hive.auto.convert.join.noconditionaltask.size. > However, the current implementation will also merge multiple mapjoin > operators into a single task regardless of whether the sum of the table sizes > will exceed the configured value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
[ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-3996: - Attachment: HIVE-3996_4.patch > Correctly enforce the memory limit on the multi-table map-join > -- > > Key: HIVE-3996 > URL: https://issues.apache.org/jira/browse/HIVE-3996 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-3996_2.patch, HIVE-3996_3.patch, HIVE-3996_4.patch, > HIVE-3996.patch > > > Currently with HIVE-3784, the joins are converted to map-joins based on > checks of the table size against the config variable: > hive.auto.convert.join.noconditionaltask.size. > However, the current implementation will also merge multiple mapjoin > operators into a single task regardless of whether the sum of the table sizes > will exceed the configured value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
[ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3996: - Status: Open (was: Patch Available) comments > Correctly enforce the memory limit on the multi-table map-join > -- > > Key: HIVE-3996 > URL: https://issues.apache.org/jira/browse/HIVE-3996 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-3996_2.patch, HIVE-3996_3.patch, HIVE-3996.patch > > > Currently with HIVE-3784, the joins are converted to map-joins based on > checks of the table size against the config variable: > hive.auto.convert.join.noconditionaltask.size. > However, the current implementation will also merge multiple mapjoin > operators into a single task regardless of whether the sum of the table sizes > will exceed the configured value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
[ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-3996: - Status: Patch Available (was: Open) > Correctly enforce the memory limit on the multi-table map-join > -- > > Key: HIVE-3996 > URL: https://issues.apache.org/jira/browse/HIVE-3996 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-3996_2.patch, HIVE-3996_3.patch, HIVE-3996.patch > > > Currently with HIVE-3784, the joins are converted to map-joins based on > checks of the table size against the config variable: > hive.auto.convert.join.noconditionaltask.size. > However, the current implementation will also merge multiple mapjoin > operators into a single task regardless of whether the sum of the table sizes > will exceed the configured value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
[ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-3996: - Attachment: HIVE-3996_3.patch Added a test case that demonstrates the issue when combining map-joins. This is an almost exact replica of the join32.q test with the size altered but, current code would generate the same plan as join32.q when the sum of the sizes of the tables would exceed the size configured by noConditionalTask.size. > Correctly enforce the memory limit on the multi-table map-join > -- > > Key: HIVE-3996 > URL: https://issues.apache.org/jira/browse/HIVE-3996 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-3996_2.patch, HIVE-3996_3.patch, HIVE-3996.patch > > > Currently with HIVE-3784, the joins are converted to map-joins based on > checks of the table size against the config variable: > hive.auto.convert.join.noconditionaltask.size. > However, the current implementation will also merge multiple mapjoin > operators into a single task regardless of whether the sum of the table sizes > will exceed the configured value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
[ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3996: - Status: Open (was: Patch Available) > Correctly enforce the memory limit on the multi-table map-join > -- > > Key: HIVE-3996 > URL: https://issues.apache.org/jira/browse/HIVE-3996 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-3996_2.patch, HIVE-3996.patch > > > Currently with HIVE-3784, the joins are converted to map-joins based on > checks of the table size against the config variable: > hive.auto.convert.join.noconditionaltask.size. > However, the current implementation will also merge multiple mapjoin > operators into a single task regardless of whether the sum of the table sizes > will exceed the configured value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
[ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-3996: - Status: Patch Available (was: Open) > Correctly enforce the memory limit on the multi-table map-join > -- > > Key: HIVE-3996 > URL: https://issues.apache.org/jira/browse/HIVE-3996 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-3996_2.patch, HIVE-3996.patch > > > Currently with HIVE-3784, the joins are converted to map-joins based on > checks of the table size against the config variable: > hive.auto.convert.join.noconditionaltask.size. > However, the current implementation will also merge multiple mapjoin > operators into a single task regardless of whether the sum of the table sizes > will exceed the configured value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
[ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-3996: - Attachment: HIVE-3996_2.patch Updated patch which improves existing tests with my patch. > Correctly enforce the memory limit on the multi-table map-join > -- > > Key: HIVE-3996 > URL: https://issues.apache.org/jira/browse/HIVE-3996 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-3996_2.patch, HIVE-3996.patch > > > Currently with HIVE-3784, the joins are converted to map-joins based on > checks of the table size against the config variable: > hive.auto.convert.join.noconditionaltask.size. > However, the current implementation will also merge multiple mapjoin > operators into a single task regardless of whether the sum of the table sizes > will exceed the configured value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
[ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-3996: - Attachment: HIVE-3996.patch This is the code changes only. Am adding tests and running the full suite of unit tests. > Correctly enforce the memory limit on the multi-table map-join > -- > > Key: HIVE-3996 > URL: https://issues.apache.org/jira/browse/HIVE-3996 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-3996.patch > > > Currently with HIVE-3784, the joins are converted to map-joins based on > checks of the table size against the config variable: > hive.auto.convert.join.noconditionaltask.size. > However, the current implementation will also merge multiple mapjoin > operators into a single task regardless of whether the sum of the table sizes > will exceed the configured value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
