[jira] [Updated] (HIVE-15530) Optimize the column stats update logic in table alteration
[ https://issues.apache.org/jira/browse/HIVE-15530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Micah Whitacre updated HIVE-15530: -- Component/s: Metastore > Optimize the column stats update logic in table alteration > -- > > Key: HIVE-15530 > URL: https://issues.apache.org/jira/browse/HIVE-15530 > Project: Hive > Issue Type: Bug > Components: Hive, Metastore >Reporter: Yibing Shi >Assignee: Yibing Shi > Fix For: 2.3.0 > > Attachments: HIVE-15530.1.patch, HIVE-15530.2.patch, > HIVE-15530.3.patch, HIVE-15530.4.patch, HIVE-15530.5.patch > > > Currently when a table is altered, if any of below conditions is true, HMS > would try to update column statistics for the table: > # database name is changed > # table name is changed > # old columns and new columns are not the same > As a result, when a column is added to a table, Hive also tries to update > column statistics, which is not necessary. We can loose the last condition by > checking whether all existing columns are changed or not. If not, we don't > have to update stats info. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-15530) Optimize the column stats update logic in table alteration
[ https://issues.apache.org/jira/browse/HIVE-15530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyu Tang updated HIVE-15530: --- Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) Committed to branch 2.2.0. Thanks [~Yibing] for the patch, [~aihuaxu] and [~pxiong] for reviews. > Optimize the column stats update logic in table alteration > -- > > Key: HIVE-15530 > URL: https://issues.apache.org/jira/browse/HIVE-15530 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Yibing Shi >Assignee: Yibing Shi > Fix For: 2.2.0 > > Attachments: HIVE-15530.1.patch, HIVE-15530.2.patch, > HIVE-15530.3.patch, HIVE-15530.4.patch, HIVE-15530.5.patch > > > Currently when a table is altered, if any of below conditions is true, HMS > would try to update column statistics for the table: > # database name is changed > # table name is changed > # old columns and new columns are not the same > As a result, when a column is added to a table, Hive also tries to update > column statistics, which is not necessary. We can loose the last condition by > checking whether all existing columns are changed or not. If not, we don't > have to update stats info. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15530) Optimize the column stats update logic in table alteration
[ https://issues.apache.org/jira/browse/HIVE-15530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yibing Shi updated HIVE-15530: -- Attachment: HIVE-15530.5.patch Attach a new patch based on [~ctang.ma]'s comment > Optimize the column stats update logic in table alteration > -- > > Key: HIVE-15530 > URL: https://issues.apache.org/jira/browse/HIVE-15530 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Yibing Shi >Assignee: Yibing Shi > Attachments: HIVE-15530.1.patch, HIVE-15530.2.patch, > HIVE-15530.3.patch, HIVE-15530.4.patch, HIVE-15530.5.patch > > > Currently when a table is altered, if any of below conditions is true, HMS > would try to update column statistics for the table: > # database name is changed > # table name is changed > # old columns and new columns are not the same > As a result, when a column is added to a table, Hive also tries to update > column statistics, which is not necessary. We can loose the last condition by > checking whether all existing columns are changed or not. If not, we don't > have to update stats info. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15530) Optimize the column stats update logic in table alteration
[ https://issues.apache.org/jira/browse/HIVE-15530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yibing Shi updated HIVE-15530: -- Attachment: HIVE-15530.4.patch Thanks [~aihuaxu] for looking into the patch. I have corrected the license declarement of the new files based on your suggestion. > Optimize the column stats update logic in table alteration > -- > > Key: HIVE-15530 > URL: https://issues.apache.org/jira/browse/HIVE-15530 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Yibing Shi >Assignee: Yibing Shi > Attachments: HIVE-15530.1.patch, HIVE-15530.2.patch, > HIVE-15530.3.patch, HIVE-15530.4.patch > > > Currently when a table is altered, if any of below conditions is true, HMS > would try to update column statistics for the table: > # database name is changed > # table name is changed > # old columns and new columns are not the same > As a result, when a column is added to a table, Hive also tries to update > column statistics, which is not necessary. We can loose the last condition by > checking whether all existing columns are changed or not. If not, we don't > have to update stats info. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15530) Optimize the column stats update logic in table alteration
[ https://issues.apache.org/jira/browse/HIVE-15530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yibing Shi updated HIVE-15530: -- Attachment: HIVE-15530.3.patch Try to fix the broken patch > Optimize the column stats update logic in table alteration > -- > > Key: HIVE-15530 > URL: https://issues.apache.org/jira/browse/HIVE-15530 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Yibing Shi >Assignee: Yibing Shi > Attachments: HIVE-15530.1.patch, HIVE-15530.2.patch, > HIVE-15530.3.patch > > > Currently when a table is altered, if any of below conditions is true, HMS > would try to update column statistics for the table: > # database name is changed > # table name is changed > # old columns and new columns are not the same > As a result, when a column is added to a table, Hive also tries to update > column statistics, which is not necessary. We can loose the last condition by > checking whether all existing columns are changed or not. If not, we don't > have to update stats info. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15530) Optimize the column stats update logic in table alteration
[ https://issues.apache.org/jira/browse/HIVE-15530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yibing Shi updated HIVE-15530: -- Attachment: HIVE-15530.2.patch Add unit tests > Optimize the column stats update logic in table alteration > -- > > Key: HIVE-15530 > URL: https://issues.apache.org/jira/browse/HIVE-15530 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Yibing Shi >Assignee: Yibing Shi > Attachments: HIVE-15530.1.patch, HIVE-15530.2.patch > > > Currently when a table is altered, if any of below conditions is true, HMS > would try to update column statistics for the table: > # database name is changed > # table name is changed > # old columns and new columns are not the same > As a result, when a column is added to a table, Hive also tries to update > column statistics, which is not necessary. We can loose the last condition by > checking whether all existing columns are changed or not. If not, we don't > have to update stats info. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15530) Optimize the column stats update logic in table alteration
[ https://issues.apache.org/jira/browse/HIVE-15530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yibing Shi updated HIVE-15530: -- Status: Patch Available (was: Open) > Optimize the column stats update logic in table alteration > -- > > Key: HIVE-15530 > URL: https://issues.apache.org/jira/browse/HIVE-15530 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Yibing Shi > Attachments: HIVE-15530.1.patch > > > Currently when a table is altered, if any of below conditions is true, HMS > would try to update column statistics for the table: > # database name is changed > # table name is changed > # old columns and new columns are not the same > As a result, when a column is added to a table, Hive also tries to update > column statistics, which is not necessary. We can loose the last condition by > checking whether all existing columns are changed or not. If not, we don't > have to update stats info. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15530) Optimize the column stats update logic in table alteration
[ https://issues.apache.org/jira/browse/HIVE-15530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yibing Shi updated HIVE-15530: -- Attachment: HIVE-15530.1.patch > Optimize the column stats update logic in table alteration > -- > > Key: HIVE-15530 > URL: https://issues.apache.org/jira/browse/HIVE-15530 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Yibing Shi > Attachments: HIVE-15530.1.patch > > > Currently when a table is altered, if any of below conditions is true, HMS > would try to update column statistics for the table: > # database name is changed > # table name is changed > # old columns and new columns are not the same > As a result, when a column is added to a table, Hive also tries to update > column statistics, which is not necessary. We can loose the last condition by > checking whether all existing columns are changed or not. If not, we don't > have to update stats info. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15530) Optimize the column stats update logic in table alteration
[ https://issues.apache.org/jira/browse/HIVE-15530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yibing Shi updated HIVE-15530: -- Description: Currently when a table is altered, if any of below conditions is true, HMS would try to update column statistics for the table: # database name is changed # table name is changed # old columns and new columns are not the same As a result, when a column is added to a table, Hive also tries to update column statistics, which is not necessary. We can loose the last condition by checking whether all existing columns are changed or not. If not, we don't have to update stats info. was: Currently when a table is altered, if any of below conditions is false, HMS would try to update column statistics for the table: # database name is changed # table name is changed # old columns and new columns are not the same As a result, when a column is added to a table, Hive also tries to update column statistics, which is not necessary. We can loose the last condition by checking whether all existing columns are changed or not. If not, we don't have to update stats info. > Optimize the column stats update logic in table alteration > -- > > Key: HIVE-15530 > URL: https://issues.apache.org/jira/browse/HIVE-15530 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Yibing Shi > > Currently when a table is altered, if any of below conditions is true, HMS > would try to update column statistics for the table: > # database name is changed > # table name is changed > # old columns and new columns are not the same > As a result, when a column is added to a table, Hive also tries to update > column statistics, which is not necessary. We can loose the last condition by > checking whether all existing columns are changed or not. If not, we don't > have to update stats info. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
