[jira] [Updated] (HIVE-25178) Reduce number of getPartition calls during loadDynamicPartitions
[ https://issues.apache.org/jira/browse/HIVE-25178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-25178: Labels: performance (was: ) > Reduce number of getPartition calls during loadDynamicPartitions > > > Key: HIVE-25178 > URL: https://issues.apache.org/jira/browse/HIVE-25178 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Rajesh Balamohan >Priority: Major > Labels: performance > > When dynamic partitions are loaded, Hive::loadDynamicPartition loads all > partitions from HMS causing heavy load on it. This becomes worse when large > number of partitions are present in tables. > Only relevant partitions being loaded in dynamic partitions can be queried > from HMS for partition existence. > [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2958] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25178) Reduce number of getPartition calls during loadDynamicPartitions
[ https://issues.apache.org/jira/browse/HIVE-25178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-25178: -- Labels: performance pull-request-available (was: performance) > Reduce number of getPartition calls during loadDynamicPartitions > > > Key: HIVE-25178 > URL: https://issues.apache.org/jira/browse/HIVE-25178 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Rajesh Balamohan >Priority: Major > Labels: performance, pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > When dynamic partitions are loaded, Hive::loadDynamicPartition loads all > partitions from HMS causing heavy load on it. This becomes worse when large > number of partitions are present in tables. > Only relevant partitions being loaded in dynamic partitions can be queried > from HMS for partition existence. > [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2958] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25178) Reduce number of getPartition calls during loadDynamicPartitions for External Tables
[ https://issues.apache.org/jira/browse/HIVE-25178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Narayanan Venkateswaran updated HIVE-25178: --- Summary: Reduce number of getPartition calls during loadDynamicPartitions for External Tables (was: Reduce number of getPartition calls during loadDynamicPartitions) > Reduce number of getPartition calls during loadDynamicPartitions for External > Tables > > > Key: HIVE-25178 > URL: https://issues.apache.org/jira/browse/HIVE-25178 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Rajesh Balamohan >Assignee: Narayanan Venkateswaran >Priority: Major > Labels: performance, pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > When dynamic partitions are loaded, Hive::loadDynamicPartition loads all > partitions from HMS causing heavy load on it. This becomes worse when large > number of partitions are present in tables. > Only relevant partitions being loaded in dynamic partitions can be queried > from HMS for partition existence. > [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2958] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25178) Reduce number of getPartition calls during loadDynamicPartitions for External Tables
[ https://issues.apache.org/jira/browse/HIVE-25178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Narayanan Venkateswaran updated HIVE-25178: --- Attachment: 0001-HIVE-25178-Reduce-number-of-getPartition-calls-durin.patch > Reduce number of getPartition calls during loadDynamicPartitions for External > Tables > > > Key: HIVE-25178 > URL: https://issues.apache.org/jira/browse/HIVE-25178 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Rajesh Balamohan >Assignee: Narayanan Venkateswaran >Priority: Major > Labels: performance, pull-request-available > Attachments: > 0001-HIVE-25178-Reduce-number-of-getPartition-calls-durin.patch > > Time Spent: 20m > Remaining Estimate: 0h > > When dynamic partitions are loaded, Hive::loadDynamicPartition loads all > partitions from HMS causing heavy load on it. This becomes worse when large > number of partitions are present in tables. > Only relevant partitions being loaded in dynamic partitions can be queried > from HMS for partition existence. > [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2958] -- This message was sent by Atlassian Jira (v8.3.4#803005)
