[jira] [Updated] (HIVE-27505) Added batching with retry mechanism in Hive.getAllPartitionsOf API
[ https://issues.apache.org/jira/browse/HIVE-27505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Ahuja updated HIVE-27505: Summary: Added batching with retry mechanism in Hive.getAllPartitionsOf API (was: Added batching with retry mechanism in Hive.getAllPartitionsOf) > Added batching with retry mechanism in Hive.getAllPartitionsOf API > -- > > Key: HIVE-27505 > URL: https://issues.apache.org/jira/browse/HIVE-27505 > Project: Hive > Issue Type: Bug >Reporter: Vikram Ahuja >Assignee: Vikram Ahuja >Priority: Major > Labels: pull-request-available > > Hive.getAllPartitionsOf() tries to list partitions in one go, it can cause > memory related issues with wide tables(>2000 columns) with large amount of > partitions when data is being transferred from HMS to HS2 using Thrift. > Optimise the flow to use PartitionIterable so that partitions can be fetched > in batches rather than one go. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27505) Added batching with retry mechanism in Hive.getAllPartitionsOf API
[ https://issues.apache.org/jira/browse/HIVE-27505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HIVE-27505: Release Note: (was: Merged to master. Thanks for the patch Vikram Ahuja and review zhangbutao !!) > Added batching with retry mechanism in Hive.getAllPartitionsOf API > -- > > Key: HIVE-27505 > URL: https://issues.apache.org/jira/browse/HIVE-27505 > Project: Hive > Issue Type: Bug >Reporter: Vikram Ahuja >Assignee: Vikram Ahuja >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > > Hive.getAllPartitionsOf() tries to list partitions in one go, it can cause > memory related issues with wide tables(>2000 columns) with large amount of > partitions when data is being transferred from HMS to HS2 using Thrift. > Optimise the flow to use PartitionIterable so that partitions can be fetched > in batches rather than one go. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27505) Added batching with retry mechanism in Hive.getAllPartitionsOf API
[ https://issues.apache.org/jira/browse/HIVE-27505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pravin Sinha updated HIVE-27505: Issue Type: Improvement (was: Bug) > Added batching with retry mechanism in Hive.getAllPartitionsOf API > -- > > Key: HIVE-27505 > URL: https://issues.apache.org/jira/browse/HIVE-27505 > Project: Hive > Issue Type: Improvement >Reporter: Vikram Ahuja >Assignee: Vikram Ahuja >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > > Hive.getAllPartitionsOf() tries to list partitions in one go, it can cause > memory related issues with wide tables(>2000 columns) with large amount of > partitions when data is being transferred from HMS to HS2 using Thrift. > Optimise the flow to use PartitionIterable so that partitions can be fetched > in batches rather than one go. -- This message was sent by Atlassian Jira (v8.20.10#820010)
