[
https://issues.apache.org/jira/browse/HIVE-26437?focusedWorklogId=814234&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-814234
]
ASF GitHub Bot logged work on HIVE-26437:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 06/Oct/22 06:14
Start Date: 06/Oct/22 06:14
Worklog Time Spent: 10m
Work Description: atsaonerk opened a new pull request, #3644:
URL: https://github.com/apache/hive/pull/3644
Currently partitions of table is dump in parallel manner. But if table is
not partitioned, it is dumped serially.
This change introduces parallelism at table level as well. A single thread
pool which is currently being used
for partition level is also used for table level. The table level dump task
is added to same thread pool.
The degree of parallelism depends upon config parameter
REPL_PARTITIONS_DUMP_PARALLELISM whose defaul value is 100.
The new ExportService is introduced with this change which would be
responsible for exporting table and partitions during repl dump. The
ExportService is initialized and configured with thread pools by HiveServer2
service.
A new Hiveconfig variable ie "REPL_TABLE_DUMP_PARALLELISM is introduced to
define the number of threads which would be created in thread pool.
The ExportService which is created as singleton instance would be used by
ReplDumpTask.
Issue Time Tracking
-------------------
Worklog Id: (was: 814234)
Time Spent: 0.5h (was: 20m)
> dump unpartitioned Tables in parallel
> -------------------------------------
>
> Key: HIVE-26437
> URL: https://issues.apache.org/jira/browse/HIVE-26437
> Project: Hive
> Issue Type: Improvement
> Components: Hive
> Reporter: Amit Saonerkar
> Assignee: Amit Saonerkar
> Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
