[
https://issues.apache.org/jira/browse/HIVE-23032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17060441#comment-17060441
]
Denys Kuzmenko edited comment on HIVE-23032 at 3/18/20, 4:06 PM:
-----------------------------------------------------------------
[~pvary], very close results for MySQL, however in case of Oracle/Postgres it
gives better performance.
4 tables x 1800 partitions
PostgreSQL v9.3
{code}
batch insert
lockReq: {count=200, sum=720917, min=1790, average=3604.585000, max=6952}
total: {count=200, sum=430610, min=749, average=2153.050000, max=5913}
batch insert + reWriteBatchedInserts
lockReq: {count=200, sum=488272, min=1370, average=2441.360000, max=3813}
total: {count=200, sum=283623, min=667, average=1418.115000, max=2780}
Multi-row insert
lockReq: {count=200, sum=771952, min=1853, average=3859.760000, max=7817}
total: {count=200, sum=455352, min=768, average=2276.760000, max=5880}
{code}
MySQL v5.1.46
{code}
batch insert + rewriteBatchedStatements
lockReq: {count=1000, sum=717697, min=408, average=717.697000, max=1814}
total: {count=1000, sum=2490528, min=1347, average=2490.528000, max=4341}
Multi-row insert
lockReq: {count=1000, sum=710006, min=393, average=710.01, max=2242}
total: {count=1000, sum=2229274, min=1223, average=2229.27, max=5600}
{code}
was (Author: dkuzmenko):
[~pvary], same for MySQL, however for Oracle/Postgres it gives better
performance.
*postgres:9.3* (4 tables x 1800 partitions)
{code}
batch insert
lockReq: {count=200, sum=720917, min=1790, average=3604.585000, max=6952}
total: {count=200, sum=430610, min=749, average=2153.050000, max=5913}
batch insert + reWriteBatchedInserts
lockReq: {count=200, sum=488272, min=1370, average=2441.360000, max=3813}
total: {count=200, sum=283623, min=667, average=1418.115000, max=2780}
Multi-row insert
lockReq: {count=200, sum=771952, min=1853, average=3859.760000, max=7817}
total: {count=200, sum=455352, min=768, average=2276.760000, max=5880}
{code}
MySQL v5.1.46
{code}
batch insert + rewriteBatchedStatements
lockReq: {count=1000, sum=717697, min=408, average=717.697000, max=1814}
total: {count=1000, sum=2490528, min=1347, average=2490.528000, max=4341}
Multi-row insert
lockReq: {count=1000, sum=710006, min=393, average=710.01, max=2242}
total: {count=1000, sum=2229274, min=1223, average=2229.27, max=5600}
{code}
> Add batching in Lock generation
> -------------------------------
>
> Key: HIVE-23032
> URL: https://issues.apache.org/jira/browse/HIVE-23032
> Project: Hive
> Issue Type: Improvement
> Components: Locking
> Reporter: Denys Kuzmenko
> Assignee: Denys Kuzmenko
> Priority: Major
> Attachments: HIVE-23032.1.patch
>
>
> Replace multi-row insert in Oracle with batching. Performance tests showed
> significant performance improvement after turning batching on.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
