[ https://issues.apache.org/jira/browse/HIVE-23032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17060441#comment-17060441 ]
Denys Kuzmenko edited comment on HIVE-23032 at 3/18/20, 4:06 PM: ----------------------------------------------------------------- [~pvary], very close results for MySQL, however in case of Oracle/Postgres it gives better performance. 4 tables x 1800 partitions PostgreSQL v9.3 {code} batch insert lockReq: {count=200, sum=720917, min=1790, average=3604.585000, max=6952} total: {count=200, sum=430610, min=749, average=2153.050000, max=5913} batch insert + reWriteBatchedInserts lockReq: {count=200, sum=488272, min=1370, average=2441.360000, max=3813} total: {count=200, sum=283623, min=667, average=1418.115000, max=2780} Multi-row insert lockReq: {count=200, sum=771952, min=1853, average=3859.760000, max=7817} total: {count=200, sum=455352, min=768, average=2276.760000, max=5880} {code} MySQL v5.1.46 {code} batch insert + rewriteBatchedStatements lockReq: {count=1000, sum=717697, min=408, average=717.697000, max=1814} total: {count=1000, sum=2490528, min=1347, average=2490.528000, max=4341} Multi-row insert lockReq: {count=1000, sum=710006, min=393, average=710.01, max=2242} total: {count=1000, sum=2229274, min=1223, average=2229.27, max=5600} {code} was (Author: dkuzmenko): [~pvary], same for MySQL, however for Oracle/Postgres it gives better performance. *postgres:9.3* (4 tables x 1800 partitions) {code} batch insert lockReq: {count=200, sum=720917, min=1790, average=3604.585000, max=6952} total: {count=200, sum=430610, min=749, average=2153.050000, max=5913} batch insert + reWriteBatchedInserts lockReq: {count=200, sum=488272, min=1370, average=2441.360000, max=3813} total: {count=200, sum=283623, min=667, average=1418.115000, max=2780} Multi-row insert lockReq: {count=200, sum=771952, min=1853, average=3859.760000, max=7817} total: {count=200, sum=455352, min=768, average=2276.760000, max=5880} {code} MySQL v5.1.46 {code} batch insert + rewriteBatchedStatements lockReq: {count=1000, sum=717697, min=408, average=717.697000, max=1814} total: {count=1000, sum=2490528, min=1347, average=2490.528000, max=4341} Multi-row insert lockReq: {count=1000, sum=710006, min=393, average=710.01, max=2242} total: {count=1000, sum=2229274, min=1223, average=2229.27, max=5600} {code} > Add batching in Lock generation > ------------------------------- > > Key: HIVE-23032 > URL: https://issues.apache.org/jira/browse/HIVE-23032 > Project: Hive > Issue Type: Improvement > Components: Locking > Reporter: Denys Kuzmenko > Assignee: Denys Kuzmenko > Priority: Major > Attachments: HIVE-23032.1.patch > > > Replace multi-row insert in Oracle with batching. Performance tests showed > significant performance improvement after turning batching on. -- This message was sent by Atlassian Jira (v8.3.4#803005)