-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72283/
-----------------------------------------------------------
(Updated ápr. 2, 2020, 10:23 de)
Review request for hive, Denys Kuzmenko and Marton Bod.
Changes
-------
Updated with comments
Bugs: HIVE-23076
https://issues.apache.org/jira/browse/HIVE-23076
Repository: hive-git
Description
-------
Add batching for openTxn request for better performance
Diffs (updated)
-----
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java
74ef88545e
Diff: https://reviews.apache.org/r/72283/diff/3/
Changes: https://reviews.apache.org/r/72283/diff/2-3/
Testing
-------
Tested it locally against all of the supported RDBMS types:
mysql no patch
Operation Mean Med Min Max Err%
openTxn0-1 2.094 1.821 1.462 4.786 31.06
openTxn0-2 2.419 2.161 1.720 5.867 32.43
openTxn0-10 2.578 2.289 1.973 7.204 28.74
openTxn0-100 6.948 6.835 5.254 11.03 15.91
openTxn0-1000 51.31 50.49 33.56 93.10 16.27
openTxn115k-1 26.94 23.69 22.24 169.6 56.13
openTxn115k-2 25.26 23.81 22.42 50.68 16.90
openTxn115k-10 26.20 24.29 23.01 60.73 21.94
openTxn125k-100 29.14 28.18 25.81 43.63 11.16
mysql patch
Operation Mean Med Min Max Err%
openTxn0-1 2.264 1.964 1.652 6.023 35.59
openTxn0-2 2.538 2.289 1.932 6.013 29.41
openTxn0-10 2.982 2.641 2.177 8.829 32.54
openTxn0-100 6.775 6.386 5.012 21.73 27.10
openTxn0-1000 42.96 42.93 30.89 61.92 14.46
openTxn115k-1 24.29 23.27 22.40 73.62 21.64
openTxn115k-2 24.05 23.58 22.46 28.60 5.651
openTxn115k-10 24.48 24.02 22.94 29.97 6.075
openTxn125k-100 27.91 27.51 25.78 42.50 6.905
postgres no patch
Operation Mean Med Min Max Err%
openTxn0-1 3.734 2.883 2.506 11.46 55.16
openTxn0-2 3.834 3.111 2.633 15.50 53.22
openTxn0-10 5.005 4.178 3.449 16.80 47.56
openTxn0-100 9.823 7.755 6.833 79.34 79.96
openTxn0-1000 75.51 72.03 58.62 207.9 23.98
openTxn115k-1 21.79 19.45 18.43 66.76 29.10
openTxn115k-2 21.91 20.14 18.88 51.42 20.92
openTxn115k-10 22.43 20.85 19.38 45.18 18.58
openTxn125k-100 27.71 25.36 23.19 54.99 21.46
postgres patch
Operation Mean Med Min Max Err%
openTxn0-1 1.688 1.423 1.130 7.814 55.91
openTxn0-2 1.982 1.662 1.306 7.786 47.13
openTxn0-10 2.680 2.564 1.761 5.069 26.93
openTxn0-100 8.340 7.535 5.351 30.00 37.97
openTxn0-1000 41.73 37.55 24.38 107.8 33.87
openTxn115k-1 12.24 11.65 10.21 26.23 19.75
openTxn115k-2 13.07 11.86 10.76 68.95 47.37
openTxn115k-10 13.03 12.23 11.06 54.88 34.23
openTxn125k-100 15.62 14.03 12.46 102.9 58.21
Oracle no patch
Operation Mean Med Min Max Err%
openTxn0-1 14.85 13.91 11.50 27.26 19.49
openTxn0-2 17.89 17.13 14.56 27.00 13.53
openTxn0-10 23.12 21.38 17.91 67.37 25.46
openTxn0-100 114.1 99.03 82.62 214.0 35.61
openTxn0-1000 4123 3952 3593 5790 15.96
openTxn115k-1 16.74 16.88 14.01 21.75 14.52
openTxn115k-2 20.28 18.34 16.51 30.34 23.09
openTxn115k-10 22.42 21.07 19.87 31.39 15.74
openTxn125k-100 88.13 87.88 78.95 100.4 7.990
Oracle patch
Operation Mean Med Min Max Err%
openTxn0-1 15.87 14.07 12.21 80.44 48.32
openTxn0-2 17.06 16.14 12.80 33.52 19.47
openTxn0-10 16.89 15.62 12.34 37.92 25.18
openTxn0-100 18.99 20.03 15.69 21.46 10.72
openTxn0-1000 39.13 39.83 29.69 49.96 19.32
openTxn11k-1 14.49 14.16 13.25 17.34 8.577
openTxn11k-2 16.92 14.43 13.76 29.30 32.29
openTxn11k-10 15.99 14.15 12.47 31.26 34.44
openTxn12k-100 15.55 15.14 13.73 18.15 9.721
SQL Server no patch
Operation Mean Med Min Max Err%
openTxn0-1 23.82 21.35 17.84 75.85 32.13
openTxn0-2 25.71 24.54 19.21 56.05 21.78
openTxn0-10 27.04 25.68 21.39 48.45 18.22
openTxn0-100 64.38 61.65 48.65 129.8 18.90
openTxn0-1000 1026 886.4 696.6 3944 43.44
openTxn115k-1 72.94 65.50 56.21 141.7 25.70
openTxn115k-2 64.45 62.46 57.09 84.71 8.815
openTxn115k-10 68.21 65.07 58.18 103.2 11.66
openTxn125k-100 95.63 91.63 83.61 190.2 15.46
SQL Server patch
Operation Mean Med Min Max Err%
openTxn0-1 17.41 16.92 14.01 28.99 12.16
openTxn0-2 24.43 22.69 19.60 57.17 21.47
openTxn0-10 29.91 27.99 23.43 55.37 20.02
openTxn0-100 41.81 38.68 28.40 104.2 31.17
openTxn0-1000 112.4 107.0 69.80 203.7 25.56
openTxn115k-1 22.42 20.90 17.93 95.42 35.30
openTxn115k-2 28.95 27.91 22.22 45.23 14.59
openTxn115k-10 33.16 32.07 25.92 62.41 15.66
openTxn125k-100 62.16 69.24 30.08 187.6 42.01
Thanks,
Peter Vary
