wecharyu opened a new pull request, #4517: URL: https://github.com/apache/hive/pull/4517
### What changes were proposed in this pull request? 1. Rename `rename_partition` to `alter_partition_core`. 2. Implemented update in batches for all the tables involved in alter partitions. ### Why are the changes needed? For the performance improvement: 1. JDO update need fetch all fields of old partitions while direct sql only need to get the specific fields. 2. JDO persistence produces many redundant queries while direct sql does not. ### Does this PR introduce _any_ user-facing change? No. ### Is the change a dependency upgrade? No. ### How was this patch tested? 1. passing all existing UT. 2. add a benchmark test: ```bash java -jar ./hmsbench-jar-with-dependencies.jar -H localhost --savedata /tmp/benchdata --sanitize -N 10 -N 100 -N 1000 -o bench_results_direct.csv -C -d testbench_http --params=100 -E 'drop.*' -E 'renameTable.*' -E 'getTableObjectsByName.*' -E 'listTables.*' -E 'listPartitions.*' -E 'getPartitions.*' -E 'getPartitionsByNames.*' -E 'getPartitionNames.*' -E 'listPartition' -E 'getPartition' -E 'getNid' -E 'listDatabases' -E 'getTable' -E 'createTable' -E 'addPartitions.*' ``` - JDO ```bash Operation Mean Med Min Max Err% addPartition 27.77 25.05 22.09 50.14 21.88 alterPartitions.10 98.23 97.49 92.21 132.7 4.700 alterPartitions.100 771.6 780.4 713.3 858.3 4.208 alterPartitions.1000 7414 7362 7207 7740 2.050 ``` - direct sql ```bash Operation Mean Med Min Max Err% addPartition 23.50 20.54 19.55 61.42 31.05 alterPartitions.10 71.33 70.60 67.16 86.74 3.685 alterPartitions.100 428.1 425.2 412.0 479.0 3.052 alterPartitions.1000 3986 3983 3862 4161 1.655 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
