[
https://issues.apache.org/jira/browse/HIVE-19637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran resolved HIVE-19637.
------------------------------------------
Resolution: Fixed
Fix Version/s: 3.1.0
Committed to master. Thanks for the review!
> Add slow test report script to testutils
> ----------------------------------------
>
> Key: HIVE-19637
> URL: https://issues.apache.org/jira/browse/HIVE-19637
> Project: Hive
> Issue Type: Sub-task
> Components: Test
> Affects Versions: 3.1.0
> Reporter: Prasanth Jayachandran
> Assignee: Prasanth Jayachandran
> Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19637.1.patch
>
>
> Wrote the attached utility script to find top K slow tests from precommit
> test url. Would like to get that committed to testutils so that its useful
> for everyone.
> {code:title=ascii mode}
> $ python gen-report.py -b 11102 -a
> Processing 1073 test xml reports from
> http://104.198.109.242/logs/PreCommit-HIVE-Build-11102/test-results/..
> Top 25 testsuite in terms of execution time (in seconds).. [Total time:
> 73882.661 seconds]
> ##########################################################################################################
> ██████████████████████████████████████████████████ 20806 TestCliDriver
> ███████████████████████ 9601
> TestMiniLlapLocalCliDriver
> ███████████████████ 8210 TestSparkCliDriver
> ██████ 2744 TestMinimrCliDriver
> █████ 2262
> TestEncryptedHDFSCliDriver
> ████ 2021
> TestMiniSparkOnYarnCliDriver
> ████ 1808 TestHiveCli
> ███ 1566
> TestMiniLlapCliDriver
> ███ 1345
> TestReplicationScenarios
> ██ 1238
> TestMiniDruidCliDriver
> ██ 940
> TestNegativeCliDriver
> ██ 865 TestHBaseCliDriver
> █ 681
> TestMiniTezCliDriver
> █ 555
> TestTxnCommands2WithSplitUpdateAndVectorization
> █ 543 TestCompactor
> █ 528 TestTxnCommands2
> 378 TestStreaming
> 374
> TestBlobstoreCliDriver
> 328
> TestNegativeMinimrCliDriver
> 302
> TestTxnCommandsWithSplitUpdateAndVectorization
> 301 TestHCatClient
> 299 TestTxnCommands
> 261 TestTxnLoadData
> 258 TestAcidOnTez
> 240
> TestHBaseNegativeCliDriver
> Top 25 testcases in terms of execution time (in seconds).. [Total time:
> 63102.607 seconds]
> ###############################################################################################################################################
> ██████████████████████████████████████████████████ 680
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_reducers_power_two]
> █████████████████████████████████████████████ 623
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_map_operators]
> ███████████████████████████████ 429
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_dyn_part]
> ███████████████████████████ 374
> TestSparkCliDriver_testCliDriver[vectorization_short_regress]
> ███████████████████████████ 374
> TestMiniLlapLocalCliDriver_testCliDriver[vectorization_short_regress]
> ████████████████████████ 330
> TestMiniDruidCliDriver_testCliDriver[druidmini_dynamic_partition]
> █████████████████ 238
> TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join5]
> ████████████████ 227
> TestMiniDruidCliDriver_testCliDriver[druidmini_test_insert]
> ███████████████ 214
> TestEncryptedHDFSCliDriver_testCliDriver[encryption_auto_purge_tables]
> ███████████████ 211
> TestMiniLlapCliDriver_testCliDriver[unionDistinct_1]
> ███████████████ 210
> TestMiniSparkOnYarnCliDriver_testCliDriver[vector_outer_join5]
> ███████████████ 206
> TestMinimrCliDriver_testCliDriver[bucket_num_reducers_acid]
> ██████████████ 202
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_merge]
> ██████████████ 198
> TestCliDriver_testCliDriver[typechangetest]
> ████████████ 172
> TestEncryptedHDFSCliDriver_testCliDriver[encryption_drop_table]
> ████████████ 164
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_num_buckets]
> ███████████ 158
> TestCliDriver_testCliDriver[mm_all]
> ███████████ 155
> TestMiniSparkOnYarnCliDriver_testCliDriver[spark_dynamic_partition_pruning]
> ██████████ 145
> TestMiniSparkOnYarnCliDriver_testCliDriver[spark_vectorized_dynamic_partition_pruning]
> ██████████ 141
> TestMiniLlapCliDriver_testCliDriver[mm_all]
> ██████████ 140
> TestMiniDruidCliDriver_testCliDriver[druidmini_mv]
> ██████████ 137
> TestSparkCliDriver_testCliDriver[auto_join_filters]
> █████████ 135
> TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join3]
> █████████ 124
> TestMiniSparkOnYarnCliDriver_testCliDriver[vector_outer_join3]
> ████████ 121
> TestCliDriver_testCliDriver[type_change_test_int_vectorized]
> {code}
> {code:title=text mode}
> $ python gen-report.py -b 11102
> Processing 1073 test xml reports from
> http://104.198.109.242/logs/PreCommit-HIVE-Build-11102/test-results/..
> Top 25 testsuite in terms of execution time (in seconds).. [Total time:
> 73882.661 seconds]
> TestCliDriver 20805.579
> TestMiniLlapLocalCliDriver 9601.362
> TestSparkCliDriver 8210.062
> TestMinimrCliDriver 2743.746
> TestEncryptedHDFSCliDriver 2261.866
> TestMiniSparkOnYarnCliDriver 2021.468
> TestHiveCli 1807.56
> TestMiniLlapCliDriver 1565.858
> TestReplicationScenarios 1345.344
> TestMiniDruidCliDriver 1237.776
> TestNegativeCliDriver 940.321
> TestHBaseCliDriver 864.707
> TestMiniTezCliDriver 681.457
> TestTxnCommands2WithSplitUpdateAndVectorization 555.382
> TestCompactor 543.173
> TestTxnCommands2 527.84
> TestStreaming 378.325
> TestBlobstoreCliDriver 374.134
> TestNegativeMinimrCliDriver 328.128
> TestTxnCommandsWithSplitUpdateAndVectorization 301.718
> TestHCatClient 301.324
> TestTxnCommands 298.694
> TestTxnLoadData 260.841
> TestAcidOnTez 258.262
> TestHBaseNegativeCliDriver 240.198
> Top 25 testcases in terms of execution time (in seconds).. [Total time:
> 63102.607 seconds]
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_reducers_power_two]
> 680.326
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_map_operators] 623.404
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_dyn_part] 429.358
> TestSparkCliDriver_testCliDriver[vectorization_short_regress] 374.491
> TestMiniLlapLocalCliDriver_testCliDriver[vectorization_short_regress] 374.164
> TestMiniDruidCliDriver_testCliDriver[druidmini_dynamic_partition] 329.945
> TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join5] 238.5
> TestMiniDruidCliDriver_testCliDriver[druidmini_test_insert] 226.773
> TestEncryptedHDFSCliDriver_testCliDriver[encryption_auto_purge_tables]
> 214.09
> TestMiniLlapCliDriver_testCliDriver[unionDistinct_1] 210.803
> TestMiniSparkOnYarnCliDriver_testCliDriver[vector_outer_join5] 209.894
> TestMinimrCliDriver_testCliDriver[bucket_num_reducers_acid] 206.339
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_merge] 201.754
> TestCliDriver_testCliDriver[typechangetest] 198.371
> TestEncryptedHDFSCliDriver_testCliDriver[encryption_drop_table] 172.267
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_num_buckets] 163.617
> TestCliDriver_testCliDriver[mm_all] 158.401
> TestMiniSparkOnYarnCliDriver_testCliDriver[spark_dynamic_partition_pruning]
> 155.255
> TestMiniSparkOnYarnCliDriver_testCliDriver[spark_vectorized_dynamic_partition_pruning]
> 145.481
> TestMiniLlapCliDriver_testCliDriver[mm_all] 141.369
> TestMiniDruidCliDriver_testCliDriver[druidmini_mv] 139.815
> TestSparkCliDriver_testCliDriver[auto_join_filters] 137.391
> TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join3] 135.227
> TestMiniSparkOnYarnCliDriver_testCliDriver[vector_outer_join3] 124.469
> TestCliDriver_testCliDriver[type_change_test_int_vectorized] 120.85
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
