Prasanth Jayachandran created HIVE-19637:
--------------------------------------------
Summary: Add slow test report script to testutils
Key: HIVE-19637
URL: https://issues.apache.org/jira/browse/HIVE-19637
Project: Hive
Issue Type: Sub-task
Components: Test
Affects Versions: 3.1.0
Reporter: Prasanth Jayachandran
Assignee: Prasanth Jayachandran
Wrote the attached utility script to find top K slow tests from precommit test
url. Would like to get that committed to testutils so that its useful for
everyone.
{code:title=ascii mode}
$ python gen-report.py -b 11102 -a
Processing 1073 test xml reports from
http://104.198.109.242/logs/PreCommit-HIVE-Build-11102/test-results/..
Top 25 testsuite in terms of execution time (in seconds).. [Total time:
73882.661 seconds]
##########################################################################################################
██████████████████████████████████████████████████ 20806 TestCliDriver
███████████████████████ 9601
TestMiniLlapLocalCliDriver
███████████████████ 8210 TestSparkCliDriver
██████ 2744 TestMinimrCliDriver
█████ 2262
TestEncryptedHDFSCliDriver
████ 2021
TestMiniSparkOnYarnCliDriver
████ 1808 TestHiveCli
███ 1566 TestMiniLlapCliDriver
███ 1345
TestReplicationScenarios
██ 1238
TestMiniDruidCliDriver
██ 940 TestNegativeCliDriver
██ 865 TestHBaseCliDriver
█ 681 TestMiniTezCliDriver
█ 555
TestTxnCommands2WithSplitUpdateAndVectorization
█ 543 TestCompactor
█ 528 TestTxnCommands2
378 TestStreaming
374
TestBlobstoreCliDriver
328
TestNegativeMinimrCliDriver
302
TestTxnCommandsWithSplitUpdateAndVectorization
301 TestHCatClient
299 TestTxnCommands
261 TestTxnLoadData
258 TestAcidOnTez
240
TestHBaseNegativeCliDriver
Top 25 testcases in terms of execution time (in seconds).. [Total time:
63102.607 seconds]
###############################################################################################################################################
██████████████████████████████████████████████████ 680
TestMinimrCliDriver_testCliDriver[infer_bucket_sort_reducers_power_two]
█████████████████████████████████████████████ 623
TestMinimrCliDriver_testCliDriver[infer_bucket_sort_map_operators]
███████████████████████████████ 429
TestMinimrCliDriver_testCliDriver[infer_bucket_sort_dyn_part]
███████████████████████████ 374
TestSparkCliDriver_testCliDriver[vectorization_short_regress]
███████████████████████████ 374
TestMiniLlapLocalCliDriver_testCliDriver[vectorization_short_regress]
████████████████████████ 330
TestMiniDruidCliDriver_testCliDriver[druidmini_dynamic_partition]
█████████████████ 238
TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join5]
████████████████ 227
TestMiniDruidCliDriver_testCliDriver[druidmini_test_insert]
███████████████ 214
TestEncryptedHDFSCliDriver_testCliDriver[encryption_auto_purge_tables]
███████████████ 211
TestMiniLlapCliDriver_testCliDriver[unionDistinct_1]
███████████████ 210
TestMiniSparkOnYarnCliDriver_testCliDriver[vector_outer_join5]
███████████████ 206
TestMinimrCliDriver_testCliDriver[bucket_num_reducers_acid]
██████████████ 202
TestMinimrCliDriver_testCliDriver[infer_bucket_sort_merge]
██████████████ 198
TestCliDriver_testCliDriver[typechangetest]
████████████ 172
TestEncryptedHDFSCliDriver_testCliDriver[encryption_drop_table]
████████████ 164
TestMinimrCliDriver_testCliDriver[infer_bucket_sort_num_buckets]
███████████ 158
TestCliDriver_testCliDriver[mm_all]
███████████ 155
TestMiniSparkOnYarnCliDriver_testCliDriver[spark_dynamic_partition_pruning]
██████████ 145
TestMiniSparkOnYarnCliDriver_testCliDriver[spark_vectorized_dynamic_partition_pruning]
██████████ 141
TestMiniLlapCliDriver_testCliDriver[mm_all]
██████████ 140
TestMiniDruidCliDriver_testCliDriver[druidmini_mv]
██████████ 137
TestSparkCliDriver_testCliDriver[auto_join_filters]
█████████ 135
TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join3]
█████████ 124
TestMiniSparkOnYarnCliDriver_testCliDriver[vector_outer_join3]
████████ 121
TestCliDriver_testCliDriver[type_change_test_int_vectorized]
{code}
{code:title=text mode}
$ python gen-report.py -b 11102
Processing 1073 test xml reports from
http://104.198.109.242/logs/PreCommit-HIVE-Build-11102/test-results/..
Top 25 testsuite in terms of execution time (in seconds).. [Total time:
73882.661 seconds]
TestCliDriver 20805.579
TestMiniLlapLocalCliDriver 9601.362
TestSparkCliDriver 8210.062
TestMinimrCliDriver 2743.746
TestEncryptedHDFSCliDriver 2261.866
TestMiniSparkOnYarnCliDriver 2021.468
TestHiveCli 1807.56
TestMiniLlapCliDriver 1565.858
TestReplicationScenarios 1345.344
TestMiniDruidCliDriver 1237.776
TestNegativeCliDriver 940.321
TestHBaseCliDriver 864.707
TestMiniTezCliDriver 681.457
TestTxnCommands2WithSplitUpdateAndVectorization 555.382
TestCompactor 543.173
TestTxnCommands2 527.84
TestStreaming 378.325
TestBlobstoreCliDriver 374.134
TestNegativeMinimrCliDriver 328.128
TestTxnCommandsWithSplitUpdateAndVectorization 301.718
TestHCatClient 301.324
TestTxnCommands 298.694
TestTxnLoadData 260.841
TestAcidOnTez 258.262
TestHBaseNegativeCliDriver 240.198
Top 25 testcases in terms of execution time (in seconds).. [Total time:
63102.607 seconds]
TestMinimrCliDriver_testCliDriver[infer_bucket_sort_reducers_power_two] 680.326
TestMinimrCliDriver_testCliDriver[infer_bucket_sort_map_operators] 623.404
TestMinimrCliDriver_testCliDriver[infer_bucket_sort_dyn_part] 429.358
TestSparkCliDriver_testCliDriver[vectorization_short_regress] 374.491
TestMiniLlapLocalCliDriver_testCliDriver[vectorization_short_regress] 374.164
TestMiniDruidCliDriver_testCliDriver[druidmini_dynamic_partition] 329.945
TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join5] 238.5
TestMiniDruidCliDriver_testCliDriver[druidmini_test_insert] 226.773
TestEncryptedHDFSCliDriver_testCliDriver[encryption_auto_purge_tables] 214.09
TestMiniLlapCliDriver_testCliDriver[unionDistinct_1] 210.803
TestMiniSparkOnYarnCliDriver_testCliDriver[vector_outer_join5] 209.894
TestMinimrCliDriver_testCliDriver[bucket_num_reducers_acid] 206.339
TestMinimrCliDriver_testCliDriver[infer_bucket_sort_merge] 201.754
TestCliDriver_testCliDriver[typechangetest] 198.371
TestEncryptedHDFSCliDriver_testCliDriver[encryption_drop_table] 172.267
TestMinimrCliDriver_testCliDriver[infer_bucket_sort_num_buckets] 163.617
TestCliDriver_testCliDriver[mm_all] 158.401
TestMiniSparkOnYarnCliDriver_testCliDriver[spark_dynamic_partition_pruning]
155.255
TestMiniSparkOnYarnCliDriver_testCliDriver[spark_vectorized_dynamic_partition_pruning]
145.481
TestMiniLlapCliDriver_testCliDriver[mm_all] 141.369
TestMiniDruidCliDriver_testCliDriver[druidmini_mv] 139.815
TestSparkCliDriver_testCliDriver[auto_join_filters] 137.391
TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join3] 135.227
TestMiniSparkOnYarnCliDriver_testCliDriver[vector_outer_join3] 124.469
TestCliDriver_testCliDriver[type_change_test_int_vectorized] 120.85
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
