Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
gavinchou merged PR #53408: URL: https://github.com/apache/doris/pull/53408 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
github-actions[bot] commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3106212001 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3104023009 # BE UT Coverage Report Increment line coverage `93.75% (30/32)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/73395f0e752343ec0b81d8131979d2d22d167ccf_73395f0e752343ec0b81d8131979d2d22d167ccf/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/73395f0e752343ec0b81d8131979d2d22d167ccf_73395f0e752343ec0b81d8131979d2d22d167ccf/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.48% (15924/27702) | | Line Coverage | 46.28% (143029/309082) | | Region Coverage | 35.78% (107867/301473) | | Branch Coverage | 38.25% (47552/124328) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3103053210 ClickBench: Total hot run time: 32.18 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 73395f0e752343ec0b81d8131979d2d22d167ccf, data reload: false query1 0.040.030.03 query2 0.070.050.04 query3 0.250.070.07 query4 1.630.110.11 query5 0.440.400.40 query6 1.180.650.66 query7 0.020.020.01 query8 0.050.040.04 query9 0.590.520.52 query10 0.600.580.56 query11 0.160.110.12 query12 0.150.110.11 query13 0.630.610.62 query14 0.800.820.82 query15 0.880.860.87 query16 0.400.400.39 query17 1.041.051.07 query18 0.210.200.19 query19 1.981.831.86 query20 0.020.010.02 query21 15.38 0.890.54 query22 0.771.110.72 query23 14.92 1.350.64 query24 6.710.640.53 query25 0.530.190.08 query26 0.490.170.14 query27 0.050.040.05 query28 10.01 0.930.43 query29 12.55 4.003.28 query30 3.082.992.96 query31 2.830.580.38 query32 3.220.550.48 query33 3.073.113.01 query34 15.87 5.394.81 query35 4.914.914.91 query36 0.680.500.49 query37 0.090.070.06 query38 0.040.040.04 query39 0.030.020.02 query40 0.180.160.14 query41 0.080.030.02 query42 0.040.020.03 query43 0.040.030.03 Total cold run time: 106.71 s Total hot run time: 32.18 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3103024604 TPC-DS: Total hot run time: 187529 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 73395f0e752343ec0b81d8131979d2d22d167ccf, data reload: false query1 1018405 415 405 query2 6522165816441644 query3 6752225 221 221 query4 26491 23720 23346 23346 query5 4332636 483 483 query6 309 226 206 206 query7 4638487 292 292 query8 294 247 217 217 query9 8590285028382838 query10 464 344 297 297 query11 15695 14939 14759 14759 query12 165 117 112 112 query13 1654533 410 410 query14 8653601459765976 query15 210 198 170 170 query16 7140660 432 432 query17 1041725 600 600 query18 2003415 372 372 query19 188 186 158 158 query20 122 119 114 114 query21 208 121 100 100 query22 4169421839793979 query23 34057 33016 33126 33016 query24 8153239524042395 query25 546 480 445 445 query26 1241280 160 160 query27 2757502 347 347 query28 4352217021572157 query29 775 567 460 460 query30 286 220 192 192 query31 921 816 741 741 query32 106 77 72 72 query33 560 378 327 327 query34 785 856 501 501 query35 790 823 735 735 query36 955 994 897 897 query37 121 108 86 86 query38 4195415141484148 query39 1473140014241400 query40 221 128 116 116 query41 59 55 54 54 query42 121 111 112 111 query43 507 498 449 449 query44 1329840 842 840 query45 172 173 167 167 query46 839 1025629 629 query47 1782180917231723 query48 375 435 300 300 query49 727 466 376 376 query50 640 687 421 421 query51 5617555054125412 query52 117 117 103 103 query53 235 272 198 198 query54 595 603 530 530 query55 92 91 92 91 query56 330 328 309 309 query57 1192119711361136 query58 283 279 275 275 query59 2562266126312631 query60 350 372 322 322 query61 128 122 121 121 query62 783 723 671 671 query63 233 204 194 194 query64 43461003677 677 query65 4352416441864164 query66 1154407 349 349 query67 15831 15440 15374 15374 query68 7978895 584 584 query69 519 325 281 281 query70 1216114710921092 query71 463 340 321 321 query72 5591482449084824 query73 710 663 353 353 query74 9010908589278927 query75 3775308626492649 query76 36341131732 732 query77 792 394 312 312 query78 991510114 92829282 query79 2190887 596 596 query80 606 538 472 472 query81 462 255 226 226 query82 430 136 106 106 query83 259 277 270 270 query84 279 113 81 81 query85 919 360 324 324 query86 338 324 303 303 query87 4410447643134313 query88 322322152215 query89 384 326 290 290 query90 1925224 222 222 query91 156 140 110 110 query92 82 74 65 65 query93 1129984 619 619 query94 690 396 315 315 query95 408 319 310 310 query96 483 580 275 275 query97 2696272126342634 query98 247 217 208 208 query99 1637144412651265 Total cold run time: 274259 ms Total hot run time: 187529 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3102955820 TPC-H: Total hot run time: 34106 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 73395f0e752343ec0b81d8131979d2d22d167ccf, data reload: false -- Round 1 -- q1 17592 546350735073 q2 1923285 180 180 q3 10316 1294708 708 q4 10224 1033524 524 q5 7523245823632363 q6 181 161 128 128 q7 912 756 610 610 q8 9292130110851085 q9 6885523951575157 q10 6888240919691969 q11 470 299 274 274 q12 348 352 221 221 q13 17791 375330923092 q14 231 227 222 222 q15 563 478 475 475 q16 417 441 378 378 q17 584 896 363 363 q18 7470722271507150 q19 1214982 568 568 q20 352 350 228 228 q21 3820257423402340 q22 10691044998 998 Total cold run time: 106065 ms Total hot run time: 34106 ms - Round 2, with runtime_filter_mode=off - q1 5224521051145114 q2 248 331 219 219 q3 2182269722852285 q4 1353179713691369 q5 4227453446104534 q6 221 173 124 124 q7 2031195418551855 q8 2646261726092609 q9 7268735871967196 q10 3084329331433143 q11 576 509 491 491 q12 683 806 663 663 q13 3500404833943394 q14 314 309 286 286 q15 536 485 462 462 q16 465 527 458 458 q17 1180161113811381 q18 8013783475607560 q19 781 808 855 808 q20 2068204718811881 q21 5118455243894389 q22 10911041977 977 Total cold run time: 52809 ms Total hot run time: 51198 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
Yukang-Lian commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3102562981 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3101818314 ClickBench: Total hot run time: 32.91 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 92947d9a8ef30f367a76266ac54a653bbccfbae9, data reload: false query1 0.040.040.03 query2 0.080.040.04 query3 0.250.070.07 query4 1.620.110.11 query5 0.410.430.39 query6 1.160.670.67 query7 0.030.020.02 query8 0.050.040.03 query9 0.600.520.51 query10 0.580.570.57 query11 0.160.110.11 query12 0.150.110.11 query13 0.630.620.61 query14 0.810.840.82 query15 0.900.870.88 query16 0.400.390.40 query17 1.051.071.05 query18 0.220.210.20 query19 2.011.911.89 query20 0.010.020.01 query21 15.38 0.930.56 query22 0.761.230.72 query23 14.85 1.350.63 query24 7.040.940.81 query25 0.500.260.18 query26 0.620.180.15 query27 0.050.060.05 query28 9.590.970.44 query29 12.70 4.093.31 query30 3.113.043.01 query31 2.820.610.38 query32 3.260.560.49 query33 3.133.103.20 query34 16.10 5.504.84 query35 4.894.934.92 query36 0.690.510.49 query37 0.100.070.07 query38 0.050.040.03 query39 0.030.020.02 query40 0.170.140.14 query41 0.080.020.02 query42 0.040.020.03 query43 0.040.030.03 Total cold run time: 107.16 s Total hot run time: 32.91 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3101798350 TPC-DS: Total hot run time: 188595 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 92947d9a8ef30f367a76266ac54a653bbccfbae9, data reload: false query1 1014387 397 387 query2 6521175118311751 query3 6754235 225 225 query4 26246 23928 23244 23244 query5 4321623 480 480 query6 288 208 213 208 query7 4629492 293 293 query8 268 233 233 233 query9 8575288628902886 query10 473 339 288 288 query11 15455 15000 14769 14769 query12 159 121 111 111 query13 1659555 411 411 query14 8722613360946094 query15 216 185 169 169 query16 7370639 478 478 query17 1228756 615 615 query18 1994424 335 335 query19 198 195 170 170 query20 132 135 119 119 query21 214 127 112 112 query22 4228419942594199 query23 33878 32979 33012 32979 query24 8209239623992396 query25 543 477 401 401 query26 1234270 161 161 query27 2744503 347 347 query28 4349219121532153 query29 804 575 455 455 query30 286 224 197 197 query31 904 819 751 751 query32 82 79 76 76 query33 547 391 336 336 query34 800 836 528 528 query35 806 849 747 747 query36 1003994 952 952 query37 122 106 88 88 query38 4168416341184118 query39 1501144313961396 query40 221 125 115 115 query41 64 57 54 54 query42 120 105 111 105 query43 500 497 493 493 query44 1357850 844 844 query45 182 169 169 169 query46 844 1011635 635 query47 1778185917431743 query48 383 408 294 294 query49 710 480 381 381 query50 655 687 421 421 query51 5468542154205420 query52 125 119 105 105 query53 240 282 193 193 query54 604 605 530 530 query55 100 89 85 85 query56 316 317 323 317 query57 1198119311471147 query58 284 273 278 273 query59 2732276226732673 query60 353 335 332 332 query61 163 128 125 125 query62 790 732 658 658 query63 224 193 185 185 query64 43661048686 686 query65 4284415641774156 query66 1151464 345 345 query67 15888 15444 15629 15444 query68 8265910 566 566 query69 493 334 302 302 query70 1264120711351135 query71 508 328 364 328 query72 5697485049144850 query73 747 636 358 358 query74 9075884689838846 query75 3948309627182718 query76 37701147797 797 query77 783 402 320 320 query78 10049 10054 93099309 query79 4195875 611 611 query80 659 580 558 558 query81 467 268 222 222 query82 503 140 113 113 query83 286 261 238 238 query84 289 102 79 79 query85 788 364 323 323 query86 355 346 307 307 query87 4508443943884388 query88 2926224822302230 query89 444 323 299 299 query90 1934237 232 232 query91 143 150 113 113 query92 88 78 67 67 query93 2414979 630 630 query94 676 413 311 311 query95 398 333 322 322 query96 486 596 283 283 query97 2687277626322632 query98 249 215 220 215 query99 1470140813071307 Total cold run time: 278308 ms Total hot run time: 188595 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3101751924 TPC-H: Total hot run time: 34057 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 92947d9a8ef30f367a76266ac54a653bbccfbae9, data reload: false -- Round 1 -- q1 17614 570651145114 q2 1930283 182 182 q3 10365 1319712 712 q4 10265 1023516 516 q5 7807247423152315 q6 180 160 127 127 q7 915 731 614 614 q8 9306133110811081 q9 7121513551325132 q10 6885236619671967 q11 472 298 273 273 q12 359 367 220 220 q13 17760 371730823082 q14 241 230 218 218 q15 551 487 475 475 q16 422 429 380 380 q17 620 885 384 384 q18 7567708771737087 q19 1213983 577 577 q20 362 363 231 231 q21 4036318324102410 q22 10761023960 960 Total cold run time: 107067 ms Total hot run time: 34057 ms - Round 2, with runtime_filter_mode=off - q1 5201515751765157 q2 247 325 220 220 q3 2159270423162316 q4 1392180713431343 q5 4424451245334512 q6 223 173 129 129 q7 2025196117591759 q8 2697270825912591 q9 7373739773937393 q10 3088336129122912 q11 566 507 489 489 q12 716 796 607 607 q13 3770404233633363 q14 306 301 298 298 q15 534 486 512 486 q16 612 486 442 442 q17 1182151714221422 q18 8157759575387538 q19 828 810 847 810 q20 2000196917921792 q21 4781430444564304 q22 10991028958 958 Total cold run time: 53380 ms Total hot run time: 50841 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
github-actions[bot] commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3101455755 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
github-actions[bot] commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3101455579 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
luwei16 commented on code in PR #53408:
URL: https://github.com/apache/doris/pull/53408#discussion_r2217102516
##
be/src/olap/olap_server.cpp:
##
@@ -684,6 +684,22 @@ void StorageEngine::_compaction_tasks_producer_callback() {
last_base_score_update_time = cur_time;
}
}
+std::unique_ptr& thread_pool =
+(compaction_type == CompactionType::CUMULATIVE_COMPACTION)
+? _cumu_compaction_thread_pool
+: _base_compaction_thread_pool;
+if (config::compaction_num_per_round != 0) {
+_compaction_num_per_round = config::compaction_num_per_round;
+} else if (thread_pool->get_queue_size() == 0) {
+// If all tasks in the thread pool queue are executed,
+// double the number of tasks generated each time,
+// with a maximum of
config::max_automatic_compaction_num_per_round tasks per generation.
Review Comment:
Need to consider the case where there are too many tasks in the queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
Yukang-Lian commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3101441488 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3100696034 ClickBench: Total hot run time: 32.47 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit ca838aa4c9a22d8a1fd62554f4f1dd490f48a260, data reload: false query1 0.040.040.04 query2 0.070.040.04 query3 0.240.070.07 query4 1.640.110.11 query5 0.450.430.42 query6 1.150.670.67 query7 0.020.020.02 query8 0.050.030.03 query9 0.590.530.51 query10 0.570.570.57 query11 0.160.110.11 query12 0.150.120.12 query13 0.630.620.61 query14 0.780.820.80 query15 0.880.880.84 query16 0.390.380.40 query17 1.081.081.08 query18 0.230.210.22 query19 1.951.811.92 query20 0.010.010.01 query21 15.39 0.880.56 query22 0.771.040.78 query23 14.99 1.340.61 query24 6.611.530.75 query25 0.490.260.07 query26 0.660.180.13 query27 0.050.060.06 query28 9.850.980.44 query29 12.55 3.993.29 query30 3.022.972.95 query31 2.820.580.37 query32 3.230.550.48 query33 3.183.093.12 query34 15.99 5.414.84 query35 4.904.884.79 query36 0.660.500.49 query37 0.090.070.06 query38 0.050.030.04 query39 0.040.020.02 query40 0.170.140.14 query41 0.070.020.02 query42 0.040.020.02 query43 0.040.030.03 Total cold run time: 106.74 s Total hot run time: 32.47 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3100688191 TPC-DS: Total hot run time: 186969 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit ca838aa4c9a22d8a1fd62554f4f1dd490f48a260, data reload: false query1 993 391 411 391 query2 6533165716451645 query3 6748216 220 216 query4 26424 23575 22936 22936 query5 4376628 466 466 query6 305 209 205 205 query7 4631500 307 307 query8 274 237 213 213 query9 8612285928652859 query10 476 332 270 270 query11 15927 14979 14763 14763 query12 172 114 115 114 query13 1669523 411 411 query14 8617592257845784 query15 209 191 158 158 query16 7293635 476 476 query17 1194706 565 565 query18 1976401 304 304 query19 185 183 163 163 query20 131 121 109 109 query21 211 122 105 105 query22 4302429840724072 query23 33739 32975 33175 32975 query24 8140236323492349 query25 528 484 438 438 query26 1092272 157 157 query27 2740505 356 356 query28 4407217621552155 query29 760 574 452 452 query30 277 222 185 185 query31 909 823 744 744 query32 83 76 73 73 query33 556 370 325 325 query34 794 843 504 504 query35 793 839 726 726 query36 978 1011886 886 query37 116 110 87 87 query38 4197417341564156 query39 1522143514641435 query40 225 128 117 117 query41 66 54 56 54 query42 122 111 113 111 query43 517 496 473 473 query44 1325839 844 839 query45 179 167 164 164 query46 837 1000625 625 query47 1764185417701770 query48 394 438 328 328 query49 738 471 388 388 query50 658 695 410 410 query51 5469546054075407 query52 118 109 107 107 query53 234 251 193 193 query54 586 604 532 532 query55 87 88 90 88 query56 316 319 303 303 query57 1201121011331133 query58 273 262 278 262 query59 2530266725022502 query60 353 335 324 324 query61 131 124 124 124 query62 784 714 641 641 query63 226 193 187 187 query64 4211776 776 query65 4292460142434243 query66 1199429 352 352 query67 15771 15678 15582 15582 query68 7861887 545 545 query69 508 338 293 293 query70 1210110911441109 query71 457 410 310 310 query72 5600485251534852 query73 732 721 358 358 query74 9240911486608660 query75 3467310226122612 query76 32331146713 713 query77 750 395 326 326 query78 10032 10244 93759375 query79 2595828 584 584 query80 624 526 492 492 query81 464 253 214 214 query82 430 139 109 109 query83 282 250 234 234 query84 289 99 82 82 query85 798 355 321 321 query86 338 302 300 300 query87 4340442143104310 query88 3309228722702270 query89 390 323 290 290 query90 1920218 215 215 query91 141 137 110 110 query92 90 72 66 66 query93 1736955 597 597 query94 712 417 293 293 query95 391 319 308 308 query96 501 587 287 287 query97 2761279526642664 query98 237 220 210 210 query99 1454141312901290 Total cold run time: 274387 ms Total hot run time: 186969 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3100669634 TPC-H: Total hot run time: 34923 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit ca838aa4c9a22d8a1fd62554f4f1dd490f48a260, data reload: false -- Round 1 -- q1 17571 556351995199 q2 1933279 182 182 q3 10318 1274714 714 q4 10222 1011510 510 q5 7759244324212421 q6 191 160 129 129 q7 920 757 609 609 q8 9322128811011101 q9 7514512451805124 q10 6903236719931993 q11 487 293 275 275 q12 346 354 229 229 q13 17791 370530733073 q14 231 226 218 218 q15 564 478 485 478 q16 417 444 372 372 q17 622 887 378 378 q18 7571714071197119 q19 1240961 577 577 q20 341 347 228 228 q21 4198324029992999 q22 10901033995 995 Total cold run time: 107551 ms Total hot run time: 34923 ms - Round 2, with runtime_filter_mode=off - q1 5197516251105110 q2 236 324 228 228 q3 2136264422732273 q4 1378177813321332 q5 4485445646134456 q6 223 175 138 138 q7 1992191017921792 q8 2635257925232523 q9 7386730873387308 q10 3121344629542954 q11 587 515 509 509 q12 697 780 602 602 q13 3703392234733473 q14 293 342 273 273 q15 538 511 486 486 q16 449 507 438 438 q17 1190156514081408 q18 7983775575987598 q19 872 875 940 875 q20 1925196918211821 q21 4775443542654265 q22 1075105110181018 Total cold run time: 52876 ms Total hot run time: 50880 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
Yukang-Lian commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3100524445 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3097241449 ClickBench: Total hot run time: 32.42 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit ca838aa4c9a22d8a1fd62554f4f1dd490f48a260, data reload: false query1 0.050.030.03 query2 0.080.040.04 query3 0.250.080.08 query4 1.620.110.12 query5 0.440.430.42 query6 1.150.660.68 query7 0.020.020.02 query8 0.050.040.04 query9 0.590.540.53 query10 0.590.570.58 query11 0.160.120.11 query12 0.160.120.13 query13 0.630.610.60 query14 0.800.820.82 query15 0.890.870.87 query16 0.390.390.39 query17 1.081.081.09 query18 0.240.220.21 query19 1.901.901.83 query20 0.020.010.02 query21 15.39 0.890.53 query22 0.761.130.68 query23 14.97 1.360.66 query24 6.801.410.32 query25 0.300.220.13 query26 0.620.170.15 query27 0.050.060.06 query28 9.470.970.44 query29 12.58 3.943.28 query30 3.113.113.09 query31 2.830.600.38 query32 3.250.570.48 query33 3.133.103.09 query34 16.14 5.394.79 query35 4.914.904.89 query36 0.700.510.51 query37 0.100.070.07 query38 0.070.060.05 query39 0.040.040.03 query40 0.180.160.15 query41 0.090.040.03 query42 0.050.030.03 query43 0.050.040.04 Total cold run time: 106.7 s Total hot run time: 32.42 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3097222771 TPC-DS: Total hot run time: 191196 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit ca838aa4c9a22d8a1fd62554f4f1dd490f48a260, data reload: false query1 1002399 424 399 query2 6583176417291729 query3 6755238 225 225 query4 26047 23630 23724 23630 query5 4489722 598 598 query6 329 257 253 253 query7 4642523 307 307 query8 385 351 339 339 query9 8606296229382938 query10 513 387 323 323 query11 15920 15044 14821 14821 query12 189 148 130 130 query13 1690570 455 455 query14 9765608459485948 query15 216 202 183 183 query16 7667639 510 510 query17 1199746 597 597 query18 2058445 332 332 query19 202 200 177 177 query20 160 133 128 128 query21 234 139 126 126 query22 4252432040984098 query23 34176 33375 33105 33105 query24 8083240523992399 query25 561 496 424 424 query26 1243286 175 175 query27 2693515 369 369 query28 4385220321882188 query29 763 604 474 474 query30 297 236 205 205 query31 969 870 773 773 query32 92 85 89 85 query33 584 415 367 367 query34 813 852 546 546 query35 809 850 747 747 query36 981 1015941 941 query37 123 116 99 99 query38 4147413641934136 query39 1574148914701470 query40 236 142 134 134 query41 110 103 102 102 query42 132 123 120 120 query43 529 517 496 496 query44 1372884 868 868 query45 193 187 182 182 query46 854 1018649 649 query47 1821185317721772 query48 406 455 341 341 query49 838 557 485 485 query50 659 705 445 445 query51 5598568554325432 query52 124 118 111 111 query53 265 286 216 216 query54 623 612 556 556 query55 97 93 96 93 query56 351 354 347 347 query57 1243123811621162 query58 315 309 317 309 query59 2629266625232523 query60 388 395 374 374 query61 153 152 160 152 query62 806 746 678 678 query63 247 215 214 214 query64 43681105731 731 query65 4295420942424209 query66 1254589 505 505 query67 15855 15660 15562 15562 query68 8077902 560 560 query69 503 338 315 315 query70 1296111711391117 query71 469 358 331 331 query72 5322464846384638 query73 742 604 381 381 query74 9102923690769076 query75 3798323727962796 query76 36181162749 749 query77 867 458 377 377 query78 10024 10496 95229522 query79 1802816 592 592 query80 665 599 529 529 query81 489 278 238 238 query82 437 145 121 121 query83 283 282 283 282 query84 256 121 92 92 query85 813 450 355 355 query86 367 336 330 330 query87 4469449243504350 query88 3034241924232419 query89 403 360 304 304 query90 1945243 242 242 query91 156 154 127 127 query92 98 83 81 81 query93 1065920 590 590 query94 689 407 313 313 query95 410 332 326 326 query96 500 608 292 292 query97 2799279827372737 query98 251 234 227 227 query99 1460141012701270 Total cold run time: 277166 ms Total hot run time: 191196 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3097177706 TPC-H: Total hot run time: 34472 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit ca838aa4c9a22d8a1fd62554f4f1dd490f48a260, data reload: false -- Round 1 -- q1 17606 527251335133 q2 1946294 206 206 q3 10331 1298732 732 q4 10268 1028538 538 q5 7991245123622362 q6 193 171 142 142 q7 956 774 629 629 q8 9352137911211121 q9 7399511350955095 q10 6950240920012001 q11 495 306 288 288 q12 369 372 233 233 q13 17785 374731203120 q14 258 242 234 234 q15 611 543 521 521 q16 449 447 389 389 q17 577 877 375 375 q18 7681710471267104 q19 15101003616 616 q20 362 369 240 240 q21 3849258723932393 q22 1077104610001000 Total cold run time: 108015 ms Total hot run time: 34472 ms - Round 2, with runtime_filter_mode=off - q1 5209522251615161 q2 266 351 245 245 q3 2220265223492349 q4 1429182413831383 q5 4502442744834427 q6 228 180 142 142 q7 2011199517311731 q8 2668251226062512 q9 7315726072587258 q10 3113344728242824 q11 583 560 495 495 q12 717 755 694 694 q13 3726400433033303 q14 302 318 303 303 q15 585 532 516 516 q16 465 514 460 460 q17 1218156014251425 q18 7844760975887588 q19 818 847 959 847 q20 2017215819311931 q21 5216466143824382 q22 1121102710011001 Total cold run time: 53573 ms Total hot run time: 50977 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
Yukang-Lian commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3096783423 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3094640936 ClickBench: Total hot run time: 32.38 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit d6e51388727cc2fa37c53325176b464881e1f569, data reload: false query1 0.050.040.04 query2 0.080.040.04 query3 0.240.080.08 query4 1.620.110.11 query5 0.440.410.40 query6 1.170.660.67 query7 0.030.020.02 query8 0.050.040.04 query9 0.600.540.51 query10 0.600.570.58 query11 0.160.110.11 query12 0.160.130.12 query13 0.630.620.61 query14 0.810.830.81 query15 0.900.860.86 query16 0.390.400.39 query17 1.091.091.07 query18 0.220.220.22 query19 1.961.851.81 query20 0.020.020.02 query21 15.40 0.910.55 query22 0.781.210.75 query23 14.83 1.350.64 query24 6.451.600.32 query25 0.360.230.07 query26 0.650.170.15 query27 0.050.050.06 query28 9.550.970.44 query29 12.55 4.043.30 query30 3.163.093.07 query31 2.820.600.38 query32 3.230.550.49 query33 3.083.183.06 query34 16.05 5.494.83 query35 4.884.874.90 query36 0.690.520.49 query37 0.100.080.08 query38 0.060.050.05 query39 0.040.040.03 query40 0.190.160.16 query41 0.090.030.03 query42 0.040.040.04 query43 0.050.050.04 Total cold run time: 106.32 s Total hot run time: 32.38 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3094638060 TPC-DS: Total hot run time: 190709 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit d6e51388727cc2fa37c53325176b464881e1f569, data reload: false query1 1014407 429 407 query2 6596187017801780 query3 6767233 226 226 query4 26326 23945 23478 23478 query5 4449692 529 529 query6 310 257 238 238 query7 4653522 301 301 query8 373 339 334 334 query9 8685291929442919 query10 517 380 324 324 query11 15313 15537 14998 14998 query12 179 138 130 130 query13 1694572 455 455 query14 9626605161936051 query15 238 267 174 174 query16 7669664 468 468 query17 1258732 607 607 query18 2041432 370 370 query19 214 200 173 173 query20 145 138 131 131 query21 236 148 128 128 query22 4508436642194219 query23 34015 32912 33019 32912 query24 8195237923412341 query25 539 487 444 444 query26 1241282 167 167 query27 2687510 367 367 query28 4361218221682168 query29 748 574 458 458 query30 294 231 210 210 query31 972 883 809 809 query32 90 89 96 89 query33 602 406 366 366 query34 847 860 529 529 query35 810 842 767 767 query36 994 1017927 927 query37 136 118 94 94 query38 4160418740894089 query39 1527147314701470 query40 257 144 130 130 query41 113 133 103 103 query42 131 120 121 120 query43 532 547 491 491 query44 1346863 868 863 query45 188 182 181 181 query46 855 1000651 651 query47 1844186218061806 query48 411 456 334 334 query49 799 601 499 499 query50 665 715 425 425 query51 5430555955725559 query52 129 121 110 110 query53 251 281 214 214 query54 616 618 554 554 query55 97 92 92 92 query56 358 360 345 345 query57 1256123311741174 query58 318 312 319 312 query59 2698298127152715 query60 403 368 361 361 query61 153 150 145 145 query62 816 767 679 679 query63 247 213 210 210 query64 43381094765 765 query65 4306418441814181 query66 1280591 514 514 query67 15941 15586 15415 15415 query68 8063909 558 558 query69 502 346 306 306 query70 1226110811531108 query71 486 412 338 338 query72 5733474547464745 query73 749 639 371 371 query74 8871924986288628 query75 3808322627262726 query76 36221136714 714 query77 846 456 379 379 query78 10016 10328 94269426 query79 2174844 617 617 query80 779 596 529 529 query81 493 275 241 241 query82 433 146 115 115 query83 282 279 277 277 query84 253 117 98 98 query85 810 409 450 409 query86 353 349 306 306 query87 4464436144114361 query88 3239238723822382 query89 401 339 307 307 query90 1964237 235 235 query91 149 150 128 128 query92 97 85 78 78 query93 1192960 604 604 query94 690 417 321 321 query95 425 330 329 329 query96 498 586 291 291 query97 2751279326872687 query98 261 251 229 229 query99 1480139412711271 Total cold run time: 277917 ms Total hot run time: 190709 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3094632415 TPC-H: Total hot run time: 34493 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit d6e51388727cc2fa37c53325176b464881e1f569, data reload: false -- Round 1 -- q1 17585 516451885164 q2 1935303 213 213 q3 10262 1303721 721 q4 10225 1019512 512 q5 7522245923902390 q6 193 170 140 140 q7 941 775 617 617 q8 93421372 q9 7508514551305130 q10 6895237220052005 q11 515 304 280 280 q12 352 364 234 234 q13 17808 367631273127 q14 247 250 231 231 q15 573 525 531 525 q16 445 443 394 394 q17 606 892 394 394 q18 7611713671487136 q19 1253975 589 589 q20 357 381 256 256 q21 3867322223102310 q22 1100106310141014 Total cold run time: 107142 ms Total hot run time: 34493 ms - Round 2, with runtime_filter_mode=off - q1 5115512551385125 q2 268 355 243 243 q3 2151270322892289 q4 1339176213251325 q5 4347443444284428 q6 260 178 135 135 q7 2108191219091909 q8 2712259226572592 q9 7314737172717271 q10 3185327828442844 q11 574 522 507 507 q12 682 761 611 611 q13 3570390533613361 q14 308 305 292 292 q15 543 684 563 563 q16 463 495 448 448 q17 1161160013471347 q18 7820767877607678 q19 813 812 855 812 q20 2026210519231923 q21 5059444843544354 q22 1128104410431043 Total cold run time: 52946 ms Total hot run time: 51100 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
dataroaring commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3094578169 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3088988840 ClickBench: Total hot run time: 32.16 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 5065f5a94c3710a27c48b483dcdf34618b2947da, data reload: false query1 0.050.040.04 query2 0.080.040.04 query3 0.250.080.08 query4 1.620.110.11 query5 0.460.440.41 query6 1.140.650.68 query7 0.030.020.02 query8 0.050.030.04 query9 0.590.530.52 query10 0.600.590.57 query11 0.170.120.11 query12 0.150.120.12 query13 0.640.630.60 query14 0.810.810.81 query15 0.890.870.87 query16 0.380.380.39 query17 1.061.091.07 query18 0.240.210.21 query19 1.941.821.88 query20 0.010.010.01 query21 15.40 0.870.54 query22 0.751.300.70 query23 14.79 1.380.60 query24 7.281.510.34 query25 0.380.200.06 query26 0.610.170.14 query27 0.070.050.06 query28 9.580.900.44 query29 12.56 3.873.23 query30 3.123.033.03 query31 2.830.590.39 query32 3.240.550.48 query33 3.083.173.15 query34 16.26 5.404.76 query35 4.864.864.86 query36 0.690.510.50 query37 0.110.080.08 query38 0.060.050.04 query39 0.040.030.04 query40 0.190.160.16 query41 0.080.040.04 query42 0.040.030.03 query43 0.040.040.04 Total cold run time: 107.22 s Total hot run time: 32.16 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3088966748 TPC-DS: Total hot run time: 189844 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 5065f5a94c3710a27c48b483dcdf34618b2947da, data reload: false query1 1038393 415 393 query2 6555178817841784 query3 6751230 220 220 query4 26036 23560 23344 23344 query5 4798657 508 508 query6 325 223 220 220 query7 4638508 295 295 query8 349 335 323 323 query9 8652265226592652 query10 519 375 315 315 query11 15840 15097 14767 14767 query12 203 127 126 126 query13 1695577 439 439 query14 10067 600259845984 query15 231 210 187 187 query16 7693638 510 510 query17 1339728 574 574 query18 2056433 322 322 query19 200 205 172 172 query20 136 135 131 131 query21 233 186 137 137 query22 4387430043804300 query23 34128 33268 33103 33103 query24 8421243324112411 query25 520 472 426 426 query26 1133276 157 157 query27 2701517 357 357 query28 4335220021642164 query29 749 588 485 485 query30 311 237 209 209 query31 985 896 797 797 query32 83 80 74 74 query33 603 381 353 353 query34 801 868 535 535 query35 802 866 751 751 query36 973 1020942 942 query37 112 118 86 86 query38 4217411241404112 query39 1518148114461446 query40 247 140 130 130 query41 106 103 99 99 query42 131 120 119 119 query43 551 520 516 516 query44 1314845 852 845 query45 203 182 182 182 query46 850 1022659 659 query47 1875189618131813 query48 413 453 324 324 query49 816 553 463 463 query50 681 726 432 432 query51 5540554255215521 query52 123 115 112 112 query53 255 275 215 215 query54 601 592 529 529 query55 91 89 85 85 query56 324 333 323 323 query57 1229123611691169 query58 306 294 296 294 query59 2720279525982598 query60 353 347 337 337 query61 153 145 153 145 query62 840 738 664 664 query63 240 214 205 205 query64 42311055734 734 query65 4271418041894180 query66 1212638 495 495 query67 15756 15777 15624 15624 query68 7996920 555 555 query69 496 326 289 289 query70 1245114911491149 query71 486 338 321 321 query72 5291467947074679 query73 723 583 372 372 query74 9139918987358735 query75 3794325527582758 query76 36241170726 726 query77 854 436 350 350 query78 10124 10142 94179417 query79 2964852 602 602 query80 702 579 503 503 query81 482 274 242 242 query82 477 133 113 113 query83 307 291 267 267 query84 311 119 90 90 query85 832 389 360 360 query86 360 348 296 296 query87 4588457443134313 query88 2948234723482347 query89 440 354 310 310 query90 1984223 222 222 query91 152 158 141 141 query92 86 120 67 67 query93 2031947 600 600 query94 696 436 324 324 query95 396 314 314 314 query96 486 592 284 284 query97 2738279726632663 query98 237 224 215 215 query99 1482143012971297 Total cold run time: 280036 ms Total hot run time: 189844 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3088932426 TPC-H: Total hot run time: 34373 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 5065f5a94c3710a27c48b483dcdf34618b2947da, data reload: false -- Round 1 -- q1 17609 527750825082 q2 1949302 195 195 q3 10472 1337733 733 q4 10248 1020527 527 q5 7978246023622362 q6 192 170 138 138 q7 965 788 621 621 q8 9339135511221122 q9 6937513751345134 q10 6924238819701970 q11 504 309 292 292 q12 358 369 229 229 q13 17780 373930813081 q14 244 244 225 225 q15 588 524 521 521 q16 437 437 388 388 q17 587 888 357 357 q18 7661725572587255 q19 12631001590 590 q20 363 358 237 237 q21 3719257223262326 q22 10391044988 988 Total cold run time: 107156 ms Total hot run time: 34373 ms - Round 2, with runtime_filter_mode=off - q1 5141511551105110 q2 267 355 241 241 q3 2228268022982298 q4 1337175913101310 q5 4233413841144114 q6 208 174 131 131 q7 1878183616771677 q8 2533248224202420 q9 6883678068096780 q10 2953315527132713 q11 600 513 526 513 q12 641 760 615 615 q13 371930723072 q14 282 294 269 269 q15 542 499 493 493 q16 460 483 441 441 q17 1103150113401340 q18 7246714870387038 q19 778 878 1070878 q20 1893194818071807 q21 4635436741924192 q22 10761042990 990 Total cold run time: 50250 ms Total hot run time: 48442 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
Yukang-Lian commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3088585911 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
hello-stephen commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3084618523 # BE Regression && UT Coverage Report Increment line coverage `90.24% (37/41)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/53408_921eb2e5dc9fc166bc92317d91c3d1d13fda2604_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/53408_921eb2e5dc9fc166bc92317d91c3d1d13fda2604_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 81.05% (21831/26936) | | Line Coverage | 73.61% (225672/306571) | | Region Coverage | 61.17% (187760/306962) | | Branch Coverage | 65.02% (81140/124789) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
hello-stephen commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3084600763 # BE UT Coverage Report Increment line coverage `19.51% (8/41)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/921eb2e5dc9fc166bc92317d91c3d1d13fda2604_921eb2e5dc9fc166bc92317d91c3d1d13fda2604/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/921eb2e5dc9fc166bc92317d91c3d1d13fda2604_921eb2e5dc9fc166bc92317d91c3d1d13fda2604/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.39% (15757/27456) | | Line Coverage | 46.15% (141703/307041) | | Region Coverage | 35.64% (106631/299231) | | Branch Coverage | 38.15% (47076/123399) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
dataroaring commented on code in PR #53408:
URL: https://github.com/apache/doris/pull/53408#discussion_r2213410027
##
be/src/olap/olap_server.cpp:
##
@@ -684,6 +684,24 @@ void StorageEngine::_compaction_tasks_producer_callback() {
last_base_score_update_time = cur_time;
}
}
+std::unique_ptr& thread_pool =
+(compaction_type == CompactionType::CUMULATIVE_COMPACTION)
+? _cumu_compaction_thread_pool
+: _base_compaction_thread_pool;
+// If all tasks in the thread pool queue are executed,
+// double the number of tasks generated each time,
+// with a maximum of 64 tasks per generation.
+if (thread_pool->get_queue_size() == 0) {
+if (_compaction_num_per_round < 64) {
+_compaction_num_per_round *= 2;
Review Comment:
max should be a config.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3084027535 ClickBench: Total hot run time: 32.38 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 921eb2e5dc9fc166bc92317d91c3d1d13fda2604, data reload: false query1 0.040.030.04 query2 0.080.040.04 query3 0.250.070.07 query4 1.630.100.10 query5 0.440.420.43 query6 1.170.660.69 query7 0.020.020.01 query8 0.040.030.04 query9 0.590.530.52 query10 0.570.590.58 query11 0.160.110.12 query12 0.150.110.11 query13 0.640.610.60 query14 0.790.820.80 query15 0.890.880.86 query16 0.390.390.39 query17 1.071.041.06 query18 0.230.220.20 query19 1.961.841.88 query20 0.010.010.01 query21 15.38 0.920.58 query22 0.751.060.61 query23 15.15 1.380.65 query24 7.281.800.62 query25 0.550.100.13 query26 0.670.160.14 query27 0.070.050.06 query28 9.330.870.44 query29 12.54 4.093.36 query30 3.032.982.93 query31 2.820.600.38 query32 3.230.550.47 query33 3.113.143.13 query34 16.12 5.384.81 query35 4.784.864.84 query36 0.700.510.52 query37 0.090.070.07 query38 0.040.040.03 query39 0.030.030.03 query40 0.180.140.14 query41 0.080.030.03 query42 0.030.030.02 query43 0.040.030.02 Total cold run time: 107.12 s Total hot run time: 32.38 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3084010176 TPC-DS: Total hot run time: 186395 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 921eb2e5dc9fc166bc92317d91c3d1d13fda2604, data reload: false query1 983 393 392 392 query2 6520162416641624 query3 6741213 212 212 query4 26356 23686 23343 23343 query5 4364556 428 428 query6 314 213 198 198 query7 4629494 305 305 query8 259 222 213 213 query9 8615261826182618 query10 483 320 261 261 query11 15390 15515 14856 14856 query12 153 109 108 108 query13 1638528 389 389 query14 9159588859295888 query15 212 209 172 172 query16 7232651 424 424 query17 1212723 601 601 query18 1984409 324 324 query19 195 194 165 165 query20 129 115 110 110 query21 218 120 108 108 query22 4251429140074007 query23 34110 32838 33186 32838 query24 8441236923342334 query25 544 510 384 384 query26 1237273 146 146 query27 2756503 354 354 query28 4313215621272127 query29 745 551 421 421 query30 279 220 197 197 query31 924 842 773 773 query32 70 71 60 60 query33 545 359 296 296 query34 818 829 511 511 query35 798 841 755 755 query36 967 992 915 915 query37 103 96 74 74 query38 4116410541034103 query39 1493141214241412 query40 210 114 104 104 query41 57 54 51 51 query42 121 109 104 104 query43 518 493 487 487 query44 1303809 799 799 query45 177 168 163 163 query46 840 1009629 629 query47 1771182817631763 query48 383 453 320 320 query49 750 479 388 388 query50 640 683 406 406 query51 5490550155025501 query52 111 104 100 100 query53 215 256 183 183 query54 562 577 496 496 query55 80 79 79 79 query56 292 297 279 279 query57 1182120511511151 query58 263 265 274 265 query59 2676269425602560 query60 328 308 301 301 query61 124 121 122 121 query62 795 715 654 654 query63 221 190 182 182 query64 43161011700 700 query65 4310419141984191 query66 1150417 312 312 query67 15771 15660 15448 15448 query68 8261894 540 540 query69 488 311 270 270 query70 1201116010771077 query71 464 329 286 286 query72 5623472348044723 query73 736 639 353 353 query74 8960915287718771 query75 3939321627332733 query76 37651175720 720 query77 784 368 293 293 query78 987910276 93079307 query79 2151809 586 586 query80 650 515 443 443 query81 523 250 229 229 query82 456 137 98 98 query83 273 255 238 238 query84 285 99 77 77 query85 790 346 306 306 query86 334 307 319 307 query87 4425444543954395 query88 2849223023022230 query89 399 324 285 285 query90 1943202 201 201 query91 137 142 110 110 query92 75 62 55 55 query93 1173941 595 595 query94 668 413 309 309 query95 375 291 281 281 query96 489 568 275 275 query97 2762283126032603 query98 233 205 214 205 query99 1464139313111311 Total cold run time: 274506 ms Total hot run time: 186395 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3083972392 TPC-H: Total hot run time: 34014 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 921eb2e5dc9fc166bc92317d91c3d1d13fda2604, data reload: false -- Round 1 -- q1 17562 519050405040 q2 1927296 189 189 q3 10286 1310707 707 q4 10215 1041523 523 q5 7640242723632363 q6 191 162 127 127 q7 897 738 622 622 q8 9321129911841184 q9 6878513650665066 q10 6889240019771977 q11 502 290 279 279 q12 343 347 209 209 q13 17773 366631173117 q14 234 224 224 224 q15 563 486 484 484 q16 418 436 361 361 q17 600 857 353 353 q18 7641723171237123 q19 1221962 571 571 q20 338 330 223 223 q21 3902252123162316 q22 10461015956 956 Total cold run time: 106387 ms Total hot run time: 34014 ms - Round 2, with runtime_filter_mode=off - q1 5182507151155071 q2 253 317 216 216 q3 2163268023252325 q4 1369177213811381 q5 4218443344704433 q6 225 172 129 129 q7 2028200118261826 q8 2738261425582558 q9 7484735773227322 q10 3133329829202920 q11 571 510 525 510 q12 694 779 652 652 q13 3558391233403340 q14 297 312 300 300 q15 537 491 478 478 q16 460 519 452 452 q17 1157153013921392 q18 8212770075807580 q19 862 831 865 831 q20 1967200718461846 q21 4730438643104310 q22 10771033988 988 Total cold run time: 52915 ms Total hot run time: 50860 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
Yukang-Lian commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3083900502 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3083637562 ClickBench: Total hot run time: 32.42 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 93e0caec68f662f205ab10006afb4809b4095d68, data reload: false query1 0.040.030.04 query2 0.080.040.04 query3 0.240.080.07 query4 1.620.100.10 query5 0.440.430.42 query6 1.170.690.70 query7 0.030.020.02 query8 0.040.050.04 query9 0.590.530.52 query10 0.570.580.58 query11 0.160.100.11 query12 0.150.120.12 query13 0.650.610.62 query14 0.800.850.85 query15 0.940.900.90 query16 0.390.400.39 query17 1.091.061.06 query18 0.230.210.20 query19 2.021.861.94 query20 0.020.020.01 query21 15.37 0.910.55 query22 0.751.170.73 query23 14.91 1.380.58 query24 7.491.460.33 query25 0.330.160.09 query26 0.670.160.15 query27 0.060.050.06 query28 8.960.910.46 query29 12.67 4.033.33 query30 3.052.982.94 query31 2.830.580.40 query32 3.230.600.49 query33 3.223.153.20 query34 16.11 5.584.82 query35 4.914.924.91 query36 0.700.520.50 query37 0.090.060.06 query38 0.050.050.04 query39 0.030.020.02 query40 0.180.160.14 query41 0.080.030.02 query42 0.030.030.02 query43 0.030.030.03 Total cold run time: 107.02 s Total hot run time: 32.42 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3083620383 TPC-DS: Total hot run time: 186748 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 93e0caec68f662f205ab10006afb4809b4095d68, data reload: false query1 988 384 400 384 query2 6500170816281628 query3 6738217 213 213 query4 27313 23503 22958 22958 query5 4329606 444 444 query6 313 214 201 201 query7 4619496 302 302 query8 274 220 221 220 query9 8592262726122612 query10 469 318 274 274 query11 15639 14985 14838 14838 query12 160 110 109 109 query13 1650544 414 414 query14 8633591958455845 query15 212 201 169 169 query16 7149673 482 482 query17 1215725 588 588 query18 1990411 310 310 query19 190 187 160 160 query20 124 121 117 117 query21 216 127 106 106 query22 4205442340354035 query23 34149 33262 33146 33146 query24 8455236923752369 query25 543 448 397 397 query26 1243265 146 146 query27 2771503 342 342 query28 4307214421212121 query29 767 547 463 463 query30 278 220 192 192 query31 937 853 750 750 query32 68 63 62 62 query33 558 366 307 307 query34 795 834 514 514 query35 775 814 717 717 query36 992 1005921 921 query37 110 99 71 71 query38 4257423542014201 query39 1485141314391413 query40 206 117 105 105 query41 59 57 51 51 query42 121 105 114 105 query43 499 502 462 462 query44 1331808 819 808 query45 174 167 163 163 query46 844 995 617 617 query47 1765182217441744 query48 376 461 308 308 query49 724 485 421 421 query50 676 704 385 385 query51 5527546454745464 query52 112 111 101 101 query53 219 256 182 182 query54 563 560 502 502 query55 85 82 81 81 query56 296 283 276 276 query57 1186118911551155 query58 268 256 258 256 query59 2591263625062506 query60 321 321 312 312 query61 156 122 127 122 query62 791 720 666 666 query63 227 190 183 183 query64 4382999 686 686 query65 4272419042164190 query66 1131394 301 301 query67 15953 15888 15383 15383 query68 8046895 527 527 query69 467 300 270 270 query70 1175121511091109 query71 471 323 299 299 query72 5638483248064806 query73 713 638 357 357 query74 8903908489598959 query75 3802318926512651 query76 36471155736 736 query77 797 389 309 309 query78 10117 10091 94189418 query79 5010809 557 557 query80 657 522 452 452 query81 463 255 226 226 query82 666 125 98 98 query83 288 246 234 234 query84 290 110 84 84 query85 810 453 323 323 query86 341 316 303 303 query87 4492448343814381 query88 3242228722992287 query89 449 325 294 294 query90 1938208 204 204 query91 143 150 128 128 query92 74 64 56 56 query93 2917948 606 606 query94 677 419 310 310 query95 379 297 287 287 query96 501 578 283 283 query97 2718277926402640 query98 241 207 210 207 query99 1437142712711271 Total cold run time: 280372 ms Total hot run time: 186748 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
doris-robot commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3083589446 TPC-H: Total hot run time: 34246 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 93e0caec68f662f205ab10006afb4809b4095d68, data reload: false -- Round 1 -- q1 17581 540851805180 q2 1924287 178 178 q3 10322 1307739 739 q4 10224 1018544 544 q5 7526243123482348 q6 183 162 130 130 q7 922 764 601 601 q8 9318136410891089 q9 7239508450675067 q10 6893237420062006 q11 480 283 285 283 q12 346 360 213 213 q13 17758 370430873087 q14 250 241 224 224 q15 552 488 496 488 q16 422 428 378 378 q17 588 881 355 355 q18 7498722871617161 q19 1203936 574 574 q20 356 355 221 221 q21 3778328824062406 q22 10541043974 974 Total cold run time: 106417 ms Total hot run time: 34246 ms - Round 2, with runtime_filter_mode=off - q1 5354533853825338 q2 253 338 231 231 q3 2212269322892289 q4 1328179113501350 q5 4424460845464546 q6 232 167 126 126 q7 1992194518501850 q8 2649262724812481 q9 7260723971247124 q10 3120331729172917 q11 597 511 518 511 q12 697 782 638 638 q13 3608408535013501 q14 296 308 263 263 q15 508 553 472 472 q16 470 516 460 460 q17 1212159113781378 q18 8040762975647564 q19 858 822 1002822 q20 2085199818691869 q21 4895445744034403 q22 1074102410131013 Total cold run time: 53164 ms Total hot run time: 51146 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
Yukang-Lian commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3083336965 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Enhancement](Compaction) Support auto set compaction task num per round [doris]
Yukang-Lian commented on PR #53408: URL: https://github.com/apache/doris/pull/53408#issuecomment-3083322611 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
