Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 merged PR #52879: URL: https://github.com/apache/doris/pull/52879 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
hello-stephen commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3082069193 # BE Regression && UT Coverage Report Increment line coverage `97.96% (96/98)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/52879_36f2eb433af77dbd0208bb18c7a3bc3ced58ced9_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/52879_36f2eb433af77dbd0208bb18c7a3bc3ced58ced9_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 81.10% (21753/26822) | | Line Coverage | 73.70% (223551/303331) | | Region Coverage | 61.31% (185622/302760) | | Branch Coverage | 65.23% (79905/122505) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
hello-stephen commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3078894628 # BE Regression && UT Coverage Report Increment line coverage `97.96% (96/98)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/52879_36f2eb433af77dbd0208bb18c7a3bc3ced58ced9_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/52879_36f2eb433af77dbd0208bb18c7a3bc3ced58ced9_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 81.10% (21753/26822) | | Line Coverage | 73.70% (223551/303331) | | Region Coverage | 61.31% (185622/302760) | | Branch Coverage | 65.23% (79905/122505) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3078807064 # BE UT Coverage Report Increment line coverage `32.67% (33/101)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/36f2eb433af77dbd0208bb18c7a3bc3ced58ced9_36f2eb433af77dbd0208bb18c7a3bc3ced58ced9/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/36f2eb433af77dbd0208bb18c7a3bc3ced58ced9_36f2eb433af77dbd0208bb18c7a3bc3ced58ced9/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.50% (15723/27346) | | Line Coverage | 46.13% (140209/303942) | | Region Coverage | 35.41% (104495/295102) | | Branch Coverage | 38.03% (46080/121161) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3078540628 ClickBench: Total hot run time: 31.95 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 36f2eb433af77dbd0208bb18c7a3bc3ced58ced9, data reload: false query1 0.040.030.04 query2 0.080.040.04 query3 0.240.070.07 query4 1.620.100.11 query5 0.440.430.41 query6 1.150.670.68 query7 0.020.020.02 query8 0.050.030.04 query9 0.590.510.50 query10 0.580.570.56 query11 0.160.100.11 query12 0.150.120.12 query13 0.630.610.61 query14 0.810.820.80 query15 0.880.870.91 query16 0.400.390.39 query17 1.071.071.07 query18 0.220.210.21 query19 1.911.811.89 query20 0.010.010.01 query21 15.38 0.900.55 query22 0.761.050.65 query23 15.09 1.390.58 query24 6.561.420.46 query25 0.530.180.07 query26 0.580.160.15 query27 0.070.050.06 query28 9.860.950.44 query29 12.51 4.093.34 query30 3.012.982.88 query31 2.820.620.38 query32 3.230.550.48 query33 3.173.063.14 query34 16.01 5.434.82 query35 4.874.854.81 query36 0.700.510.48 query37 0.090.070.06 query38 0.060.040.04 query39 0.030.030.02 query40 0.160.150.14 query41 0.070.020.03 query42 0.030.020.02 query43 0.030.040.03 Total cold run time: 106.67 s Total hot run time: 31.95 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3078509458 TPC-DS: Total hot run time: 186876 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 36f2eb433af77dbd0208bb18c7a3bc3ced58ced9, data reload: false query1 1001392 441 392 query2 6517172516411641 query3 6742209 226 209 query4 26362 23867 23504 23504 query5 4355592 433 433 query6 308 216 206 206 query7 4623490 289 289 query8 269 225 220 220 query9 8614262626292626 query10 506 326 282 282 query11 15765 15037 14744 14744 query12 158 103 105 103 query13 1633514 395 395 query14 9165585358325832 query15 201 203 169 169 query16 7342617 483 483 query17 1174705 583 583 query18 1989394 296 296 query19 193 187 172 172 query20 122 116 114 114 query21 217 126 111 111 query22 4182414940634063 query23 34041 32976 32780 32780 query24 8440237223572357 query25 549 471 422 422 query26 1228270 152 152 query27 2773503 350 350 query28 4351216621122112 query29 783 581 470 470 query30 282 224 189 189 query31 955 846 773 773 query32 74 65 63 63 query33 576 360 332 332 query34 808 844 514 514 query35 801 816 738 738 query36 976 996 910 910 query37 111 102 89 89 query38 4088411241404112 query39 1526144414211421 query40 219 129 110 110 query41 66 74 58 58 query42 124 107 107 107 query43 510 480 492 480 query44 1328823 817 817 query45 179 175 168 168 query46 880 1020622 622 query47 1790179017261726 query48 384 420 307 307 query49 721 447 390 390 query50 616 689 416 416 query51 5410559255385538 query52 106 114 100 100 query53 224 248 185 185 query54 577 577 508 508 query55 86 79 81 79 query56 310 366 303 303 query57 1170119311051105 query58 264 256 259 256 query59 2522259125172517 query60 332 325 306 306 query61 129 126 127 126 query62 801 721 681 681 query63 222 185 180 180 query64 43561017667 667 query65 4287423842004200 query66 1153421 323 323 query67 15866 15704 15597 15597 query68 8903898 532 532 query69 479 328 271 271 query70 1240114910901090 query71 470 326 306 306 query72 5311470346564656 query73 756 569 349 349 query74 8891893790518937 query75 4173321326842684 query76 36131155719 719 query77 794 375 295 295 query78 988410100 93809380 query79 3078816 585 585 query80 655 507 450 450 query81 474 252 219 219 query82 262 136 97 97 query83 284 248 243 243 query84 287 106 90 90 query85 757 370 325 325 query86 350 324 284 284 query87 4385442542824282 query88 2858225322622253 query89 431 321 293 293 query90 2065210 208 208 query91 140 137 120 120 query92 81 62 55 55 query93 2091932 585 585 query94 680 438 318 318 query95 382 293 291 291 query96 478 579 288 288 query97 2734276326422642 query98 226 212 201 201 query99 1413140513051305 Total cold run time: 277057 ms Total hot run time: 186876 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3078448881 TPC-H: Total hot run time: 34017 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 36f2eb433af77dbd0208bb18c7a3bc3ced58ced9, data reload: false -- Round 1 -- q1 17567 527451125112 q2 1920286 181 181 q3 10329 1285717 717 q4 10225 1025530 530 q5 7462242623512351 q6 181 166 133 133 q7 899 732 624 624 q8 9296131710481048 q9 6924518451475147 q10 6877239519771977 q11 466 296 276 276 q12 352 349 215 215 q13 17767 370830883088 q14 223 241 219 219 q15 554 480 468 468 q16 427 438 380 380 q17 586 865 349 349 q18 7533717471277127 q19 1217962 570 570 q20 344 336 225 225 q21 3670319523352335 q22 1063992 945 945 Total cold run time: 105882 ms Total hot run time: 34017 ms - Round 2, with runtime_filter_mode=off - q1 5226514051795140 q2 242 330 226 226 q3 2212269222862286 q4 1309176513231323 q5 4272454144124412 q6 215 170 131 131 q7 2057191418011801 q8 2651260225752575 q9 7282725973107259 q10 3141334529512951 q11 604 550 503 503 q12 705 902 595 595 q13 3657395633363336 q14 280 316 283 283 q15 521 479 496 479 q16 455 528 442 442 q17 1191155914521452 q18 7795767376727672 q19 795 807 819 807 q20 2042203518051805 q21 4821429943894299 q22 10411013991 991 Total cold run time: 52514 ms Total hot run time: 50768 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
github-actions[bot] commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3078267851 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
github-actions[bot] commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3078267741 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3078070075 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
yiguolei commented on code in PR #52879:
URL: https://github.com/apache/doris/pull/52879#discussion_r2209523907
##
be/src/pipeline/exec/olap_scan_operator.cpp:
##
@@ -450,30 +455,34 @@ Status
OlapScanLocalState::_init_scanners(std::list* sc
return Status::OK();
}
-Status OlapScanLocalState::hold_tablets() {
-if (!_tablets.empty()) {
-return Status::OK();
-}
-
-MonotonicStopWatch timer;
-timer.start();
-_tablets.resize(_scan_ranges.size());
-_read_sources.resize(_scan_ranges.size());
-
-if (config::is_cloud_mode()) {
-std::vector sync_statistics(_scan_ranges.size());
-std::vector> tasks {};
-tasks.reserve(_scan_ranges.size());
-int64_t duration_ns {0};
-{
-SCOPED_RAW_TIMER(&duration_ns);
+Status OlapScanLocalState::_sync_cloud_tablets(RuntimeState* state) {
+if (config::is_cloud_mode() && !_sync_tablet) {
+_pending_tablets_num = _scan_ranges.size();
+if (_pending_tablets_num > 0) {
+_cloud_tablet_dependency = Dependency::create_shared(
+_parent->operator_id(), _parent->node_id(),
"CLOUD_TABLET_DEP");
+_tablets.resize(_scan_ranges.size());
+_tasks.reserve(_scan_ranges.size());
+_sync_statistics.resize(_scan_ranges.size());
+SCOPED_RAW_TIMER(&_duration_ns);
Review Comment:
这个_duration_ns 似乎没用，因为底下这个调用是异步的。
可能我们需要跟踪一下那个dependency的时间，比如他set ready的时候，记录一个时间，这样才是准确的。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
yiguolei commented on code in PR #52879:
URL: https://github.com/apache/doris/pull/52879#discussion_r2209182859
##
be/src/pipeline/exec/olap_scan_operator.cpp:
##
@@ -46,6 +46,12 @@
namespace doris::pipeline {
#include "common/compile_check_begin.h"
+Status OlapScanLocalState::init(RuntimeState* state, LocalStateInfo& info) {
+RETURN_IF_ERROR(Base::init(state, info));
+RETURN_IF_ERROR(_sync_cloud_tablets(state));
Review Comment:
这个_sync_cloud_tablets 实际是一个执行逻辑，但是我们把他放到了一个初始化的逻辑里，可能会有一些不可预期的问题
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 closed pull request #51701: [Improvement](cloud) Hold tablets in another RPC thread URL: https://github.com/apache/doris/pull/51701 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3057275046 # BE UT Coverage Report Increment line coverage `33.33% (33/99)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/26262626b0b87532253da053283f6bc364a62c23_26262626b0b87532253da053283f6bc364a62c23/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/26262626b0b87532253da053283f6bc364a62c23_26262626b0b87532253da053283f6bc364a62c23/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.05% (15509/27185) | | Line Coverage | 45.89% (138431/301683) | | Region Coverage | 35.34% (103656/293286) | | Branch Coverage | 37.77% (45582/120690) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3056987211 ClickBench: Total hot run time: 29.88 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 26262626b0b87532253da053283f6bc364a62c23, data reload: false query1 0.040.040.04 query2 0.080.050.04 query3 0.240.070.07 query4 1.630.110.11 query5 0.410.420.40 query6 1.180.660.65 query7 0.020.020.01 query8 0.040.040.03 query9 0.600.520.53 query10 0.570.570.58 query11 0.160.110.11 query12 0.150.120.11 query13 0.620.610.62 query14 0.790.800.82 query15 0.900.870.89 query16 0.380.380.40 query17 1.071.121.04 query18 0.220.220.21 query19 1.911.841.87 query20 0.010.010.02 query21 15.39 0.910.54 query22 0.751.020.58 query23 15.21 1.390.61 query24 7.091.721.19 query25 0.560.190.11 query26 0.660.170.14 query27 0.060.060.05 query28 10.18 0.830.44 query29 12.58 4.063.32 query30 0.250.090.07 query31 2.830.620.39 query32 3.250.550.47 query33 3.133.123.16 query34 16.08 5.424.73 query35 4.844.874.92 query36 0.680.500.49 query37 0.090.080.06 query38 0.060.050.04 query39 0.030.020.03 query40 0.170.160.14 query41 0.080.030.02 query42 0.030.030.03 query43 0.030.030.03 Total cold run time: 105.05 s Total hot run time: 29.88 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3056964151 TPC-DS: Total hot run time: 187594 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 26262626b0b87532253da053283f6bc364a62c23, data reload: false query1 1018404 398 398 query2 6526169016731673 query3 6735214 215 214 query4 26749 24001 23431 23431 query5 4352572 441 441 query6 313 224 199 199 query7 4618496 288 288 query8 257 224 221 221 query9 8630262526332625 query10 475 329 278 278 query11 15378 15039 14844 14844 query12 161 103 105 103 query13 1651533 419 419 query14 8587611960466046 query15 205 186 167 167 query16 7543424 248 248 query17 1363712 584 584 query18 1970410 290 290 query19 190 188 155 155 query20 133 118 116 116 query21 229 121 103 103 query22 4168415742644157 query23 34374 33177 33051 33051 query24 8461236123842361 query25 541 493 380 380 query26 1230264 145 145 query27 2783517 339 339 query28 4279214121082108 query29 737 557 429 429 query30 285 218 189 189 query31 918 844 777 777 query32 75 62 63 62 query33 590 342 289 289 query34 798 836 522 522 query35 613 631 571 571 query36 922 971 874 874 query37 107 101 79 79 query38 4166405841604058 query39 1467139214941392 query40 207 114 102 102 query41 59 54 51 51 query42 127 108 111 108 query43 496 494 475 475 query44 1307832 832 832 query45 180 174 167 167 query46 845 1047613 613 query47 1750180117001700 query48 395 409 303 303 query49 736 482 394 394 query50 628 691 417 417 query51 5484558654195419 query52 113 107 97 97 query53 223 263 196 196 query54 564 584 494 494 query55 83 80 82 80 query56 304 280 289 280 query57 1163119211331133 query58 265 248 256 248 query59 2567269625602560 query60 328 319 312 312 query61 129 127 125 125 query62 790 710 655 655 query63 226 177 185 177 query64 44131173989 989 query65 4277417641674167 query66 1140435 331 331 query67 15777 15488 15416 15416 query68 7915891 530 530 query69 527 413 267 267 query70 1161106710651065 query71 421 315 306 306 query72 5603469747234697 query73 661 606 353 353 query74 8852893391538933 query75 3505318927082708 query76 32621128732 732 query77 531 390 275 275 query78 11135 11094 10128 10128 query79 2033818 575 575 query80 578 503 488 488 query81 447 252 222 222 query82 432 124 104 104 query83 270 267 234 234 query84 250 102 91 91 query85 803 367 317 317 query86 349 294 296 294 query87 4308440343624362 query88 3718231522962296 query89 387 321 291 291 query90 1908212 211 211 query91 142 141 115 115 query92 81 60 54 54 query93 1573943 589 589 query94 700 312 200 200 query95 372 294 286 286 query96 492 565 288 288 query97 2749274526262626 query98 253 214 197 197 query99 1365138912931293 Total cold run time: 274956 ms Total hot run time: 187594 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3056919359 TPC-H: Total hot run time: 33413 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 26262626b0b87532253da053283f6bc364a62c23, data reload: false -- Round 1 -- q1 17559 524750815081 q2 1938282 183 183 q3 10414 1298710 710 q4 10256 1014530 530 q5 7847244723702370 q6 176 161 128 128 q7 924 740 600 600 q8 9345129710771077 q9 7564516851865168 q10 6898236019601960 q11 495 298 274 274 q12 340 345 212 212 q13 17767 377130723072 q14 236 234 219 219 q15 537 476 485 476 q16 433 431 369 369 q17 599 881 394 394 q18 7592715671777156 q19 1379957 559 559 q20 346 341 230 230 q21 3959318223542354 q22 367 314 291 291 Total cold run time: 106971 ms Total hot run time: 33413 ms - Round 2, with runtime_filter_mode=off - q1 5119507050985070 q2 241 329 226 226 q3 2181268222642264 q4 1357180013401340 q5 4231445344534453 q6 216 171 125 125 q7 2053199917901790 q8 2654263925242524 q9 7385739673587358 q10 3128329328642864 q11 579 526 503 503 q12 711 1009606 606 q13 3519392934033403 q14 284 337 301 301 q15 535 477 479 477 q16 466 496 457 457 q17 1183161713741374 q18 8067781375167516 q19 798 797 917 797 q20 1949201018281828 q21 4715449643544354 q22 618 599 529 529 Total cold run time: 51989 ms Total hot run time: 50159 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-305813 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
hello-stephen commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3056599064 # BE UT Coverage Report Increment line coverage `33.33% (33/99)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/2e2f61a9583d32208f5ff3d0564f014a16589ca5_2e2f61a9583d32208f5ff3d0564f014a16589ca5/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/2e2f61a9583d32208f5ff3d0564f014a16589ca5_2e2f61a9583d32208f5ff3d0564f014a16589ca5/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.01% (15500/27188) | | Line Coverage | 45.85% (138284/301624) | | Region Coverage | 35.27% (103433/293263) | | Branch Coverage | 37.71% (45511/120685) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3056278780 ClickBench: Total hot run time: 30.47 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 2e2f61a9583d32208f5ff3d0564f014a16589ca5, data reload: false query1 0.040.040.04 query2 0.070.040.04 query3 0.250.080.07 query4 1.610.110.11 query5 0.440.420.42 query6 1.190.670.65 query7 0.020.020.02 query8 0.040.030.04 query9 0.600.500.52 query10 0.570.570.56 query11 0.150.110.11 query12 0.160.120.11 query13 0.630.600.61 query14 0.810.810.82 query15 0.890.880.88 query16 0.380.390.39 query17 1.111.051.10 query18 0.240.220.21 query19 1.941.871.82 query20 0.020.010.02 query21 15.40 0.880.55 query22 0.761.230.85 query23 14.81 1.350.61 query24 6.861.621.75 query25 0.490.160.14 query26 0.590.160.14 query27 0.060.050.06 query28 10.70 0.800.44 query29 12.52 4.013.33 query30 0.250.090.06 query31 2.840.590.38 query32 3.230.550.47 query33 3.083.053.14 query34 16.13 5.404.78 query35 4.804.754.86 query36 0.690.500.49 query37 0.080.060.07 query38 0.050.040.04 query39 0.030.020.02 query40 0.170.150.14 query41 0.080.020.02 query42 0.040.020.02 query43 0.040.030.03 Total cold run time: 104.86 s Total hot run time: 30.47 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3056260217 TPC-DS: Total hot run time: 185832 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 2e2f61a9583d32208f5ff3d0564f014a16589ca5, data reload: false query1 1009376 395 376 query2 6504169516411641 query3 6749226 213 213 query4 26230 23278 22888 22888 query5 4352567 422 422 query6 298 227 198 198 query7 4646498 292 292 query8 289 225 213 213 query9 8597262726342627 query10 451 332 284 284 query11 15310 15177 14791 14791 query12 156 101 103 101 query13 1636529 408 408 query14 8451602059215921 query15 215 197 164 164 query16 7442426 269 269 query17 1339691 573 573 query18 1989389 299 299 query19 197 193 161 161 query20 119 119 114 114 query21 214 123 105 105 query22 4049407240404040 query23 34670 33240 33124 33124 query24 8471238824072388 query25 566 503 453 453 query26 1223273 149 149 query27 2738514 352 352 query28 4307215021292129 query29 765 562 448 448 query30 288 232 196 196 query31 905 822 759 759 query32 107 63 59 59 query33 553 328 287 287 query34 808 854 514 514 query35 608 645 549 549 query36 931 993 913 913 query37 116 100 78 78 query38 4134417341014101 query39 1486140114341401 query40 210 117 111 111 query41 57 55 63 55 query42 123 109 106 106 query43 508 503 474 474 query44 1358831 836 831 query45 178 168 166 166 query46 855 1015623 623 query47 1738178017351735 query48 385 437 303 303 query49 733 478 428 428 query50 660 708 423 423 query51 4214424141854185 query52 109 110 94 94 query53 225 265 181 181 query54 577 566 507 507 query55 79 85 85 85 query56 314 312 281 281 query57 1163117911201120 query58 262 253 261 253 query59 2543262025902590 query60 316 320 303 303 query61 122 126 120 120 query62 843 711 659 659 query63 232 226 192 192 query64 44111190814 814 query65 4274418942074189 query66 1120407 307 307 query67 16072 15406 15398 15398 query68 5797894 526 526 query69 525 309 263 263 query70 1212113410701070 query71 429 318 300 300 query72 5756473948504739 query73 691 664 352 352 query74 8954919189458945 query75 3211322126922692 query76 31831170749 749 query77 478 399 294 294 query78 10947 10996 10277 10277 query79 1204869 587 587 query80 657 520 453 453 query81 452 261 224 224 query82 190 133 100 100 query83 258 261 249 249 query84 252 113 93 93 query85 840 446 395 395 query86 338 311 258 258 query87 4319447342984298 query88 2957241723152315 query89 367 324 294 294 query90 1784219 211 211 query91 143 145 109 109 query92 66 63 60 60 query93 1055975 598 598 query94 622 311 200 200 query95 374 288 282 282 query96 503 580 286 286 query97 2660269225972597 query98 222 206 207 206 query99 1351137712771277 Total cold run time: 268396 ms Total hot run time: 185832 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3056214502 TPC-H: Total hot run time: 34393 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 2e2f61a9583d32208f5ff3d0564f014a16589ca5, data reload: false -- Round 1 -- q1 17581 551552195219 q2 1939299 183 183 q3 10297 1414764 764 q4 10234 1086536 536 q5 7742262325842584 q6 198 170 132 132 q7 996 766 598 598 q8 9311154012741274 q9 6821525452955254 q10 6949241919631963 q11 493 306 285 285 q12 354 392 215 215 q13 17767 386731603160 q14 237 225 213 213 q15 566 479 485 479 q16 432 432 373 373 q17 644 953 406 406 q18 7647732370617061 q19 15061079606 606 q20 351 365 225 225 q21 4416336725542554 q22 382 330 309 309 Total cold run time: 106863 ms Total hot run time: 34393 ms - Round 2, with runtime_filter_mode=off - q1 5498541953785378 q2 255 343 220 220 q3 2212274423362336 q4 1332182513431343 q5 4637451946044519 q6 259 183 128 128 q7 2090198717841784 q8 2989281427482748 q9 7407743073397339 q10 3187349729132913 q11 622 519 491 491 q12 717 786 626 626 q13 3632416133293329 q14 289 313 301 301 q15 542 487 476 476 q16 463 497 454 454 q17 1286172814741474 q18 8224787274867486 q19 852 843 926 843 q20 1937199418321832 q21 5006443843894389 q22 679 630 545 545 Total cold run time: 54115 ms Total hot run time: 50954 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3055861809 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3055754329 ClickBench: Total hot run time: 29.2 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 18e61f180f4b25eecdbf783a7d0b39a08439aa3f, data reload: false query1 0.050.040.04 query2 0.080.040.04 query3 0.250.090.08 query4 1.610.110.11 query5 0.440.440.44 query6 1.160.670.65 query7 0.030.020.02 query8 0.040.040.03 query9 0.600.500.52 query10 0.570.570.56 query11 0.150.120.11 query12 0.150.120.11 query13 0.630.600.61 query14 0.800.820.80 query15 0.900.890.88 query16 0.390.390.40 query17 1.081.041.05 query18 0.230.220.21 query19 2.011.821.82 query20 0.010.010.01 query21 15.42 0.910.53 query22 0.761.120.66 query23 15.02 1.380.62 query24 8.590.610.39 query25 0.410.230.17 query26 0.760.180.15 query27 0.070.050.06 query28 8.800.890.46 query29 12.56 3.913.37 query30 0.270.100.07 query31 2.820.600.40 query32 3.230.560.47 query33 3.003.063.06 query34 15.95 5.364.73 query35 4.764.834.82 query36 0.710.500.48 query37 0.090.070.07 query38 0.050.040.04 query39 0.040.020.03 query40 0.170.140.13 query41 0.080.020.03 query42 0.030.020.03 query43 0.040.030.03 Total cold run time: 104.81 s Total hot run time: 29.2 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3055738012 TPC-DS: Total hot run time: 185496 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 18e61f180f4b25eecdbf783a7d0b39a08439aa3f, data reload: false query1 996 383 405 383 query2 6521166916901669 query3 6756210 209 209 query4 26957 23621 23190 23190 query5 4316567 433 433 query6 308 212 196 196 query7 4630483 303 303 query8 285 229 213 213 query9 8653261226142612 query10 437 322 278 278 query11 15402 15043 14988 14988 query12 154 109 110 109 query13 1660532 393 393 query14 9451586859445868 query15 200 197 168 168 query16 7654430 262 262 query17 1477715 575 575 query18 2022414 319 319 query19 199 199 164 164 query20 128 120 115 115 query21 215 124 110 110 query22 4341433741204120 query23 33781 32858 32994 32858 query24 8397232623552326 query25 522 465 386 386 query26 1223264 146 146 query27 2734504 340 340 query28 4358211421192114 query29 707 536 429 429 query30 282 221 189 189 query31 964 824 758 758 query32 69 60 60 60 query33 544 337 274 274 query34 784 824 528 528 query35 595 642 550 550 query36 925 957 868 868 query37 111 97 76 76 query38 4128408241634082 query39 1484143614141414 query40 203 113 104 104 query41 54 54 54 54 query42 132 116 127 116 query43 498 502 455 455 query44 1304831 832 831 query45 171 164 160 160 query46 826 1012631 631 query47 1760181017541754 query48 386 410 295 295 query49 740 458 384 384 query50 628 688 407 407 query51 4105420842034203 query52 110 97 93 93 query53 247 247 184 184 query54 566 579 498 498 query55 89 79 79 79 query56 290 300 303 300 query57 1209120011401140 query58 257 241 275 241 query59 2593264526512645 query60 319 313 290 290 query61 125 127 121 121 query62 802 703 658 658 query63 223 185 185 185 query64 43371174829 829 query65 4284414141624141 query66 1048413 331 331 query67 16052 15576 15184 15184 query68 8107873 534 534 query69 493 378 262 262 query70 1202109010841084 query71 459 312 286 286 query72 5673467248274672 query73 746 661 360 360 query74 8818884689808846 query75 3836318026652665 query76 36931133708 708 query77 788 373 295 295 query78 11052 10959 10269 10269 query79 2207809 588 588 query80 567 502 435 435 query81 490 260 218 218 query82 490 122 96 96 query83 248 241 309 241 query84 256 101 83 83 query85 855 356 387 356 query86 383 307 310 307 query87 4460437244054372 query88 3609230122972297 query89 381 311 291 291 query90 1917211 201 201 query91 139 143 109 109 query92 74 59 60 59 query93 1677929 591 591 query94 692 317 202 202 query95 377 289 284 284 query96 480 559 278 278 query97 2725274426212621 query98 222 202 212 202 query99 1400142912961296 Total cold run time: 276244 ms Total hot run time: 185496 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3055705269 TPC-H: Total hot run time: 32972 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 18e61f180f4b25eecdbf783a7d0b39a08439aa3f, data reload: false -- Round 1 -- q1 17595 522250345034 q2 1945266 180 180 q3 10348 1292713 713 q4 10250 980 539 539 q5 8126251923032303 q6 181 160 126 126 q7 876 735 606 606 q8 9319129310741074 q9 6789515750155015 q10 6866237819571957 q11 467 298 272 272 q12 332 341 210 210 q13 17757 364730643064 q14 226 215 211 211 q15 558 476 488 476 q16 427 426 370 370 q17 600 862 388 388 q18 7532713970637063 q19 1432944 544 544 q20 339 342 217 217 q21 3712312523132313 q22 342 316 297 297 Total cold run time: 106019 ms Total hot run time: 32972 ms - Round 2, with runtime_filter_mode=off - q1 5069502850635028 q2 236 324 214 214 q3 2131266222892289 q4 1358176213181318 q5 4201422444764224 q6 208 169 123 123 q7 2062188817551755 q8 2591262125292529 q9 7243726373207263 q10 3084334628332833 q11 577 533 485 485 q12 700 793 637 637 q13 3568393833153315 q14 277 288 285 285 q15 516 478 466 466 q16 447 501 443 443 q17 1353159213531353 q18 7909774875077507 q19 775 844 1001844 q20 2092209718731873 q21 4892430444584304 q22 627 589 563 563 Total cold run time: 51916 ms Total hot run time: 49651 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3055616423 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
hello-stephen commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3055455378 # BE UT Coverage Report Increment line coverage `29.73% (33/111)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/45a839e27eea5ddcfd498e23f618ca49cce3799e_45a839e27eea5ddcfd498e23f618ca49cce3799e/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/45a839e27eea5ddcfd498e23f618ca49cce3799e_45a839e27eea5ddcfd498e23f618ca49cce3799e/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.00% (15500/27195) | | Line Coverage | 45.85% (138284/301592) | | Region Coverage | 35.32% (103453/292888) | | Branch Coverage | 37.76% (45503/120498) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3055210106 ClickBench: Total hot run time: 28.86 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 45a839e27eea5ddcfd498e23f618ca49cce3799e, data reload: false query1 0.040.040.03 query2 0.070.040.04 query3 0.240.070.08 query4 1.630.120.11 query5 0.430.410.42 query6 1.170.650.66 query7 0.030.020.02 query8 0.040.040.03 query9 0.600.510.53 query10 0.580.560.58 query11 0.160.100.10 query12 0.150.120.12 query13 0.610.610.61 query14 0.780.810.80 query15 0.890.910.84 query16 0.390.400.40 query17 1.101.061.08 query18 0.230.220.21 query19 2.031.791.81 query20 0.020.010.01 query21 15.42 0.880.53 query22 0.751.150.64 query23 15.00 1.350.65 query24 7.800.720.36 query25 0.480.170.07 query26 0.600.160.13 query27 0.060.040.05 query28 9.000.940.46 query29 12.55 3.983.27 query30 0.250.080.07 query31 2.840.620.38 query32 3.230.560.46 query33 3.123.063.22 query34 15.95 5.444.77 query35 4.834.894.79 query36 0.700.510.48 query37 0.090.070.07 query38 0.050.040.04 query39 0.030.020.02 query40 0.180.150.13 query41 0.070.020.03 query42 0.040.020.02 query43 0.040.040.03 Total cold run time: 104.27 s Total hot run time: 28.86 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3055199151 TPC-DS: Total hot run time: 186490 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 45a839e27eea5ddcfd498e23f618ca49cce3799e, data reload: false query1 1008390 390 390 query2 6547166617031666 query3 6742216 218 216 query4 26475 23600 23424 23424 query5 4318588 440 440 query6 308 211 199 199 query7 4630492 295 295 query8 274 228 212 212 query9 8598265026622650 query10 457 340 290 290 query11 15369 14983 14793 14793 query12 151 105 106 105 query13 1656528 410 410 query14 8532592360005923 query15 197 194 167 167 query16 7561427 252 252 query17 1353706 561 561 query18 1980388 304 304 query19 188 187 152 152 query20 121 117 120 117 query21 209 120 105 105 query22 4315416241314131 query23 34524 33013 33052 33013 query24 8467235723332333 query25 532 478 409 409 query26 1233262 143 143 query27 2788505 342 342 query28 4344216321352135 query29 741 541 431 431 query30 286 229 195 195 query31 906 849 755 755 query32 70 60 58 58 query33 547 336 293 293 query34 795 843 522 522 query35 597 636 558 558 query36 923 1020867 867 query37 115 108 78 78 query38 4170416442084164 query39 1498142713991399 query40 207 114 107 107 query41 56 57 53 53 query42 122 111 105 105 query43 487 486 486 486 query44 1333835 834 834 query45 177 169 162 162 query46 843 1024641 641 query47 1731182217201720 query48 387 434 308 308 query49 730 495 393 393 query50 629 697 418 418 query51 4146425841284128 query52 108 108 97 97 query53 217 254 191 191 query54 566 579 505 505 query55 83 81 82 81 query56 311 304 302 302 query57 1175118611301130 query58 256 255 252 252 query59 2652267525722572 query60 330 313 320 313 query61 154 128 138 128 query62 795 729 645 645 query63 226 185 183 183 query64 43771216836 836 query65 4289417241884172 query66 1105406 319 319 query67 15816 15464 15308 15308 query68 5997883 541 541 query69 507 304 273 273 query70 1206113411421134 query71 402 326 311 311 query72 5311472248184722 query73 684 638 354 354 query74 9196917790889088 query75 3196330227222722 query76 31431149748 748 query77 482 356 299 299 query78 10919 11192 10260 10260 query79 1029779 590 590 query80 615 519 453 453 query81 484 254 222 222 query82 190 122 94 94 query83 257 239 241 239 query84 246 106 83 83 query85 742 354 372 354 query86 339 293 279 279 query87 4554446643114311 query88 2872231123282311 query89 387 316 282 282 query90 1771213 211 211 query91 137 137 110 110 query92 60 58 54 54 query93 1002947 600 600 query94 617 317 199 199 query95 379 292 296 292 query96 488 566 275 275 query97 2676277826902690 query98 219 206 225 206 query99 1342142512971297 Total cold run time: 268282 ms Total hot run time: 186490 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3055175242 TPC-H: Total hot run time: 33375 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 45a839e27eea5ddcfd498e23f618ca49cce3799e, data reload: false -- Round 1 -- q1 17620 522250365036 q2 1924282 197 197 q3 10378 1292719 719 q4 10256 1033509 509 q5 7970239723782378 q6 177 168 130 130 q7 901 745 597 597 q8 9300129311221122 q9 7027515051395139 q10 6892238719651965 q11 493 303 286 286 q12 346 343 214 214 q13 17766 368330493049 q14 220 230 214 214 q15 590 480 474 474 q16 422 424 376 376 q17 595 876 381 381 q18 7904718972567189 q19 1220950 556 556 q20 344 358 220 220 q21 3734317623372337 q22 348 325 287 287 Total cold run time: 106427 ms Total hot run time: 33375 ms - Round 2, with runtime_filter_mode=off - q1 5073509651215096 q2 244 320 221 221 q3 2176266622392239 q4 1346176412981298 q5 4223435144894351 q6 212 169 129 129 q7 2065205917781778 q8 2681277626902690 q9 7347734773387338 q10 3090333928762876 q11 572 506 514 506 q12 684 762 633 633 q13 3575386335123512 q14 281 305 274 274 q15 526 470 487 470 q16 440 501 456 456 q17 1170153714041404 q18 8101754975057505 q19 775 808 918 808 q20 2093226319061906 q21 4908437542624262 q22 657 618 529 529 Total cold run time: 52239 ms Total hot run time: 50281 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3054822056 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
hello-stephen commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3053311051 # BE UT Coverage Report Increment line coverage `30.56% (33/108)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/a00770a4b8bf8c31baeed3a4b0df79cb82d63542_a00770a4b8bf8c31baeed3a4b0df79cb82d63542/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/a00770a4b8bf8c31baeed3a4b0df79cb82d63542_a00770a4b8bf8c31baeed3a4b0df79cb82d63542/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.00% (15506/27203) | | Line Coverage | 45.86% (138309/301581) | | Region Coverage | 35.33% (103477/292868) | | Branch Coverage | 37.77% (45506/120482) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3053122922 ClickBench: Total hot run time: 29.6 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit a00770a4b8bf8c31baeed3a4b0df79cb82d63542, data reload: false query1 0.040.040.03 query2 0.080.040.04 query3 0.250.070.07 query4 1.610.110.11 query5 0.440.420.42 query6 1.170.660.66 query7 0.020.020.02 query8 0.040.040.04 query9 0.620.510.51 query10 0.570.560.57 query11 0.160.110.10 query12 0.140.110.12 query13 0.630.610.61 query14 0.780.840.82 query15 0.890.860.86 query16 0.390.390.40 query17 1.091.091.13 query18 0.230.220.22 query19 2.021.841.87 query20 0.020.010.01 query21 15.41 0.880.55 query22 0.771.190.63 query23 14.96 1.410.62 query24 7.431.480.89 query25 0.550.270.10 query26 0.600.170.13 query27 0.050.050.06 query28 9.340.860.45 query29 12.57 3.993.30 query30 0.240.090.06 query31 2.830.600.38 query32 3.230.550.46 query33 3.143.143.09 query34 16.03 5.354.75 query35 4.814.804.82 query36 0.680.520.49 query37 0.090.070.07 query38 0.050.050.04 query39 0.030.020.03 query40 0.180.140.13 query41 0.080.030.03 query42 0.030.030.02 query43 0.040.030.03 Total cold run time: 104.33 s Total hot run time: 29.6 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3053107197 TPC-DS: Total hot run time: 186936 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit a00770a4b8bf8c31baeed3a4b0df79cb82d63542, data reload: false query1 1016411 413 411 query2 6498175417521752 query3 6748216 218 216 query4 26527 23966 23318 23318 query5 4292589 439 439 query6 297 227 194 194 query7 4628513 289 289 query8 276 209 232 209 query9 8636265626672656 query10 479 334 268 268 query11 16199 15174 15032 15032 query12 152 108 105 105 query13 1644545 426 426 query14 8823596960785969 query15 226 192 163 163 query16 7422440 266 266 query17 1321731 611 611 query18 2013415 320 320 query19 207 204 160 160 query20 127 121 116 116 query21 212 129 114 114 query22 4071445245304452 query23 35019 33800 32977 32977 query24 8414234624082346 query25 584 451 390 390 query26 1228260 146 146 query27 2760527 355 355 query28 4296214521282128 query29 758 542 424 424 query30 285 223 195 195 query31 916 850 774 774 query32 93 60 67 60 query33 550 346 271 271 query34 804 860 533 533 query35 602 616 567 567 query36 928 985 882 882 query37 119 110 76 76 query38 4150416040424042 query39 1507141313971397 query40 210 114 105 105 query41 55 55 50 50 query42 119 106 105 105 query43 498 516 478 478 query44 1355863 856 856 query45 178 169 166 166 query46 816 1016631 631 query47 1719176817081708 query48 388 420 317 317 query49 734 495 412 412 query50 625 684 424 424 query51 4247422641914191 query52 115 107 92 92 query53 230 263 183 183 query54 578 585 503 503 query55 87 83 85 83 query56 297 313 303 303 query57 1203118211261126 query58 266 252 270 252 query59 2595269925492549 query60 342 333 314 314 query61 127 134 121 121 query62 762 717 648 648 query63 226 187 186 186 query64 43871137920 920 query65 4408419541994195 query66 1134399 315 315 query67 1 15522 15314 15314 query68 8337915 524 524 query69 516 308 277 277 query70 1219105710771057 query71 496 327 291 291 query72 5536473648264736 query73 734 624 353 353 query74 9208880087558755 query75 4025323426732673 query76 37101185728 728 query77 858 381 294 294 query78 11330 11794 10445 10445 query79 2262861 592 592 query80 635 534 451 451 query81 463 274 233 233 query82 398 130 103 103 query83 293 257 243 243 query84 295 112 98 98 query85 790 357 308 308 query86 340 301 296 296 query87 4469443443804380 query88 3294233123002300 query89 384 315 286 286 query90 1917211 219 211 query91 141 142 108 108 query92 77 62 58 58 query93 1150936 594 594 query94 674 321 247 247 query95 373 299 281 281 query96 495 626 285 285 query97 2760274026762676 query98 234 214 206 206 query99 1457138712511251 Total cold run time: 276953 ms Total hot run time: 186936 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3053062566 TPC-H: Total hot run time: 33413 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit a00770a4b8bf8c31baeed3a4b0df79cb82d63542, data reload: false -- Round 1 -- q1 17560 519250865086 q2 1919279 206 206 q3 10301 1304747 747 q4 10215 1056507 507 q5 7539240923722372 q6 183 157 129 129 q7 900 740 599 599 q8 9323131511601160 q9 6784507851265078 q10 6881239819831983 q11 483 289 277 277 q12 347 353 210 210 q13 17806 368130903090 q14 234 227 215 215 q15 543 495 480 480 q16 433 418 383 383 q17 581 876 364 364 q18 7454713371987133 q19 1201952 551 551 q20 351 356 217 217 q21 3861317623372337 q22 363 330 289 289 Total cold run time: 105262 ms Total hot run time: 33413 ms - Round 2, with runtime_filter_mode=off - q1 5151508251175082 q2 240 327 219 219 q3 2178270322782278 q4 1356179612941294 q5 4266431446064314 q6 210 166 123 123 q7 1980196517911791 q8 2664259927172599 q9 7266729971627162 q10 3083323328452845 q11 588 508 508 508 q12 722 826 663 663 q13 3796399035083508 q14 297 309 290 290 q15 513 477 480 477 q16 450 488 442 442 q17 1232161314611461 q18 7907766176867661 q19 827 892 919 892 q20 2037211919581958 q21 5181453845844538 q22 664 630 589 589 Total cold run time: 52608 ms Total hot run time: 50694 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3052968624 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3052543482 ClickBench: Total hot run time: 29.72 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 69dc2f77e656cfcbb8020a9820d1a79c008eca97, data reload: false query1 0.040.030.03 query2 0.080.040.04 query3 0.240.080.07 query4 1.620.100.11 query5 0.440.410.41 query6 1.170.660.64 query7 0.030.020.01 query8 0.050.040.03 query9 0.600.520.52 query10 0.560.570.57 query11 0.160.110.11 query12 0.150.120.11 query13 0.620.610.61 query14 0.780.810.82 query15 0.910.870.85 query16 0.380.380.40 query17 1.071.041.06 query18 0.230.220.21 query19 2.001.801.86 query20 0.010.010.02 query21 15.39 0.900.53 query22 0.761.160.77 query23 14.84 1.350.65 query24 6.841.380.91 query25 0.500.180.05 query26 0.550.170.15 query27 0.070.060.05 query28 9.480.960.44 query29 12.55 3.963.36 query30 0.250.080.06 query31 2.840.600.39 query32 3.230.570.48 query33 3.053.203.08 query34 16.08 5.444.81 query35 4.904.824.89 query36 0.690.500.48 query37 0.090.070.06 query38 0.050.040.04 query39 0.030.020.03 query40 0.170.140.14 query41 0.090.020.03 query42 0.030.030.03 query43 0.040.030.03 Total cold run time: 103.66 s Total hot run time: 29.72 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3052524615 TPC-DS: Total hot run time: 186456 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 69dc2f77e656cfcbb8020a9820d1a79c008eca97, data reload: false query1 972 398 384 384 query2 6504184917211721 query3 6739214 223 214 query4 26925 24377 23294 23294 query5 4738568 433 433 query6 311 214 204 204 query7 4634505 290 290 query8 268 225 214 214 query9 8643265326352635 query10 502 313 267 267 query11 15944 15191 14827 14827 query12 155 114 103 103 query13 1664515 402 402 query14 8911588459025884 query15 212 195 172 172 query16 7464444 266 266 query17 1275721 598 598 query18 2000411 303 303 query19 206 194 160 160 query20 128 120 119 119 query21 214 122 107 107 query22 4240443744484437 query23 35083 33676 33091 33091 query24 8442238423962384 query25 522 454 395 395 query26 1220260 141 141 query27 2758508 347 347 query28 4285213621132113 query29 742 570 429 429 query30 282 225 193 193 query31 917 833 757 757 query32 72 61 59 59 query33 548 346 263 263 query34 781 836 510 510 query35 617 629 560 560 query36 936 974 877 877 query37 114 98 73 73 query38 4122411040974097 query39 1474140013941394 query40 203 116 105 105 query41 55 54 50 50 query42 120 103 109 103 query43 512 511 502 502 query44 1308843 830 830 query45 180 168 162 162 query46 832 998 615 615 query47 1747178516981698 query48 375 409 321 321 query49 729 470 377 377 query50 622 680 408 408 query51 4200432842084208 query52 102 105 95 95 query53 222 258 189 189 query54 607 569 507 507 query55 88 83 84 83 query56 291 292 295 292 query57 1175118811101110 query58 259 260 256 256 query59 2691269825822582 query60 325 338 295 295 query61 126 122 116 116 query62 782 707 679 679 query63 218 178 188 178 query64 43471205836 836 query65 4281418341784178 query66 1125408 357 357 query67 15713 15501 15235 15235 query68 8282899 525 525 query69 518 324 264 264 query70 1187110111301101 query71 512 307 295 295 query72 5539479950274799 query73 712 658 345 345 query74 9176911989778977 query75 3879319726892689 query76 37451154728 728 query77 779 357 314 314 query78 11011 1 10352 10352 query79 1941793 602 602 query80 594 509 445 445 query81 463 257 222 222 query82 424 120 94 94 query83 250 241 234 234 query84 243 101 84 84 query85 773 357 307 307 query86 347 290 289 289 query87 4381443443244324 query88 3295230322922292 query89 385 313 283 283 query90 1928203 201 201 query91 147 138 109 109 query92 67 58 59 58 query93 1457968 599 599 query94 669 315 199 199 query95 374 284 277 277 query96 497 583 275 275 query97 2706275926142614 query98 228 200 197 197 query99 1564141812781278 Total cold run time: 276872 ms Total hot run time: 186456 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3052490858 TPC-H: Total hot run time: 33178 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 69dc2f77e656cfcbb8020a9820d1a79c008eca97, data reload: false -- Round 1 -- q1 17557 528250275027 q2 1934275 199 199 q3 10295 1296759 759 q4 10212 1019530 530 q5 7503249123012301 q6 179 160 127 127 q7 907 749 589 589 q8 9315131311031103 q9 6952505250775052 q10 6881239019481948 q11 477 283 270 270 q12 341 357 224 224 q13 17766 368231463146 q14 225 221 211 211 q15 524 483 481 481 q16 420 432 369 369 q17 575 885 373 373 q18 7644708871087088 q19 1193946 542 542 q20 343 333 218 218 q21 3801315623252325 q22 356 321 296 296 Total cold run time: 105400 ms Total hot run time: 33178 ms - Round 2, with runtime_filter_mode=off - q1 5108508650795079 q2 244 321 220 220 q3 2219271423172317 q4 1351177713611361 q5 4247430045984300 q6 218 172 131 131 q7 2058203218471847 q8 2733275727052705 q9 7350730673897306 q10 3135330328882888 q11 591 527 490 490 q12 691 807 615 615 q13 3608404734263426 q14 275 312 267 267 q15 551 489 480 480 q16 453 510 439 439 q17 1186155114681468 q18 8120788276847684 q19 812 782 868 782 q20 2016205420042004 q21 4987473045524552 q22 603 605 626 605 Total cold run time: 52556 ms Total hot run time: 50966 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3052187101 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
yiguolei commented on code in PR #52879:
URL: https://github.com/apache/doris/pull/52879#discussion_r2194367005
##
be/src/pipeline/pipeline_task.cpp:
##
@@ -406,11 +404,12 @@ Status PipelineTask::execute(bool* done) {
});
// `_wake_up_early` must be after `_wait_to_start()`
if (_wait_to_start() || _wake_up_early) {
-if (config::enable_prefetch_tablet) {
-RETURN_IF_ERROR(_source->hold_tablets(_state));
-}
return Status::OK();
}
+if (!_hold_cloud_tablet && !_wake_up_early) {
+RETURN_IF_ERROR(_source->hold_tablets(_state));
Review Comment:
为什么要在pipelinetask 这个通用的地方去感知hold tablets 这种逻辑呢？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
yiguolei commented on code in PR #52879:
URL: https://github.com/apache/doris/pull/52879#discussion_r2194357151
##
be/src/pipeline/exec/olap_scan_operator.cpp:
##
@@ -450,55 +449,72 @@ Status
OlapScanLocalState::_init_scanners(std::list* sc
return Status::OK();
}
-Status OlapScanLocalState::hold_tablets() {
-if (!_tablets.empty()) {
+Status OlapScanLocalState::sync_cloud_tablets(RuntimeState* state) {
+if (config::is_cloud_mode() && !_sync_tablet) {
+_cloud_tablet_dependencies.resize(_scan_ranges.size());
+for (size_t i = 0; i < _scan_ranges.size(); i++) {
+_cloud_tablet_dependencies[i] =
+Dependency::create_shared(_parent->operator_id(),
_parent->node_id(),
+ "CLOUD_TABLET_DEP_" +
std::to_string(i));
+}
+
+_tablets.resize(_scan_ranges.size());
+_tasks.reserve(_scan_ranges.size());
+_sync_statistics.resize(_scan_ranges.size());
+SCOPED_RAW_TIMER(&_duration_ns);
+for (size_t i = 0; i < _scan_ranges.size(); i++) {
+auto* sync_stats = &_sync_statistics[i];
+int64_t version = 0;
+std::from_chars(_scan_ranges[i]->version.data(),
+_scan_ranges[i]->version.data() +
_scan_ranges[i]->version.size(),
+version);
+auto task_ctx = state->get_task_execution_context();
+_tasks.emplace_back([this, sync_stats, version, i, task_ctx]() {
+auto task_lock = task_ctx.lock();
+if (task_lock == nullptr) {
+return Status::OK();
+}
+Defer defer([&] {
this->_cloud_tablet_dependencies[i]->set_ready(); });
Review Comment:
用一个atomic 记录一下总数，如果都结束了，再设置一下depdency ready
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
yiguolei commented on code in PR #52879:
URL: https://github.com/apache/doris/pull/52879#discussion_r2194355270
##
be/src/pipeline/exec/olap_scan_operator.cpp:
##
@@ -450,55 +449,72 @@ Status
OlapScanLocalState::_init_scanners(std::list* sc
return Status::OK();
}
-Status OlapScanLocalState::hold_tablets() {
-if (!_tablets.empty()) {
+Status OlapScanLocalState::sync_cloud_tablets(RuntimeState* state) {
+if (config::is_cloud_mode() && !_sync_tablet) {
+_cloud_tablet_dependencies.resize(_scan_ranges.size());
+for (size_t i = 0; i < _scan_ranges.size(); i++) {
+_cloud_tablet_dependencies[i] =
+Dependency::create_shared(_parent->operator_id(),
_parent->node_id(),
+ "CLOUD_TABLET_DEP_" +
std::to_string(i));
+}
+
+_tablets.resize(_scan_ranges.size());
+_tasks.reserve(_scan_ranges.size());
+_sync_statistics.resize(_scan_ranges.size());
+SCOPED_RAW_TIMER(&_duration_ns);
+for (size_t i = 0; i < _scan_ranges.size(); i++) {
+auto* sync_stats = &_sync_statistics[i];
+int64_t version = 0;
+std::from_chars(_scan_ranges[i]->version.data(),
+_scan_ranges[i]->version.data() +
_scan_ranges[i]->version.size(),
+version);
+auto task_ctx = state->get_task_execution_context();
+_tasks.emplace_back([this, sync_stats, version, i, task_ctx]() {
+auto task_lock = task_ctx.lock();
+if (task_lock == nullptr) {
+return Status::OK();
+}
+Defer defer([&] {
this->_cloud_tablet_dependencies[i]->set_ready(); });
+auto tablet =
+
DORIS_TRY(ExecEnv::get_tablet(_scan_ranges[i]->tablet_id, sync_stats));
+_tablets[i] = {std::move(tablet), version};
+SyncOptions options;
+options.query_version = version;
+options.merge_schema = true;
+
RETURN_IF_ERROR(std::dynamic_pointer_cast(_tablets[i].tablet)
+->sync_rowsets(options, sync_stats));
+// FIXME(plat1ko): Avoid pointer cast
+
ExecEnv::GetInstance()->storage_engine().to_cloud().tablet_hotspot().count(
+*_tablets[i].tablet);
+return Status::OK();
+});
+}
+RETURN_IF_ERROR(cloud::bthread_fork_join(
+_tasks, config::init_scanner_sync_rowsets_parallelism,
&_cloud_tablet_future));
+_sync_tablet = true;
return Status::OK();
}
+return Status::OK();
+}
+Status OlapScanLocalState::hold_tablets() {
MonotonicStopWatch timer;
timer.start();
-_tablets.resize(_scan_ranges.size());
_read_sources.resize(_scan_ranges.size());
if (config::is_cloud_mode()) {
-std::vector sync_statistics(_scan_ranges.size());
-std::vector> tasks {};
-tasks.reserve(_scan_ranges.size());
-int64_t duration_ns {0};
-{
-SCOPED_RAW_TIMER(&duration_ns);
-for (size_t i = 0; i < _scan_ranges.size(); i++) {
-auto* sync_stats = &sync_statistics[i];
-int64_t version = 0;
-std::from_chars(_scan_ranges[i]->version.data(),
-_scan_ranges[i]->version.data() +
_scan_ranges[i]->version.size(),
-version);
-tasks.emplace_back([this, sync_stats, version, i]() {
-auto tablet =
-
DORIS_TRY(ExecEnv::get_tablet(_scan_ranges[i]->tablet_id, sync_stats));
-_tablets[i] = {std::move(tablet), version};
-SyncOptions options;
-options.query_version = version;
-options.merge_schema = true;
-
RETURN_IF_ERROR(std::dynamic_pointer_cast(_tablets[i].tablet)
-->sync_rowsets(options,
sync_stats));
-// FIXME(plat1ko): Avoid pointer cast
-
ExecEnv::GetInstance()->storage_engine().to_cloud().tablet_hotspot().count(
-*_tablets[i].tablet);
-return Status::OK();
-});
-}
-RETURN_IF_ERROR(
-cloud::bthread_fork_join(tasks,
config::init_scanner_sync_rowsets_parallelism));
-}
-COUNTER_UPDATE(_sync_rowset_timer, duration_ns);
+DCHECK(std::all_of(
+_cloud_tablet_dependencies.begin(),
_cloud_tablet_dependencies.end(),
+[&](DependencySPtr dep) -> bool { return
!dep->is_blocked_by(nullptr); }));
+DCHECK(_cloud_tablet_future.valid() &&
_cloud_tablet_future.get().ok());
Review Comment:
这个future 实际只是用来
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
yiguolei commented on code in PR #52879:
URL: https://github.com/apache/doris/pull/52879#discussion_r2194342571
##
be/src/pipeline/exec/olap_scan_operator.cpp:
##
@@ -450,55 +449,72 @@ Status
OlapScanLocalState::_init_scanners(std::list* sc
return Status::OK();
}
-Status OlapScanLocalState::hold_tablets() {
-if (!_tablets.empty()) {
+Status OlapScanLocalState::sync_cloud_tablets(RuntimeState* state) {
+if (config::is_cloud_mode() && !_sync_tablet) {
+_cloud_tablet_dependencies.resize(_scan_ranges.size());
+for (size_t i = 0; i < _scan_ranges.size(); i++) {
+_cloud_tablet_dependencies[i] =
Review Comment:
不要这么多的dependency，比如如果查询1个tablet，这里就会有1个dependency，此时我们pipeline
每次遍历的时候，就都得遍历一遍1个，还是很废的。
实际我们就是一个dependency，就是当所有的tablet 都sync之后
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
yiguolei commented on code in PR #52879:
URL: https://github.com/apache/doris/pull/52879#discussion_r2194340062
##
be/src/pipeline/exec/olap_scan_operator.cpp:
##
@@ -450,55 +449,72 @@ Status
OlapScanLocalState::_init_scanners(std::list* sc
return Status::OK();
}
-Status OlapScanLocalState::hold_tablets() {
-if (!_tablets.empty()) {
+Status OlapScanLocalState::sync_cloud_tablets(RuntimeState* state) {
+if (config::is_cloud_mode() && !_sync_tablet) {
Review Comment:
为什么需要_sync_tablet 这个标记位?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
yiguolei commented on code in PR #52879:
URL: https://github.com/apache/doris/pull/52879#discussion_r2194291959
##
be/src/pipeline/exec/scan_operator.h:
##
@@ -80,6 +80,7 @@ class ScanLocalStateBase : public PipelineXLocalState<> {
virtual Status clone_conjunct_ctxs(vectorized::VExprContextSPtrs&
conjuncts) = 0;
virtual void set_scan_ranges(RuntimeState* state,
const std::vector&
scan_ranges) = 0;
+virtual Status sync_cloud_tablets(RuntimeState* state) = 0;
Review Comment:
这个方法应该只有内表才需要，不能放到scan local state 的基类里
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3051208225 # BE UT Coverage Report Increment line coverage `25.81% (24/93)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/c9645b47164d8cde7fa6492b07b4b5eb0f5264b5_c9645b47164d8cde7fa6492b07b4b5eb0f5264b5/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/c9645b47164d8cde7fa6492b07b4b5eb0f5264b5_c9645b47164d8cde7fa6492b07b4b5eb0f5264b5/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.34% (15513/27056) | | Line Coverage | 46.34% (141036/304377) | | Region Coverage | 45.59% (71305/156396) | | Branch Coverage | 40.33% (37587/93196) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3051092061 ClickBench: Total hot run time: 29.3 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit c9645b47164d8cde7fa6492b07b4b5eb0f5264b5, data reload: false query1 0.050.040.04 query2 0.080.040.04 query3 0.250.080.07 query4 1.620.110.11 query5 0.430.440.42 query6 1.140.670.65 query7 0.030.020.01 query8 0.040.040.03 query9 0.610.510.54 query10 0.560.580.57 query11 0.150.100.11 query12 0.150.120.12 query13 0.620.610.62 query14 0.790.810.82 query15 0.890.880.87 query16 0.400.390.39 query17 1.071.051.07 query18 0.230.210.21 query19 1.991.861.87 query20 0.010.010.01 query21 15.40 0.940.55 query22 0.771.150.66 query23 14.98 1.340.60 query24 7.921.500.51 query25 0.510.290.12 query26 0.600.170.13 query27 0.070.060.06 query28 9.080.900.45 query29 12.56 3.953.33 query30 0.260.100.06 query31 2.840.610.40 query32 3.230.560.47 query33 3.123.073.19 query34 16.01 5.394.76 query35 4.904.844.87 query36 0.690.500.49 query37 0.090.070.07 query38 0.060.040.03 query39 0.040.020.03 query40 0.180.140.14 query41 0.080.020.02 query42 0.040.020.02 query43 0.030.020.02 Total cold run time: 104.57 s Total hot run time: 29.3 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3051084919 TPC-DS: Total hot run time: 186002 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit c9645b47164d8cde7fa6492b07b4b5eb0f5264b5, data reload: false query1 993 402 384 384 query2 6516170716571657 query3 6755215 212 212 query4 26560 23343 23271 23271 query5 4357586 431 431 query6 286 212 205 205 query7 4633480 284 284 query8 288 218 220 218 query9 8636260626142606 query10 482 346 283 283 query11 15834 15195 14846 14846 query12 154 104 103 103 query13 1644499 385 385 query14 8428584258625842 query15 190 182 186 182 query16 7210421 245 245 query17 1370698 565 565 query18 1976416 324 324 query19 196 176 151 151 query20 123 118 118 118 query21 208 123 108 108 query22 4217423545254235 query23 34931 33890 33162 33162 query24 8415235923732359 query25 531 463 385 385 query26 1220260 147 147 query27 2800505 349 349 query28 4326213221012101 query29 763 553 422 422 query30 284 214 188 188 query31 907 876 753 753 query32 71 60 60 60 query33 542 323 269 269 query34 774 849 515 515 query35 590 654 572 572 query36 926 978 904 904 query37 115 104 76 76 query38 4098412139813981 query39 1477142614161416 query40 208 122 106 106 query41 106 50 49 49 query42 118 104 104 104 query43 497 497 474 474 query44 1283832 829 829 query45 174 170 161 161 query46 822 996 615 615 query47 1745178216971697 query48 378 407 308 308 query49 730 478 387 387 query50 616 683 403 403 query51 4094423442064206 query52 113 101 90 90 query53 275 265 183 183 query54 555 561 490 490 query55 83 83 78 78 query56 278 293 281 281 query57 1160117910931093 query58 257 248 249 248 query59 2512270825212521 query60 337 312 314 312 query61 125 120 123 120 query62 807 708 652 652 query63 214 191 182 182 query64 4508115710511051 query65 4239416241974162 query66 1158415 322 322 query67 15720 15596 15372 15372 query68 8098888 547 547 query69 509 314 355 314 query70 1220111610921092 query71 468 320 289 289 query72 5612482048374820 query73 694 615 357 357 query74 9121881089528810 query75 3765320027222722 query76 36281134709 709 query77 773 378 291 291 query78 10867 4 10256 10256 query79 2602812 577 577 query80 618 514 439 439 query81 467 265 216 216 query82 446 118 90 90 query83 283 309 230 230 query84 292 96 87 87 query85 779 354 314 314 query86 399 312 288 288 query87 4541441443094309 query88 3441230522912291 query89 383 320 276 276 query90 1865215 211 211 query91 138 141 111 111 query92 75 56 53 53 query93 1918931 588 588 query94 668 313 200 200 query95 372 284 278 278 query96 488 570 276 276 query97 2707274526432643 query98 237 213 206 206 query99 1415140313021302 Total cold run time: 275827 ms Total hot run time: 186002 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3051067093 TPC-H: Total hot run time: 33354 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit c9645b47164d8cde7fa6492b07b4b5eb0f5264b5, data reload: false -- Round 1 -- q1 17580 521651145114 q2 1930291 185 185 q3 10582 1338736 736 q4 10290 1032506 506 q5 8595236523892365 q6 191 165 128 128 q7 877 746 602 602 q8 9299124910641064 q9 6849508050925080 q10 6886236919791979 q11 470 296 286 286 q12 340 340 222 222 q13 17766 372930853085 q14 222 236 231 231 q15 535 489 476 476 q16 424 410 377 377 q17 596 861 376 376 q18 7441718371337133 q19 1201955 557 557 q20 335 344 226 226 q21 3673317623342334 q22 355 327 292 292 Total cold run time: 106437 ms Total hot run time: 33354 ms - Round 2, with runtime_filter_mode=off - q1 5108510750755075 q2 236 332 227 227 q3 2160269722602260 q4 1372177313111311 q5 4195442445794424 q6 217 169 129 129 q7 2019196218151815 q8 2704271226262626 q9 7365721872157215 q10 3110329728722872 q11 576 509 610 509 q12 688 773 637 637 q13 3494402134473447 q14 283 311 300 300 q15 541 507 490 490 q16 452 511 445 445 q17 1192163413971397 q18 7859780275747574 q19 759 846 1102846 q20 2014207019511951 q21 5045454946264549 q22 668 645 572 572 Total cold run time: 52057 ms Total hot run time: 50671 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3051025626 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3050873217 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3049589082 # BE UT Coverage Report Increment line coverage `27.59% (24/87)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/4eebf9cf16903cbbd8f3f3003acb5144ad38e9f3_4eebf9cf16903cbbd8f3f3003acb5144ad38e9f3/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/4eebf9cf16903cbbd8f3f3003acb5144ad38e9f3_4eebf9cf16903cbbd8f3f3003acb5144ad38e9f3/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.32% (15508/27056) | | Line Coverage | 46.30% (140927/304358) | | Region Coverage | 45.55% (71253/156412) | | Branch Coverage | 40.27% (37545/93222) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3049196888 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3048220459 # BE UT Coverage Report Increment line coverage `27.27% (24/88)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/b5f1f090ae980496484f828f69f1efb94b37f565_b5f1f090ae980496484f828f69f1efb94b37f565/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/b5f1f090ae980496484f828f69f1efb94b37f565_b5f1f090ae980496484f828f69f1efb94b37f565/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.31% (15502/27048) | | Line Coverage | 46.30% (140886/304290) | | Region Coverage | 45.56% (71240/156376) | | Branch Coverage | 40.27% (37535/93206) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3047667619 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3047509421 ClickBench: Total hot run time: 29.78 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 088a6678ed1c3311734942b3a7371bf95786258c, data reload: false query1 0.040.030.03 query2 0.080.040.04 query3 0.240.070.08 query4 1.610.110.10 query5 0.450.430.41 query6 1.160.660.65 query7 0.030.020.02 query8 0.050.040.04 query9 0.620.510.51 query10 0.570.570.56 query11 0.150.120.12 query12 0.150.120.12 query13 0.620.610.62 query14 0.840.800.81 query15 0.890.860.85 query16 0.410.370.39 query17 1.041.021.06 query18 0.220.210.21 query19 1.941.841.90 query20 0.020.010.01 query21 15.42 0.890.54 query22 0.761.190.69 query23 14.90 1.330.59 query24 6.921.141.06 query25 0.520.150.13 query26 0.580.160.14 query27 0.060.040.05 query28 9.740.910.44 query29 12.56 4.023.33 query30 0.260.090.07 query31 2.850.600.38 query32 3.230.550.47 query33 3.143.073.08 query34 16.32 5.444.80 query35 4.904.894.84 query36 0.690.500.49 query37 0.090.070.06 query38 0.050.040.04 query39 0.030.030.02 query40 0.170.140.13 query41 0.080.030.03 query42 0.030.020.02 query43 0.030.020.03 Total cold run time: 104.46 s Total hot run time: 29.78 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3047498759 TPC-DS: Total hot run time: 184087 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 088a6678ed1c3311734942b3a7371bf95786258c, data reload: false query1 1002384 393 384 query2 6525174216891689 query3 6747211 214 211 query4 26191 23619 23236 23236 query5 4374585 429 429 query6 303 213 230 213 query7 4631496 289 289 query8 284 210 234 210 query9 8607262626452626 query10 489 311 266 266 query11 15710 15499 14844 14844 query12 153 111 104 104 query13 1646520 390 390 query14 9084562755325532 query15 197 185 167 167 query16 7242634 468 468 query17 1184707 566 566 query18 1975390 297 297 query19 185 183 199 183 query20 124 117 117 117 query21 214 117 105 105 query22 4225420644764206 query23 34808 33810 32990 32990 query24 8373235024402350 query25 524 452 420 420 query26 1227274 148 148 query27 2748515 361 361 query28 4350213021122112 query29 767 546 425 425 query30 277 223 179 179 query31 914 828 742 742 query32 73 57 58 57 query33 549 348 298 298 query34 796 829 506 506 query35 777 794 745 745 query36 952 1001886 886 query37 99 91 68 68 query38 4197407440424042 query39 1449140914051405 query40 220 126 99 99 query41 53 54 48 48 query42 121 102 103 102 query43 494 507 461 461 query44 1313820 830 820 query45 203 164 159 159 query46 865 987 621 621 query47 1797177917361736 query48 376 409 299 299 query49 769 471 391 391 query50 642 673 405 405 query51 4203412242314122 query52 113 106 98 98 query53 234 261 182 182 query54 558 566 504 504 query55 82 79 81 79 query56 324 325 272 272 query57 1186118411051105 query58 265 248 239 239 query59 2629269726762676 query60 320 320 311 311 query61 128 126 125 125 query62 805 719 652 652 query63 226 202 181 181 query64 43521003670 670 query65 4260415141684151 query66 1164417 336 336 query67 15955 15424 15472 15424 query68 7847875 524 524 query69 488 311 268 268 query70 1142110210841084 query71 424 318 312 312 query72 5810464745074507 query73 635 556 354 354 query74 8861913286668666 query75 3471315326392639 query76 32571134691 691 query77 528 402 286 286 query78 10185 10310 93139313 query79 2413781 571 571 query80 622 507 420 420 query81 476 267 233 233 query82 437 122 91 91 query83 281 249 230 230 query84 297 112 80 80 query85 773 404 315 315 query86 374 313 272 272 query87 4411443142984298 query88 3656226222812262 query89 380 306 286 286 query90 1910204 205 204 query91 142 143 110 110 query92 75 57 57 57 query93 1903935 586 586 query94 696 417 297 297 query95 370 290 278 278 query96 490 580 277 277 query97 2726277126002600 query98 229 203 197 197 query99 1460141012451245 Total cold run time: 274398 ms Total hot run time: 184087 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3047476775 TPC-H: Total hot run time: 5 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 088a6678ed1c3311734942b3a7371bf95786258c, data reload: false -- Round 1 -- q1 17603 520249964996 q2 1934281 186 186 q3 10321 1286720 720 q4 10226 1052521 521 q5 7537237323442344 q6 185 157 128 128 q7 884 736 594 594 q8 9299128810561056 q9 6827515451325132 q10 6874238619361936 q11 500 288 280 280 q12 342 354 220 220 q13 17776 368231183118 q14 221 234 219 219 q15 549 477 480 477 q16 419 456 377 377 q17 605 880 359 359 q18 7623745272477247 q19 1207949 564 564 q20 327 333 219 219 q21 3893252123392339 q22 352 318 303 303 Total cold run time: 105504 ms Total hot run time: 5 ms - Round 2, with runtime_filter_mode=off - q1 5096509650595059 q2 240 325 229 229 q3 2190271022762276 q4 1407180314151415 q5 4218420146324201 q6 220 172 131 131 q7 2026200817851785 q8 2682276526622662 q9 7393735276367352 q10 3079325928132813 q11 563 524 527 524 q12 691 763 654 654 q13 3553388633343334 q14 316 319 311 311 q15 550 491 475 475 q16 450 495 451 451 q17 1235161113621362 q18 7954772676297629 q19 825 817 857 817 q20 2065219418911891 q21 5082465544624462 q22 700 629 540 540 Total cold run time: 52535 ms Total hot run time: 50373 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3047337139 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3047302851 ClickBench: Total hot run time: 29.72 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit e936d8fd51eaeaa6d523611f9846c05e6b102836, data reload: false query1 0.040.030.03 query2 0.080.040.04 query3 0.240.080.07 query4 1.620.110.12 query5 0.420.430.44 query6 1.170.680.67 query7 0.030.020.02 query8 0.040.030.04 query9 0.600.500.52 query10 0.570.570.58 query11 0.160.110.11 query12 0.150.110.12 query13 0.630.610.61 query14 0.820.810.85 query15 0.900.900.89 query16 0.390.390.38 query17 1.051.041.07 query18 0.230.220.22 query19 1.931.901.92 query20 0.010.010.02 query21 15.38 0.900.54 query22 0.771.120.67 query23 14.97 1.390.59 query24 6.881.530.83 query25 0.480.150.07 query26 0.620.170.13 query27 0.060.050.06 query28 9.690.910.44 query29 12.56 4.023.37 query30 0.250.090.06 query31 2.840.590.39 query32 3.250.560.46 query33 3.153.163.14 query34 16.09 5.454.77 query35 4.874.844.86 query36 0.700.510.49 query37 0.100.070.06 query38 0.050.040.03 query39 0.030.030.02 query40 0.170.150.15 query41 0.080.030.02 query42 0.040.020.02 query43 0.040.030.03 Total cold run time: 104.15 s Total hot run time: 29.72 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3047296312 TPC-DS: Total hot run time: 185766 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit e936d8fd51eaeaa6d523611f9846c05e6b102836, data reload: false query1 1033390 401 390 query2 6536168817041688 query3 6743230 220 220 query4 26370 23663 23359 23359 query5 4371598 455 455 query6 303 214 214 214 query7 4629509 296 296 query8 270 222 217 217 query9 8631266126492649 query10 474 328 280 280 query11 15541 15105 14760 14760 query12 158 104 100 100 query13 1672527 431 431 query14 8509595759325932 query15 209 194 172 172 query16 7750442 261 261 query17 1332742 585 585 query18 2020406 333 333 query19 198 191 178 178 query20 120 123 116 116 query21 219 126 120 120 query22 4151431039853985 query23 34059 33107 32855 32855 query24 8538237523932375 query25 530 482 382 382 query26 815 270 151 151 query27 2730514 343 343 query28 4367218121492149 query29 682 568 431 431 query30 288 215 199 199 query31 938 845 771 771 query32 70 63 59 59 query33 542 328 279 279 query34 816 854 515 515 query35 622 665 551 551 query36 952 1003879 879 query37 114 101 71 71 query38 4165412040734073 query39 1472139914081399 query40 208 114 99 99 query41 58 55 53 53 query42 126 113 107 107 query43 516 508 488 488 query44 1309842 842 842 query45 174 175 168 168 query46 825 1014621 621 query47 1775176117501750 query48 387 417 309 309 query49 708 476 386 386 query50 644 702 421 421 query51 4253416442914164 query52 109 109 92 92 query53 224 255 195 195 query54 582 567 501 501 query55 80 81 81 81 query56 307 307 304 304 query57 1162121211211121 query58 261 253 264 253 query59 2672276625912591 query60 332 325 306 306 query61 155 123 126 123 query62 787 724 655 655 query63 231 186 189 186 query64 35341201892 892 query65 4253418342194183 query66 963 430 324 324 query67 15854 15580 15379 15379 query68 8526869 532 532 query69 510 310 270 270 query70 1199108910441044 query71 463 320 298 298 query72 5772474548444745 query73 729 636 351 351 query74 9263908686718671 query75 3891327227062706 query76 35911152710 710 query77 788 369 292 292 query78 10947 11367 10321 10321 query79 1143821 588 588 query80 618 518 453 453 query81 470 320 219 219 query82 376 127 98 98 query83 257 256 234 234 query84 247 107 82 82 query85 771 364 315 315 query86 332 292 296 292 query87 4348442242984298 query88 2976231923172317 query89 375 309 289 289 query90 1965217 205 205 query91 138 158 110 110 query92 70 59 56 56 query93 1070981 600 600 query94 689 318 196 196 query95 381 301 289 289 query96 499 568 286 286 query97 2775278826382638 query98 228 207 204 204 query99 1496140112401240 Total cold run time: 272331 ms Total hot run time: 185766 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3047278976 TPC-H: Total hot run time: 33231 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit e936d8fd51eaeaa6d523611f9846c05e6b102836, data reload: false -- Round 1 -- q1 17711 514350315031 q2 1929278 199 199 q3 10568 1280718 718 q4 10250 1022507 507 q5 8311246023562356 q6 186 163 126 126 q7 940 736 595 595 q8 9307129511191119 q9 7543525750815081 q10 6904239219651965 q11 483 296 268 268 q12 347 355 217 217 q13 17761 366130873087 q14 233 235 221 221 q15 548 479 477 477 q16 421 423 380 380 q17 599 875 356 356 q18 7800716473187164 q19 1674942 547 547 q20 322 341 221 221 q21 3720255623102310 q22 376 318 286 286 Total cold run time: 107933 ms Total hot run time: 33231 ms - Round 2, with runtime_filter_mode=off - q1 5133506150615061 q2 247 330 219 219 q3 2176269523042304 q4 1456177913501350 q5 4217456944164416 q6 224 175 126 126 q7 2088197917911791 q8 2694252425282524 q9 7380727973897279 q10 3131340329402940 q11 565 502 480 480 q12 677 743 672 672 q13 3651391935743574 q14 276 287 274 274 q15 517 477 485 477 q16 464 482 431 431 q17 1218177913741374 q18 7801769876697669 q19 805 811 983 811 q20 1975207618301830 q21 4802434942874287 q22 638 592 563 563 Total cold run time: 52135 ms Total hot run time: 50452 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3047134482 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3044679937 # BE UT Coverage Report Increment line coverage `42.37% (25/59)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/2021a3c92292e983707281c16b7a3828c743a752_2021a3c92292e983707281c16b7a3828c743a752/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/2021a3c92292e983707281c16b7a3828c743a752_2021a3c92292e983707281c16b7a3828c743a752/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.31% (15498/27043) | | Line Coverage | 46.29% (140853/304278) | | Region Coverage | 45.55% (71220/156372) | | Branch Coverage | 40.26% (37526/93208) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3044496732 ClickBench: Total hot run time: 29.84 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 2021a3c92292e983707281c16b7a3828c743a752, data reload: false query1 0.050.040.04 query2 0.080.040.04 query3 0.240.070.08 query4 1.620.120.11 query5 0.420.420.42 query6 1.180.680.65 query7 0.030.020.02 query8 0.050.030.03 query9 0.590.500.50 query10 0.560.570.58 query11 0.160.110.11 query12 0.150.120.11 query13 0.630.610.62 query14 0.800.840.81 query15 0.890.870.88 query16 0.390.400.40 query17 1.101.041.04 query18 0.230.220.21 query19 2.021.851.83 query20 0.020.010.02 query21 15.38 0.890.54 query22 0.751.260.75 query23 14.76 1.340.67 query24 6.671.640.91 query25 0.500.190.16 query26 0.570.170.13 query27 0.060.060.05 query28 9.580.920.44 query29 12.55 3.953.30 query30 0.250.100.07 query31 2.830.610.38 query32 3.230.560.48 query33 3.063.073.30 query34 16.10 5.424.79 query35 4.774.884.83 query36 0.690.520.49 query37 0.100.070.07 query38 0.050.040.03 query39 0.030.030.02 query40 0.180.150.14 query41 0.080.030.02 query42 0.030.020.02 query43 0.040.030.03 Total cold run time: 103.47 s Total hot run time: 29.84 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3044472929 TPC-DS: Total hot run time: 185577 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 2021a3c92292e983707281c16b7a3828c743a752, data reload: false query1 1018387 397 387 query2 6528169017031690 query3 6746217 215 215 query4 26054 23641 23254 23254 query5 4309559 439 439 query6 293 202 189 189 query7 4626476 285 285 query8 260 215 203 203 query9 8622265426482648 query10 454 346 285 285 query11 15202 15208 14783 14783 query12 160 107 104 104 query13 1656531 404 404 query14 8544581558475815 query15 203 182 176 176 query16 7546429 260 260 query17 1335687 560 560 query18 1991394 301 301 query19 183 214 153 153 query20 120 117 119 117 query21 214 125 102 102 query22 3988409440544054 query23 33952 33003 33131 33003 query24 8483236223852362 query25 546 486 391 391 query26 1235267 150 150 query27 2786513 343 343 query28 4277214221302130 query29 764 553 425 425 query30 287 222 197 197 query31 922 843 740 740 query32 73 68 60 60 query33 548 323 303 303 query34 793 851 541 541 query35 608 661 555 555 query36 917 965 873 873 query37 119 104 78 78 query38 4180411840864086 query39 1474141514061406 query40 204 113 110 110 query41 55 57 51 51 query42 119 107 105 105 query43 512 486 458 458 query44 1338850 825 825 query45 173 171 163 163 query46 837 1032630 630 query47 1742180017241724 query48 406 434 313 313 query49 741 460 390 390 query50 638 693 416 416 query51 4124421841004100 query52 113 106 92 92 query53 232 261 183 183 query54 575 562 490 490 query55 83 78 85 78 query56 287 311 288 288 query57 1147120811181118 query58 270 248 254 248 query59 2546264825072507 query60 317 309 305 305 query61 123 120 122 120 query62 790 717 673 673 query63 227 185 188 185 query64 44921163803 803 query65 4254417241674167 query66 1133394 329 329 query67 15749 15724 15237 15237 query68 7525877 539 539 query69 537 312 261 261 query70 1190111011191110 query71 386 333 295 295 query72 5774476548904765 query73 673 660 357 357 query74 9141897789338933 query75 3191322227442744 query76 31551134724 724 query77 492 390 282 282 query78 11077 1 10282 10282 query79 1393822 589 589 query80 631 530 448 448 query81 477 262 227 227 query82 349 126 94 94 query83 250 305 239 239 query84 248 109 92 92 query85 750 352 314 314 query86 310 304 305 304 query87 4420452642754275 query88 2926231723362317 query89 386 313 283 283 query90 1780216 215 215 query91 136 143 113 113 query92 63 59 69 59 query93 1099961 597 597 query94 633 317 201 201 query95 374 299 285 285 query96 502 554 282 282 query97 2705277226872687 query98 226 212 220 212 query99 1332141512771277 Total cold run time: 269374 ms Total hot run time: 185577 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3044414780 TPC-H: Total hot run time: 33223 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 2021a3c92292e983707281c16b7a3828c743a752, data reload: false -- Round 1 -- q1 17576 520851045104 q2 1931286 192 192 q3 10486 1320720 720 q4 10314 1024505 505 q5 8956239823732373 q6 213 156 126 126 q7 895 753 599 599 q8 9303127110861086 q9 6868509451525094 q10 6949238219491949 q11 513 296 276 276 q12 337 361 209 209 q13 17773 370830753075 q14 225 224 223 223 q15 563 482 468 468 q16 450 426 390 390 q17 619 863 365 365 q18 7683707970047004 q19 15691006562 562 q20 327 341 216 216 q21 3630322123822382 q22 357 330 305 305 Total cold run time: 107537 ms Total hot run time: 33223 ms - Round 2, with runtime_filter_mode=off - q1 5170510051155100 q2 243 328 220 220 q3 2214264022972297 q4 1391177413201320 q5 4226448744904487 q6 223 170 153 153 q7 2052199218011801 q8 2680251125062506 q9 7245722472667224 q10 3125330529002900 q11 571 519 489 489 q12 739 801 714 714 q13 3568381734063406 q14 271 303 277 277 q15 699 515 477 477 q16 426 479 440 440 q17 1189153914151415 q18 7839777176327632 q19 843 828 947 828 q20 2060207920012001 q21 4848432042824282 q22 647 591 543 543 Total cold run time: 52269 ms Total hot run time: 50512 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3044267196 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3043936792 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Thearas commented on PR #52879: URL: https://github.com/apache/doris/pull/52879#issuecomment-3043936955 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
gavinchou commented on code in PR #51701:
URL: https://github.com/apache/doris/pull/51701#discussion_r2158938321
##
be/src/pipeline/exec/olap_scan_operator.cpp:
##
@@ -446,107 +445,145 @@ Status
OlapScanLocalState::_init_scanners(std::list* sc
return Status::OK();
}
-Status OlapScanLocalState::hold_tablets() {
+Status OlapScanLocalState::hold_tablets(RuntimeState* state) {
if (!_tablets.empty()) {
+// In cloud mode, we should make sure none-or-all tablets are loaded
into cache.
+DCHECK(!config::is_cloud_mode() ||
+ !std::any_of(
+ _cloud_tablet_dependencies.begin(),
_cloud_tablet_dependencies.end(),
+ [&](DependencySPtr dep) -> bool { return
dep->is_blocked_by(nullptr); }));
return Status::OK();
}
-MonotonicStopWatch timer;
-timer.start();
_tablets.resize(_scan_ranges.size());
_read_sources.resize(_scan_ranges.size());
if (config::is_cloud_mode()) {
-std::vector sync_statistics(_scan_ranges.size());
-std::vector> tasks {};
-tasks.reserve(_scan_ranges.size());
-int64_t duration_ns {0};
-{
-SCOPED_RAW_TIMER(&duration_ns);
-for (size_t i = 0; i < _scan_ranges.size(); i++) {
-auto* sync_stats = &sync_statistics[i];
-int64_t version = 0;
-std::from_chars(_scan_ranges[i]->version.data(),
+/**
+ * In cloud-mode, cold tablet replica need to be read from remote
storage before execution. This
+ * behaviour will be harmful for pipeline execution due to its heavy
IO. So we shuold make sure
+ * it is separated from execution thread.
+ */
+auto task_ctx = state->get_task_execution_context();
+for (size_t i = 0; i < _scan_ranges.size(); i++) {
+
RETURN_IF_ERROR(ExecEnv::GetInstance()->fragment_mgr()->get_cloud_tablet_thread_pool()->submit_func(
+[&, i, state, task_ctx]() {
Review Comment:
separate the lambda to a function, too long to read and to many indentions
##
be/src/pipeline/exec/olap_scan_operator.cpp:
##
@@ -446,107 +445,145 @@ Status
OlapScanLocalState::_init_scanners(std::list* sc
return Status::OK();
}
-Status OlapScanLocalState::hold_tablets() {
+Status OlapScanLocalState::hold_tablets(RuntimeState* state) {
if (!_tablets.empty()) {
+// In cloud mode, we should make sure none-or-all tablets are loaded
into cache.
+DCHECK(!config::is_cloud_mode() ||
+ !std::any_of(
+ _cloud_tablet_dependencies.begin(),
_cloud_tablet_dependencies.end(),
+ [&](DependencySPtr dep) -> bool { return
dep->is_blocked_by(nullptr); }));
return Status::OK();
}
-MonotonicStopWatch timer;
-timer.start();
_tablets.resize(_scan_ranges.size());
_read_sources.resize(_scan_ranges.size());
if (config::is_cloud_mode()) {
-std::vector sync_statistics(_scan_ranges.size());
-std::vector> tasks {};
-tasks.reserve(_scan_ranges.size());
-int64_t duration_ns {0};
-{
-SCOPED_RAW_TIMER(&duration_ns);
-for (size_t i = 0; i < _scan_ranges.size(); i++) {
-auto* sync_stats = &sync_statistics[i];
-int64_t version = 0;
-std::from_chars(_scan_ranges[i]->version.data(),
+/**
+ * In cloud-mode, cold tablet replica need to be read from remote
storage before execution. This
+ * behaviour will be harmful for pipeline execution due to its heavy
IO. So we shuold make sure
+ * it is separated from execution thread.
+ */
+auto task_ctx = state->get_task_execution_context();
+for (size_t i = 0; i < _scan_ranges.size(); i++) {
+
RETURN_IF_ERROR(ExecEnv::GetInstance()->fragment_mgr()->get_cloud_tablet_thread_pool()->submit_func(
+[&, i, state, task_ctx]() {
Review Comment:
separate the lambda to a function, too long to read and too many indentions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
hello-stephen commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2986771276 # BE Regression && UT Coverage Report Increment line coverage `26.78% (49/183)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/51701_9a56d75115942561891de61be8fb68c45e67746b_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/51701_9a56d75115942561891de61be8fb68c45e67746b_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 60.92% (16017/26292) | | Line Coverage | 50.40% (150319/298264) | | Region Coverage | 47.72% (85875/179943) | | Branch Coverage | 41.28% (42212/102254) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
hello-stephen commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2986765038 # BE UT Coverage Report Increment line coverage `15.05% (28/186)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/9a56d75115942561891de61be8fb68c45e67746b_9a56d75115942561891de61be8fb68c45e67746b/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/9a56d75115942561891de61be8fb68c45e67746b_9a56d75115942561891de61be8fb68c45e67746b/report/index.html) | Category | Coverage | |---|| | Function Coverage | 56.34% (15049/26711) | | Line Coverage | 45.10% (134605/298439) | | Region Coverage | 44.24% (67692/152997) | | Branch Coverage | 38.82% (34730/89460) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2983697290 ClickBench: Total hot run time: 29.3 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 9a56d75115942561891de61be8fb68c45e67746b, data reload: false query1 0.040.040.03 query2 0.060.040.04 query3 0.240.070.07 query4 1.610.110.11 query5 0.430.450.43 query6 1.160.660.66 query7 0.020.020.02 query8 0.040.040.03 query9 0.580.540.51 query10 0.570.570.56 query11 0.150.110.11 query12 0.150.120.12 query13 0.630.650.61 query14 0.820.800.82 query15 0.900.860.89 query16 0.390.400.39 query17 1.081.091.03 query18 0.230.200.20 query19 1.941.891.86 query20 0.010.010.02 query21 15.40 0.890.55 query22 0.751.350.72 query23 14.75 1.380.64 query24 6.571.560.43 query25 0.550.230.08 query26 0.620.160.14 query27 0.060.050.05 query28 9.620.950.44 query29 12.56 3.953.33 query30 0.250.090.07 query31 2.820.600.38 query32 3.240.540.49 query33 3.103.093.11 query34 16.03 5.354.84 query35 4.884.824.80 query36 0.680.510.48 query37 0.090.070.06 query38 0.050.040.03 query39 0.030.030.02 query40 0.180.150.14 query41 0.080.020.02 query42 0.030.020.02 query43 0.040.030.03 Total cold run time: 103.43 s Total hot run time: 29.3 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2983682696 TPC-DS: Total hot run time: 185265 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 9a56d75115942561891de61be8fb68c45e67746b, data reload: false query1 1013398 383 383 query2 6527184818211821 query3 6759221 214 214 query4 26826 23735 23031 23031 query5 4350600 449 449 query6 315 204 212 204 query7 4625492 284 284 query8 270 233 228 228 query9 8661261426452614 query10 486 343 276 276 query11 15388 15014 14842 14842 query12 163 106 105 105 query13 1661527 420 420 query14 9447616761176117 query15 213 194 177 177 query16 7696642 475 475 query17 1197730 579 579 query18 2030406 352 352 query19 190 172 167 167 query20 119 121 113 113 query21 214 127 108 108 query22 4179430139653965 query23 34016 32987 33008 32987 query24 8417234823942348 query25 525 469 379 379 query26 1226264 149 149 query27 2721505 335 335 query28 4339211921022102 query29 718 554 443 443 query30 289 220 191 191 query31 949 855 761 761 query32 95 64 63 63 query33 541 347 309 309 query34 787 831 533 533 query35 775 797 723 723 query36 951 975 877 877 query37 111 104 77 77 query38 4080410740044004 query39 1463137714541377 query40 204 113 105 105 query41 63 64 56 56 query42 124 119 116 116 query43 489 505 479 479 query44 1297821 819 819 query45 174 168 167 167 query46 822 1015632 632 query47 1806176817211721 query48 378 420 314 314 query49 741 498 377 377 query50 628 666 417 417 query51 4110414140964096 query52 107 106 104 104 query53 229 258 186 186 query54 580 601 526 526 query55 90 86 91 86 query56 299 305 281 281 query57 1171120210921092 query58 264 256 254 254 query59 2699269926122612 query60 342 325 307 307 query61 124 125 125 125 query62 809 717 680 680 query63 225 193 184 184 query64 42561038668 668 query65 4263410541704105 query66 1058395 306 306 query67 15658 15437 15361 15361 query68 7893881 528 528 query69 477 316 269 269 query70 1148109411061094 query71 506 313 298 298 query72 5476474347404740 query73 693 613 347 347 query74 9125914589938993 query75 3843320927032703 query76 37021195770 770 query77 788 366 284 284 query78 10028 10185 93349334 query79 2039832 590 590 query80 575 506 444 444 query81 474 262 228 228 query82 438 122 96 96 query83 250 258 232 232 query84 264 105 94 94 query85 805 353 345 345 query86 337 312 298 298 query87 4572443743504350 query88 3509228122882281 query89 380 316 278 278 query90 1961216 208 208 query91 142 146 108 108 query92 75 64 58 58 query93 1604959 583 583 query94 688 426 297 297 query95 371 293 285 285 query96 493 573 284 284 query97 2702271526182618 query98 222 219 198 198 query99 1337137812721272 Total cold run time: 274259 ms Total hot run time: 185265 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2983648794 TPC-H: Total hot run time: 34676 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 9a56d75115942561891de61be8fb68c45e67746b, data reload: false -- Round 1 -- q1 17606 514849834983 q2 1931279 196 196 q3 10292 1328730 730 q4 10207 1019512 512 q5 7622249424262426 q6 181 165 135 135 q7 925 761 631 631 q8 9342125411341134 q9 6784513254245132 q10 6892238619441944 q11 481 288 278 278 q12 346 360 223 223 q13 17751 373531003100 q14 242 225 206 206 q15 551 499 506 499 q16 449 441 376 376 q17 587 886 352 352 q18 7779709070717071 q19 1741956 562 562 q20 342 329 230 230 q21 4156324830073007 q22 1054998 949 949 Total cold run time: 107261 ms Total hot run time: 34676 ms - Round 2, with runtime_filter_mode=off - q1 5167522850265026 q2 239 312 227 227 q3 2151266523012301 q4 1365184413521352 q5 4242426044754260 q6 216 169 129 129 q7 2007192517811781 q8 2644270825632563 q9 7197705471377054 q10 3040323828192819 q11 581 510 505 505 q12 705 750 638 638 q13 3616396732813281 q14 268 291 262 262 q15 535 502 485 485 q16 433 500 450 450 q17 1139150413891389 q18 7794764173737373 q19 820 844 1047844 q20 1981205118681868 q21 4924444744314431 q22 1066992 1028992 Total cold run time: 52130 ms Total hot run time: 50030 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-298360 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2983189456 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
hello-stephen commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2983006321 # BE Regression && UT Coverage Report Increment line coverage `21.02% (37/176)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/51701_d638982356e2b265a924e720b5b995dc905117a7_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/51701_d638982356e2b265a924e720b5b995dc905117a7_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 61.17% (16081/26290) | | Line Coverage | 50.71% (151244/298232) | | Region Coverage | 48.06% (86472/179915) | | Branch Coverage | 41.60% (42530/102234) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
hello-stephen commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2982920551 # BE Regression && UT Coverage Report Increment line coverage `21.02% (37/176)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/51701_d638982356e2b265a924e720b5b995dc905117a7_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/51701_d638982356e2b265a924e720b5b995dc905117a7_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 61.17% (16081/26290) | | Line Coverage | 50.71% (151244/298232) | | Region Coverage | 48.06% (86472/179915) | | Branch Coverage | 41.60% (42530/102234) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
hello-stephen commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2982903807 # BE UT Coverage Report Increment line coverage `12.29% (22/179)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/d638982356e2b265a924e720b5b995dc905117a7_d638982356e2b265a924e720b5b995dc905117a7/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/d638982356e2b265a924e720b5b995dc905117a7_d638982356e2b265a924e720b5b995dc905117a7/report/index.html) | Category | Coverage | |---|| | Function Coverage | 56.33% (15048/26713) | | Line Coverage | 45.10% (134567/298392) | | Region Coverage | 44.24% (67671/152961) | | Branch Coverage | 38.82% (34718/89436) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2982770231 ClickBench: Total hot run time: 29.64 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit d638982356e2b265a924e720b5b995dc905117a7, data reload: false query1 0.030.030.03 query2 0.080.030.04 query3 0.230.060.06 query4 1.610.100.11 query5 0.420.410.44 query6 1.170.650.66 query7 0.020.020.01 query8 0.040.040.04 query9 0.580.510.51 query10 0.570.560.56 query11 0.160.110.11 query12 0.140.110.11 query13 0.630.610.61 query14 0.810.860.80 query15 0.900.860.87 query16 0.380.380.38 query17 1.081.061.03 query18 0.220.210.21 query19 2.001.811.85 query20 0.010.010.01 query21 15.40 0.940.54 query22 0.751.120.71 query23 14.96 1.370.66 query24 7.391.220.83 query25 0.510.110.05 query26 0.590.160.14 query27 0.050.050.05 query28 9.370.940.44 query29 12.54 4.003.36 query30 0.260.090.06 query31 2.820.610.40 query32 3.250.560.47 query33 3.093.083.08 query34 15.99 5.404.83 query35 4.884.864.87 query36 0.690.500.49 query37 0.090.070.07 query38 0.050.040.04 query39 0.040.030.02 query40 0.180.140.14 query41 0.080.020.02 query42 0.030.030.02 query43 0.040.030.03 Total cold run time: 104.13 s Total hot run time: 29.64 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2982761522 TPC-DS: Total hot run time: 193032 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit d638982356e2b265a924e720b5b995dc905117a7, data reload: false query1 13801030991 991 query2 6418193818651865 query3 11152 472046404640 query4 25294 24085 22971 22971 query5 4590618 474 474 query6 319 215 207 207 query7 3995504 285 285 query8 277 227 223 223 query9 8534267826862678 query10 448 318 283 283 query11 15299 14992 14829 14829 query12 160 107 114 107 query13 1563534 429 429 query14 8922618560346034 query15 203 199 166 166 query16 7539626 481 481 query17 1179752 611 611 query18 2100460 329 329 query19 211 203 172 172 query20 132 129 120 120 query21 221 171 106 106 query22 4526464245134513 query23 34369 33651 33616 33616 query24 8549237924662379 query25 514 454 434 434 query26 1203267 166 166 query27 2810523 363 363 query28 4758223122152215 query29 695 553 450 450 query30 286 241 192 192 query31 925 852 797 797 query32 68 69 67 67 query33 532 391 337 337 query34 797 875 514 514 query35 785 849 743 743 query36 966 993 909 909 query37 109 101 73 73 query38 4284430642024202 query39 1517146814781468 query40 224 152 112 112 query41 66 58 56 56 query42 122 114 118 114 query43 519 529 483 483 query44 1321848 834 834 query45 185 176 168 168 query46 846 1025662 662 query47 1871188017921792 query48 395 429 318 318 query49 721 494 408 408 query50 667 724 407 407 query51 4197430243004300 query52 119 112 107 107 query53 236 261 186 186 query54 590 580 516 516 query55 82 80 86 80 query56 298 303 335 303 query57 1244125011531153 query58 275 257 254 254 query59 2777292827442744 query60 326 323 310 310 query61 137 125 136 125 query62 773 741 684 684 query63 233 229 195 195 query64 40891025674 674 query65 4378423243174232 query66 945 402 303 303 query67 15797 15751 15547 15547 query68 8443896 532 532 query69 478 305 266 266 query70 1251107111121071 query71 471 312 299 299 query72 5593467646574657 query73 717 601 353 353 query74 8974914690659065 query75 3888318926532653 query76 37101173743 743 query77 785 374 283 283 query78 10081 10172 95399539 query79 2307825 584 584 query80 612 501 478 478 query81 497 265 225 225 query82 466 131 96 96 query83 275 243 233 233 query84 293 103 84 84 query85 782 339 302 302 query86 343 304 277 277 query87 4474438743424342 query88 3022223122182218 query89 394 312 287 287 query90 1878203 202 202 query91 134 142 114 114 query92 75 60 57 57 query93 1585940 579 579 query94 664 390 307 307 query95 365 282 281 281 query96 502 566 281 281 query97 2719272926452645 query98 227 215 200 200 query99 1429138712731273 Total cold run time: 278496 ms Total hot run time: 193032 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2982739379 TPC-H: Total hot run time: 34278 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit d638982356e2b265a924e720b5b995dc905117a7, data reload: false -- Round 1 -- q1 17610 520249984998 q2 1946277 169 169 q3 10441 1292762 762 q4 10218 1031541 541 q5 7542236023762360 q6 171 157 129 129 q7 913 764 604 604 q8 9324131411421142 q9 6796513751495137 q10 6937239219851985 q11 484 289 280 280 q12 358 349 218 218 q13 17755 368930873087 q14 231 234 212 212 q15 573 490 486 486 q16 438 422 371 371 q17 608 874 354 354 q18 7565715471997154 q19 1836977 589 589 q20 331 346 225 225 q21 4186319424532453 q22 1075102310221022 Total cold run time: 107338 ms Total hot run time: 34278 ms - Round 2, with runtime_filter_mode=off - q1 5160506850975068 q2 243 319 221 221 q3 2170267723132313 q4 1413183213711371 q5 4253414043704140 q6 226 173 128 128 q7 2025193918301830 q8 2648263125292529 q9 7173711171937111 q10 3075321228312831 q11 591 521 502 502 q12 718 821 616 616 q13 3557387832323232 q14 274 293 276 276 q15 538 506 503 503 q16 473 508 445 445 q17 1218158213761376 q18 7790749974597459 q19 850 839 892 839 q20 2043205219021902 q21 5038433043334330 q22 1133107310661066 Total cold run time: 52609 ms Total hot run time: 50088 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2982668093 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2982564288 # BE UT Coverage Report Increment line coverage `11.86% (21/177)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/16337656e16fe0c07d061f20806234621991f37e_16337656e16fe0c07d061f20806234621991f37e/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/16337656e16fe0c07d061f20806234621991f37e_16337656e16fe0c07d061f20806234621991f37e/report/index.html) | Category | Coverage | |---|| | Function Coverage | 56.33% (15048/26714) | | Line Coverage | 45.09% (134560/298396) | | Region Coverage | 44.24% (67670/152963) | | Branch Coverage | 38.81% (34710/89436) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2982443499 ClickBench: Total hot run time: 29.57 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 16337656e16fe0c07d061f20806234621991f37e, data reload: false query1 0.030.030.03 query2 0.060.040.03 query3 0.230.070.07 query4 1.620.100.10 query5 0.420.420.42 query6 1.180.670.67 query7 0.020.020.02 query8 0.040.030.03 query9 0.580.520.52 query10 0.560.590.57 query11 0.160.110.11 query12 0.150.120.12 query13 0.640.620.60 query14 0.800.820.81 query15 0.890.850.85 query16 0.380.380.38 query17 1.051.061.12 query18 0.220.210.20 query19 1.981.801.80 query20 0.010.010.02 query21 15.41 0.940.57 query22 0.751.200.68 query23 14.92 1.390.62 query24 7.520.960.74 query25 0.500.280.12 query26 0.590.160.13 query27 0.060.060.05 query28 9.300.930.46 query29 12.62 3.943.30 query30 0.250.100.06 query31 2.820.590.39 query32 3.230.550.49 query33 2.973.103.11 query34 16.04 5.414.81 query35 4.874.824.86 query36 0.690.490.50 query37 0.090.070.07 query38 0.060.040.03 query39 0.030.030.03 query40 0.180.140.14 query41 0.080.020.02 query42 0.030.020.03 query43 0.040.040.03 Total cold run time: 104.07 s Total hot run time: 29.57 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2982435447 TPC-DS: Total hot run time: 185389 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 16337656e16fe0c07d061f20806234621991f37e, data reload: false query1 995 397 400 397 query2 6533191518891889 query3 6749224 220 220 query4 26582 23185 23276 23185 query5 4350623 479 479 query6 297 211 209 209 query7 4623505 286 286 query8 263 227 225 225 query9 8646261726532617 query10 487 365 284 284 query11 15808 15039 14809 14809 query12 163 109 106 106 query13 1654539 424 424 query14 10060 612061696120 query15 207 202 176 176 query16 7222645 493 493 query17 1204763 577 577 query18 1995399 300 300 query19 184 179 152 152 query20 128 115 121 115 query21 214 120 105 105 query22 4088404340284028 query23 33904 33098 33068 33068 query24 7794236123752361 query25 545 454 407 407 query26 1243278 149 149 query27 2638494 356 356 query28 4309214021102110 query29 787 555 426 426 query30 288 229 191 191 query31 960 866 744 744 query32 80 69 65 65 query33 573 393 316 316 query34 817 829 526 526 query35 795 804 732 732 query36 929 984 884 884 query37 108 97 76 76 query38 4150418240114011 query39 1479140514221405 query40 247 126 110 110 query41 65 58 60 58 query42 128 120 115 115 query43 517 527 507 507 query44 1347844 850 844 query45 186 187 180 180 query46 867 1049652 652 query47 1775180517421742 query48 405 434 344 344 query49 796 506 404 404 query50 672 687 418 418 query51 4226421540834083 query52 113 111 97 97 query53 223 250 178 178 query54 579 580 510 510 query55 86 83 82 82 query56 300 328 284 284 query57 1155122711091109 query58 265 256 255 255 query59 2584265926242624 query60 325 317 323 317 query61 128 121 123 121 query62 788 709 664 664 query63 224 188 181 181 query64 43461026665 665 query65 4253417442344174 query66 1132410 319 319 query67 15645 15393 15293 15293 query68 8043889 527 527 query69 470 323 279 279 query70 1180112710741074 query71 459 327 288 288 query72 5665470346894689 query73 672 593 353 353 query74 8901885788118811 query75 3748326726672667 query76 36101179738 738 query77 777 367 287 287 query78 10020 10210 93689368 query79 2006796 583 583 query80 580 518 441 441 query81 483 257 224 224 query82 433 131 97 97 query83 259 259 239 239 query84 249 164 95 95 query85 782 362 313 313 query86 329 323 278 278 query87 4315435242264226 query88 3439223622342234 query89 374 313 293 293 query90 1952209 205 205 query91 146 145 113 113 query92 79 58 62 58 query93 1477920 574 574 query94 625 404 303 303 query95 384 294 285 285 query96 495 572 277 277 query97 2736280826462646 query98 241 207 203 203 query99 1368140412681268 Total cold run time: 273525 ms Total hot run time: 185389 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2982399145 TPC-H: Total hot run time: 34062 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 16337656e16fe0c07d061f20806234621991f37e, data reload: false -- Round 1 -- q1 17631 518550925092 q2 1931308 184 184 q3 10290 1365732 732 q4 10242 1023513 513 q5 7563241023802380 q6 186 173 131 131 q7 909 752 614 614 q8 9323130311901190 q9 6809508151445081 q10 6899238019511951 q11 475 291 285 285 q12 345 351 225 225 q13 17761 369430603060 q14 234 243 224 224 q15 553 479 475 475 q16 418 428 362 362 q17 584 867 358 358 q18 7679716671267126 q19 1224958 562 562 q20 342 344 220 220 q21 3700255623232323 q22 10901048974 974 Total cold run time: 106188 ms Total hot run time: 34062 ms - Round 2, with runtime_filter_mode=off - q1 5141510350805080 q2 246 322 220 220 q3 2177269623202320 q4 1364177213491349 q5 4233409844624098 q6 220 168 129 129 q7 2041198517801780 q8 2636247624642464 q9 7181717671987176 q10 3098328128642864 q11 565 549 498 498 q12 662 746 620 620 q13 3508398833283328 q14 277 297 268 268 q15 528 475 470 470 q16 444 474 445 445 q17 1175150713921392 q18 7814758074137413 q19 820 842 994 842 q20 2018205619361936 q21 5260459643174317 q22 10851025977 977 Total cold run time: 52493 ms Total hot run time: 49986 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2982343546 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2982341171 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
hello-stephen commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2980946793 # BE Regression && UT Coverage Report Increment line coverage `21.05% (36/171)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/51701_a02235dd425171b70f4912fbf8c5fa38235fb80c_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/51701_a02235dd425171b70f4912fbf8c5fa38235fb80c_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 60.24% (15838/26291) | | Line Coverage | 49.51% (147647/298231) | | Region Coverage | 46.74% (84097/179922) | | Branch Coverage | 40.08% (40980/102236) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
hello-stephen commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2980753756 # BE UT Coverage Report Increment line coverage `12.07% (21/174)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/a02235dd425171b70f4912fbf8c5fa38235fb80c_a02235dd425171b70f4912fbf8c5fa38235fb80c/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/a02235dd425171b70f4912fbf8c5fa38235fb80c_a02235dd425171b70f4912fbf8c5fa38235fb80c/report/index.html) | Category | Coverage | |---|| | Function Coverage | 56.34% (15050/26714) | | Line Coverage | 45.11% (134597/298391) | | Region Coverage | 44.25% (67693/152963) | | Branch Coverage | 38.82% (34718/89436) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
hello-stephen commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2980472391 # BE Regression && UT Coverage Report Increment line coverage ` ` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/51701_d1e33cacee9ad756004bdf3d11e1e5a1546fe092_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/51701_d1e33cacee9ad756004bdf3d11e1e5a1546fe092_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 60.24% (15837/26291) | | Line Coverage | 49.47% (147521/298224) | | Region Coverage | 46.69% (83997/179922) | | Branch Coverage | 40.05% (40943/102236) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
Gabriel39 commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2980344269 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2979979187 ClickBench: Total hot run time: 29.7 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit d1e33cacee9ad756004bdf3d11e1e5a1546fe092, data reload: false query1 0.040.030.03 query2 0.070.040.04 query3 0.230.070.07 query4 1.620.100.11 query5 0.420.420.41 query6 1.180.640.67 query7 0.020.020.01 query8 0.050.030.03 query9 0.570.520.52 query10 0.570.570.57 query11 0.150.110.11 query12 0.150.110.12 query13 0.630.600.61 query14 0.800.810.84 query15 0.880.850.89 query16 0.390.380.40 query17 1.101.071.06 query18 0.220.200.20 query19 1.951.831.81 query20 0.010.010.02 query21 15.40 0.900.55 query22 0.761.280.73 query23 14.79 1.380.67 query24 7.481.001.25 query25 0.500.160.08 query26 0.610.170.14 query27 0.060.050.06 query28 9.330.880.45 query29 12.55 4.023.30 query30 0.260.090.06 query31 2.840.590.38 query32 3.230.560.48 query33 3.083.143.13 query34 16.04 5.374.75 query35 4.814.834.75 query36 0.680.510.49 query37 0.080.060.06 query38 0.050.040.04 query39 0.030.020.03 query40 0.180.140.14 query41 0.090.030.02 query42 0.030.020.02 query43 0.030.030.03 Total cold run time: 103.96 s Total hot run time: 29.7 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2979964757 TPC-DS: Total hot run time: 192618 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit d1e33cacee9ad756004bdf3d11e1e5a1546fe092, data reload: false query1 13871023998 998 query2 6523189818771877 query3 11006 466346004600 query4 25498 23819 23322 23322 query5 3745583 457 457 query6 296 216 206 206 query7 3992515 292 292 query8 287 231 226 226 query9 8488267626412641 query10 474 345 285 285 query11 15435 15106 14855 14855 query12 157 112 112 112 query13 1557513 397 397 query14 8859613561546135 query15 210 195 168 168 query16 7154626 486 486 query17 1195693 562 562 query18 1985432 313 313 query19 202 191 164 164 query20 130 129 122 122 query21 208 131 111 111 query22 4432447643454345 query23 34641 33738 33393 33393 query24 8622244524582445 query25 573 508 432 432 query26 1192274 154 154 query27 2818524 369 369 query28 4665218421772177 query29 787 593 486 486 query30 282 228 199 199 query31 975 867 763 763 query32 71 69 60 60 query33 550 385 322 322 query34 826 944 543 543 query35 816 843 765 765 query36 973 1014911 911 query37 111 111 85 85 query38 4190432341804180 query39 1531145014431443 query40 221 130 117 117 query41 62 59 55 55 query42 124 115 121 115 query43 516 527 481 481 query44 1409851 849 849 query45 193 187 166 166 query46 857 1021660 660 query47 1810187317721772 query48 396 423 327 327 query49 733 487 402 402 query50 671 717 409 409 query51 4200425841824182 query52 110 112 105 105 query53 242 267 185 185 query54 592 590 521 521 query55 95 81 84 81 query56 311 300 294 294 query57 1224124211681168 query58 280 294 264 264 query59 2636277826412641 query60 351 342 322 322 query61 130 121 123 121 query62 803 717 669 669 query63 236 194 197 194 query64 40811044693 693 query65 4391430342514251 query66 1101406 306 306 query67 16188 15661 15331 15331 query68 8356897 520 520 query69 485 303 278 278 query70 1228112111321121 query71 475 331 291 291 query72 5609482247494749 query73 703 649 362 362 query74 8879911488798879 query75 3814321126802680 query76 37111199761 761 query77 790 386 293 293 query78 10069 10089 94389438 query79 2305832 583 583 query80 580 499 448 448 query81 492 259 221 221 query82 430 121 95 95 query83 289 252 239 239 query84 298 106 83 83 query85 779 354 315 315 query86 331 295 268 268 query87 4348438543004300 query88 3653228222602260 query89 396 321 291 291 query90 1880215 219 215 query91 146 144 111 111 query92 73 62 61 61 query93 1671976 579 579 query94 677 429 376 376 query95 368 291 278 278 query96 496 565 285 285 query97 2690276526762676 query98 234 205 230 205 query99 1471144112751275 Total cold run time: 278532 ms Total hot run time: 192618 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [Improvement](cloud) Hold tablets in another RPC thread [doris]
doris-robot commented on PR #51701: URL: https://github.com/apache/doris/pull/51701#issuecomment-2979932014 TPC-H: Total hot run time: 33952 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit d1e33cacee9ad756004bdf3d11e1e5a1546fe092, data reload: false -- Round 1 -- q1 17604 516950385038 q2 1946287 188 188 q3 10369 1258747 747 q4 10218 1017515 515 q5 7571235423232323 q6 185 163 132 132 q7 888 745 624 624 q8 9328130011371137 q9 6754507851595078 q10 6892237619711971 q11 501 289 279 279 q12 344 342 215 215 q13 17765 365730783078 q14 225 232 217 217 q15 570 480 474 474 q16 430 434 369 369 q17 584 872 359 359 q18 7887715972467159 q19 2547973 556 556 q20 323 321 216 216 q21 3589311523432343 q22 10381047934 934 Total cold run time: 107558 ms Total hot run time: 33952 ms - Round 2, with runtime_filter_mode=off - q1 5329503150505031 q2 235 325 235 235 q3 2133266522822282 q4 1333170613791379 q5 4245419343664193 q6 217 179 136 136 q7 2019194217681768 q8 2607265926162616 q9 7254708171077081 q10 3065318928022802 q11 580 515 493 493 q12 665 783 616 616 q13 3489384832593259 q14 286 302 279 279 q15 520 488 474 474 q16 447 490 441 441 q17 1159156613721372 q18 7705760975757575 q19 830 798 893 798 q20 1995204118771877 q21 4888438742954295 q22 11001060999 999 Total cold run time: 52101 ms Total hot run time: 50001 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
