Re: [PR] [fix](load) reset routine load task EOF sign correctly [doris]
dataroaring merged PR #50048: URL: https://github.com/apache/doris/pull/50048 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](load) reset routine load task EOF sign correctly [doris]
github-actions[bot] commented on PR #50048: URL: https://github.com/apache/doris/pull/50048#issuecomment-2814652517 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](load) reset routine load task EOF sign correctly [doris]
github-actions[bot] commented on PR #50048: URL: https://github.com/apache/doris/pull/50048#issuecomment-2814652568 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](load) reset routine load task EOF sign correctly [doris]
doris-robot commented on PR #50048: URL: https://github.com/apache/doris/pull/50048#issuecomment-2812385744 ClickBench: Total hot run time: 29.29 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 8e7f684334528a48637f02cda785bfa8dde4d153, data reload: false query1 0.040.040.03 query2 0.120.110.11 query3 0.250.200.20 query4 1.590.200.19 query5 0.600.600.59 query6 1.180.700.72 query7 0.020.020.02 query8 0.040.040.04 query9 0.570.520.52 query10 0.570.570.57 query11 0.150.110.11 query12 0.150.120.11 query13 0.600.590.60 query14 1.151.251.15 query15 0.870.850.87 query16 0.380.380.38 query17 1.041.021.05 query18 0.210.200.18 query19 1.901.851.78 query20 0.010.010.01 query21 15.42 0.940.55 query22 0.741.050.66 query23 15.07 1.420.63 query24 6.701.970.55 query25 0.490.250.12 query26 0.550.160.14 query27 0.060.050.05 query28 9.390.910.43 query29 12.59 3.963.31 query30 0.250.090.06 query31 2.830.600.39 query32 3.220.540.48 query33 3.023.063.11 query34 15.76 5.094.46 query35 4.524.514.45 query36 0.650.500.49 query37 0.090.070.06 query38 0.050.040.03 query39 0.020.020.02 query40 0.170.140.13 query41 0.070.030.02 query42 0.040.020.02 query43 0.040.030.03 Total cold run time: 103.18 s Total hot run time: 29.29 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](load) reset routine load task EOF sign correctly [doris]
doris-robot commented on PR #50048: URL: https://github.com/apache/doris/pull/50048#issuecomment-2812373490 TPC-DS: Total hot run time: 192557 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 8e7f684334528a48637f02cda785bfa8dde4d153, data reload: false query1 1433106710951067 query2 6143178318041783 query3 11006 442644414426 query4 25745 23592 23560 23560 query5 4526646 442 442 query6 295 196 202 196 query7 3988493 278 278 query8 289 241 229 229 query9 8517252125562521 query10 475 314 265 265 query11 15305 15022 14996 14996 query12 154 108 108 108 query13 1573500 397 397 query14 8910614662936146 query15 200 190 167 167 query16 7652619 469 469 query17 1197766 621 621 query18 2032422 311 311 query19 200 195 165 165 query20 144 122 124 122 query21 211 129 113 113 query22 4612443242624262 query23 34380 33568 33502 33502 query24 8866244523812381 query25 521 476 401 401 query26 1328281 148 148 query27 2788498 334 334 query28 4784219421382138 query29 709 563 443 443 query30 280 222 188 188 query31 903 885 786 786 query32 74 64 65 64 query33 537 365 318 318 query34 795 884 516 516 query35 843 848 769 769 query36 956 1008893 893 query37 117 97 71 71 query38 4167425342944253 query39 1477145414271427 query40 205 120 104 104 query41 59 58 51 51 query42 120 111 117 111 query43 500 506 475 475 query44 1290813 804 804 query45 184 173 163 163 query46 847 1022641 641 query47 1855185617921792 query48 382 419 303 303 query49 760 510 421 421 query50 652 703 411 411 query51 4248429241634163 query52 112 113 113 113 query53 230 275 190 190 query54 597 583 507 507 query55 80 84 82 82 query56 322 305 328 305 query57 1210119311041104 query58 255 255 256 255 query59 2765282026652665 query60 333 318 295 295 query61 136 125 128 125 query62 755 766 666 666 query63 225 186 184 184 query64 40481048696 696 query65 4536433243354332 query66 978 404 309 309 query67 15798 15553 15304 15304 query68 8227872 501 501 query69 486 306 282 282 query70 1158109011151090 query71 471 310 289 289 query72 5616468248414682 query73 694 649 352 352 query74 8891911789478947 query75 3912318126732673 query76 37461183741 741 query77 788 360 275 275 query78 991610080 94049404 query79 2226823 554 554 query80 583 507 444 444 query81 475 256 221 221 query82 456 127 99 99 query83 250 249 233 233 query84 246 100 82 82 query85 776 345 325 325 query86 377 297 282 282 query87 4454438643094309 query88 3729220421922192 query89 390 309 328 309 query90 1907204 208 204 query91 137 143 111 111 query92 77 60 55 55 query93 1942950 573 573 query94 645 419 299 299 query95 376 296 283 283 query96 485 567 273 273 query97 3104321131403140 query98 219 215 202 202 query99 1526141712841284 Total cold run time: 279899 ms Total hot run time: 192557 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [fix](load) reset routine load task EOF sign correctly [doris]
doris-robot commented on PR #50048: URL: https://github.com/apache/doris/pull/50048#issuecomment-2812346261 TPC-H: Total hot run time: 35002 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 8e7f684334528a48637f02cda785bfa8dde4d153, data reload: false -- Round 1 -- q1 25934 525252525252 q2 2072291 180 180 q3 10378 1339711 711 q4 10242 546 546 q5 7592265426602654 q6 200 176 133 133 q7 1000786 623 623 q8 9319148412171217 q9 6912529752525252 q10 6945235919031903 q11 497 301 283 283 q12 372 396 225 225 q13 17758 379930793079 q14 246 232 216 216 q15 548 495 477 477 q16 450 454 407 407 q17 592 914 354 354 q18 7508723271777177 q19 12271076599 599 q20 337 343 218 218 q21 4379345425342534 q22 10771013962 962 Total cold run time: 115585 ms Total hot run time: 35002 ms - Round 2, with runtime_filter_mode=off - q1 5493546854765468 q2 249 344 228 228 q3 2191272023282328 q4 1531197614431443 q5 4507439844334398 q6 260 191 130 130 q7 2058191717581758 q8 2880281627762776 q9 7250695372176953 q10 3095323727642764 q11 644 528 494 494 q12 713 766 614 614 q13 3581404333463346 q14 298 304 265 265 q15 542 485 481 481 q16 496 515 464 464 q17 1231170214781478 q18 7745759374937493 q19 886 777 829 777 q20 1946199318871887 q21 5475495047434743 q22 1099106110701061 Total cold run time: 54170 ms Total hot run time: 51349 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](load) reset routine load task EOF sign correctly [doris]
sollhui commented on PR #50048: URL: https://github.com/apache/doris/pull/50048#issuecomment-2812255620 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](load) reset routine load task EOF sign correctly [doris]
liaoxin01 commented on code in PR #50048:
URL: https://github.com/apache/doris/pull/50048#discussion_r2046999271
##
fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadTaskInfo.java:
##
@@ -171,6 +172,11 @@ public void
handleTaskByTxnCommitAttachment(RLTaskTxnCommitAttachment rlTaskTxnC
}
private void judgeEof(RLTaskTxnCommitAttachment rlTaskTxnCommitAttachment)
{
+if (DebugPointUtil.isEnable("RoutineLoadTaskInfo.judgeEof")) {
+this.isEof = false;
+return;
+}
+this.isEof = false;
Review Comment:
```suggestion
if (rlTaskTxnCommitAttachment.getTotalRows() <
routineLoadJob.getMaxBatchRows()
&& rlTaskTxnCommitAttachment.getReceivedBytes() <
routineLoadJob.getMaxBatchSizeBytes()
&& rlTaskTxnCommitAttachment.getTaskExecutionTimeMs() <
this.timeoutMs) {
this.isEof = true;
} else {
this.isEof = false;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](load) reset routine load task EOF sign correctly [doris]
doris-robot commented on PR #50048: URL: https://github.com/apache/doris/pull/50048#issuecomment-2804082423 TPC-H: Total hot run time: 34009 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit e10802887d2cf0c4b5aa42532f0de32fa03e7aa4, data reload: false -- Round 1 -- q1 25838 503350045004 q2 2072279 188 188 q3 10390 1228670 670 q4 10218 992 516 516 q5 7533229423162294 q6 182 158 130 130 q7 908 824 600 600 q8 9310127911301130 q9 6817506351285063 q10 6843229218721872 q11 505 283 271 271 q12 356 364 221 221 q13 17773 367130573057 q14 230 228 212 212 q15 531 487 474 474 q16 633 616 594 594 q17 595 850 357 357 q18 7437713170637063 q19 1502961 588 588 q20 333 328 224 224 q21 4161339625052505 q22 10621037976 976 Total cold run time: 115229 ms Total hot run time: 34009 ms - Round 2, with runtime_filter_mode=off - q1 5157509650865086 q2 244 334 242 242 q3 2225263622742274 q4 1409177913961396 q5 4441440043584358 q6 219 168 129 129 q7 1993191717491749 q8 2597264624792479 q9 7312720270367036 q10 2968316727132713 q11 604 508 482 482 q12 664 756 588 588 q13 3472389032543254 q14 308 332 283 283 q15 527 470 482 470 q16 642 670 669 669 q17 1153154313731373 q18 7715768074037403 q19 813 815 804 804 q20 1884196118551855 q21 5340490048674867 q22 1075106310131013 Total cold run time: 52762 ms Total hot run time: 50523 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](load) reset routine load task EOF sign correctly [doris]
doris-robot commented on PR #50048: URL: https://github.com/apache/doris/pull/50048#issuecomment-2804125066 ClickBench: Total hot run time: 30.79 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit e10802887d2cf0c4b5aa42532f0de32fa03e7aa4, data reload: false query1 0.040.040.03 query2 0.120.100.12 query3 0.250.190.19 query4 1.590.190.19 query5 0.600.590.59 query6 1.200.710.72 query7 0.020.020.02 query8 0.040.030.03 query9 0.570.510.51 query10 0.580.580.58 query11 0.150.110.10 query12 0.140.110.11 query13 0.610.590.60 query14 2.672.842.80 query15 0.950.860.84 query16 0.380.380.40 query17 1.031.031.06 query18 0.210.190.20 query19 1.921.921.81 query20 0.010.010.01 query21 15.35 0.950.56 query22 0.741.200.70 query23 14.86 1.350.61 query24 7.341.390.34 query25 0.450.110.24 query26 0.710.170.14 query27 0.060.040.05 query28 8.660.870.43 query29 12.56 4.033.36 query30 0.260.090.07 query31 2.820.600.39 query32 3.210.540.47 query33 3.003.033.01 query34 15.72 5.084.48 query35 4.494.524.48 query36 0.660.500.49 query37 0.080.060.06 query38 0.040.040.03 query39 0.030.020.02 query40 0.170.140.12 query41 0.080.020.02 query42 0.040.030.02 query43 0.040.030.03 Total cold run time: 104.45 s Total hot run time: 30.79 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](load) reset routine load task EOF sign correctly [doris]
doris-robot commented on PR #50048: URL: https://github.com/apache/doris/pull/50048#issuecomment-2804112248 TPC-DS: Total hot run time: 193059 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit e10802887d2cf0c4b5aa42532f0de32fa03e7aa4, data reload: false query1 1422110610891089 query2 6122189118911891 query3 11000 452845474528 query4 56338 24481 22982 22982 query5 4909499 456 456 query6 346 209 187 187 query7 4955486 278 278 query8 306 245 229 229 query9 5977259226092592 query10 432 335 268 268 query11 15113 14961 14797 14797 query12 165 117 106 106 query13 1089526 409 409 query14 10172 645663646364 query15 214 207 176 176 query16 7170650 513 513 query17 1119758 583 583 query18 1606429 340 340 query19 209 202 169 169 query20 126 132 122 122 query21 232 127 109 109 query22 4420442042524252 query23 34137 33603 33684 33603 query24 6529242724942427 query25 473 475 405 405 query26 703 267 160 160 query27 2325506 331 331 query28 3052243124352431 query29 571 551 449 449 query30 276 226 192 192 query31 862 890 774 774 query32 81 67 64 64 query33 459 383 330 330 query34 758 893 516 516 query35 847 851 751 751 query36 939 985 893 893 query37 120 97 79 79 query38 4170428643144286 query39 1508144814701448 query40 215 135 109 109 query41 63 52 53 52 query42 124 109 115 109 query43 511 530 486 486 query44 1297829 815 815 query45 185 182 168 168 query46 851 1027632 632 query47 1831188517801780 query48 385 422 303 303 query49 691 518 405 405 query50 733 721 418 418 query51 4228431141924192 query52 121 110 103 103 query53 237 275 187 187 query54 587 593 520 520 query55 85 87 95 87 query56 300 304 311 304 query57 1159118910991099 query58 263 253 261 253 query59 2710279926762676 query60 340 335 330 330 query61 130 132 138 132 query62 736 748 663 663 query63 221 190 184 184 query64 17231056705 705 query65 4304425642404240 query66 699 420 304 304 query67 15836 15522 15664 15522 query68 7240829 532 532 query69 549 300 280 280 query70 1214108810851085 query71 524 306 297 297 query72 5540471748524717 query73 1372627 351 351 query74 9029916188038803 query75 3755318726772677 query76 42901188764 764 query77 635 359 328 328 query78 994710088 93149314 query79 2449802 554 554 query80 619 499 445 445 query81 484 258 226 226 query82 466 130 93 93 query83 349 258 242 242 query84 296 107 94 94 query85 807 352 312 312 query86 369 295 283 283 query87 4411446943234323 query88 3056222522382225 query89 402 305 282 282 query90 1957207 210 207 query91 144 138 114 114 query92 77 67 58 58 query93 1175923 584 584 query94 732 423 309 309 query95 371 283 286 283 query96 498 565 269 269 query97 3136322031713171 query98 234 209 199 199 query99 1431137712811281 Total cold run time: 299375 ms Total hot run time: 193059 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] fix(load): reset routine load task EOF sign correctly [doris]
sollhui commented on PR #50048: URL: https://github.com/apache/doris/pull/50048#issuecomment-2803993435 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] fix(load): reset routine load task EOF sign correctly [doris]
hello-stephen commented on PR #50048: URL: https://github.com/apache/doris/pull/50048#issuecomment-2803990968 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
