Re: [PR] [enhance](memory) back pressure writing when memory usage is high in sink operation [doris]
liaoxin01 merged PR #58530: URL: https://github.com/apache/doris/pull/58530 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [enhance](memory) back pressure writing when memory usage is high in sink operation [doris]
hello-stephen commented on PR #58530: URL: https://github.com/apache/doris/pull/58530#issuecomment-3605547060 # BE Regression && UT Coverage Report Increment line coverage `100.00% (7/7)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/58530_b0051fc60604597f6f85160d7e053a5b270312d1_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/58530_b0051fc60604597f6f85160d7e053a5b270312d1_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 72.25% (24799/34326) | | Line Coverage | 58.88% (259856/441316) | | Region Coverage | 53.77% (215880/401503) | | Branch Coverage | 55.25% (92259/166985) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [enhance](memory) back pressure writing when memory usage is high in sink operation [doris]
hello-stephen commented on PR #58530: URL: https://github.com/apache/doris/pull/58530#issuecomment-3602793642 # BE Regression && UT Coverage Report Increment line coverage `100.00% (7/7)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/58530_b0051fc60604597f6f85160d7e053a5b270312d1_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/58530_b0051fc60604597f6f85160d7e053a5b270312d1_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 72.22% (24791/34326) | | Line Coverage | 58.83% (259638/441316) | | Region Coverage | 53.66% (215430/401503) | | Branch Coverage | 55.20% (92176/166985) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [enhance](memory) back pressure writing when memory usage is high in sink operation [doris]
hello-stephen commented on PR #58530: URL: https://github.com/apache/doris/pull/58530#issuecomment-3602400670 # BE UT Coverage Report Increment line coverage `0.00% (0/7)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/b0051fc60604597f6f85160d7e053a5b270312d1_b0051fc60604597f6f85160d7e053a5b270312d1/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/b0051fc60604597f6f85160d7e053a5b270312d1_b0051fc60604597f6f85160d7e053a5b270312d1/report/index.html) | Category | Coverage | |---|| | Function Coverage | 53.51% (18734/35010) | | Line Coverage | 39.10% (172714/441711) | | Region Coverage | 33.70% (133702/396693) | | Branch Coverage | 34.66% (57566/166088) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [enhance](memory) back pressure writing when memory usage is high in sink operation [doris]
doris-robot commented on PR #58530: URL: https://github.com/apache/doris/pull/58530#issuecomment-3602232188 ClickBench: Total hot run time: 27.17 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit b0051fc60604597f6f85160d7e053a5b270312d1, data reload: false query1 0.050.050.05 query2 0.110.050.05 query3 0.260.090.09 query4 1.610.100.11 query5 0.260.260.27 query6 1.170.670.64 query7 0.030.020.02 query8 0.050.040.04 query9 0.580.500.50 query10 0.560.550.56 query11 0.160.110.11 query12 0.140.120.11 query13 0.630.600.60 query14 0.990.980.98 query15 0.810.790.80 query16 0.390.420.40 query17 1.040.981.06 query18 0.230.220.22 query19 1.921.851.86 query20 0.010.020.01 query21 15.48 0.290.14 query22 4.630.050.04 query23 16.05 0.270.10 query24 2.170.610.20 query25 0.080.100.05 query26 0.140.140.14 query27 0.060.060.04 query28 5.361.221.02 query29 12.56 4.033.19 query30 0.270.160.13 query31 2.810.620.39 query32 3.230.550.46 query33 3.013.013.00 query34 16.73 5.214.53 query35 4.544.574.56 query36 0.660.500.48 query37 0.100.070.06 query38 0.070.040.04 query39 0.050.030.03 query40 0.170.140.13 query41 0.090.030.03 query42 0.040.030.03 query43 0.040.040.03 Total cold run time: 99.34 s Total hot run time: 27.17 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [enhance](memory) back pressure writing when memory usage is high in sink operation [doris]
doris-robot commented on PR #58530: URL: https://github.com/apache/doris/pull/58530#issuecomment-3602212360 TPC-DS: Total hot run time: 181969 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit b0051fc60604597f6f85160d7e053a5b270312d1, data reload: false query1 1091398 387 387 query2 6586120011521152 query3 6744223 226 223 query4 25466 23325 23002 23002 query5 4909672 486 486 query6 335 238 231 231 query7 4666516 319 319 query8 305 262 257 257 query9 8736263826562638 query10 536 355 315 315 query11 15325 15395 14928 14928 query12 203 123 117 117 query13 1696568 447 447 query14 9694604660716046 query15 215 200 185 185 query16 7721712 528 528 query17 1217780 627 627 query18 2043432 340 340 query19 224 205 195 195 query20 134 136 128 128 query21 220 138 117 117 query22 3959395838773877 query23 32863 31965 32021 31965 query24 8565241324342413 query25 658 544 496 496 query26 1248284 170 170 query27 2707510 341 341 query28 4329218621802180 query29 822 661 538 538 query30 325 248 215 215 query31 812 711 625 625 query32 88 83 75 75 query33 622 446 327 327 query34 822 885 546 546 query35 772 849 758 758 query36 891 923 851 851 query37 134 118 90 90 query38 3906385738723857 query39 1482160314071407 query40 240 136 119 119 query41 67 63 63 63 query42 126 118 114 114 query43 439 458 432 432 query44 1318743 748 743 query45 198 188 184 184 query46 893 1033645 645 query47 1702170316341634 query48 408 432 349 349 query49 753 514 428 428 query50 697 711 415 415 query51 3892389238333833 query52 112 118 104 104 query53 249 259 204 204 query54 313 313 286 286 query55 99 95 92 92 query56 362 336 322 322 query57 1134116611061106 query58 288 284 275 275 query59 2369244323182318 query60 362 375 340 340 query61 162 165 161 161 query62 775 719 655 655 query63 227 193 195 193 query64 45011224900 900 query65 4047398439843984 query66 1112444 339 339 query67 15202 15071 14724 14724 query68 8417999 626 626 query69 512 349 310 310 query70 1094100710261007 query71 498 347 316 316 query72 5839508648034803 query73 706 585 349 349 query74 8774882786358635 query75 3672302925392539 query76 38091175770 770 query77 833 410 313 313 query78 9475969187268726 query79 1987850 593 593 query80 654 584 514 514 query81 482 280 246 246 query82 433 153 120 120 query83 265 266 244 244 query84 258 123 100 100 query85 924 495 459 459 query86 343 281 307 281 query87 4024404139743974 query88 3685228022732273 query89 395 336 289 289 query90 1968223 215 215 query91 175 175 142 142 query92 91 72 65 65 query93 1280992 657 657 query94 732 442 335 335 query95 503 420 405 405 query96 536 554 286 286 query97 2641267325632563 query98 242 212 212 212 query99 1639139512411241 Total cold run time: 272004 ms Total hot run time: 181969 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [enhance](memory) back pressure writing when memory usage is high in sink operation [doris]
doris-robot commented on PR #58530: URL: https://github.com/apache/doris/pull/58530#issuecomment-3602164129 TPC-H: Total hot run time: 34234 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit b0051fc60604597f6f85160d7e053a5b270312d1, data reload: false -- Round 1 -- q1 17616 511849044904 q2 2057319 200 200 q3 10245 1339750 750 q4 10228 878 320 320 q5 7517243721462146 q6 193 174 135 135 q7 957 781 633 633 q8 9361138810991099 q9 7015533653255325 q10 6784217817821782 q11 515 315 285 285 q12 335 376 237 237 q13 17761 364930183018 q14 232 239 221 221 q15 595 530 520 520 q16 875 854 808 808 q17 681 785 527 527 q18 7408716270947094 q19 1116983 613 613 q20 351 350 224 224 q21 3987325424582458 q22 1014976 935 935 Total cold run time: 106843 ms Total hot run time: 34234 ms - Round 2, with runtime_filter_mode=off - q1 4947494049044904 q2 313 393 312 312 q3 2134267323332333 q4 1323176412841284 q5 4223441146554411 q6 222 172 128 128 q7 2036200418051805 q8 2639254225342534 q9 7539756174797479 q10 3045325228102810 q11 589 585 552 552 q12 708 724 575 575 q13 3441391833403340 q14 283 291 283 283 q15 546 523 517 517 q16 888 993 890 890 q17 1183154813851385 q18 8064767675257525 q19 879 884 889 884 q20 2012216219021902 q21 4952445641644164 q22 10681007971 971 Total cold run time: 53034 ms Total hot run time: 50988 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [enhance](memory) back pressure writing when memory usage is high in sink operation [doris]
sollhui commented on PR #58530: URL: https://github.com/apache/doris/pull/58530#issuecomment-3602032366 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [enhance](memory) back pressure writing when memory usage is high in sink operation [doris]
sollhui commented on PR #58530: URL: https://github.com/apache/doris/pull/58530#issuecomment-3596551727 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [enhance](memory) back pressure writing when memory usage is high in sink operation [doris]
sollhui commented on PR #58530: URL: https://github.com/apache/doris/pull/58530#issuecomment-3596476383 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [enhance](memory) back pressure writing when memory usage is high in sink operation [doris]
doris-robot commented on PR #58530: URL: https://github.com/apache/doris/pull/58530#issuecomment-3591720494 # BE UT Coverage Report Increment line coverage `0.00% (0/5)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/95610cbd2df07c4d4cd4ad3ed2496b9ec8cea287_95610cbd2df07c4d4cd4ad3ed2496b9ec8cea287/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/95610cbd2df07c4d4cd4ad3ed2496b9ec8cea287_95610cbd2df07c4d4cd4ad3ed2496b9ec8cea287/report/index.html) | Category | Coverage | |---|| | Function Coverage | 52.90% (18513/34995) | | Line Coverage | 38.45% (169645/441190) | | Region Coverage | 33.23% (131651/396156) | | Branch Coverage | 34.18% (56683/165817) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [enhance](memory) back pressure writing when memory usage is high in sink operation [doris]
doris-robot commented on PR #58530: URL: https://github.com/apache/doris/pull/58530#issuecomment-3591714431 ClickBench: Total hot run time: 27.46 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 95610cbd2df07c4d4cd4ad3ed2496b9ec8cea287, data reload: false query1 0.050.070.05 query2 0.090.050.05 query3 0.260.090.09 query4 1.600.110.11 query5 0.270.260.26 query6 1.160.630.62 query7 0.040.020.03 query8 0.060.040.05 query9 0.580.510.50 query10 0.570.560.56 query11 0.160.110.12 query12 0.140.120.11 query13 0.620.600.62 query14 0.980.971.01 query15 0.800.810.81 query16 0.400.430.42 query17 1.001.061.04 query18 0.250.210.21 query19 1.951.851.86 query20 0.020.010.01 query21 15.44 0.290.14 query22 4.750.050.04 query23 16.14 0.270.10 query24 1.010.550.23 query25 0.070.050.06 query26 0.140.140.13 query27 0.070.040.05 query28 2.971.251.03 query29 12.62 3.983.29 query30 0.270.140.11 query31 2.820.600.41 query32 3.240.540.47 query33 2.983.093.05 query34 16.96 5.184.56 query35 4.604.544.54 query36 0.650.500.49 query37 0.100.060.06 query38 0.060.040.04 query39 0.050.030.03 query40 0.180.150.13 query41 0.080.030.04 query42 0.040.040.03 query43 0.040.030.04 Total cold run time: 96.28 s Total hot run time: 27.46 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [enhance](memory) back pressure writing when memory usage is high in sink operation [doris]
doris-robot commented on PR #58530: URL: https://github.com/apache/doris/pull/58530#issuecomment-3591711348 TPC-DS: Total hot run time: 181233 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 95610cbd2df07c4d4cd4ad3ed2496b9ec8cea287, data reload: false query1 1072405 384 384 query2 6601121811681168 query3 6750229 220 220 query4 25145 23593 22670 22670 query5 5254658 502 502 query6 340 246 224 224 query7 4656495 299 299 query8 308 251 245 245 query9 8712257626012576 query10 558 358 307 307 query11 15256 15129 14677 14677 query12 187 122 113 113 query13 1673580 474 474 query14 9962596759105910 query15 209 200 188 188 query16 7685707 545 545 query17 1281772 646 646 query18 2064435 355 355 query19 218 246 185 185 query20 127 124 124 124 query21 213 137 115 115 query22 4090396239093909 query23 32875 31874 31853 31853 query24 8517243423692369 query25 598 513 448 448 query26 904 266 168 168 query27 2738497 349 349 query28 4344214321322132 query29 718 631 494 494 query30 306 243 207 207 query31 816 720 621 621 query32 89 79 72 72 query33 586 366 326 326 query34 797 875 541 541 query35 759 827 735 735 query36 874 957 832 832 query37 118 116 114 114 query38 3939385737473747 query39 1486142114121412 query40 231 131 117 117 query41 69 74 63 63 query42 131 117 114 114 query43 432 456 411 411 query44 1298761 761 761 query45 200 193 184 184 query46 860 1001647 647 query47 1676175216401640 query48 406 439 325 325 query49 722 553 430 430 query50 655 686 412 412 query51 3860399638373837 query52 116 117 104 104 query53 245 269 212 212 query54 310 299 277 277 query55 94 96 95 95 query56 327 323 344 323 query57 1130114810791079 query58 293 275 271 271 query59 2313238323322332 query60 359 363 340 340 query61 168 157 159 157 query62 773 697 649 649 query63 224 216 197 197 query64 37631176911 911 query65 4014396239343934 query66 880 442 337 337 query67 15302 15031 14774 14774 query68 5866973 637 637 query69 520 353 302 302 query70 11571073985 985 query71 442 347 320 320 query72 5427484050354840 query73 670 572 357 357 query74 8900878488288784 query75 3053308625652565 query76 32581123772 772 query77 522 416 321 321 query78 9487972389668966 query79 1929843 578 578 query80 730 610 530 530 query81 522 270 246 246 query82 186 145 120 120 query83 264 275 260 260 query84 264 129 162 129 query85 887 497 442 442 query86 379 297 278 278 query87 4018406939213921 query88 2884226422612261 query89 390 357 302 302 query90 1994220 216 216 query91 171 173 142 142 query92 77 70 66 66 query93 1540975 657 657 query94 705 439 319 319 query95 508 401 404 401 query96 512 529 276 276 query97 2614267225712571 query98 248 223 216 216 query99 1285139512641264 Total cold run time: 265085 ms Total hot run time: 181233 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [enhance](memory) back pressure writing when memory usage is high in sink operation [doris]
doris-robot commented on PR #58530: URL: https://github.com/apache/doris/pull/58530#issuecomment-3591702460 TPC-H: Total hot run time: 33908 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 95610cbd2df07c4d4cd4ad3ed2496b9ec8cea287, data reload: false -- Round 1 -- q1 17657 507049024902 q2 2038312 203 203 q3 10275 1301727 727 q4 10235 976 379 379 q5 7607259622552255 q6 187 179 144 144 q7 965 842 658 658 q8 9357132510191019 q9 6922526652145214 q10 6861222318111811 q11 499 319 299 299 q12 332 364 225 225 q13 17797 367530383038 q14 237 245 211 211 q15 590 508 512 508 q16 897 863 808 808 q17 597 751 525 525 q18 7494718269286928 q19 1225951 581 581 q20 349 335 227 227 q21 3579338523042304 q22 988 993 942 942 Total cold run time: 106688 ms Total hot run time: 33908 ms - Round 2, with runtime_filter_mode=off - q1 4973492249014901 q2 317 382 314 314 q3 2155262622522252 q4 1320176013081308 q5 4215430844694308 q6 217 178 136 136 q7 2023206618921892 q8 2693246624482448 q9 7553745574797455 q10 3034320928532853 q11 597 509 490 490 q12 678 792 580 580 q13 3556397033513351 q14 280 333 275 275 q15 548 511 515 511 q16 907 966 908 908 q17 1146144114031403 q18 7916756574307430 q19 789 754 756 754 q20 1893202117761776 q21 4578433841284128 q22 1088107610011001 Total cold run time: 52476 ms Total hot run time: 50474 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [enhance](memory) back pressure writing when memory usage is high in sink operation [doris]
sollhui commented on PR #58530: URL: https://github.com/apache/doris/pull/58530#issuecomment-3591681485 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [enhance](memory) back pressure writing when memory usage is high in sink operation [doris]
sollhui commented on PR #58530: URL: https://github.com/apache/doris/pull/58530#issuecomment-3591680096 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [enhance](memory) back pressure writing when memory usage is high in sink operation [doris]
Thearas commented on PR #58530: URL: https://github.com/apache/doris/pull/58530#issuecomment-3591680030 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
