Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
yiguolei merged PR #52553: URL: https://github.com/apache/doris/pull/52553 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
github-actions[bot] commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3100828181 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
github-actions[bot] commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3100828054 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
hello-stephen commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3099579697 # BE Regression && UT Coverage Report Increment line coverage `80.72% (67/83)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/52553_48eb3f27ae3fe4c1b41b77929c9365461561b1ab_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/52553_48eb3f27ae3fe4c1b41b77929c9365461561b1ab_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 81.04% (22006/27153) | | Line Coverage | 73.67% (227201/308408) | | Region Coverage | 61.27% (189352/309031) | | Branch Coverage | 65.03% (81716/125650) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3099123806 # BE UT Coverage Report Increment line coverage `80.72% (67/83)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/48eb3f27ae3fe4c1b41b77929c9365461561b1ab_48eb3f27ae3fe4c1b41b77929c9365461561b1ab/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/48eb3f27ae3fe4c1b41b77929c9365461561b1ab_48eb3f27ae3fe4c1b41b77929c9365461561b1ab/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.47% (15899/27667) | | Line Coverage | 46.25% (142857/308863) | | Region Coverage | 35.76% (107745/301299) | | Branch Coverage | 38.22% (47488/124258) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3097925762 ClickBench: Total hot run time: 32.86 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 48eb3f27ae3fe4c1b41b77929c9365461561b1ab, data reload: false query1 0.040.040.04 query2 0.080.040.05 query3 0.240.070.08 query4 1.620.110.12 query5 0.430.420.42 query6 1.160.650.67 query7 0.030.020.02 query8 0.040.040.05 query9 0.590.540.52 query10 0.590.580.58 query11 0.160.110.12 query12 0.150.120.12 query13 0.630.620.61 query14 0.800.850.81 query15 0.890.860.86 query16 0.380.390.40 query17 1.041.051.03 query18 0.210.210.21 query19 1.941.851.86 query20 0.010.020.02 query21 15.37 0.900.56 query22 0.801.230.79 query23 14.76 1.380.62 query24 6.420.641.27 query25 0.470.220.08 query26 0.580.170.14 query27 0.060.050.06 query28 10.00 0.940.44 query29 12.56 3.913.32 query30 3.223.173.08 query31 2.820.590.38 query32 3.260.550.48 query33 3.043.173.10 query34 16.18 5.384.86 query35 4.874.914.92 query36 0.720.520.50 query37 0.100.080.08 query38 0.060.050.04 query39 0.040.030.03 query40 0.190.170.15 query41 0.090.040.04 query42 0.050.040.03 query43 0.050.040.05 Total cold run time: 106.74 s Total hot run time: 32.86 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3097911330 TPC-DS: Total hot run time: 189136 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 48eb3f27ae3fe4c1b41b77929c9365461561b1ab, data reload: false query1 999 400 411 400 query2 6582164317311643 query3 6751222 228 222 query4 26876 23588 23418 23418 query5 4493746 599 599 query6 332 242 224 224 query7 4640508 308 308 query8 372 294 285 285 query9 8629286428302830 query10 491 376 306 306 query11 15871 14917 14812 14812 query12 173 125 124 124 query13 1670535 410 410 query14 9394608560496049 query15 215 201 173 173 query16 7255657 429 429 query17 1211733 598 598 query18 2043444 337 337 query19 196 210 207 207 query20 144 134 127 127 query21 229 142 129 129 query22 4081411739903990 query23 33809 32977 32959 32959 query24 8079238024122380 query25 522 492 415 415 query26 735 295 176 176 query27 2638517 348 348 query28 4277218121692169 query29 701 602 501 501 query30 302 240 202 202 query31 959 848 790 790 query32 92 88 84 84 query33 571 407 360 360 query34 795 847 523 523 query35 825 870 759 759 query36 1018986 938 938 query37 166 113 95 95 query38 4123412640264026 query39 1515146314331433 query40 239 144 138 138 query41 106 103 99 99 query42 136 120 123 120 query43 537 529 484 484 query44 1334845 854 845 query45 193 186 178 178 query46 847 1009640 640 query47 1771184917821782 query48 392 438 331 331 query49 754 559 471 471 query50 676 718 439 439 query51 5510561253945394 query52 118 121 115 115 query53 251 282 221 221 query54 660 631 575 575 query55 104 97 100 97 query56 394 362 340 340 query57 1196124011581158 query58 317 308 315 308 query59 2544263224672467 query60 370 376 370 370 query61 159 151 151 151 query62 830 735 681 681 query63 248 211 211 211 query64 31561056743 743 query65 4244422342124212 query66 1252593 514 514 query67 15903 15505 15445 15445 query68 7703892 544 544 query69 514 349 305 305 query70 1239116811511151 query71 478 344 326 326 query72 5813475548754755 query73 705 642 357 357 query74 90159006 query75 3870311326902690 query76 35281142738 738 query77 857 450 380 380 query78 10085 10205 91999199 query79 2244842 579 579 query80 665 595 546 546 query81 482 271 244 244 query82 477 142 119 119 query83 282 280 278 278 query84 261 113 90 90 query85 854 409 359 359 query86 384 343 315 315 query87 4388447042454245 query88 3649232522862286 query89 408 332 309 309 query90 1979235 237 235 query91 165 158 129 129 query92 104 86 82 82 query93 1774963 610 610 query94 727 448 310 310 query95 417 339 326 326 query96 495 583 281 281 query97 2705271126252625 query98 258 225 220 220 query99 1376138813081308 Total cold run time: 275916 ms Total hot run time: 189136 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3097881964 TPC-H: Total hot run time: 34541 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 48eb3f27ae3fe4c1b41b77929c9365461561b1ab, data reload: false -- Round 1 -- q1 17572 528450985098 q2 1945297 201 201 q3 10283 1321736 736 q4 10227 1038535 535 q5 7558250223902390 q6 191 170 140 140 q7 939 776 627 627 q8 9354136211631163 q9 7529509951725099 q10 6921241119891989 q11 484 314 298 298 q12 368 372 230 230 q13 17766 373231153115 q14 236 256 234 234 q15 581 518 522 518 q16 437 439 405 405 q17 610 907 365 365 q18 7515718371507150 q19 12531004623 623 q20 360 372 238 238 q21 4200322924082408 q22 10991041979 979 Total cold run time: 107428 ms Total hot run time: 34541 ms - Round 2, with runtime_filter_mode=off - q1 5218515351345134 q2 269 353 239 239 q3 2153274323272327 q4 1408184513401340 q5 4454445844174417 q6 219 177 134 134 q7 2022197918061806 q8 2719269525432543 q9 7277742171837183 q10 3173331428672867 q11 612 501 492 492 q12 734 799 620 620 q13 3711392833843384 q14 286 304 286 286 q15 557 509 500 500 q16 473 493 461 461 q17 1162152513891389 q18 8090765274987498 q19 1045889 959 889 q20 2027208719471947 q21 5216464644334433 q22 1077105410081008 Total cold run time: 53902 ms Total hot run time: 50897 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
xinyiZzz commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3097417926 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
xinyiZzz commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3096582308 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3096211664 ClickBench: Total hot run time: 33.03 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 6b6ea355bcce8bd3e8d63c638f3e4d2480d76abe, data reload: false query1 0.040.030.04 query2 0.080.040.05 query3 0.240.070.07 query4 1.620.100.11 query5 0.440.420.41 query6 1.140.650.68 query7 0.020.020.03 query8 0.060.040.04 query9 0.590.530.52 query10 0.580.570.56 query11 0.170.110.11 query12 0.160.120.11 query13 0.640.610.62 query14 0.810.830.87 query15 0.900.880.88 query16 0.380.380.40 query17 1.041.091.05 query18 0.210.210.20 query19 1.921.841.80 query20 0.010.010.01 query21 15.39 0.920.54 query22 0.771.210.85 query23 14.78 1.390.64 query24 6.560.801.36 query25 0.450.230.17 query26 0.630.170.14 query27 0.080.050.06 query28 9.380.920.44 query29 12.55 4.003.31 query30 3.193.093.12 query31 2.830.590.39 query32 3.240.560.47 query33 3.083.083.12 query34 16.04 5.414.85 query35 4.834.864.89 query36 0.710.520.50 query37 0.100.080.08 query38 0.060.050.06 query39 0.050.030.03 query40 0.190.170.17 query41 0.090.030.03 query42 0.050.030.03 query43 0.050.040.04 Total cold run time: 106.15 s Total hot run time: 33.03 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3096197024 TPC-DS: Total hot run time: 191119 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 6b6ea355bcce8bd3e8d63c638f3e4d2480d76abe, data reload: false query1 1021397 400 397 query2 6537176217551755 query3 6748229 225 225 query4 26692 23825 23053 23053 query5 4473747 600 600 query6 334 251 222 222 query7 4646524 311 311 query8 394 344 329 329 query9 8642287428262826 query10 525 370 325 325 query11 15936 15122 14814 14814 query12 202 140 136 136 query13 1716565 450 450 query14 9861602759495949 query15 219 208 186 186 query16 7649641 470 470 query17 1202733 629 629 query18 2051444 339 339 query19 205 199 170 170 query20 168 135 134 134 query21 239 142 128 128 query22 4352464045424542 query23 35053 33869 33626 33626 query24 8184239823902390 query25 540 488 449 449 query26 1236283 167 167 query27 2703522 360 360 query28 4332218021412141 query29 754 580 466 466 query30 299 231 207 207 query31 975 867 787 787 query32 93 89 82 82 query33 593 402 353 353 query34 817 858 523 523 query35 811 851 795 795 query36 984 1045908 908 query37 129 113 95 95 query38 4096412840534053 query39 1502144616101446 query40 235 142 136 136 query41 107 100 98 98 query42 134 121 120 120 query43 525 512 478 478 query44 1324877 854 854 query45 191 184 180 180 query46 838 1017644 644 query47 1861184118361836 query48 396 443 326 326 query49 816 570 462 462 query50 662 707 442 442 query51 5556570554505450 query52 126 122 110 110 query53 255 277 204 204 query54 614 609 534 534 query55 95 95 95 95 query56 345 354 341 341 query57 1215125811681168 query58 317 317 304 304 query59 2599265025752575 query60 385 379 360 360 query61 154 152 149 149 query62 813 777 741 741 query63 251 212 211 211 query64 43271088784 784 query65 4315423641914191 query66 1251593 519 519 query67 15955 15613 15525 15525 query68 8839908 541 541 query69 490 375 314 314 query70 1288117411201120 query71 479 355 333 333 query72 5698483948244824 query73 755 627 371 371 query74 9065905190539051 query75 4234313226662666 query76 37231153752 752 query77 854 466 393 393 query78 10081 10225 93239323 query79 5849810 564 564 query80 694 593 522 522 query81 480 274 233 233 query82 705 149 119 119 query83 316 289 289 289 query84 304 127 106 106 query85 821 403 373 373 query86 374 341 334 334 query87 4479442944714429 query88 2931235923382338 query89 477 342 301 301 query90 1980242 241 241 query91 151 159 125 125 query92 104 89 78 78 query93 3190951 594 594 query94 691 417 324 324 query95 415 338 334 334 query96 488 598 282 282 query97 2780278226252625 query98 252 224 225 224 query99 1426142813211321 Total cold run time: 286988 ms Total hot run time: 191119 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3096158332 TPC-H: Total hot run time: 34599 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 6b6ea355bcce8bd3e8d63c638f3e4d2480d76abe, data reload: false -- Round 1 -- q1 17617 528151545154 q2 1941293 202 202 q3 10353 1315725 725 q4 10246 1022529 529 q5 7577247023452345 q6 202 176 141 141 q7 954 780 635 635 q8 9320131011691169 q9 6959516451985164 q10 6924237720012001 q11 483 306 294 294 q12 363 386 244 244 q13 17794 369431113111 q14 240 251 236 236 q15 585 528 532 528 q16 441 450 394 394 q17 641 890 399 399 q18 7643726371107110 q19 1248986 609 609 q20 384 376 261 261 q21 4073259323602360 q22 11051053988 988 Total cold run time: 107093 ms Total hot run time: 34599 ms - Round 2, with runtime_filter_mode=off - q1 5251518351935183 q2 268 346 242 242 q3 2190268122832283 q4 1363184013261326 q5 4404453145764531 q6 235 177 141 141 q7 2029194718221822 q8 2671273226222622 q9 7327760572157215 q10 3159327929002900 q11 592 513 487 487 q12 726 798 648 648 q13 3666397933113311 q14 292 303 288 288 q15 586 512 506 506 q16 475 504 472 472 q17 1184163713661366 q18 8145767075037503 q19 921 904 916 904 q20 2057206419151915 q21 5071466346674663 q22 1124111711121112 Total cold run time: 53736 ms Total hot run time: 51440 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
xinyiZzz commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3095862146 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
hello-stephen commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3095800293 # BE UT Coverage Report Increment line coverage `80.43% (74/92)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/08920acb61c1f6daf1ea4ffe70a0b936608ff5b1_08920acb61c1f6daf1ea4ffe70a0b936608ff5b1/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/08920acb61c1f6daf1ea4ffe70a0b936608ff5b1_08920acb61c1f6daf1ea4ffe70a0b936608ff5b1/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.46% (15898/27668) | | Line Coverage | 46.26% (142848/308825) | | Region Coverage | 35.76% (107724/301277) | | Branch Coverage | 38.22% (47486/124238) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3095305145 ClickBench: Total hot run time: 32.49 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 08920acb61c1f6daf1ea4ffe70a0b936608ff5b1, data reload: false query1 0.040.040.04 query2 0.080.040.05 query3 0.250.080.08 query4 1.610.110.11 query5 0.450.420.42 query6 1.170.660.66 query7 0.030.020.02 query8 0.040.040.03 query9 0.590.540.51 query10 0.590.570.57 query11 0.170.110.12 query12 0.150.120.12 query13 0.630.630.61 query14 0.790.830.82 query15 0.890.860.87 query16 0.390.380.40 query17 1.021.071.04 query18 0.210.210.20 query19 1.921.851.79 query20 0.020.020.01 query21 15.37 0.900.53 query22 0.801.200.85 query23 14.78 1.370.63 query24 6.681.970.51 query25 0.480.230.10 query26 0.580.160.13 query27 0.060.060.06 query28 9.580.910.43 query29 12.58 3.903.24 query30 3.113.093.06 query31 2.810.580.38 query32 3.240.550.48 query33 3.073.163.07 query34 16.06 5.454.81 query35 4.874.884.88 query36 0.680.510.49 query37 0.110.080.07 query38 0.060.050.05 query39 0.040.030.04 query40 0.190.170.16 query41 0.090.040.04 query42 0.040.030.04 query43 0.050.040.04 Total cold run time: 106.37 s Total hot run time: 32.49 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3095296291 TPC-DS: Total hot run time: 189413 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 08920acb61c1f6daf1ea4ffe70a0b936608ff5b1, data reload: false query1 1034399 415 399 query2 6574176717431743 query3 6753228 229 228 query4 26562 23735 22975 22975 query5 4467660 562 562 query6 323 233 225 225 query7 4640507 291 291 query8 362 312 302 302 query9 8644287028462846 query10 533 397 324 324 query11 15833 15089 14832 14832 query12 185 137 136 136 query13 1694577 440 440 query14 9481607561026075 query15 224 207 180 180 query16 7715691 520 520 query17 1204738 604 604 query18 2053425 348 348 query19 204 196 175 175 query20 149 131 130 130 query21 234 141 123 123 query22 4226428341904190 query23 33870 33067 32919 32919 query24 8151236924032369 query25 541 491 434 434 query26 947 283 170 170 query27 2702515 349 349 query28 4367216721372137 query29 679 573 469 469 query30 304 246 203 203 query31 972 879 799 799 query32 89 88 85 85 query33 598 401 367 367 query34 815 855 526 526 query35 814 839 758 758 query36 995 1032941 941 query37 127 119 96 96 query38 4119420940224022 query39 1508147814651465 query40 236 148 132 132 query41 109 103 102 102 query42 137 125 128 125 query43 528 505 499 499 query44 1381851 886 851 query45 191 181 182 181 query46 850 1020649 649 query47 1840187517831783 query48 404 443 329 329 query49 794 558 473 473 query50 674 711 436 436 query51 5608560055335533 query52 132 131 115 115 query53 251 289 208 208 query54 619 617 556 556 query55 92 97 98 97 query56 348 357 333 333 query57 1236123011641164 query58 320 316 315 315 query59 2640277026022602 query60 375 372 384 372 query61 158 154 153 153 query62 818 735 663 663 query63 241 212 209 209 query64 35961127743 743 query65 4297419441734173 query66 1068585 524 524 query67 15875 15592 15316 15316 query68 8774911 543 543 query69 509 356 319 319 query70 1289116511581158 query71 485 350 329 329 query72 5642476049374760 query73 805 698 364 364 query74 8976917186328632 query75 4196313926572657 query76 35971139734 734 query77 847 455 375 375 query78 10086 10148 94209420 query79 2762895 579 579 query80 728 599 536 536 query81 478 269 241 241 query82 465 151 119 119 query83 313 301 275 275 query84 295 112 98 98 query85 793 409 360 360 query86 361 306 322 306 query87 4495444943594359 query88 2977235823042304 query89 417 331 302 302 query90 1940238 233 233 query91 158 154 132 132 query92 92 84 82 82 query93 1145940 596 596 query94 699 433 319 319 query95 417 343 336 336 query96 499 594 283 283 query97 2712274426332633 query98 243 234 222 222 query99 1480144714141414 Total cold run time: 278185 ms Total hot run time: 189413 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3095274230 TPC-H: Total hot run time: 34417 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 08920acb61c1f6daf1ea4ffe70a0b936608ff5b1, data reload: false -- Round 1 -- q1 17649 528050775077 q2 1950319 198 198 q3 10295 1362736 736 q4 10256 1055534 534 q5 7990249823872387 q6 191 170 144 144 q7 966 769 629 629 q8 9345141911331133 q9 7115518051615161 q10 6939242519701970 q11 518 292 289 289 q12 377 361 226 226 q13 17783 375831403140 q14 251 231 218 218 q15 561 506 507 506 q16 442 425 376 376 q17 595 903 367 367 q18 7732711672127116 q19 1304972 617 617 q20 369 348 232 232 q21 3975323223922392 q22 11051036969 969 Total cold run time: 107708 ms Total hot run time: 34417 ms - Round 2, with runtime_filter_mode=off - q1 5157510350915091 q2 266 335 232 232 q3 2187270222832283 q4 1371180413761376 q5 4499445745344457 q6 233 187 143 143 q7 2003196417741774 q8 2740256729012567 q9 7426726473717264 q10 3136332128802880 q11 626 534 524 524 q12 685 761 632 632 q13 3689410333883388 q14 296 335 307 307 q15 557 509 508 508 q16 462 502 473 473 q17 1186174114251425 q18 8045770476037603 q19 867 858 1086858 q20 2064209422132094 q21 5055442743084308 q22 1130105410231023 Total cold run time: 53680 ms Total hot run time: 51210 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
xinyiZzz commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3095151556 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3094835930 # BE UT Coverage Report Increment line coverage `80.43% (74/92)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/e3fb4ab2c6420dd8dc2141ea402912585af8_e3fb4ab2c6420dd8dc2141ea402912585af8/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/e3fb4ab2c6420dd8dc2141ea402912585af8_e3fb4ab2c6420dd8dc2141ea402912585af8/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.46% (15892/27659) | | Line Coverage | 46.25% (142744/308620) | | Region Coverage | 35.73% (107580/301083) | | Branch Coverage | 38.23% (47445/124114) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3094730639 ClickBench: Total hot run time: 32.77 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit e3fb4ab2c6420dd8dc2141ea402912585af8, data reload: false query1 0.050.040.04 query2 0.080.040.04 query3 0.250.080.08 query4 1.620.110.11 query5 0.430.400.39 query6 1.170.650.68 query7 0.020.020.02 query8 0.050.030.04 query9 0.590.530.52 query10 0.590.580.58 query11 0.160.110.12 query12 0.150.120.12 query13 0.630.620.61 query14 0.840.800.82 query15 0.900.850.89 query16 0.410.380.41 query17 1.081.061.09 query18 0.210.220.21 query19 1.961.861.88 query20 0.010.020.01 query21 15.38 0.940.54 query22 0.771.070.76 query23 14.98 1.380.63 query24 6.841.530.79 query25 0.480.160.09 query26 0.600.170.13 query27 0.060.060.05 query28 9.820.920.44 query29 12.71 3.943.27 query30 3.133.093.05 query31 2.820.580.39 query32 3.230.550.49 query33 3.093.163.04 query34 15.90 5.424.88 query35 4.864.884.87 query36 0.690.500.50 query37 0.100.070.07 query38 0.060.050.04 query39 0.040.030.03 query40 0.190.160.15 query41 0.080.030.03 query42 0.040.030.02 query43 0.050.040.04 Total cold run time: 107.12 s Total hot run time: 32.77 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3094728162 TPC-DS: Total hot run time: 189793 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit e3fb4ab2c6420dd8dc2141ea402912585af8, data reload: false query1 1024386 453 386 query2 6909173517671735 query3 6758230 236 230 query4 26536 23380 23103 23103 query5 4440730 584 584 query6 322 239 219 219 query7 4646531 309 309 query8 377 341 337 337 query9 8637291028682868 query10 524 376 313 313 query11 15576 15073 14732 14732 query12 182 132 131 131 query13 1705575 431 431 query14 9647593059235923 query15 211 193 176 176 query16 7672684 493 493 query17 1571779 582 582 query18 2033437 357 357 query19 202 203 174 174 query20 138 134 136 134 query21 238 163 130 130 query22 4401433543334333 query23 33972 33133 33217 33133 query24 8183239723952395 query25 566 507 425 425 query26 1025290 168 168 query27 2688530 365 365 query28 4336218121542154 query29 662 596 484 484 query30 300 236 201 201 query31 977 887 779 779 query32 97 86 81 81 query33 579 402 367 367 query34 800 898 515 515 query35 797 852 737 737 query36 10011023941 941 query37 144 111 100 100 query38 4072417841364136 query39 1497144114431441 query40 235 144 133 133 query41 109 103 101 101 query42 133 119 124 119 query43 515 522 481 481 query44 1369846 847 846 query45 193 189 180 180 query46 872 1055645 645 query47 1863189818231823 query48 399 455 323 323 query49 761 565 495 495 query50 687 759 429 429 query51 5616551156965511 query52 122 116 109 109 query53 248 290 210 210 query54 603 619 541 541 query55 94 93 92 92 query56 359 350 331 331 query57 1249126511701170 query58 320 309 311 309 query59 2635270625422542 query60 380 366 351 351 query61 150 149 142 142 query62 828 722 671 671 query63 249 222 207 207 query64 36151104770 770 query65 4285416941454145 query66 1148598 523 523 query67 15779 15656 15415 15415 query68 8340914 532 532 query69 498 337 302 302 query70 1272115911121112 query71 486 350 324 324 query72 5351485047634763 query73 746 628 367 367 query74 8952911188188818 query75 3894320728072807 query76 36031209737 737 query77 855 448 407 407 query78 10037 10193 94149414 query79 1805822 590 590 query80 723 599 527 527 query81 477 271 242 242 query82 420 145 117 117 query83 314 278 269 269 query84 301 116 103 103 query85 791 394 356 356 query86 379 338 316 316 query87 4447448342974297 query88 2912231822792279 query89 421 343 302 302 query90 1946242 235 235 query91 148 158 124 124 query92 97 84 78 78 query93 1168945 602 602 query94 695 436 325 325 query95 420 327 332 327 query96 493 649 286 286 query97 2759279526172617 query98 235 223 217 217 query99 1473142512851285 Total cold run time: 276719 ms Total hot run time: 189793 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3094722436 TPC-H: Total hot run time: 34563 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit e3fb4ab2c6420dd8dc2141ea402912585af8, data reload: false -- Round 1 -- q1 17593 520050465046 q2 1942299 202 202 q3 10275 1296691 691 q4 10231 1002534 534 q5 8045247023882388 q6 192 170 137 137 q7 969 768 621 621 q8 9342141611701170 q9 6929516251625162 q10 6892239319751975 q11 496 313 284 284 q12 362 367 228 228 q13 17780 371431333133 q14 252 250 232 232 q15 594 542 525 525 q16 437 446 412 412 q17 587 874 371 371 q18 7600725072977250 q19 1412995 591 591 q20 355 359 242 242 q21 3759316223612361 q22 1075106910081008 Total cold run time: 107119 ms Total hot run time: 34563 ms - Round 2, with runtime_filter_mode=off - q1 5110509251105092 q2 271 354 246 246 q3 2184266223062306 q4 1351178413201320 q5 4353454445124512 q6 216 176 133 133 q7 1990199117831783 q8 2654252925732529 q9 7427742474877424 q10 3168325728482848 q11 573 528 500 500 q12 679 780 651 651 q13 3563411632633263 q14 283 321 290 290 q15 547 491 500 491 q16 456 523 472 472 q17 1194155114001400 q18 7928769077277690 q19 823 827 872 827 q20 2018210320362036 q21 5028455744214421 q22 1100110010131013 Total cold run time: 52916 ms Total hot run time: 51247 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
xinyiZzz commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3094678900 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
xinyiZzz commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3094646739 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
hello-stephen commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3094508906 # BE UT Coverage Report Increment line coverage `80.43% (74/92)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/77bc4cb632154a9334fecb3084330372aa565b33_77bc4cb632154a9334fecb3084330372aa565b33/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/77bc4cb632154a9334fecb3084330372aa565b33_77bc4cb632154a9334fecb3084330372aa565b33/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.44% (15765/27444) | | Line Coverage | 46.17% (141757/307014) | | Region Coverage | 35.67% (106872/299572) | | Branch Coverage | 38.16% (47166/123585) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3094418750 ClickBench: Total hot run time: 33.04 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 77bc4cb632154a9334fecb3084330372aa565b33, data reload: false query1 0.050.040.04 query2 0.080.050.05 query3 0.250.070.08 query4 1.630.110.12 query5 0.450.420.40 query6 1.160.650.68 query7 0.030.030.02 query8 0.050.040.04 query9 0.600.510.53 query10 0.590.580.58 query11 0.160.120.12 query12 0.160.120.12 query13 0.630.620.61 query14 0.800.840.83 query15 0.890.870.86 query16 0.410.400.41 query17 1.091.071.10 query18 0.210.210.20 query19 1.911.891.89 query20 0.020.020.01 query21 15.36 0.900.54 query22 0.791.200.66 query23 14.93 1.420.63 query24 7.271.620.83 query25 0.440.200.19 query26 0.560.180.15 query27 0.070.060.07 query28 9.190.890.45 query29 12.54 3.963.33 query30 3.153.103.06 query31 2.840.590.40 query32 3.260.570.48 query33 3.123.173.11 query34 16.08 5.444.85 query35 4.844.834.86 query36 0.700.510.49 query37 0.100.080.07 query38 0.050.050.05 query39 0.040.030.03 query40 0.190.150.16 query41 0.090.030.04 query42 0.040.040.03 query43 0.040.040.04 Total cold run time: 106.86 s Total hot run time: 33.04 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3094415168 TPC-DS: Total hot run time: 189255 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 77bc4cb632154a9334fecb3084330372aa565b33, data reload: false query1 1013399 416 399 query2 6545172916751675 query3 6750229 221 221 query4 26683 23338 23655 23338 query5 4472702 545 545 query6 323 236 220 220 query7 4639544 314 314 query8 381 339 336 336 query9 8658262826102610 query10 518 356 309 309 query11 15761 15158 14908 14908 query12 177 131 125 125 query13 1689550 445 445 query14 9385599858985898 query15 215 221 178 178 query16 7710689 501 501 query17 1228736 588 588 query18 2032418 318 318 query19 196 199 168 168 query20 137 128 126 126 query21 235 140 126 126 query22 4318417140974097 query23 33940 33042 32951 32951 query24 8449240824372408 query25 532 479 414 414 query26 825 316 177 177 query27 2653504 341 341 query28 4391217721512151 query29 697 578 450 450 query30 297 236 203 203 query31 968 881 791 791 query32 86 75 79 75 query33 589 381 338 338 query34 795 858 515 515 query35 817 830 782 782 query36 999 1006923 923 query37 116 110 83 83 query38 4141412739933993 query39 1597146514381438 query40 233 144 129 129 query41 107 103 101 101 query42 127 118 111 111 query43 515 528 491 491 query44 1305829 819 819 query45 187 184 183 183 query46 861 1020681 681 query47 1823191217951795 query48 393 428 325 325 query49 774 554 449 449 query50 660 716 440 440 query51 5718564655725572 query52 122 119 109 109 query53 239 270 205 205 query54 593 592 533 533 query55 89 86 87 86 query56 324 338 322 322 query57 1225127811621162 query58 312 295 298 295 query59 2549265325742574 query60 385 355 332 332 query61 158 151 172 151 query62 822 735 657 657 query63 241 211 217 211 query64 35041079728 728 query65 4301413041934130 query66 1063593 564 564 query67 16020 15781 15588 15588 query68 7884898 556 556 query69 497 326 293 293 query70 1217113210961096 query71 490 322 318 318 query72 5343494848254825 query73 720 616 367 367 query74 9226890386758675 query75 3888319327702770 query76 36351166723 723 query77 860 437 415 415 query78 996810065 94159415 query79 2257819 603 603 query80 649 570 505 505 query81 486 281 237 237 query82 446 150 113 113 query83 313 298 265 265 query84 296 114 98 98 query85 832 397 362 362 query86 361 337 296 296 query87 4495443244034403 query88 3487230522932293 query89 423 343 305 305 query90 1962227 225 225 query91 203 159 126 126 query92 89 76 69 69 query93 1841891 595 595 query94 688 429 308 308 query95 396 316 311 311 query96 492 579 285 285 query97 2678273926402640 query98 241 227 212 212 query99 1452142713301330 Total cold run time: 277422 ms Total hot run time: 189255 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3094406806 TPC-H: Total hot run time: 35042 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 77bc4cb632154a9334fecb3084330372aa565b33, data reload: false -- Round 1 -- q1 17600 545052805280 q2 1951296 211 211 q3 10275 1418763 763 q4 10238 1074528 528 q5 7547270624762476 q6 215 180 140 140 q7 1007797 636 636 q8 9358151912031203 q9 6946532853785328 q10 6959246019801980 q11 507 307 292 292 q12 366 400 231 231 q13 17775 383230823082 q14 243 243 243 243 q15 613 536 526 526 q16 435 460 391 391 q17 598 938 363 363 q18 7607734471397139 q19 12551118639 639 q20 379 359 237 237 q21 4256324623592359 q22 11051056995 995 Total cold run time: 107235 ms Total hot run time: 35042 ms - Round 2, with runtime_filter_mode=off - q1 5471536754185367 q2 286 366 248 248 q3 2212269522962296 q4 1425188414311431 q5 4554444045404440 q6 303 185 136 136 q7 2112194917591759 q8 2962279027572757 q9 7268723472337233 q10 3247332029412941 q11 637 508 511 508 q12 723 796 618 618 q13 3721409935333533 q14 284 307 285 285 q15 581 530 511 511 q16 479 512 438 438 q17 1245187115091509 q18 7889770575027502 q19 844 848 1004848 q20 2010211321202113 q21 5290511146424642 q22 1157116110921092 Total cold run time: 54700 ms Total hot run time: 52207 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
xinyiZzz commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3094252679 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
xinyiZzz commented on code in PR #52553: URL: https://github.com/apache/doris/pull/52553#discussion_r2215126784 ## be/src/vec/common/pod_array.h: ## @@ -228,12 +261,6 @@ class PODArrayBase : private boost::noncopyable, ptrdiff_t end_diff = c_end - c_start; ptrdiff_t res_mem_diff = c_res_mem - c_start; -// Realloc can do 2 possible things: -// - expand existing memory region -// - allocate new memory block and free the old one -// Because we don't know which option will be picked we need to make sure there is enough -// memory for all options. -check_memory(res_mem_diff); Review Comment: done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
xinyiZzz commented on code in PR #52553:
URL: https://github.com/apache/doris/pull/52553#discussion_r2215104679
##
be/src/vec/common/pod_array.h:
##
@@ -265,6 +292,14 @@ class PODArrayBase : private boost::noncopyable,
realloc(allocated_bytes() * 2,
std::forward(allocator_params)...);
}
+/// Virtual memory allocated from Allocator, usually only used by PODArray
itself.
+size_t capacity_bytes() const {
Review Comment:
这里 protected 就是只有 PodArrayBase 及其子类可以看到了
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
yiguolei commented on code in PR #52553: URL: https://github.com/apache/doris/pull/52553#discussion_r2215094023 ## be/src/vec/common/pod_array.h: ## @@ -228,12 +261,6 @@ class PODArrayBase : private boost::noncopyable, ptrdiff_t end_diff = c_end - c_start; ptrdiff_t res_mem_diff = c_res_mem - c_start; -// Realloc can do 2 possible things: -// - expand existing memory region -// - allocate new memory block and free the old one -// Because we don't know which option will be picked we need to make sure there is enough -// memory for all options. -check_memory(res_mem_diff); Review Comment: 这里为什么不需要check了？ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
yiguolei commented on code in PR #52553:
URL: https://github.com/apache/doris/pull/52553#discussion_r2215089294
##
be/src/vec/common/pod_array.h:
##
@@ -265,6 +292,14 @@ class PODArrayBase : private boost::noncopyable,
realloc(allocated_bytes() * 2,
std::forward(allocator_params)...);
}
+/// Virtual memory allocated from Allocator, usually only used by PODArray
itself.
+size_t capacity_bytes() const {
Review Comment:
这个方法得是一个private的，不要让外部看到
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
yiguolei commented on code in PR #52553:
URL: https://github.com/apache/doris/pull/52553#discussion_r2215086929
##
be/src/vec/common/pod_array.h:
##
@@ -441,10 +480,7 @@ class PODArray : public PODArrayBasec_end; }
-void set_end_ptr(void* ptr) {
-this->c_end = (char*)ptr;
-this->reset_resident_memory();
Review Comment:
这里为什么不改res memory了？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
xinyiZzz commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3084412499 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
xinyiZzz commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3084405165 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
hello-stephen commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3077673660 # BE Regression && UT Coverage Report Increment line coverage `76.92% (50/65)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/52553_5ddf94122c066ff0748e472c85bc5148c168a956_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/52553_5ddf94122c066ff0748e472c85bc5148c168a956_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 81.18% (21791/26843) | | Line Coverage | 73.75% (223884/303562) | | Region Coverage | 61.42% (186157/303064) | | Branch Coverage | 65.24% (80001/122624) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
hello-stephen commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3077605413 # BE Regression && UT Coverage Report Increment line coverage `76.92% (50/65)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/52553_5ddf94122c066ff0748e472c85bc5148c168a956_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/52553_5ddf94122c066ff0748e472c85bc5148c168a956_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 81.18% (21791/26843) | | Line Coverage | 73.75% (223884/303562) | | Region Coverage | 61.42% (186157/303064) | | Branch Coverage | 65.24% (80001/122624) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
hello-stephen commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3077104751 # BE Regression && UT Coverage Report Increment line coverage ` ` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/52553_5ddf94122c066ff0748e472c85bc5148c168a956_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/52553_5ddf94122c066ff0748e472c85bc5148c168a956_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage || | Line Coverage || | Region Coverage || | Branch Coverage || -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
hello-stephen commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3076780028 # BE Regression && UT Coverage Report Increment line coverage ` ` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/52553_5ddf94122c066ff0748e472c85bc5148c168a956_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/52553_5ddf94122c066ff0748e472c85bc5148c168a956_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage || | Line Coverage || | Region Coverage || | Branch Coverage || -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
hello-stephen commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3076779464 # BE Regression && UT Coverage Report Increment line coverage ` ` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/52553_5ddf94122c066ff0748e472c85bc5148c168a956_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/52553_5ddf94122c066ff0748e472c85bc5148c168a956_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage || | Line Coverage || | Region Coverage || | Branch Coverage || -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
hello-stephen commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3076774866 # BE Regression && UT Coverage Report Increment line coverage ` ` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/52553_5ddf94122c066ff0748e472c85bc5148c168a956_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/52553_5ddf94122c066ff0748e472c85bc5148c168a956_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage || | Line Coverage || | Region Coverage || | Branch Coverage || -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
hello-stephen commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3076757239 # BE Regression && UT Coverage Report Increment line coverage ` ` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/52553_5ddf94122c066ff0748e472c85bc5148c168a956_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/52553_5ddf94122c066ff0748e472c85bc5148c168a956_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage || | Line Coverage || | Region Coverage || | Branch Coverage || -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
hello-stephen commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3076636982 # BE Regression && UT Coverage Report Increment line coverage ` ` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/52553_5ddf94122c066ff0748e472c85bc5148c168a956_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/52553_5ddf94122c066ff0748e472c85bc5148c168a956_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage || | Line Coverage || | Region Coverage || | Branch Coverage || -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
hello-stephen commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3073996294 # BE Regression && UT Coverage Report Increment line coverage ` ` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/52553_5ddf94122c066ff0748e472c85bc5148c168a956_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/52553_5ddf94122c066ff0748e472c85bc5148c168a956_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage || | Line Coverage || | Region Coverage || | Branch Coverage || -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3073919733 ClickBench: Total hot run time: 32.47 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 5ddf94122c066ff0748e472c85bc5148c168a956, data reload: false query1 0.040.030.03 query2 0.080.040.04 query3 0.240.080.07 query4 1.630.110.11 query5 0.430.420.43 query6 1.170.670.66 query7 0.030.020.02 query8 0.050.040.04 query9 0.610.520.52 query10 0.570.580.56 query11 0.170.110.11 query12 0.140.110.12 query13 0.630.610.61 query14 0.810.820.83 query15 0.890.870.88 query16 0.390.390.38 query17 1.041.071.04 query18 0.210.190.20 query19 1.961.811.91 query20 0.010.010.01 query21 15.40 0.890.55 query22 0.751.200.65 query23 14.96 1.410.65 query24 6.671.360.81 query25 0.510.220.09 query26 0.670.170.15 query27 0.070.050.06 query28 9.030.880.45 query29 12.55 3.953.24 query30 3.022.972.94 query31 2.820.580.38 query32 3.230.560.47 query33 3.023.063.12 query34 16.04 5.404.85 query35 4.904.914.89 query36 0.700.500.49 query37 0.090.060.06 query38 0.050.050.04 query39 0.040.020.02 query40 0.170.140.15 query41 0.090.020.02 query42 0.030.020.02 query43 0.040.030.03 Total cold run time: 105.95 s Total hot run time: 32.47 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3073899403 TPC-DS: Total hot run time: 184049 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 5ddf94122c066ff0748e472c85bc5148c168a956, data reload: false query1 975 375 378 375 query2 6500176517041704 query3 6741212 215 212 query4 26749 23713 23176 23176 query5 4368555 438 438 query6 314 209 208 208 query7 4614480 290 290 query8 258 216 211 211 query9 8631250725152507 query10 448 326 270 270 query11 15731 14981 14745 14745 query12 147 100 108 100 query13 1664505 373 373 query14 8878607360476047 query15 205 190 171 171 query16 7194436 264 264 query17 1320725 587 587 query18 1978413 305 305 query19 196 195 165 165 query20 124 122 117 117 query21 215 128 110 110 query22 4166434840584058 query23 33847 32942 32907 32907 query24 8483232823522328 query25 563 526 382 382 query26 1225264 147 147 query27 2785502 324 324 query28 4300207420602060 query29 764 555 433 433 query30 286 216 196 196 query31 922 848 753 753 query32 75 62 57 57 query33 557 384 267 267 query34 774 832 494 494 query35 601 627 547 547 query36 957 1013908 908 query37 112 98 80 80 query38 4104418440194019 query39 1468141114191411 query40 213 120 103 103 query41 56 56 53 53 query42 122 103 103 103 query43 504 512 491 491 query44 1292790 798 790 query45 174 167 176 167 query46 834 1055608 608 query47 1776184517231723 query48 370 395 293 293 query49 734 444 378 378 query50 626 686 424 424 query51 5548556654495449 query52 107 102 93 93 query53 219 254 177 177 query54 571 554 485 485 query55 80 80 81 80 query56 281 301 286 286 query57 1204120411411141 query58 252 255 259 255 query59 2568267826112611 query60 327 320 290 290 query61 126 118 121 118 query62 790 694 645 645 query63 219 186 181 181 query64 44471205832 832 query65 4278415941984159 query66 1151401 321 321 query67 16053 15595 15563 15563 query68 8475864 519 519 query69 503 308 258 258 query70 1245114711321132 query71 458 321 289 289 query72 5753242651872426 query73 785 753 347 347 query74 9190894786728672 query75 3791317126742674 query76 36101134708 708 query77 788 374 285 285 query78 10908 10971 10206 10206 query79 2120824 571 571 query80 582 513 436 436 query81 470 252 223 223 query82 474 129 95 95 query83 255 255 249 249 query84 251 109 98 98 query85 803 370 357 357 query86 383 315 287 287 query87 4350433542864286 query88 3636224922292229 query89 382 317 287 287 query90 1871225 214 214 query91 140 147 109 109 query92 78 61 58 58 query93 1592924 592 592 query94 679 316 199 199 query95 383 296 291 291 query96 493 558 269 269 query97 2703271726082608 query98 233 204 207 204 query99 1679141313061306 Total cold run time: 277224 ms Total hot run time: 184049 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3073894450 # BE UT Coverage Report Increment line coverage `76.92% (50/65)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/5ddf94122c066ff0748e472c85bc5148c168a956_5ddf94122c066ff0748e472c85bc5148c168a956/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/5ddf94122c066ff0748e472c85bc5148c168a956_5ddf94122c066ff0748e472c85bc5148c168a956/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.53% (15744/27367) | | Line Coverage | 46.16% (140420/304170) | | Region Coverage | 35.47% (104793/295406) | | Branch Coverage | 38.06% (46159/121280) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3073853717 TPC-H: Total hot run time: 33320 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 5ddf94122c066ff0748e472c85bc5148c168a956, data reload: false -- Round 1 -- q1 17561 525051115111 q2 1951276 181 181 q3 10302 1285727 727 q4 10226 994 525 525 q5 7533239323482348 q6 178 160 127 127 q7 892 744 591 591 q8 9317129510281028 q9 7478514352005143 q10 6902238319611961 q11 478 305 281 281 q12 342 344 210 210 q13 17793 366430823082 q14 232 232 220 220 q15 545 487 486 486 q16 424 417 381 381 q17 598 891 380 380 q18 7589715171187118 q19 1217951 570 570 q20 353 351 230 230 q21 4009315123312331 q22 354 310 289 289 Total cold run time: 106274 ms Total hot run time: 33320 ms - Round 2, with runtime_filter_mode=off - q1 5552515651375137 q2 240 326 222 222 q3 2151268222832283 q4 1387180213341334 q5 4222445944894459 q6 219 171 137 137 q7 2035194818531853 q8 2664260726362607 q9 7389728673577286 q10 3144343628372837 q11 607 525 537 525 q12 666 765 646 646 q13 3670419833473347 q14 281 318 270 270 q15 515 472 476 472 q16 462 498 455 455 q17 1183158613791379 q18 8209759077837590 q19 774 962 1125962 q20 2072207119341934 q21 4849446243764376 q22 593 591 558 558 Total cold run time: 52884 ms Total hot run time: 50669 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
xinyiZzz commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3073245446 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3071997681 # BE UT Coverage Report Increment line coverage `76.92% (50/65)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/084ea2e3ff6167c0238bc06e1df21eaae3fa1157_084ea2e3ff6167c0238bc06e1df21eaae3fa1157/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/084ea2e3ff6167c0238bc06e1df21eaae3fa1157_084ea2e3ff6167c0238bc06e1df21eaae3fa1157/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.53% (15740/27362) | | Line Coverage | 46.20% (140370/303834) | | Region Coverage | 35.48% (104747/295197) | | Branch Coverage | 38.08% (46119/12) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3071811989 ClickBench: Total hot run time: 32.85 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 084ea2e3ff6167c0238bc06e1df21eaae3fa1157, data reload: false query1 0.050.030.03 query2 0.070.040.04 query3 0.240.070.07 query4 1.650.110.12 query5 0.430.440.40 query6 1.160.680.66 query7 0.030.020.02 query8 0.040.030.03 query9 0.600.520.51 query10 0.570.590.57 query11 0.160.110.11 query12 0.150.120.11 query13 0.630.610.61 query14 0.820.820.84 query15 0.880.890.86 query16 0.390.390.39 query17 1.051.021.05 query18 0.200.190.20 query19 1.951.831.84 query20 0.020.010.02 query21 15.39 0.890.55 query22 0.761.090.93 query23 14.73 1.370.63 query24 7.051.070.89 query25 0.490.300.17 query26 0.600.170.15 query27 0.070.050.05 query28 10.02 0.890.46 query29 12.53 4.003.37 query30 3.053.032.89 query31 2.810.610.38 query32 3.230.560.48 query33 3.223.093.21 query34 16.10 5.384.76 query35 4.864.834.86 query36 0.700.520.50 query37 0.090.070.07 query38 0.050.040.04 query39 0.030.020.03 query40 0.190.140.14 query41 0.080.020.03 query42 0.030.030.02 query43 0.040.030.02 Total cold run time: 107.21 s Total hot run time: 32.85 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3071805016 TPC-DS: Total hot run time: 186104 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 084ea2e3ff6167c0238bc06e1df21eaae3fa1157, data reload: false query1 1003393 396 393 query2 6541177117771771 query3 6754215 220 215 query4 26707 23362 23063 23063 query5 4379582 434 434 query6 307 222 195 195 query7 4632486 294 294 query8 257 236 209 209 query9 8622256425762564 query10 484 324 280 280 query11 15614 15130 14872 14872 query12 164 105 108 105 query13 1643519 393 393 query14 8941591459585914 query15 212 195 175 175 query16 7336651 453 453 query17 1202747 594 594 query18 1992407 307 307 query19 202 185 158 158 query20 131 120 112 112 query21 211 126 106 106 query22 4147413341134113 query23 34188 33076 33008 33008 query24 8443237223822372 query25 532 518 375 375 query26 1219276 146 146 query27 2745504 325 325 query28 4337207620422042 query29 762 550 432 432 query30 281 222 187 187 query31 906 842 764 764 query32 77 62 61 61 query33 582 337 324 324 query34 805 836 522 522 query35 782 843 748 748 query36 962 987 895 895 query37 110 101 73 73 query38 4167420440394039 query39 1546139714211397 query40 203 117 105 105 query41 58 55 62 55 query42 120 103 126 103 query43 518 498 467 467 query44 1346797 802 797 query45 177 184 173 173 query46 843 1010612 612 query47 1764183017371737 query48 375 433 289 289 query49 729 460 395 395 query50 640 684 429 429 query51 5461551655465516 query52 106 106 93 93 query53 219 257 179 179 query54 584 579 494 494 query55 83 82 81 81 query56 297 287 271 271 query57 1170118611151115 query58 253 244 255 244 query59 2693278826482648 query60 322 313 306 306 query61 127 121 127 121 query62 777 719 668 668 query63 224 192 188 188 query64 4338993 674 674 query65 4272417342134173 query66 1144413 306 306 query67 15728 15612 15433 15433 query68 8110872 523 523 query69 533 317 267 267 query70 1222115111101110 query71 456 309 297 297 query72 5587469848544698 query73 717 626 340 340 query74 9097877287028702 query75 3795317726572657 query76 36551176720 720 query77 791 388 286 286 query78 10134 10227 92559255 query79 2181885 588 588 query80 594 498 452 452 query81 476 322 229 229 query82 445 125 99 99 query83 257 249 234 234 query84 248 113 89 89 query85 822 359 315 315 query86 390 335 289 289 query87 4412440943474347 query88 3482221522032203 query89 383 324 286 286 query90 1893212 216 212 query91 142 143 109 109 query92 84 63 56 56 query93 1596965 580 580 query94 664 409 314 314 query95 374 296 288 288 query96 487 573 270 270 query97 2701277026862686 query98 233 202 200 200 query99 1468138913011301 Total cold run time: 275925 ms Total hot run time: 186104 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3071789773 TPC-H: Total hot run time: 33819 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 084ea2e3ff6167c0238bc06e1df21eaae3fa1157, data reload: false -- Round 1 -- q1 17580 530553305305 q2 1973277 181 181 q3 10320 1289743 743 q4 10223 1020524 524 q5 7604244723702370 q6 185 160 129 129 q7 885 767 622 622 q8 9348137011411141 q9 7364507651325076 q10 6892239919801980 q11 485 284 271 271 q12 346 351 225 225 q13 17761 367531283128 q14 225 229 224 224 q15 552 488 491 488 q16 431 420 374 374 q17 624 895 367 367 q18 7589713972567139 q19 1218945 588 588 q20 339 347 230 230 q21 4007323824202420 q22 357 322 294 294 Total cold run time: 106308 ms Total hot run time: 33819 ms - Round 2, with runtime_filter_mode=off - q1 5206515651675156 q2 243 329 221 221 q3 2202270822662266 q4 1448176613771377 q5 4245449744684468 q6 241 171 127 127 q7 2064194018651865 q8 2683260426172604 q9 7269730073427300 q10 3067326929142914 q11 617 517 499 499 q12 684 771 637 637 q13 3604401834613461 q14 436 296 267 267 q15 523 472 476 472 q16 454 475 447 447 q17 1209150614421442 q18 8050771677557716 q19 806 899 1027899 q20 2017204217851785 q21 4801436243384338 q22 648 578 532 532 Total cold run time: 52517 ms Total hot run time: 50793 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
xinyiZzz commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3071659512 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
hello-stephen commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3069943864 # BE UT Coverage Report Increment line coverage `76.92% (50/65)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/d4ba0ed9de8b32d453bf69e444a2d7cfe42a2ce5_d4ba0ed9de8b32d453bf69e444a2d7cfe42a2ce5/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/d4ba0ed9de8b32d453bf69e444a2d7cfe42a2ce5_d4ba0ed9de8b32d453bf69e444a2d7cfe42a2ce5/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.66% (15696/27220) | | Line Coverage | 46.34% (140016/302119) | | Region Coverage | 35.59% (104714/294264) | | Branch Coverage | 38.16% (46118/120850) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3069665398 ClickBench: Total hot run time: 29.33 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit d4ba0ed9de8b32d453bf69e444a2d7cfe42a2ce5, data reload: false query1 0.040.040.03 query2 0.080.040.05 query3 0.250.080.08 query4 1.600.100.11 query5 0.420.440.41 query6 1.170.660.65 query7 0.030.020.02 query8 0.050.030.04 query9 0.590.510.51 query10 0.580.570.57 query11 0.160.110.11 query12 0.140.120.12 query13 0.620.630.61 query14 0.790.820.82 query15 0.910.890.85 query16 0.380.380.38 query17 1.051.061.01 query18 0.220.200.19 query19 1.971.781.86 query20 0.010.010.02 query21 15.40 0.890.56 query22 0.731.150.62 query23 15.08 1.400.60 query24 6.501.580.79 query25 0.480.210.12 query26 0.640.160.14 query27 0.070.060.05 query28 9.450.870.43 query29 12.63 3.923.26 query30 0.240.100.07 query31 2.840.590.38 query32 3.220.570.47 query33 3.113.063.18 query34 16.11 5.404.80 query35 4.864.854.82 query36 0.680.500.50 query37 0.090.070.07 query38 0.050.040.04 query39 0.030.020.02 query40 0.170.140.13 query41 0.070.030.03 query42 0.030.020.03 query43 0.030.030.03 Total cold run time: 103.57 s Total hot run time: 29.33 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3069648895 TPC-DS: Total hot run time: 186616 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit d4ba0ed9de8b32d453bf69e444a2d7cfe42a2ce5, data reload: false query1 984 375 371 371 query2 6491161416191614 query3 6754209 217 209 query4 26385 23646 23302 23302 query5 4339568 436 436 query6 317 211 213 211 query7 4634492 284 284 query8 259 220 233 220 query9 8623259625782578 query10 490 335 287 287 query11 15212 15217 14867 14867 query12 157 107 103 103 query13 1652522 406 406 query14 8680581359155813 query15 216 196 182 182 query16 7501611 486 486 query17 1210739 607 607 query18 1997411 303 303 query19 192 192 161 161 query20 118 117 116 116 query21 213 181 106 106 query22 4276433447674334 query23 34926 33597 33173 33173 query24 8484237823952378 query25 527 441 382 382 query26 1232258 144 144 query27 2752500 329 329 query28 4382207020882070 query29 740 539 430 430 query30 276 215 184 184 query31 900 822 745 745 query32 68 61 58 58 query33 541 351 298 298 query34 784 838 508 508 query35 774 800 769 769 query36 956 1001908 908 query37 109 91 75 75 query38 4147408140734073 query39 1499141914071407 query40 201 119 101 101 query41 60 57 50 50 query42 122 105 105 105 query43 488 511 462 462 query44 1276790 801 790 query45 178 164 162 162 query46 871 1001619 619 query47 1734180217011701 query48 374 414 308 308 query49 739 478 381 381 query50 654 665 405 405 query51 5500556154525452 query52 110 104 93 93 query53 223 260 179 179 query54 572 567 496 496 query55 80 80 77 77 query56 312 278 278 278 query57 1179120311051105 query58 261 239 250 239 query59 2573264425152515 query60 318 312 297 297 query61 120 119 116 116 query62 790 732 657 657 query63 222 182 191 182 query64 4295976 639 639 query65 4274421241524152 query66 1100399 294 294 query67 15714 15812 15501 15501 query68 8709872 526 526 query69 477 319 271 271 query70 1249109711191097 query71 459 320 303 303 query72 5798476448024764 query73 718 636 346 346 query74 8835904789048904 query75 4106314326732673 query76 36711164741 741 query77 781 389 300 300 query78 998510147 92359235 query79 2187850 569 569 query80 625 498 454 454 query81 475 254 224 224 query82 463 125 97 97 query83 285 262 235 235 query84 296 106 86 86 query85 786 348 317 317 query86 346 310 318 310 query87 4357447644244424 query88 3450220522122205 query89 381 314 298 298 query90 1931213 207 207 query91 206 139 106 106 query92 69 58 55 55 query93 1674956 583 583 query94 668 407 299 299 query95 384 299 289 289 query96 493 586 272 272 query97 2738268326312631 query98 231 208 198 198 query99 1424141013141314 Total cold run time: 276394 ms Total hot run time: 186616 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3069612797 TPC-H: Total hot run time: 33403 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit d4ba0ed9de8b32d453bf69e444a2d7cfe42a2ce5, data reload: false -- Round 1 -- q1 17567 528550755075 q2 1928290 208 208 q3 10284 1310748 748 q4 10217 1027509 509 q5 7564242822922292 q6 178 161 128 128 q7 939 740 607 607 q8 9319130511561156 q9 6775522650935093 q10 6900236319461946 q11 466 274 269 269 q12 339 351 206 206 q13 17768 368230963096 q14 223 233 217 217 q15 540 479 466 466 q16 423 425 387 387 q17 589 884 365 365 q18 7627721272097209 q19 1231956 559 559 q20 334 338 211 211 q21 3780253923762376 q22 351 316 280 280 Total cold run time: 105342 ms Total hot run time: 33403 ms - Round 2, with runtime_filter_mode=off - q1 5094506951175069 q2 253 332 221 221 q3 2200272222892289 q4 1404179013291329 q5 4262414346274143 q6 220 172 128 128 q7 2052197318171817 q8 2747267825442544 q9 7439723971907190 q10 3082327928362836 q11 582 513 509 509 q12 713 783 615 615 q13 3569400234123412 q14 294 316 277 277 q15 506 495 470 470 q16 459 487 438 438 q17 1210159713851385 q18 8100767875617561 q19 823 864 892 864 q20 2055206119111911 q21 4973473945204520 q22 680 642 599 599 Total cold run time: 52717 ms Total hot run time: 50127 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
xinyiZzz commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3069199399 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3061897882 TPC-H: Total hot run time: 42597 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 0e4882b2a980c6875273542079da28020692c1d0, data reload: false -- Round 1 -- q1 17582 519750625062 q2 1917314 215 215 q3 10259 1370755 755 q4 10263 1008546 546 q5 7894239923942394 q6 178 155 129 129 q7 931 764 642 642 q8 9346141311031103 q9 7138538653795379 q10 7119255121602160 q11 481 299 268 268 q12 344 351 223 223 q13 17807 384733033303 q14 265 266 245 245 q15 548 479 475 475 q16 439 447 408 408 q17 605 874 359 359 q18 8230790778957895 q19 1366976 581 581 q20 346 351 234 234 q21 10303 10041 99109910 q22 381 350 311 311 Total cold run time: 113742 ms Total hot run time: 42597 ms - Round 2, with runtime_filter_mode=off - q1 5164521955605219 q2 426 459 416 416 q3 2571306326072607 q4 2296238518791879 q5 4931490048894889 q6 205 157 123 123 q7 5121498550024985 q8 4589462145304530 q9 7809763175227522 q10 3242342129692969 q11 1411140514361405 q12 10411122994 994 q13 3494385033273327 q14 301 311 274 274 q15 513 467 470 467 q16 664 695 654 654 q17 2009216820932093 q18 7963789278437843 q19 829 874 1014874 q20 2338244522332233 q21 12019 11798 11866 11798 q22 594 588 524 524 Total cold run time: 69530 ms Total hot run time: 67625 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
xinyiZzz commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3061531026 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3027795636 ClickBench: Total hot run time: 29.28 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 5b829932f1e4e56e8c85becd42127805f244f18c, data reload: false query1 0.040.040.04 query2 0.070.040.04 query3 0.240.080.08 query4 1.620.110.11 query5 0.430.420.42 query6 1.150.650.65 query7 0.030.020.02 query8 0.050.040.04 query9 0.600.530.53 query10 0.570.570.56 query11 0.150.110.11 query12 0.150.120.12 query13 0.620.600.61 query14 0.800.840.84 query15 0.930.870.90 query16 0.420.400.39 query17 1.061.041.08 query18 0.220.210.21 query19 2.021.922.02 query20 0.020.010.01 query21 15.35 0.920.56 query22 0.771.130.62 query23 15.03 1.390.62 query24 6.852.160.44 query25 0.410.210.06 query26 0.410.170.15 query27 0.070.060.05 query28 9.920.900.46 query29 12.59 3.863.32 query30 0.260.090.06 query31 2.840.590.38 query32 3.230.570.47 query33 3.033.133.17 query34 16.01 5.384.77 query35 4.834.784.83 query36 0.710.500.49 query37 0.080.060.06 query38 0.050.040.03 query39 0.030.020.03 query40 0.180.150.13 query41 0.070.040.03 query42 0.030.020.02 query43 0.040.030.03 Total cold run time: 103.98 s Total hot run time: 29.28 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3027753802 TPC-DS: Total hot run time: 185452 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 5b829932f1e4e56e8c85becd42127805f244f18c, data reload: false query1 1018385 384 384 query2 6549175417091709 query3 6738209 209 209 query4 25936 23836 23641 23641 query5 4322590 453 453 query6 288 237 199 199 query7 4624492 290 290 query8 275 224 211 211 query9 8658261826172617 query10 450 335 271 271 query11 15322 15046 14896 14896 query12 156 108 102 102 query13 1655532 428 428 query14 8559569457295694 query15 213 190 183 183 query16 7523634 473 473 query17 1207738 582 582 query18 2015420 315 315 query19 196 198 168 168 query20 120 124 116 116 query21 216 125 108 108 query22 4239430340524052 query23 34616 33039 33089 33039 query24 8473237224082372 query25 506 484 382 382 query26 799 273 151 151 query27 2745506 332 332 query28 4287215621302130 query29 666 559 421 421 query30 280 217 203 203 query31 908 826 782 782 query32 71 65 60 60 query33 542 353 316 316 query34 795 839 533 533 query35 787 796 721 721 query36 945 988 887 887 query37 102 102 73 73 query38 4135411541484115 query39 1480143514181418 query40 208 118 111 111 query41 54 55 50 50 query42 131 105 105 105 query43 509 515 486 486 query44 1295817 815 815 query45 193 172 167 167 query46 834 1000641 641 query47 1754181617071707 query48 387 405 303 303 query49 695 477 377 377 query50 618 685 391 391 query51 4131413940954095 query52 121 110 100 100 query53 223 254 184 184 query54 567 567 529 529 query55 84 77 83 77 query56 305 293 289 289 query57 1164117811141114 query58 276 250 253 250 query59 2682276525682568 query60 325 315 308 308 query61 121 119 119 119 query62 819 732 658 658 query63 220 193 189 189 query64 34041021724 724 query65 4260418042044180 query66 969 399 325 325 query67 15577 15941 15308 15308 query68 8144896 529 529 query69 494 300 267 267 query70 1217109710681068 query71 454 325 288 288 query72 5605468248204682 query73 727 639 351 351 query74 9276916289018901 query75 3856317726972697 query76 36031157712 712 query77 778 376 290 290 query78 10029 10223 94759475 query79 1274889 579 579 query80 710 514 438 438 query81 485 255 219 219 query82 335 123 97 97 query83 272 248 232 232 query84 292 182 88 88 query85 738 426 309 309 query86 336 293 291 291 query87 4399465042684268 query88 2812222522402225 query89 369 315 286 286 query90 2067206 201 201 query91 135 135 108 108 query92 78 57 57 57 query93 1070955 596 596 query94 677 404 305 305 query95 381 287 282 282 query96 483 574 275 275 query97 2738270226562656 query98 226 205 196 196 query99 1430136712781278 Total cold run time: 269832 ms Total hot run time: 185452 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3027714718 TPC-H: Total hot run time: 33616 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 5b829932f1e4e56e8c85becd42127805f244f18c, data reload: false -- Round 1 -- q1 17593 518250065006 q2 1920291 198 198 q3 10296 1270714 714 q4 10211 1036510 510 q5 7680243223252325 q6 177 154 126 126 q7 886 728 596 596 q8 9304135810601060 q9 6826510450775077 q10 6882237819561956 q11 495 283 271 271 q12 347 343 202 202 q13 17787 369231203120 q14 223 238 205 205 q15 542 479 480 479 q16 419 418 386 386 q17 585 868 347 347 q18 7817732770407040 q19 1503947 534 534 q20 323 343 211 211 q21 4003311022922292 q22 1007980 961 961 Total cold run time: 106826 ms Total hot run time: 33616 ms - Round 2, with runtime_filter_mode=off - q1 5143508050835080 q2 247 341 224 224 q3 2165267323142314 q4 1391180213061306 q5 4206457045514551 q6 218 170 126 126 q7 2005197117871787 q8 2704258225462546 q9 7345731072577257 q10 3139331328972897 q11 585 557 488 488 q12 688 770 645 645 q13 3799385732743274 q14 301 321 265 265 q15 536 477 473 473 q16 449 566 575 566 q17 1179153914211421 q18 7949762977807629 q19 775 791 855 791 q20 1985200818321832 q21 4771430843454308 q22 10621020976 976 Total cold run time: 52642 ms Total hot run time: 50756 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
xinyiZzz commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3027408028 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
hello-stephen commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3025348453 # BE UT Coverage Report Increment line coverage `100.00% (6/6)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/135bc28086803adb1d343231705d2ce3b3aeb58d_135bc28086803adb1d343231705d2ce3b3aeb58d/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/135bc28086803adb1d343231705d2ce3b3aeb58d_135bc28086803adb1d343231705d2ce3b3aeb58d/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.07% (15359/26912) | | Line Coverage | 46.14% (139455/302257) | | Region Coverage | 45.44% (70671/155531) | | Branch Coverage | 40.21% (37316/92804) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3025168847 TPC-DS: Total hot run time: 184665 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 135bc28086803adb1d343231705d2ce3b3aeb58d, data reload: false query1 1023390 386 386 query2 6559169116291629 query3 6741212 208 208 query4 26758 23335 23036 23036 query5 4319560 454 454 query6 320 210 214 210 query7 4622495 282 282 query8 273 219 225 219 query9 8622261126252611 query10 484 338 271 271 query11 15747 15102 14955 14955 query12 169 115 102 102 query13 1651528 394 394 query14 9530565955875587 query15 203 195 170 170 query16 7663654 490 490 query17 1209728 592 592 query18 2035407 324 324 query19 205 190 166 166 query20 120 122 121 121 query21 214 118 104 104 query22 4162413140654065 query23 34097 32995 32985 32985 query24 8359239623412341 query25 520 455 384 384 query26 1228295 146 146 query27 2725503 338 338 query28 4296212521122112 query29 738 570 419 419 query30 278 221 191 191 query31 936 820 755 755 query32 71 62 61 61 query33 543 350 315 315 query34 779 827 516 516 query35 779 813 740 740 query36 933 991 892 892 query37 110 98 76 76 query38 4264416440424042 query39 1494142214091409 query40 218 115 100 100 query41 57 71 49 49 query42 133 111 106 106 query43 477 515 472 472 query44 1303827 812 812 query45 176 169 161 161 query46 830 1006639 639 query47 1763181317161716 query48 390 415 295 295 query49 731 456 383 383 query50 625 689 408 408 query51 4070410541834105 query52 135 108 95 95 query53 227 245 191 191 query54 570 557 493 493 query55 80 78 79 78 query56 289 297 311 297 query57 1187118311291129 query58 275 269 257 257 query59 2564266125152515 query60 312 313 304 304 query61 120 118 118 118 query62 810 704 640 640 query63 215 181 192 181 query64 4310995 642 642 query65 4281415441824154 query66 1043420 307 307 query67 15801 15667 15898 15667 query68 8042899 513 513 query69 463 302 271 271 query70 1258110010381038 query71 453 307 300 300 query72 5657488149044881 query73 760 644 350 350 query74 9195910988488848 query75 3842317726922692 query76 37461168716 716 query77 787 372 272 272 query78 10114 10169 92409240 query79 2831830 583 583 query80 675 518 458 458 query81 472 257 224 224 query82 452 126 92 92 query83 280 253 239 239 query84 286 98 91 91 query85 799 361 363 361 query86 333 329 288 288 query87 4455443542704270 query88 3310226323132263 query89 410 323 288 288 query90 1976200 205 200 query91 154 145 114 114 query92 81 62 58 58 query93 1960918 568 568 query94 677 427 312 312 query95 377 293 281 281 query96 501 551 285 285 query97 2764279126932693 query98 223 219 219 219 query99 1443139412451245 Total cold run time: 276547 ms Total hot run time: 184665 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3025186680 ClickBench: Total hot run time: 29.96 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 135bc28086803adb1d343231705d2ce3b3aeb58d, data reload: false query1 0.050.040.04 query2 0.070.040.05 query3 0.240.080.09 query4 1.600.110.11 query5 0.450.440.41 query6 1.170.660.65 query7 0.020.020.02 query8 0.040.040.03 query9 0.610.520.51 query10 0.560.580.57 query11 0.160.120.11 query12 0.150.120.11 query13 0.630.610.61 query14 0.800.810.81 query15 0.910.910.88 query16 0.390.380.40 query17 1.091.091.05 query18 0.230.220.21 query19 1.941.911.89 query20 0.020.010.02 query21 15.38 0.920.55 query22 0.761.280.72 query23 14.81 1.390.64 query24 7.141.010.86 query25 0.490.150.13 query26 0.570.180.13 query27 0.060.040.05 query28 9.470.920.44 query29 12.55 4.073.41 query30 0.260.090.07 query31 2.860.600.39 query32 3.230.560.48 query33 3.123.083.12 query34 16.12 5.394.82 query35 4.794.884.86 query36 0.690.500.49 query37 0.090.070.07 query38 0.060.040.04 query39 0.030.030.02 query40 0.170.140.13 query41 0.080.030.02 query42 0.030.030.02 query43 0.030.030.03 Total cold run time: 103.92 s Total hot run time: 29.96 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
doris-robot commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3025136417 TPC-H: Total hot run time: 33872 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 135bc28086803adb1d343231705d2ce3b3aeb58d, data reload: false -- Round 1 -- q1 17612 519750555055 q2 1943287 183 183 q3 10363 1305725 725 q4 10231 1025511 511 q5 7574244023512351 q6 183 163 132 132 q7 909 735 596 596 q8 9317128911581158 q9 7554511751775117 q10 6902237719661966 q11 477 288 266 266 q12 343 379 210 210 q13 17771 370631043104 q14 220 227 215 215 q15 538 487 484 484 q16 432 419 374 374 q17 604 879 364 364 q18 7643719770027002 q19 1417957 540 540 q20 326 337 218 218 q21 3890255623222322 q22 1065987 979 979 Total cold run time: 107314 ms Total hot run time: 33872 ms - Round 2, with runtime_filter_mode=off - q1 5139509950675067 q2 249 332 218 218 q3 2199266823002300 q4 1384177313681368 q5 4270447445584474 q6 212 171 130 130 q7 2014196217681768 q8 2633255325792553 q9 7507740873297329 q10 3092328728812881 q11 568 505 499 499 q12 720 794 562 562 q13 3379367630763076 q14 269 281 254 254 q15 504 460 459 459 q16 423 466 417 417 q17 1112152513091309 q18 7402705270137013 q19 789 810 934 810 q20 1913196918331833 q21 4842428843614288 q22 1109104010221022 Total cold run time: 51729 ms Total hot run time: 49630 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
xinyiZzz commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3024802437 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
xinyiZzz commented on code in PR #52553:
URL: https://github.com/apache/doris/pull/52553#discussion_r2176361079
##
be/src/vec/common/pod_array.h:
##
@@ -174,15 +177,26 @@ class PODArrayBase : private boost::noncopyable,
static_assert(!TAllocator::need_check_and_tracking_memory(),
"TAllocator should specify
`NoTrackingDefaultMemoryAllocator`");
if (UNLIKELY(c_end_new - c_res_mem > 0)) {
+// 1. if (capacity_size / PRE_GROWTH_RATIO) <= PRE_MIN_GROWTH_SIZE:
// - allocated_bytes = c_end_of_storage - c_start = 4 MB;
// - used_bytes = c_end_new - c_start = 2.1 MB;
// - last tracking_res_memory = c_res_mem - c_start = 1 MB;
-// - res_mem_growth = min(allocated_bytes,
integerRoundUp(used_bytes)) - last_tracking_res_memory = 3 - 1 = 2 MB;
+// - res_mem_growth = min(allocated_bytes,
integerRoundUp(used_bytes, 1M)) - last_tracking_res_memory = 3 - 1 = 2 MB;
// - update tracking_res_memory = 1 + 2 = 3 MB;
// so after each reset_resident_memory, tracking_res_memory >=
used_bytes;
+//
+// 2. if (capacity_size / PRE_GROWTH_RATIO) > PRE_MIN_GROWTH_SIZE:
+// - allocated_bytes = c_end_of_storage - c_start = 1024 MB;
+// - used_bytes = c_end_new - c_start = 210 MB;
+// - last tracking_res_memory = c_res_mem - c_start = 102 MB;
+// - res_mem_growth = min(allocated_bytes,
integerRoundUp(used_bytes, 102M)) - last_tracking_res_memory = 306 - 102 = 204
MB;
+// - update tracking_res_memory = 102 + 204 = 306 MB;
+
+auto capacity_size = static_cast(c_end_of_storage -
c_start);
+auto min_growth_size = std::max(static_cast(capacity_size
/ PRE_GROWTH_RATIO),
+PRE_MIN_GROWTH_SIZE);
int64_t res_mem_growth =
-std::min(static_cast(c_end_of_storage - c_start),
- integerRoundUp(c_end_new - c_start,
PRE_GROWTH_SIZE)) -
+std::min(capacity_size, integerRoundUp(c_end_new -
c_start, min_growth_size)) -
Review Comment:
向上取整到 min_growth_size 的整数倍，min_growth_size = max(1M, capacity / 10)
比如 used 了 10K，实际会 tracking 1M，那么接下来若干次 use 都不会再次 tracking，直到 used 超过 1M 后再
tracking 到 2M，所以 tracking 会一直比实际 used 大一点。
这个 pr 改动后会再次降低 tracking 频率
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
yiguolei commented on code in PR #52553:
URL: https://github.com/apache/doris/pull/52553#discussion_r2176187465
##
be/src/vec/common/pod_array.h:
##
@@ -174,15 +177,26 @@ class PODArrayBase : private boost::noncopyable,
static_assert(!TAllocator::need_check_and_tracking_memory(),
"TAllocator should specify
`NoTrackingDefaultMemoryAllocator`");
if (UNLIKELY(c_end_new - c_res_mem > 0)) {
+// 1. if (capacity_size / PRE_GROWTH_RATIO) <= PRE_MIN_GROWTH_SIZE:
// - allocated_bytes = c_end_of_storage - c_start = 4 MB;
// - used_bytes = c_end_new - c_start = 2.1 MB;
// - last tracking_res_memory = c_res_mem - c_start = 1 MB;
-// - res_mem_growth = min(allocated_bytes,
integerRoundUp(used_bytes)) - last_tracking_res_memory = 3 - 1 = 2 MB;
+// - res_mem_growth = min(allocated_bytes,
integerRoundUp(used_bytes, 1M)) - last_tracking_res_memory = 3 - 1 = 2 MB;
// - update tracking_res_memory = 1 + 2 = 3 MB;
// so after each reset_resident_memory, tracking_res_memory >=
used_bytes;
+//
+// 2. if (capacity_size / PRE_GROWTH_RATIO) > PRE_MIN_GROWTH_SIZE:
+// - allocated_bytes = c_end_of_storage - c_start = 1024 MB;
+// - used_bytes = c_end_new - c_start = 210 MB;
+// - last tracking_res_memory = c_res_mem - c_start = 102 MB;
+// - res_mem_growth = min(allocated_bytes,
integerRoundUp(used_bytes, 102M)) - last_tracking_res_memory = 306 - 102 = 204
MB;
+// - update tracking_res_memory = 102 + 204 = 306 MB;
+
+auto capacity_size = static_cast(c_end_of_storage -
c_start);
+auto min_growth_size = std::max(static_cast(capacity_size
/ PRE_GROWTH_RATIO),
+PRE_MIN_GROWTH_SIZE);
int64_t res_mem_growth =
-std::min(static_cast(c_end_of_storage - c_start),
- integerRoundUp(c_end_new - c_start,
PRE_GROWTH_SIZE)) -
+std::min(capacity_size, integerRoundUp(c_end_new -
c_start, min_growth_size)) -
Review Comment:
1. 这里为啥要integerRoundUp(c_end_new - c_start, min_growth_size) ？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
xinyiZzz commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3018948243 runbuild all -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
xinyiZzz commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3018948735 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [opt](memory) Optimize PODArray memory tracking performance [doris]
hello-stephen commented on PR #52553: URL: https://github.com/apache/doris/pull/52553#issuecomment-3018948154 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
