Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
dataroaring merged PR #49383: URL: https://github.com/apache/doris/pull/49383 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2919460073 ClickBench: Total hot run time: 29.34 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 9685355ce5d95e5fb7e492c7ba539df9a1ba2ed5, data reload: false query1 0.040.030.04 query2 0.110.100.10 query3 0.250.190.20 query4 1.590.190.20 query5 0.450.410.43 query6 1.140.670.66 query7 0.020.020.01 query8 0.040.040.03 query9 0.580.510.53 query10 0.560.570.57 query11 0.150.100.11 query12 0.140.120.12 query13 0.610.610.59 query14 0.780.800.80 query15 0.860.840.84 query16 0.380.380.39 query17 1.051.081.02 query18 0.230.220.21 query19 1.931.811.85 query20 0.020.010.01 query21 15.42 0.890.53 query22 0.781.440.81 query23 14.69 1.360.66 query24 6.981.380.92 query25 0.470.150.09 query26 0.650.160.14 query27 0.060.040.05 query28 9.940.850.44 query29 12.54 4.033.31 query30 0.260.090.06 query31 2.830.600.38 query32 3.230.570.47 query33 2.973.093.21 query34 15.74 5.074.51 query35 4.564.574.54 query36 0.670.490.47 query37 0.080.060.06 query38 0.060.040.03 query39 0.030.020.02 query40 0.170.120.12 query41 0.070.020.02 query42 0.030.030.02 query43 0.030.030.03 Total cold run time: 103.19 s Total hot run time: 29.34 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2919440826 TPC-DS: Total hot run time: 186221 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 9685355ce5d95e5fb7e492c7ba539df9a1ba2ed5, data reload: false query1 1022492 490 490 query2 6585179318151793 query3 6750223 225 223 query4 26820 23939 23286 23286 query5 4329613 461 461 query6 308 202 184 184 query7 4633486 291 291 query8 262 213 231 213 query9 8624263426512634 query10 449 341 269 269 query11 16079 15248 15021 15021 query12 159 110 109 109 query13 1649532 411 411 query14 8870616162336161 query15 243 198 173 173 query16 6997636 492 492 query17 1179677 569 569 query18 1933376 290 290 query19 192 184 171 171 query20 122 115 114 114 query21 216 134 110 110 query22 4017397441123974 query23 34093 33361 33239 33239 query24 8435240823772377 query25 547 446 393 393 query26 1244269 150 150 query27 2749496 338 338 query28 4334213221242124 query29 735 565 427 427 query30 280 214 193 193 query31 942 819 762 762 query32 69 67 66 66 query33 559 357 331 331 query34 801 849 539 539 query35 787 814 710 710 query36 952 953 880 880 query37 116 99 82 82 query38 4192411741454117 query39 1479139914241399 query40 225 126 112 112 query41 71 66 68 66 query42 130 115 115 115 query43 505 503 462 462 query44 1286839 826 826 query45 198 181 172 172 query46 846 1021639 639 query47 1753177717221722 query48 393 439 320 320 query49 775 482 431 431 query50 647 673 404 404 query51 4076427740714071 query52 105 103 96 96 query53 218 248 191 191 query54 584 587 508 508 query55 86 82 83 82 query56 316 308 287 287 query57 1119114410681068 query58 262 256 259 256 query59 2628259525292529 query60 318 318 314 314 query61 128 121 141 121 query62 821 718 668 668 query63 233 193 193 193 query64 4443988 678 678 query65 4272418541644164 query66 1141419 305 305 query67 15816 15753 15306 15306 query68 8002884 536 536 query69 481 308 281 281 query70 1180111811091109 query71 451 324 301 301 query72 5598475148804751 query73 704 597 354 354 query74 9212915689758975 query75 3803320526842684 query76 36321172752 752 query77 795 378 299 299 query78 10072 10115 92909290 query79 1968819 585 585 query80 577 516 445 445 query81 494 254 219 219 query82 434 122 94 94 query83 280 253 241 241 query84 293 103 91 91 query85 771 340 316 316 query86 343 324 277 277 query87 4463440543714371 query88 3575227622492249 query89 386 320 285 285 query90 1938211 205 205 query91 145 139 111 111 query92 73 60 58 58 query93 1489955 586 586 query94 679 409 310 310 query95 380 284 285 284 query96 492 567 317 317 query97 2717271226682668 query98 225 205 205 205 query99 1411143113001300 Total cold run time: 274210 ms Total hot run time: 186221 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2919402319 TPC-H: Total hot run time: 33570 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 9685355ce5d95e5fb7e492c7ba539df9a1ba2ed5, data reload: false -- Round 1 -- q1 26078 568650525052 q2 1963274 183 183 q3 10439 1251684 684 q4 10242 993 533 533 q5 7673235923242324 q6 178 162 145 145 q7 875 723 614 614 q8 9319121910381038 q9 6789505150645051 q10 6856231718831883 q11 504 287 283 283 q12 348 348 212 212 q13 17808 370230313031 q14 233 237 212 212 q15 541 507 516 507 q16 420 435 368 368 q17 586 844 347 347 q18 7754716770287028 q19 1992938 571 571 q20 337 330 223 223 q21 3679258122912291 q22 10641021990 990 Total cold run time: 115678 ms Total hot run time: 33570 ms - Round 2, with runtime_filter_mode=off - q1 5217504450175017 q2 236 321 228 228 q3 2176266322792279 q4 1390180913351335 q5 4547441443594359 q6 214 168 127 127 q7 1970188216871687 q8 2547248524692469 q9 7306723672107210 q10 3005317227512751 q11 569 493 495 493 q12 658 741 607 607 q13 3492393733173317 q14 283 309 274 274 q15 545 497 501 497 q16 458 477 446 446 q17 1146150714061406 q18 7762751873617361 q19 755 785 886 785 q20 1943203019101910 q21 4933445643874387 q22 1077106410221022 Total cold run time: 52229 ms Total hot run time: 49967 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
hello-stephen commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2919176163 # Cloud UT Coverage Report Increment line coverage `83.44% (267/320)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/9685355ce5d95e5fb7e492c7ba539df9a1ba2ed5_9685355ce5d95e5fb7e492c7ba539df9a1ba2ed5_cloud/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/9685355ce5d95e5fb7e492c7ba539df9a1ba2ed5_9685355ce5d95e5fb7e492c7ba539df9a1ba2ed5_cloud/report/index.html) | Category | Coverage | |---|| | Function Coverage | 83.31% (1118/1342) | | Line Coverage | 66.80% (19146/28663) | | Region Coverage | 66.43% (9474/14261) | | Branch Coverage | 56.33% (5129/9106) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
dataroaring commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2919141936 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2909102256 TPC-H: Total hot run time: 33972 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit d8abf2c49f7b650636d80d0184390235c717b042, data reload: false -- Round 1 -- q1 26789 501249714971 q2 2091282 199 199 q3 10495 1274718 718 q4 10233 997 527 527 q5 7703239923212321 q6 190 163 133 133 q7 930 723 593 593 q8 9299120710721072 q9 6744510551475105 q10 6807231718931893 q11 518 292 268 268 q12 339 354 229 229 q13 17817 368331053105 q14 227 230 228 228 q15 528 508 466 466 q16 422 440 379 379 q17 601 856 370 370 q18 7671717872227178 q19 1694976 567 567 q20 331 334 230 230 q21 3943311024302430 q22 10181046990 990 Total cold run time: 116390 ms Total hot run time: 33972 ms - Round 2, with runtime_filter_mode=off - q1 5090506950515051 q2 244 326 238 238 q3 2169265023062306 q4 1323173913801380 q5 4427447644244424 q6 235 170 129 129 q7 2007193217841784 q8 2597257525012501 q9 7251709870427042 q10 3092318127452745 q11 591 510 502 502 q12 697 782 621 621 q13 3468384632883288 q14 314 323 290 290 q15 515 473 471 471 q16 432 492 454 454 q17 1150155613651365 q18 8005761775157515 q19 836 845 898 845 q20 1950200618631863 q21 4815458544614461 q22 1128106810311031 Total cold run time: 52336 ms Total hot run time: 50306 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
zhannngchen commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2908814009 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
hello-stephen commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2908870163 # Cloud UT Coverage Report Increment line coverage `83.44% (267/320)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/d8abf2c49f7b650636d80d0184390235c717b042_d8abf2c49f7b650636d80d0184390235c717b042_cloud/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/d8abf2c49f7b650636d80d0184390235c717b042_d8abf2c49f7b650636d80d0184390235c717b042_cloud/report/index.html) | Category | Coverage | |---|| | Function Coverage | 83.30% (1117/1341) | | Line Coverage | 66.42% (18997/28600) | | Region Coverage | 65.99% (9399/14242) | | Branch Coverage | 55.85% (5080/9096) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2909154998 ClickBench: Total hot run time: 29.8 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit d8abf2c49f7b650636d80d0184390235c717b042, data reload: false query1 0.040.030.03 query2 0.160.100.10 query3 0.370.200.20 query4 1.590.210.07 query5 0.420.420.42 query6 1.160.670.68 query7 0.030.020.02 query8 0.060.050.05 query9 0.620.530.53 query10 0.590.600.58 query11 0.260.130.14 query12 0.260.140.14 query13 0.640.620.62 query14 0.810.820.83 query15 0.970.870.88 query16 0.370.370.37 query17 1.051.041.07 query18 0.190.180.17 query19 2.021.911.86 query20 0.020.020.02 query21 15.44 0.970.67 query22 0.931.100.81 query23 14.73 1.550.76 query24 5.350.580.28 query25 0.170.090.08 query26 0.540.220.19 query27 0.090.090.09 query28 11.05 1.240.59 query29 12.62 4.123.41 query30 0.280.080.07 query31 2.830.630.43 query32 3.240.590.51 query33 3.033.173.19 query34 16.80 5.164.42 query35 4.514.434.54 query36 0.620.510.50 query37 0.200.160.16 query38 0.180.160.17 query39 0.060.050.05 query40 0.190.160.16 query41 0.100.060.06 query42 0.060.050.04 query43 0.060.050.05 Total cold run time: 104.71 s Total hot run time: 29.8 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2909137964 TPC-DS: Total hot run time: 193320 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit d8abf2c49f7b650636d80d0184390235c717b042, data reload: false query1 1425108710691069 query2 6191185717421742 query3 11041 454245344534 query4 54377 24982 22956 22956 query5 5226548 477 477 query6 342 206 199 199 query7 4886528 289 289 query8 325 280 260 260 query9 5786268527082685 query10 429 335 274 274 query11 15016 15312 14844 14844 query12 161 112 106 106 query13 1045546 412 412 query14 10217 639962906290 query15 209 199 185 185 query16 7064639 475 475 query17 1075726 595 595 query18 1505412 307 307 query19 203 184 164 164 query20 131 129 120 120 query21 210 137 108 108 query22 4654470044764476 query23 34364 33572 33584 33572 query24 6807248324672467 query25 457 463 411 411 query26 675 275 157 157 query27 2315512 351 351 query28 3119218221922182 query29 581 579 436 436 query30 273 224 190 190 query31 878 864 787 787 query32 88 74 62 62 query33 456 376 342 342 query34 788 857 573 573 query35 795 856 730 730 query36 952 998 894 894 query37 119 105 80 80 query38 4253431543464315 query39 1559144614901446 query40 219 121 107 107 query41 54 53 54 53 query42 133 110 115 110 query43 517 512 475 475 query44 1387864 854 854 query45 185 180 174 174 query46 866 1043670 670 query47 1817187818351835 query48 407 448 330 330 query49 668 511 448 448 query50 682 706 416 416 query51 4279429842334233 query52 112 107 106 106 query53 236 257 184 184 query54 583 620 520 520 query55 91 80 80 80 query56 306 310 302 302 query57 1171124011071107 query58 271 260 257 257 query59 2668284326382638 query60 344 340 306 306 query61 131 180 122 122 query62 744 696 672 672 query63 232 188 189 188 query64 14171015660 660 query65 4337422942374229 query66 752 398 307 307 query67 16022 15722 15539 15539 query68 6571931 520 520 query69 538 307 277 277 query70 1192109410841084 query71 518 320 286 286 query72 5748491250664912 query73 1480726 359 359 query74 8967899290478992 query75 3875320527242724 query76 42751198771 771 query77 657 373 288 288 query78 10155 10326 93789378 query79 2401833 595 595 query80 604 507 433 433 query81 490 261 219 219 query82 438 131 98 98 query83 334 248 231 231 query84 300 105 86 86 query85 761 346 310 310 query86 374 313 284 284 query87 452544054405 query88 3610227522772275 query89 406 313 280 280 query90 1844217 211 211 query91 145 141 113 113 query92 74 66 56 56 query93 1970934 581 581 query94 655 422 319 319 query95 375 301 291 291 query96 504 576 284 284 query97 2723284027012701 query98 223 206 205 205 query99 1419139212681268 Total cold run time: 298779 ms Total hot run time: 193320 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
github-actions[bot] commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2908820710 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2903729583 ClickBench: Total hot run time: 29.31 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit d8abf2c49f7b650636d80d0184390235c717b042, data reload: false query1 0.040.040.03 query2 0.120.100.11 query3 0.250.200.20 query4 1.600.190.19 query5 0.450.440.43 query6 1.550.670.68 query7 0.020.020.01 query8 0.040.050.04 query9 0.560.520.51 query10 0.590.570.58 query11 0.160.110.11 query12 0.140.120.13 query13 0.610.600.60 query14 0.810.800.81 query15 0.870.880.87 query16 0.400.390.38 query17 1.031.041.04 query18 0.230.220.21 query19 1.971.791.89 query20 0.020.010.01 query21 15.39 0.950.56 query22 0.761.130.76 query23 14.88 1.390.62 query24 7.051.650.79 query25 0.440.150.08 query26 0.630.170.14 query27 0.050.050.06 query28 9.760.940.47 query29 12.60 4.003.28 query30 0.250.100.06 query31 2.820.580.39 query32 3.230.550.46 query33 3.033.113.06 query34 15.73 5.154.52 query35 4.564.574.55 query36 0.680.500.49 query37 0.090.060.07 query38 0.060.040.04 query39 0.040.030.03 query40 0.180.150.15 query41 0.080.030.02 query42 0.030.030.02 query43 0.040.030.03 Total cold run time: 103.84 s Total hot run time: 29.31 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2903716268 TPC-DS: Total hot run time: 185815 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit d8abf2c49f7b650636d80d0184390235c717b042, data reload: false query1 1008487 478 478 query2 6601183617911791 query3 6746226 214 214 query4 26824 23387 23532 23387 query5 4544620 460 460 query6 300 210 214 210 query7 4641491 287 287 query8 302 254 237 237 query9 8618263426242624 query10 470 325 271 271 query11 15416 15162 14897 14897 query12 154 108 104 104 query13 1664509 405 405 query14 8646623762426237 query15 219 195 175 175 query16 7153619 503 503 query17 944 720 567 567 query18 1967407 309 309 query19 201 191 164 164 query20 123 118 124 118 query21 216 126 108 108 query22 4030427840024002 query23 34086 33265 33180 33180 query24 8456238323652365 query25 565 482 434 434 query26 1295265 151 151 query27 2766507 341 341 query28 4314213321182118 query29 782 542 421 421 query30 278 215 182 182 query31 910 845 761 761 query32 73 64 66 64 query33 553 376 320 320 query34 823 851 515 515 query35 778 790 723 723 query36 913 957 895 895 query37 118 102 76 76 query38 4052420740284028 query39 1494142313991399 query40 220 120 104 104 query41 56 56 65 56 query42 124 110 115 110 query43 498 503 468 468 query44 1307827 832 827 query45 176 175 176 175 query46 827 1012645 645 query47 1752179717301730 query48 376 414 303 303 query49 773 522 435 435 query50 630 691 407 407 query51 4096416341414141 query52 114 103 94 94 query53 224 249 176 176 query54 559 577 498 498 query55 90 81 84 81 query56 305 305 295 295 query57 1152118410891089 query58 274 258 261 258 query59 2516255424792479 query60 341 318 304 304 query61 149 124 127 124 query62 775 723 644 644 query63 229 188 185 185 query64 43691034751 751 query65 4337425342634253 query66 1180411 315 315 query67 15828 15465 15308 15308 query68 9291875 522 522 query69 469 316 275 275 query70 1223110710891089 query71 479 322 291 291 query72 5396467146184618 query73 700 561 362 362 query74 8929910387168716 query75 4315321426782678 query76 39991181742 742 query77 1016372 298 298 query78 10092 10264 93999399 query79 2181825 571 571 query80 641 523 432 432 query81 459 253 222 222 query82 428 125 100 100 query83 284 317 239 239 query84 286 97 82 82 query85 777 350 314 314 query86 332 314 265 265 query87 4380441642504250 query88 3226229023902290 query89 407 316 282 282 query90 1919212 203 203 query91 143 145 108 108 query92 78 60 59 59 query93 1166925 579 579 query94 682 399 324 324 query95 376 292 283 283 query96 497 577 283 283 query97 2714273726842684 query98 241 207 200 200 query99 1466139912791279 Total cold run time: 274912 ms Total hot run time: 185815 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2903687212 TPC-H: Total hot run time: 34020 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit d8abf2c49f7b650636d80d0184390235c717b042, data reload: false -- Round 1 -- q1 26198 506450235023 q2 2066285 182 182 q3 10511 1245697 697 q4 10209 1021523 523 q5 7683242123512351 q6 187 165 133 133 q7 903 743 602 602 q8 9312128110651065 q9 6799511651285116 q10 6877234919241924 q11 468 288 269 269 q12 343 351 215 215 q13 17787 370931563156 q14 230 255 216 216 q15 540 488 495 488 q16 425 433 378 378 q17 586 862 347 347 q18 7607729871657165 q19 2152971 562 562 q20 336 346 216 216 q21 3689321524052405 q22 10651023987 987 Total cold run time: 115973 ms Total hot run time: 34020 ms - Round 2, with runtime_filter_mode=off - q1 5222545550945094 q2 250 326 238 238 q3 2172268923442344 q4 1358181815221522 q5 4577442743554355 q6 213 164 129 129 q7 1963193017471747 q8 2591271525482548 q9 7226716771007100 q10 2991316127352735 q11 580 517 502 502 q12 703 779 604 604 q13 3413389733603360 q14 279 296 273 273 q15 520 467 473 467 q16 438 495 436 436 q17 1203157813511351 q18 7776751673207320 q19 777 780 814 780 q20 2004208118751875 q21 4793437343864373 q22 10791030994 994 Total cold run time: 52128 ms Total hot run time: 50147 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
mymeiyi commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2903391677 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
hello-stephen commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2903418019 # Cloud UT Coverage Report Increment line coverage `83.44% (267/320)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/d8abf2c49f7b650636d80d0184390235c717b042_d8abf2c49f7b650636d80d0184390235c717b042_cloud/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/d8abf2c49f7b650636d80d0184390235c717b042_d8abf2c49f7b650636d80d0184390235c717b042_cloud/report/index.html) | Category | Coverage | |---|| | Function Coverage | 83.30% (1117/1341) | | Line Coverage | 66.44% (19000/28599) | | Region Coverage | 65.99% (9397/14240) | | Branch Coverage | 55.86% (5080/9094) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2891277274 TPC-DS: Total hot run time: 192672 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit c4870f45fec152ffe7ee3d679732cf2800dbe045, data reload: false query1 1407107510841075 query2 6259179317951793 query3 11165 471546794679 query4 25596 23587 23367 23367 query5 4831610 457 457 query6 309 205 215 205 query7 4005498 295 295 query8 322 281 251 251 query9 8529264626742646 query10 500 328 284 284 query11 15267 15009 14839 14839 query12 176 111 104 104 query13 1566551 439 439 query14 9298609862676098 query15 207 189 169 169 query16 7615646 473 473 query17 1194818 634 634 query18 2035414 321 321 query19 207 206 232 206 query20 129 132 118 118 query21 203 124 109 109 query22 4554452443164316 query23 34954 33875 33667 33667 query24 8780241724012401 query25 509 471 397 397 query26 898 278 155 155 query27 2726509 350 350 query28 4812220922052205 query29 659 573 467 467 query30 277 217 189 189 query31 914 864 767 767 query32 72 64 61 61 query33 529 381 311 311 query34 811 857 543 543 query35 808 836 760 760 query36 980 997 888 888 query37 116 103 82 82 query38 4182429540924092 query39 1703149114711471 query40 228 121 118 118 query41 58 54 58 54 query42 119 115 111 111 query43 529 512 482 482 query44 1400880 886 880 query45 191 168 167 167 query46 869 1050655 655 query47 1816190618061806 query48 422 437 334 334 query49 753 505 426 426 query50 687 679 413 413 query51 4291428042684268 query52 107 110 109 109 query53 232 276 191 191 query54 597 588 535 535 query55 84 88 89 88 query56 304 305 308 305 query57 1166122211541154 query58 283 268 261 261 query59 2609268325982598 query60 335 326 304 304 query61 138 133 130 130 query62 882 759 688 688 query63 236 194 193 193 query64 34131074695 695 query65 4486422142254221 query66 925 442 308 308 query67 16048 15540 15326 15326 query68 9438897 540 540 query69 497 307 272 272 query70 1208113211041104 query71 467 334 297 297 query72 5347465645634563 query73 685 570 366 366 query74 8865900385998599 query75 4327319526812681 query76 37081174762 762 query77 991 374 298 298 query78 10086 10286 94569456 query79 1443810 591 591 query80 595 523 445 445 query81 495 253 218 218 query82 191 120 92 92 query83 285 255 237 237 query84 297 119 82 82 query85 750 368 316 316 query86 323 307 276 276 query87 4449443543674367 query88 2937235323792353 query89 385 327 284 284 query90 2056207 201 201 query91 146 141 113 113 query92 74 58 57 57 query93 1125959 578 578 query94 669 385 309 309 query95 373 289 315 289 query96 500 571 286 286 query97 2695272026832683 query98 228 225 195 195 query99 1429141312581258 Total cold run time: 279376 ms Total hot run time: 192672 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2891294786 ClickBench: Total hot run time: 29.84 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit c4870f45fec152ffe7ee3d679732cf2800dbe045, data reload: false query1 0.030.040.03 query2 0.150.110.12 query3 0.350.200.19 query4 1.590.200.20 query5 0.460.440.45 query6 1.150.660.66 query7 0.020.010.02 query8 0.050.050.05 query9 0.630.510.52 query10 0.570.590.58 query11 0.260.130.12 query12 0.250.130.13 query13 0.630.620.64 query14 0.790.810.82 query15 0.960.880.87 query16 0.370.360.37 query17 1.041.061.02 query18 0.180.190.18 query19 1.981.791.88 query20 0.010.020.01 query21 15.40 0.980.67 query22 0.921.060.83 query23 14.68 1.470.75 query24 5.450.560.27 query25 0.160.090.09 query26 0.560.220.19 query27 0.090.080.09 query28 10.98 1.170.60 query29 12.60 4.053.42 query30 0.280.090.07 query31 2.820.620.43 query32 3.220.600.50 query33 3.053.053.06 query34 16.49 5.194.57 query35 4.524.454.50 query36 0.630.500.50 query37 0.200.170.17 query38 0.170.160.16 query39 0.060.050.05 query40 0.190.160.16 query41 0.100.050.06 query42 0.070.060.05 query43 0.050.040.04 Total cold run time: 104.16 s Total hot run time: 29.84 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2891235601 TPC-H: Total hot run time: 34098 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit c4870f45fec152ffe7ee3d679732cf2800dbe045, data reload: false -- Round 1 -- q1 26194 527851395139 q2 2087291 187 187 q3 10579 1288717 717 q4 10251 981 532 532 q5 7889241223132313 q6 193 168 133 133 q7 952 739 627 627 q8 9338132311471147 q9 6762512151675121 q10 6880231618961896 q11 525 292 269 269 q12 355 347 212 212 q13 17782 368031153115 q14 242 238 212 212 q15 527 487 497 487 q16 420 434 368 368 q17 603 861 367 367 q18 7553723171897189 q19 1438950 567 567 q20 332 355 233 233 q21 3853315322992299 q22 10741046968 968 Total cold run time: 115829 ms Total hot run time: 34098 ms - Round 2, with runtime_filter_mode=off - q1 5202506250775062 q2 243 322 224 224 q3 2165265822652265 q4 1363186914361436 q5 4449443443744374 q6 220 171 127 127 q7 2006193317681768 q8 2592259925342534 q9 7314703672127036 q10 3041318727562756 q11 598 514 488 488 q12 666 757 607 607 q13 3529394633443344 q14 277 298 292 292 q15 533 474 467 467 q16 447 498 465 465 q17 1165160013521352 q18 7864760074287428 q19 809 846 982 846 q20 1966197918731873 q21 4970449144774477 q22 1137107510231023 Total cold run time: 52556 ms Total hot run time: 50244 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2890146500 TPC-DS: Total hot run time: 192649 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 618f0e016b3618fd04a27976bce2a05770ad1699, data reload: false query1 1411108510641064 query2 6259178918281789 query3 11003 433846124338 query4 54485 25511 23510 23510 query5 5154553 433 433 query6 336 197 193 193 query7 4866510 296 296 query8 311 262 254 254 query9 5347260826022602 query10 442 326 265 265 query11 15003 15214 14747 14747 query12 172 115 105 105 query13 1044537 410 410 query14 10174 635563146314 query15 217 201 189 189 query16 7146639 513 513 query17 1103724 580 580 query18 1567423 308 308 query19 196 202 190 190 query20 131 123 119 119 query21 217 130 109 109 query22 4602462644854485 query23 34323 33545 33266 33266 query24 6795247724102410 query25 483 498 405 405 query26 701 279 150 150 query27 2228528 354 354 query28 2938213521292129 query29 561 606 440 440 query30 278 224 193 193 query31 867 883 769 769 query32 79 66 61 61 query33 459 357 319 319 query34 773 853 546 546 query35 789 830 763 763 query36 942 1024910 910 query37 118 100 78 78 query38 4336426642714266 query39 1554145115661451 query40 216 128 114 114 query41 60 55 50 50 query42 128 113 117 113 query43 507 505 492 492 query44 1325841 838 838 query45 186 178 171 171 query46 854 1031650 650 query47 1878188018271827 query48 401 440 332 332 query49 676 544 467 467 query50 696 700 408 408 query51 4214434841874187 query52 115 108 113 108 query53 237 275 191 191 query54 601 601 529 529 query55 88 91 87 87 query56 317 322 352 322 query57 1207121611381138 query58 265 253 256 253 query59 2729279526442644 query60 327 332 319 319 query61 128 120 127 120 query62 730 753 674 674 query63 232 192 184 184 query64 15141018681 681 query65 4301426742234223 query66 766 411 301 301 query67 16266 15737 15549 15549 query68 6967880 517 517 query69 556 301 263 263 query70 1210114510611061 query71 519 323 295 295 query72 5710471147864711 query73 1507598 349 349 query74 9428915988608860 query75 3940320826992699 query76 42121198777 777 query77 672 352 285 285 query78 10248 10209 93929392 query79 2536816 581 581 query80 638 512 469 469 query81 495 257 223 223 query82 393 130 98 98 query83 257 252 252 252 query84 291 107 84 84 query85 794 352 390 352 query86 374 304 278 278 query87 4424449943114311 query88 3520228722662266 query89 407 322 280 280 query90 1800202 206 202 query91 140 152 108 108 query92 71 60 64 60 query93 2319930 573 573 query94 639 398 297 297 query95 361 293 349 293 query96 503 573 281 281 query97 2703281826532653 query98 233 211 198 198 query99 1398144412861286 Total cold run time: 299639 ms Total hot run time: 192649 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
hello-stephen commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2890833724 # Cloud UT Coverage Report Increment line coverage `85.61% (238/278)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/c4870f45fec152ffe7ee3d679732cf2800dbe045_c4870f45fec152ffe7ee3d679732cf2800dbe045_cloud/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/c4870f45fec152ffe7ee3d679732cf2800dbe045_c4870f45fec152ffe7ee3d679732cf2800dbe045_cloud/report/index.html) | Category | Coverage | |---|| | Function Coverage | 83.35% (1116/1339) | | Line Coverage | 66.51% (18969/28522) | | Region Coverage | 66.03% (9384/14212) | | Branch Coverage | 55.81% (5063/9072) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
dataroaring commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2890774550 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2890162852 ClickBench: Total hot run time: 29.6 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 618f0e016b3618fd04a27976bce2a05770ad1699, data reload: false query1 0.040.030.03 query2 0.150.100.10 query3 0.340.200.19 query4 1.590.200.20 query5 0.460.450.45 query6 1.180.640.66 query7 0.020.010.01 query8 0.080.040.04 query9 0.620.530.53 query10 0.570.600.58 query11 0.250.130.13 query12 0.250.130.13 query13 0.630.620.63 query14 0.790.830.83 query15 0.970.870.87 query16 0.380.380.37 query17 1.051.061.08 query18 0.180.190.18 query19 1.981.821.78 query20 0.010.000.01 query21 15.40 0.980.67 query22 0.911.040.83 query23 14.70 1.510.74 query24 5.440.580.29 query25 0.160.090.09 query26 0.540.220.18 query27 0.080.080.08 query28 11.00 1.230.58 query29 12.54 4.003.34 query30 0.270.080.07 query31 2.860.630.43 query32 3.240.600.49 query33 3.103.083.08 query34 16.47 5.144.40 query35 4.514.444.46 query36 0.620.500.49 query37 0.190.180.18 query38 0.170.170.15 query39 0.050.040.04 query40 0.190.160.14 query41 0.110.050.05 query42 0.070.050.05 query43 0.060.050.05 Total cold run time: 104.22 s Total hot run time: 29.6 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2890108433 TPC-H: Total hot run time: 33942 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 618f0e016b3618fd04a27976bce2a05770ad1699, data reload: false -- Round 1 -- q1 27142 506250805062 q2 2074278 189 189 q3 10577 1412725 725 q4 10219 986 530 530 q5 7793239123102310 q6 188 167 137 137 q7 927 773 615 615 q8 9359129210801080 q9 6806511950545054 q10 6881231218931893 q11 487 284 277 277 q12 351 365 225 225 q13 17776 367630583058 q14 247 231 216 216 q15 531 492 487 487 q16 420 429 391 391 q17 614 859 369 369 q18 7584721171887188 q19 1604971 563 563 q20 322 343 230 230 q21 4171314123892389 q22 10371027954 954 Total cold run time: 117110 ms Total hot run time: 33942 ms - Round 2, with runtime_filter_mode=off - q1 5196504951305049 q2 243 324 229 229 q3 2212266822932293 q4 1344183714801480 q5 4490438143974381 q6 217 168 128 128 q7 2020193917641764 q8 2586258925882588 q9 7233715471037103 q10 3045319827462746 q11 587 520 502 502 q12 697 770 638 638 q13 3514388433933393 q14 276 300 284 284 q15 545 474 475 474 q16 459 500 453 453 q17 1210152913891389 q18 7800764374227422 q19 817 845 1015845 q20 1952199518461846 q21 4855450543934393 q22 1124105410231023 Total cold run time: 52422 ms Total hot run time: 50423 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2889824108 # Cloud UT Coverage Report Increment line coverage `85.61% (238/278)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/618f0e016b3618fd04a27976bce2a05770ad1699_618f0e016b3618fd04a27976bce2a05770ad1699_cloud/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/618f0e016b3618fd04a27976bce2a05770ad1699_618f0e016b3618fd04a27976bce2a05770ad1699_cloud/report/index.html) | Category | Coverage | |---|| | Function Coverage | 83.35% (1116/1339) | | Line Coverage | 66.51% (18969/28522) | | Region Coverage | 65.97% (9375/14212) | | Branch Coverage | 55.81% (5063/9072) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
mymeiyi commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2889762750 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
mymeiyi commented on code in PR #49383: URL: https://github.com/apache/doris/pull/49383#discussion_r2094917718 ## cloud/src/meta-service/meta_service.h: ## @@ -41,6 +41,7 @@ class Transaction; constexpr std::string_view BUILT_IN_STORAGE_VAULT_NAME = "built_in_storage_vault"; static constexpr int COMPACTION_DELETE_BITMAP_LOCK_ID = -1; static constexpr int SCHEMA_CHANGE_DELETE_BITMAP_LOCK_ID = -2; +static constexpr int COMPACTION_WITHOUT_LOCK_DELETE_BITMAP_LOCK_ID = -3; Review Comment: no, this update_delete_bitmap does not hold lock -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
cosven commented on code in PR #49383:
URL: https://github.com/apache/doris/pull/49383#discussion_r2086886474
##
be/src/olap/base_tablet.cpp:
##
@@ -1768,6 +1768,148 @@ Status BaseTablet::update_delete_bitmap_without_lock(
return Status::OK();
}
+void BaseTablet::agg_delete_bitmap_for_stale_rowsets(
+Version version, DeleteBitmapKeyRanges&
remove_delete_bitmap_key_ranges) {
+if (!config::enable_agg_and_remove_pre_rowsets_delete_bitmap) {
+return;
+}
+if (!(keys_type() == UNIQUE_KEYS && enable_unique_key_merge_on_write())) {
+return;
+}
+int64_t start_version = version.first;
+int64_t end_version = version.second;
+DCHECK(start_version < end_version)
+<< ". start_version: " << start_version << ", end_version: " <<
end_version;
+// get pre rowsets
+std::vector pre_rowsets {};
+{
+std::shared_lock rdlock(_meta_lock);
+for (const auto& it2 : _rs_version_map) {
+if (it2.first.second < start_version) {
+pre_rowsets.emplace_back(it2.second);
+}
+}
+}
+std::sort(pre_rowsets.begin(), pre_rowsets.end(), Rowset::comparator);
+// do agg for pre rowsets
+DeleteBitmapPtr new_delete_bitmap =
std::make_shared(tablet_id());
+for (auto& rowset : pre_rowsets) {
+for (uint32_t seg_id = 0; seg_id < rowset->num_segments(); ++seg_id) {
+auto d = tablet_meta()->delete_bitmap().get_agg_without_cache(
+{rowset->rowset_id(), seg_id, end_version}, start_version);
+if (d->isEmpty()) {
+continue;
+}
+VLOG_DEBUG << "agg delete bitmap for tablet_id=" << tablet_id()
+ << ", rowset_id=" << rowset->rowset_id() << ", seg_id="
<< seg_id
+ << ", rowset_version=" << rowset->version().to_string()
+ << ". compaction start_version=" << start_version
+ << ", end_version=" << end_version << ",
delete_bitmap=" << d->cardinality();
+DeleteBitmap::BitmapKey start_key {rowset->rowset_id(), seg_id,
start_version};
+DeleteBitmap::BitmapKey end_key {rowset->rowset_id(), seg_id,
end_version};
+new_delete_bitmap->set(end_key, *d);
+remove_delete_bitmap_key_ranges.emplace_back(start_key, end_key);
+}
+}
+tablet_meta()->delete_bitmap().merge(*new_delete_bitmap);
+}
+
+void BaseTablet::check_agg_delete_bitmap_for_stale_rowsets(int64_t&
useless_rowset_count,
+ int64_t&
useless_rowset_version_count) {
+std::set rowset_ids;
+std::set end_versions;
+traverse_rowsets(
+[&rowset_ids, &end_versions](const RowsetSharedPtr& rs) {
+rowset_ids.emplace(rs->rowset_id());
+end_versions.emplace(rs->end_version());
+},
+true);
+
+std::set useless_rowsets;
+std::map> useless_rowset_versions;
+{
+_tablet_meta->delete_bitmap().traverse_rowset_and_version(
+// 0: rowset and rowset with version exists
+// -1: rowset does not exist
+// -2: rowset exist, rowset with version does not exist
+[&](const RowsetId& rowset_id, int64_t version) {
+if (rowset_ids.find(rowset_id) == rowset_ids.end()) {
+useless_rowsets.emplace(rowset_id);
+return -1;
+}
+if (end_versions.find(version) == end_versions.end()) {
+if (useless_rowset_versions.find(rowset_id) ==
+useless_rowset_versions.end()) {
+useless_rowset_versions[rowset_id] = {};
+}
+
useless_rowset_versions[rowset_id].emplace_back(version);
+return -2;
Review Comment:
确认一下这样理解对不对：
1. useless_rowsets 对应：无用 rowset 上的 delete bitmap 泄露。
2. useless_rowset_versions 对应：有用的 rowset 上的 delete bitmap 泄露。
所谓的“无用 rowset 上的 delete bitmap 泄露”指的就是“一个 rowset 进行 compaction 后，会遗留下来一些
delete bitmap”这种情况
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
cosven commented on code in PR #49383:
URL: https://github.com/apache/doris/pull/49383#discussion_r2086904783
##
be/src/olap/tablet_manager.cpp:
##
@@ -1133,6 +1133,39 @@ Status TabletManager::start_trash_sweep() {
for_each_tablet([](const TabletSharedPtr& tablet) {
tablet->delete_expired_stale_rowset(); },
filter_all_tablets);
+if (config::enable_check_agg_and_remove_pre_rowsets_delete_bitmap) {
+int64_t max_useless_rowset_count = 0;
+int64_t tablet_id_with_max_useless_rowset_count = 0;
+int64_t max_useless_rowset_version_count = 0;
+int64_t tablet_id_with_max_useless_rowset_version_count = 0;
+OlapStopWatch watch;
+for_each_tablet(
+[&](const TabletSharedPtr& tablet) {
+int64_t useless_rowset_count = 0;
+int64_t useless_rowset_version_count = 0;
+
tablet->check_agg_delete_bitmap_for_stale_rowsets(useless_rowset_count,
+
useless_rowset_version_count);
+if (useless_rowset_count > max_useless_rowset_count) {
+max_useless_rowset_count = useless_rowset_count;
+tablet_id_with_max_useless_rowset_count =
tablet->tablet_id();
+}
+if (useless_rowset_version_count >
max_useless_rowset_version_count) {
+max_useless_rowset_version_count =
useless_rowset_version_count;
+tablet_id_with_max_useless_rowset_version_count =
tablet->tablet_id();
+}
Review Comment:
感觉求 useless_rowset_count 的总和比较有参考价值，它的和表示了有多少个 delete bitmap 泄露了。算出
useless_rowset_count 的最大值，似乎参考意义不大？
max_useless_rowset_version_count 也是类似。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
cosven commented on code in PR #49383:
URL: https://github.com/apache/doris/pull/49383#discussion_r2086886474
##
be/src/olap/base_tablet.cpp:
##
@@ -1768,6 +1768,148 @@ Status BaseTablet::update_delete_bitmap_without_lock(
return Status::OK();
}
+void BaseTablet::agg_delete_bitmap_for_stale_rowsets(
+Version version, DeleteBitmapKeyRanges&
remove_delete_bitmap_key_ranges) {
+if (!config::enable_agg_and_remove_pre_rowsets_delete_bitmap) {
+return;
+}
+if (!(keys_type() == UNIQUE_KEYS && enable_unique_key_merge_on_write())) {
+return;
+}
+int64_t start_version = version.first;
+int64_t end_version = version.second;
+DCHECK(start_version < end_version)
+<< ". start_version: " << start_version << ", end_version: " <<
end_version;
+// get pre rowsets
+std::vector pre_rowsets {};
+{
+std::shared_lock rdlock(_meta_lock);
+for (const auto& it2 : _rs_version_map) {
+if (it2.first.second < start_version) {
+pre_rowsets.emplace_back(it2.second);
+}
+}
+}
+std::sort(pre_rowsets.begin(), pre_rowsets.end(), Rowset::comparator);
+// do agg for pre rowsets
+DeleteBitmapPtr new_delete_bitmap =
std::make_shared(tablet_id());
+for (auto& rowset : pre_rowsets) {
+for (uint32_t seg_id = 0; seg_id < rowset->num_segments(); ++seg_id) {
+auto d = tablet_meta()->delete_bitmap().get_agg_without_cache(
+{rowset->rowset_id(), seg_id, end_version}, start_version);
+if (d->isEmpty()) {
+continue;
+}
+VLOG_DEBUG << "agg delete bitmap for tablet_id=" << tablet_id()
+ << ", rowset_id=" << rowset->rowset_id() << ", seg_id="
<< seg_id
+ << ", rowset_version=" << rowset->version().to_string()
+ << ". compaction start_version=" << start_version
+ << ", end_version=" << end_version << ",
delete_bitmap=" << d->cardinality();
+DeleteBitmap::BitmapKey start_key {rowset->rowset_id(), seg_id,
start_version};
+DeleteBitmap::BitmapKey end_key {rowset->rowset_id(), seg_id,
end_version};
+new_delete_bitmap->set(end_key, *d);
+remove_delete_bitmap_key_ranges.emplace_back(start_key, end_key);
+}
+}
+tablet_meta()->delete_bitmap().merge(*new_delete_bitmap);
+}
+
+void BaseTablet::check_agg_delete_bitmap_for_stale_rowsets(int64_t&
useless_rowset_count,
+ int64_t&
useless_rowset_version_count) {
+std::set rowset_ids;
+std::set end_versions;
+traverse_rowsets(
+[&rowset_ids, &end_versions](const RowsetSharedPtr& rs) {
+rowset_ids.emplace(rs->rowset_id());
+end_versions.emplace(rs->end_version());
+},
+true);
+
+std::set useless_rowsets;
+std::map> useless_rowset_versions;
+{
+_tablet_meta->delete_bitmap().traverse_rowset_and_version(
+// 0: rowset and rowset with version exists
+// -1: rowset does not exist
+// -2: rowset exist, rowset with version does not exist
+[&](const RowsetId& rowset_id, int64_t version) {
+if (rowset_ids.find(rowset_id) == rowset_ids.end()) {
+useless_rowsets.emplace(rowset_id);
+return -1;
+}
+if (end_versions.find(version) == end_versions.end()) {
+if (useless_rowset_versions.find(rowset_id) ==
+useless_rowset_versions.end()) {
+useless_rowset_versions[rowset_id] = {};
+}
+
useless_rowset_versions[rowset_id].emplace_back(version);
+return -2;
Review Comment:
确认一下这样理解对不对：
1. useless_rowsets 对应：无用 rowset 上的 delete bitmap 泄露。
2. useless_rowset_versions 对应：有用的 rowset 上的 delete bitmap 泄露。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
hello-stephen commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2873176036 # BE Regression && UT Coverage Report Increment line coverage `26.77% (144/538)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/49383_d07492893fc4e59d788d2a9da5397b310ade123c_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/49383_d07492893fc4e59d788d2a9da5397b310ade123c_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 79.35% (20862/26290) | | Line Coverage | 72.53% (214768/296110) | | Region Coverage | 70.71% (126328/178659) | | Branch Coverage | 64.42% (65467/101622) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
hello-stephen commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2872596826 # BE UT Coverage Report Increment line coverage `1.30% (7/538)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/d07492893fc4e59d788d2a9da5397b310ade123c_d07492893fc4e59d788d2a9da5397b310ade123c/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/d07492893fc4e59d788d2a9da5397b310ade123c_d07492893fc4e59d788d2a9da5397b310ade123c/report/index.html) | Category | Coverage | |---|| | Function Coverage | 55.73% (14894/26724) | | Line Coverage | 44.50% (131795/296174) | | Region Coverage | 43.56% (66273/152151) | | Branch Coverage | 38.16% (33964/88998) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2872321959 ClickBench: Total hot run time: 29.82 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit d07492893fc4e59d788d2a9da5397b310ade123c, data reload: false query1 0.030.030.03 query2 0.140.110.10 query3 0.350.190.19 query4 1.600.200.08 query5 0.570.550.57 query6 1.180.730.73 query7 0.020.010.02 query8 0.060.040.05 query9 0.610.520.52 query10 0.560.590.57 query11 0.250.130.13 query12 0.250.130.14 query13 0.630.620.61 query14 0.800.820.81 query15 0.970.880.88 query16 0.370.370.38 query17 1.051.061.07 query18 0.190.180.19 query19 1.941.801.88 query20 0.020.020.01 query21 15.40 0.980.68 query22 0.921.020.78 query23 14.67 1.490.73 query24 5.590.560.29 query25 0.160.090.08 query26 0.560.220.18 query27 0.090.090.09 query28 11.01 1.130.58 query29 12.53 4.103.39 query30 0.280.080.06 query31 2.800.620.42 query32 3.240.590.50 query33 3.043.153.14 query34 16.39 5.094.46 query35 4.484.484.47 query36 0.640.500.49 query37 0.210.170.17 query38 0.160.160.16 query39 0.050.040.04 query40 0.190.170.16 query41 0.120.050.05 query42 0.060.050.05 query43 0.060.050.05 Total cold run time: 104.24 s Total hot run time: 29.82 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2872307419 TPC-DS: Total hot run time: 192543 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit d07492893fc4e59d788d2a9da5397b310ade123c, data reload: false query1 1427106510471047 query2 6421177617931776 query3 11033 463347864633 query4 25342 23903 23134 23134 query5 5366624 487 487 query6 309 207 215 207 query7 3991508 293 293 query8 312 261 240 240 query9 8488262126082608 query10 513 326 270 270 query11 15322 15084 15088 15084 query12 171 111 111 111 query13 1569532 414 414 query14 9463604460606044 query15 209 187 168 168 query16 7589645 433 433 query17 1182790 594 594 query18 2050425 323 323 query19 198 206 167 167 query20 180 120 117 117 query21 201 122 117 117 query22 44434492 query23 34760 33791 33393 33393 query24 8366241423812381 query25 516 464 397 397 query26 1217277 147 147 query27 2784506 341 341 query28 4533214021442140 query29 708 569 439 439 query30 270 222 194 194 query31 901 847 813 813 query32 79 63 63 63 query33 542 364 305 305 query34 800 876 543 543 query35 807 844 736 736 query36 987 977 894 894 query37 112 100 77 77 query38 4250434241754175 query39 1512148314661466 query40 209 122 109 109 query41 56 51 52 51 query42 120 101 112 101 query43 500 496 473 473 query44 1312824 829 824 query45 184 187 181 181 query46 849 1027642 642 query47 1805189117801780 query48 400 429 328 328 query49 750 513 443 443 query50 669 683 403 403 query51 4251430642554255 query52 116 110 102 102 query53 224 255 183 183 query54 613 573 518 518 query55 81 79 87 79 query56 315 303 290 290 query57 1160119311541154 query58 270 260 253 253 query59 2580280925642564 query60 332 323 315 315 query61 129 121 143 121 query62 770 723 661 661 query63 233 184 187 184 query64 39911071699 699 query65 4391442243964396 query66 971 404 301 301 query67 16378 15487 15268 15268 query68 9443879 530 530 query69 500 306 263 263 query70 1248108510841084 query71 473 309 303 303 query72 5303467946854679 query73 702 590 353 353 query74 8896908389248924 query75 4273324927272727 query76 37941183744 744 query77 996 372 280 280 query78 994610237 92609260 query79 2611816 581 581 query80 679 504 451 451 query81 470 258 218 218 query82 467 124 96 96 query83 290 243 237 237 query84 290 101 82 82 query85 763 360 314 314 query86 413 316 309 309 query87 4391439442944294 query88 2821224522592245 query89 429 317 289 289 query90 2050199 202 199 query91 153 145 109 109 query92 76 60 53 53 query93 2218931 570 570 query94 671 408 300 300 query95 370 282 279 279 query96 485 567 280 280 query97 3128324031343134 query98 252 206 202 202 query99 1406140512611261 Total cold run time: 282589 ms Total hot run time: 192543 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2872274794 TPC-H: Total hot run time: 34047 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit d07492893fc4e59d788d2a9da5397b310ade123c, data reload: false -- Round 1 -- q1 26200 503049724972 q2 2078299 185 185 q3 10550 1266701 701 q4 10236 984 517 517 q5 7852240123522352 q6 190 167 138 138 q7 929 738 629 629 q8 9332125411861186 q9 6813516150245024 q10 6874232319081908 q11 482 291 270 270 q12 353 342 211 211 q13 17761 363731573157 q14 234 221 221 221 q15 532 495 492 492 q16 442 437 376 376 q17 625 860 373 373 q18 7709720271587158 q19 1425954 546 546 q20 343 340 226 226 q21 4025340224212421 q22 1051984 987 984 Total cold run time: 116036 ms Total hot run time: 34047 ms - Round 2, with runtime_filter_mode=off - q1 5081500850455008 q2 244 327 232 232 q3 2179271422762276 q4 1334178113681368 q5 4441435643984356 q6 227 166 125 125 q7 2015192317581758 q8 2580263225692569 q9 7262724670277027 q10 3072318927412741 q11 565 516 524 516 q12 677 741 605 605 q13 3539389032923292 q14 277 298 263 263 q15 528 485 500 485 q16 449 498 444 444 q17 1132153013881388 q18 7856755274377437 q19 795 812 946 812 q20 1978208318491849 q21 5040482547854785 q22 1092103110221022 Total cold run time: 52363 ms Total hot run time: 50358 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
hello-stephen commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2871940059 # Cloud UT Coverage Report Increment line coverage `85.24% (231/271)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/d07492893fc4e59d788d2a9da5397b310ade123c_d07492893fc4e59d788d2a9da5397b310ade123c_cloud/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/d07492893fc4e59d788d2a9da5397b310ade123c_d07492893fc4e59d788d2a9da5397b310ade123c_cloud/report/index.html) | Category | Coverage | |---|| | Function Coverage | 83.23% (1107/1330) | | Line Coverage | 66.49% (18894/28416) | | Region Coverage | 66.05% (9385/14208) | | Branch Coverage | 55.88% (5066/9066) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
mymeiyi commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2871835902 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
mymeiyi commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2870523029 run cloud_p0 external -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2866193153 ClickBench: Total hot run time: 29.76 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit eae660d14b678305d893b551d65fefc19c359118, data reload: false query1 0.040.030.03 query2 0.150.120.11 query3 0.340.200.20 query4 1.590.200.20 query5 0.600.600.61 query6 1.160.710.71 query7 0.020.010.02 query8 0.050.050.05 query9 0.620.520.51 query10 0.590.600.57 query11 0.250.130.13 query12 0.250.130.13 query13 0.640.620.61 query14 0.780.820.83 query15 0.960.860.86 query16 0.360.390.37 query17 1.041.011.05 query18 0.180.180.19 query19 2.031.821.85 query20 0.010.020.01 query21 15.39 0.960.66 query22 0.921.080.77 query23 14.69 1.570.75 query24 5.600.540.27 query25 0.170.100.09 query26 0.560.220.18 query27 0.090.080.08 query28 11.00 1.160.56 query29 12.54 4.063.35 query30 0.280.090.07 query31 2.820.640.43 query32 3.240.590.50 query33 3.003.173.10 query34 16.64 5.124.42 query35 4.494.474.43 query36 0.630.510.50 query37 0.190.170.18 query38 0.160.170.15 query39 0.050.040.04 query40 0.200.160.16 query41 0.110.060.05 query42 0.060.050.05 query43 0.060.050.05 Total cold run time: 104.55 s Total hot run time: 29.76 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2866414502 # BE UT Coverage Report Increment line coverage `1.43% (7/489)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/eae660d14b678305d893b551d65fefc19c359118_eae660d14b678305d893b551d65fefc19c359118/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/eae660d14b678305d893b551d65fefc19c359118_eae660d14b678305d893b551d65fefc19c359118/report/index.html) | Category | Coverage | |---|| | Function Coverage | 55.66% (14868/26710) | | Line Coverage | 44.44% (131514/295967) | | Region Coverage | 43.50% (66141/152057) | | Branch Coverage | 38.08% (33874/88948) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2866178036 TPC-DS: Total hot run time: 192156 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit eae660d14b678305d893b551d65fefc19c359118, data reload: false query1 1435110210311031 query2 6194184219131842 query3 11035 463745394539 query4 54351 25479 23046 23046 query5 4927573 458 458 query6 323 201 198 198 query7 4864503 279 279 query8 313 252 241 241 query9 5387255025442544 query10 467 303 267 267 query11 15013 15029 14846 14846 query12 166 110 104 104 query13 1031503 401 401 query14 10191 624961896189 query15 205 191 178 178 query16 7159657 509 509 query17 1066743 582 582 query18 1656407 308 308 query19 191 195 162 162 query20 122 154 120 120 query21 201 125 106 106 query22 4592472345364536 query23 34216 33487 33317 33317 query24 6718246624192419 query25 481 475 403 403 query26 716 274 155 155 query27 2347504 333 333 query28 3099210220902090 query29 596 609 454 454 query30 266 220 192 192 query31 896 883 771 771 query32 74 76 68 68 query33 457 367 327 327 query34 789 862 545 545 query35 788 873 758 758 query36 949 1002913 913 query37 119 102 83 83 query38 4250421341484148 query39 1471146514251425 query40 220 126 112 112 query41 57 57 52 52 query42 125 111 105 105 query43 499 520 474 474 query44 1321812 813 812 query45 180 180 169 169 query46 850 1043638 638 query47 1876190518381838 query48 380 440 304 304 query49 665 512 428 428 query50 668 701 427 427 query51 4168420441834183 query52 110 110 100 100 query53 226 268 183 183 query54 574 613 538 538 query55 85 79 82 79 query56 334 317 293 293 query57 1158121911581158 query58 254 252 262 252 query59 2877286428232823 query60 348 340 304 304 query61 128 128 127 127 query62 737 796 684 684 query63 225 190 192 190 query64 17931078689 689 query65 4357422041704170 query66 697 402 311 311 query67 15835 15520 15453 15453 query68 7024876 495 495 query69 559 308 263 263 query70 1156107511091075 query71 497 317 290 290 query72 5798490048324832 query73 1521626 339 339 query74 9253906085398539 query75 4040321226982698 query76 41841191743 743 query77 704 364 346 346 query78 10150 10125 93789378 query79 2619804 552 552 query80 668 509 432 432 query81 480 247 217 217 query82 437 125 95 95 query83 245 251 233 233 query84 302 107 85 85 query85 768 357 312 312 query86 370 301 275 275 query87 4374440742724272 query88 3274223322262226 query89 393 309 277 277 query90 1793210 211 210 query91 152 143 112 112 query92 71 63 58 58 query93 2039939 570 570 query94 661 414 307 307 query95 365 291 278 278 query96 472 567 271 271 query97 3192318931153115 query98 232 222 202 202 query99 1470139712761276 Total cold run time: 299131 ms Total hot run time: 192156 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
github-actions[bot] commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2866155789 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
github-actions[bot] commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2866155691 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mow) reduce mow delete bitmap count: agg and remove delete bitmap of pre rowsets when cu compaction [doris]
doris-robot commented on PR #49383: URL: https://github.com/apache/doris/pull/49383#issuecomment-2866146129 TPC-H: Total hot run time: 34404 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit eae660d14b678305d893b551d65fefc19c359118, data reload: false -- Round 1 -- q1 26414 518751495149 q2 2100301 204 204 q3 10578 1317704 704 q4 10239 1025517 517 q5 7888247823752375 q6 192 162 130 130 q7 939 745 610 610 q8 9328144112581258 q9 6834522252055205 q10 6896231218831883 q11 501 284 273 273 q12 367 360 210 210 q13 17776 370530583058 q14 234 224 217 217 q15 519 488 476 476 q16 449 421 364 364 q17 644 918 384 384 q18 7520708071027080 q19 1372974 611 611 q20 342 348 231 231 q21 3981333124782478 q22 10611029987 987 Total cold run time: 116174 ms Total hot run time: 34404 ms - Round 2, with runtime_filter_mode=off - q1 5265522452235223 q2 247 328 226 226 q3 2184266622912291 q4 1346183714651465 q5 4460443444284428 q6 227 175 127 127 q7 2009192517721772 q8 2621259825812581 q9 7260704672187046 q10 3024316227172717 q11 579 528 475 475 q12 681 757 598 598 q13 3469387833033303 q14 271 310 263 263 q15 535 494 492 492 q16 450 484 468 468 q17 1209162813681368 q18 7768764874107410 q19 893 894 995 894 q20 1972196718201820 q21 5106473745914591 q22 1107106210381038 Total cold run time: 52683 ms Total hot run time: 50596 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
