Re: [PR] [fix](statistics)Fix sample analyze agg/mor table doesn't remove new partition flag bug. [doris]
Jibing-Li merged PR #51116: URL: https://github.com/apache/doris/pull/51116 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](statistics)Fix sample analyze agg/mor table doesn't remove new partition flag bug. [doris]
github-actions[bot] commented on PR #51116: URL: https://github.com/apache/doris/pull/51116#issuecomment-2903104256 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](statistics)Fix sample analyze agg/mor table doesn't remove new partition flag bug. [doris]
github-actions[bot] commented on PR #51116: URL: https://github.com/apache/doris/pull/51116#issuecomment-2903104292 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](statistics)Fix sample analyze agg/mor table doesn't remove new partition flag bug. [doris]
Jibing-Li commented on PR #51116: URL: https://github.com/apache/doris/pull/51116#issuecomment-2901677850 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](statistics)Fix sample analyze agg/mor table doesn't remove new partition flag bug. [doris]
doris-robot commented on PR #51116: URL: https://github.com/apache/doris/pull/51116#issuecomment-2901004002 ClickBench: Total hot run time: 28.56 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit e7c18b585e936104fde636f05f720488c4ee7937, data reload: false query1 0.040.040.03 query2 0.120.110.10 query3 0.240.200.19 query4 1.600.180.18 query5 0.510.450.44 query6 1.580.650.66 query7 0.020.020.01 query8 0.040.040.04 query9 0.580.510.50 query10 0.580.560.56 query11 0.150.110.10 query12 0.140.110.11 query13 0.610.600.61 query14 0.790.810.80 query15 0.860.860.85 query16 0.380.380.38 query17 1.000.991.06 query18 0.220.210.20 query19 1.931.811.79 query20 0.010.010.01 query21 15.47 0.900.54 query22 0.751.060.69 query23 15.04 1.370.66 query24 7.880.930.30 query25 0.420.150.12 query26 0.640.170.15 query27 0.060.050.04 query28 9.260.920.46 query29 12.57 3.953.30 query30 0.250.090.07 query31 2.820.600.39 query32 3.230.560.47 query33 3.013.053.11 query34 15.83 5.034.49 query35 4.474.504.51 query36 0.660.490.47 query37 0.080.060.07 query38 0.050.040.04 query39 0.030.020.02 query40 0.170.150.13 query41 0.080.030.03 query42 0.040.020.02 query43 0.030.030.03 Total cold run time: 104.24 s Total hot run time: 28.56 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](statistics)Fix sample analyze agg/mor table doesn't remove new partition flag bug. [doris]
doris-robot commented on PR #51116: URL: https://github.com/apache/doris/pull/51116#issuecomment-2900990676 TPC-DS: Total hot run time: 185761 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit e7c18b585e936104fde636f05f720488c4ee7937, data reload: false query1 1026474 505 474 query2 6567185518201820 query3 6738238 217 217 query4 26084 23793 23125 23125 query5 4315602 472 472 query6 326 215 197 197 query7 4624497 284 284 query8 293 250 240 240 query9 8601261326162613 query10 497 323 274 274 query11 15645 15116 14917 14917 query12 158 114 108 108 query13 1664537 404 404 query14 9433638364076383 query15 213 190 170 170 query16 7297642 467 467 query17 1191721 587 587 query18 1981396 316 316 query19 197 192 163 163 query20 123 123 123 123 query21 213 126 113 113 query22 4116414140284028 query23 33896 33065 33075 33065 query24 8452236623872366 query25 535 483 394 394 query26 1228265 150 150 query27 2747492 339 339 query28 4353213421162116 query29 749 544 447 447 query30 278 219 187 187 query31 942 866 751 751 query32 78 67 65 65 query33 562 406 313 313 query34 795 851 505 505 query35 799 815 724 724 query36 937 976 902 902 query37 112 102 77 77 query38 4058412340754075 query39 1458140714211407 query40 211 118 104 104 query41 59 51 52 51 query42 122 110 104 104 query43 494 499 467 467 query44 1301809 810 809 query45 179 174 167 167 query46 834 1018653 653 query47 1761180817311731 query48 394 419 305 305 query49 754 509 428 428 query50 654 689 395 395 query51 4215412640914091 query52 104 106 95 95 query53 217 251 186 186 query54 585 567 513 513 query55 84 80 81 80 query56 296 298 300 298 query57 1172115211151115 query58 260 253 248 248 query59 2622271426222622 query60 329 320 307 307 query61 125 127 124 124 query62 805 699 652 652 query63 225 185 188 185 query64 4339996 659 659 query65 4359421342144213 query66 1137407 315 315 query67 15858 15679 15303 15303 query68 8269877 515 515 query69 487 308 264 264 query70 1242114710661066 query71 494 326 302 302 query72 5631469346654665 query73 693 593 345 345 query74 9334907188178817 query75 3927318826862686 query76 37551202739 739 query77 791 361 286 286 query78 10004 10099 92979297 query79 2896819 584 584 query80 636 519 442 442 query81 469 263 220 220 query82 464 127 95 95 query83 284 249 233 233 query84 283 105 91 91 query85 797 354 313 313 query86 339 319 296 296 query87 4488443543054305 query88 3258238123152315 query89 419 321 292 292 query90 1957229 214 214 query91 153 137 114 114 query92 76 62 56 56 query93 1601942 581 581 query94 673 426 306 306 query95 381 301 286 286 query96 502 571 287 287 query97 2704276226562656 query98 230 213 210 210 query99 1461145412851285 Total cold run time: 275474 ms Total hot run time: 185761 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [fix](statistics)Fix sample analyze agg/mor table doesn't remove new partition flag bug. [doris]
doris-robot commented on PR #51116: URL: https://github.com/apache/doris/pull/51116#issuecomment-2900952653 TPC-H: Total hot run time: 34078 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit e7c18b585e936104fde636f05f720488c4ee7937, data reload: false -- Round 1 -- q1 26049 509750525052 q2 2067282 184 184 q3 10441 1258720 720 q4 10236 1001535 535 q5 7675244423182318 q6 190 174 137 137 q7 924 759 606 606 q8 9323130011431143 q9 6792514951165116 q10 6817231018961896 q11 480 293 274 274 q12 349 353 214 214 q13 17773 367731433143 q14 234 224 213 213 q15 548 510 506 506 q16 409 427 365 365 q17 590 858 361 361 q18 7831718370877087 q19 1212930 570 570 q20 340 329 223 223 q21 3920320624422442 q22 10621022973 973 Total cold run time: 115262 ms Total hot run time: 34078 ms - Round 2, with runtime_filter_mode=off - q1 5467506653055066 q2 247 333 228 228 q3 2186264622642264 q4 1385180014111411 q5 4576446743994399 q6 211 168 129 129 q7 1966189417431743 q8 2634253825072507 q9 7197715073447150 q10 2980319127502750 q11 594 522 509 509 q12 697 780 658 658 q13 3525389032823282 q14 280 301 260 260 q15 551 483 494 483 q16 433 479 446 446 q17 1164158813541354 q18 7590760575597559 q19 799 910 978 910 q20 1969204319081908 q21 4763426143494261 q22 10881023989 989 Total cold run time: 52302 ms Total hot run time: 50266 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](statistics)Fix sample analyze agg/mor table doesn't remove new partition flag bug. [doris]
Jibing-Li commented on code in PR #51116:
URL: https://github.com/apache/doris/pull/51116#discussion_r2102341922
##
fe/fe-core/src/main/java/org/apache/doris/statistics/TableStatsMeta.java:
##
@@ -193,17 +193,23 @@ public void update(AnalysisInfo analyzedJob, TableIf
tableIf) {
jobType = analyzedJob.jobType;
if (tableIf != null) {
if (tableIf instanceof OlapTable) {
+OlapTable olapTable = (OlapTable) tableIf;
indexesRowCount.putAll(analyzedJob.indexesRowCount);
-clearStaleIndexRowCount((OlapTable) tableIf);
+clearStaleIndexRowCount(olapTable);
+Set> columnIndexPairs =
olapTable.getColumnIndexPairs(
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [fix](statistics)Fix sample analyze agg/mor table doesn't remove new partition flag bug. [doris]
Jibing-Li commented on PR #51116: URL: https://github.com/apache/doris/pull/51116#issuecomment-2900892912 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](statistics)Fix sample analyze agg/mor table doesn't remove new partition flag bug. [doris]
morrySnow commented on code in PR #51116:
URL: https://github.com/apache/doris/pull/51116#discussion_r2101916089
##
fe/fe-core/src/main/java/org/apache/doris/statistics/TableStatsMeta.java:
##
@@ -193,17 +193,23 @@ public void update(AnalysisInfo analyzedJob, TableIf
tableIf) {
jobType = analyzedJob.jobType;
if (tableIf != null) {
if (tableIf instanceof OlapTable) {
+OlapTable olapTable = (OlapTable) tableIf;
indexesRowCount.putAll(analyzedJob.indexesRowCount);
-clearStaleIndexRowCount((OlapTable) tableIf);
+clearStaleIndexRowCount(olapTable);
+Set> columnIndexPairs =
olapTable.getColumnIndexPairs(
Review Comment:
why not get below set used in containsAll directly?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [fix](statistics)Fix sample analyze agg/mor table doesn't remove new partition flag bug. [doris]
doris-robot commented on PR #51116: URL: https://github.com/apache/doris/pull/51116#issuecomment-2897064869 ClickBench: Total hot run time: 29.02 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 038077918b3d67d39b537d204b0683877bb30bad, data reload: false query1 0.040.030.03 query2 0.130.100.11 query3 0.250.190.20 query4 1.600.200.19 query5 0.480.450.45 query6 1.150.670.66 query7 0.020.020.02 query8 0.040.030.04 query9 0.590.510.52 query10 0.560.580.57 query11 0.160.120.11 query12 0.140.110.11 query13 0.600.600.59 query14 0.790.810.80 query15 0.880.890.85 query16 0.380.410.37 query17 1.061.021.06 query18 0.230.210.21 query19 1.941.831.78 query20 0.010.020.01 query21 15.46 0.890.54 query22 0.761.310.69 query23 14.76 1.400.62 query24 6.961.060.77 query25 0.480.090.12 query26 0.640.160.16 query27 0.060.050.05 query28 9.140.940.43 query29 12.56 4.053.35 query30 0.260.090.06 query31 2.830.580.38 query32 3.230.540.48 query33 3.093.123.04 query34 15.68 5.094.47 query35 4.514.534.50 query36 0.660.510.48 query37 0.080.060.06 query38 0.050.040.04 query39 0.040.020.03 query40 0.170.130.12 query41 0.090.030.02 query42 0.030.030.02 query43 0.030.030.03 Total cold run time: 102.62 s Total hot run time: 29.02 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](statistics)Fix sample analyze agg/mor table doesn't remove new partition flag bug. [doris]
doris-robot commented on PR #51116: URL: https://github.com/apache/doris/pull/51116#issuecomment-2897047695 TPC-DS: Total hot run time: 193646 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 038077918b3d67d39b537d204b0683877bb30bad, data reload: false query1 1438109610781078 query2 6201181818271818 query3 10998 455044724472 query4 53261 25460 23769 23769 query5 5255503 447 447 query6 335 194 183 183 query7 4936512 295 295 query8 313 246 224 224 query9 5909263526412635 query10 436 338 269 269 query11 15087 15044 15062 15044 query12 166 117 109 109 query13 1065523 424 424 query14 10081 623162976231 query15 211 221 199 199 query16 6996655 537 537 query17 1113731 612 612 query18 1573416 328 328 query19 204 197 194 194 query20 136 120 115 115 query21 208 127 117 117 query22 4485445543744374 query23 34257 33548 33460 33460 query24 6892239724592397 query25 463 467 418 418 query26 752 274 161 161 query27 2451518 353 353 query28 3454215521462146 query29 586 554 446 446 query30 272 216 188 188 query31 854 880 819 819 query32 73 65 64 64 query33 478 374 304 304 query34 790 846 559 559 query35 793 856 732 732 query36 933 992 894 894 query37 115 98 76 76 query38 4232428241694169 query39 1636145414611454 query40 214 126 108 108 query41 56 54 52 52 query42 127 121 110 110 query43 506 516 478 478 query44 1301849 853 849 query45 183 178 169 169 query46 866 1038672 672 query47 1845186118141814 query48 398 439 333 333 query49 708 500 457 457 query50 652 680 413 413 query51 4332428942934289 query52 114 111 101 101 query53 225 261 189 189 query54 597 591 526 526 query55 91 85 88 85 query56 336 294 306 294 query57 1253119411361136 query58 275 277 260 260 query59 2714278826052605 query60 345 328 318 318 query61 140 123 124 123 query62 737 757 734 734 query63 223 188 198 188 query64 23021004723 723 query65 4323424642344234 query66 753 399 306 306 query67 15993 15524 15464 15464 query68 6954885 513 513 query69 540 310 266 266 query70 1137112111081108 query71 491 320 301 301 query72 5965473948644739 query73 1369646 361 361 query74 8894914789838983 query75 3885317826992699 query76 41991191744 744 query77 613 366 286 286 query78 10198 10162 92959295 query79 2476784 584 584 query80 608 507 499 499 query81 485 261 230 230 query82 436 122 99 99 query83 411 250 234 234 query84 295 112 83 83 query85 787 351 314 314 query86 379 314 286 286 query87 4417451143074307 query88 3609233323262326 query89 406 326 287 287 query90 1786218 210 210 query91 144 141 117 117 query92 78 62 57 57 query93 1898932 578 578 query94 667 416 305 305 query95 377 299 286 286 query96 518 573 286 286 query97 2761276726252625 query98 253 220 208 208 query99 1474142712861286 Total cold run time: 299557 ms Total hot run time: 193646 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [fix](statistics)Fix sample analyze agg/mor table doesn't remove new partition flag bug. [doris]
doris-robot commented on PR #51116: URL: https://github.com/apache/doris/pull/51116#issuecomment-2897007568 TPC-H: Total hot run time: 33699 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 038077918b3d67d39b537d204b0683877bb30bad, data reload: false -- Round 1 -- q1 25991 502049454945 q2 2085297 194 194 q3 10363 1241677 677 q4 10221 1005534 534 q5 7557241022712271 q6 182 165 131 131 q7 931 752 617 617 q8 9312127610941094 q9 6790504351065043 q10 6811229219101910 q11 480 281 277 277 q12 353 347 223 223 q13 17792 363631003100 q14 238 234 218 218 q15 533 486 494 486 q16 406 441 369 369 q17 604 842 358 358 q18 7500721372047204 q19 1255934 519 519 q20 340 335 224 224 q21 3732260523562356 q22 10451014949 949 Total cold run time: 114521 ms Total hot run time: 33699 ms - Round 2, with runtime_filter_mode=off - q1 5041501650325016 q2 240 331 230 230 q3 2121261022832283 q4 1367175313791379 q5 4289430944074309 q6 217 173 125 125 q7 1975192317871787 q8 2586264825682568 q9 7161723070517051 q10 3003313727622762 q11 582 538 495 495 q12 683 762 620 620 q13 3476385732333233 q14 272 305 278 278 q15 511 477 474 474 q16 459 493 456 456 q17 1149158913521352 q18 7781757474447444 q19 795 756 937 756 q20 1967203818961896 q21 4798445144404440 q22 1078105310001000 Total cold run time: 51551 ms Total hot run time: 49954 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](statistics)Fix sample analyze agg/mor table doesn't remove new partition flag bug. [doris]
Jibing-Li commented on PR #51116: URL: https://github.com/apache/doris/pull/51116#issuecomment-2896954095 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](statistics)Fix sample analyze agg/mor table doesn't remove new partition flag bug. [doris]
Thearas commented on PR #51116: URL: https://github.com/apache/doris/pull/51116#issuecomment-2896948679 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
