Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
BiteThet merged PR #50863: URL: https://github.com/apache/doris/pull/50863 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
github-actions[bot] commented on PR #50863: URL: https://github.com/apache/doris/pull/50863#issuecomment-2893195940 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
github-actions[bot] commented on PR #50863: URL: https://github.com/apache/doris/pull/50863#issuecomment-2893195845 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
doris-robot commented on PR #50863: URL: https://github.com/apache/doris/pull/50863#issuecomment-2891092524 ClickBench: Total hot run time: 28.76 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit d30d668d2f2fb395cd8b03cc34ba6182398a2b21, data reload: false query1 0.040.040.03 query2 0.130.110.10 query3 0.260.190.19 query4 1.600.190.19 query5 0.450.440.44 query6 1.490.670.67 query7 0.030.010.02 query8 0.040.040.04 query9 0.580.520.52 query10 0.580.580.56 query11 0.150.110.11 query12 0.150.120.12 query13 0.620.600.60 query14 0.780.830.79 query15 0.870.860.85 query16 0.400.380.38 query17 1.001.041.01 query18 0.220.210.21 query19 1.961.851.78 query20 0.010.010.01 query21 15.42 0.890.55 query22 0.741.120.70 query23 14.95 1.380.66 query24 6.621.370.53 query25 0.460.080.08 query26 0.630.170.13 query27 0.050.050.06 query28 9.710.900.45 query29 12.53 4.003.29 query30 0.250.090.08 query31 2.820.590.39 query32 3.230.560.47 query33 3.043.033.05 query34 15.73 5.114.44 query35 4.504.514.54 query36 0.650.500.47 query37 0.090.060.06 query38 0.050.040.03 query39 0.030.020.02 query40 0.170.150.14 query41 0.080.020.02 query42 0.030.030.02 query43 0.040.030.03 Total cold run time: 103.18 s Total hot run time: 28.76 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
doris-robot commented on PR #50863: URL: https://github.com/apache/doris/pull/50863#issuecomment-2891075946 TPC-DS: Total hot run time: 186535 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit d30d668d2f2fb395cd8b03cc34ba6182398a2b21, data reload: false query1 1004475 498 475 query2 6564178418291784 query3 6747224 223 223 query4 26526 23342 23695 23342 query5 4344618 451 451 query6 296 201 193 193 query7 4611491 309 309 query8 306 244 238 238 query9 8660266826882668 query10 495 324 274 274 query11 15357 15118 14889 14889 query12 155 111 104 104 query13 1646550 418 418 query14 8781586562415865 query15 222 195 172 172 query16 7132630 492 492 query17 941 727 587 587 query18 1972416 309 309 query19 195 190 168 168 query20 118 121 118 118 query21 212 128 112 112 query22 4347439942044204 query23 33965 33150 33156 33150 query24 8417243624262426 query25 577 490 422 422 query26 1223277 160 160 query27 2717500 407 407 query28 4335215921432143 query29 764 545 432 432 query30 278 214 192 192 query31 910 874 773 773 query32 74 63 69 63 query33 601 367 324 324 query34 793 870 542 542 query35 792 805 736 736 query36 956 992 866 866 query37 111 100 76 76 query38 4123417339993999 query39 1468139914001399 query40 217 127 110 110 query41 55 56 54 54 query42 119 114 109 109 query43 499 511 484 484 query44 1338849 854 849 query45 175 174 176 174 query46 854 1031642 642 query47 1777180517281728 query48 400 445 328 328 query49 778 513 424 424 query50 660 670 414 414 query51 4097430341194119 query52 118 111 99 99 query53 224 263 191 191 query54 579 591 517 517 query55 89 85 87 85 query56 312 307 314 307 query57 1174113910751075 query58 265 257 288 257 query59 2606274325962596 query60 343 327 320 320 query61 130 149 122 122 query62 789 704 673 673 query63 223 191 185 185 query64 44331038687 687 query65 4305423342594233 query66 1155416 336 336 query67 16120 15683 15405 15405 query68 8044902 547 547 query69 485 313 284 284 query70 1219111511221115 query71 453 320 304 304 query72 5441480547674767 query73 748 691 360 360 query74 9253916687878787 query75 3778318526972697 query76 36431181757 757 query77 797 372 289 289 query78 10054 10146 93369336 query79 2484786 586 586 query80 606 513 446 446 query81 498 257 224 224 query82 506 128 97 97 query83 268 257 233 233 query84 307 103 93 93 query85 816 349 301 301 query86 385 290 281 281 query87 4447446243464346 query88 3748234924082349 query89 373 321 287 287 query90 1881204 203 203 query91 147 146 114 114 query92 84 61 63 61 query93 1693955 590 590 query94 689 406 291 291 query95 369 298 287 287 query96 512 577 286 286 query97 2806276127092709 query98 234 203 213 203 query99 1448138112771277 Total cold run time: 274785 ms Total hot run time: 186535 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
doris-robot commented on PR #50863: URL: https://github.com/apache/doris/pull/50863#issuecomment-2891037009 TPC-H: Total hot run time: 33874 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit d30d668d2f2fb395cd8b03cc34ba6182398a2b21, data reload: false -- Round 1 -- q1 26420 510850295029 q2 2102276 190 190 q3 10514 1228699 699 q4 10237 1008525 525 q5 7708238723552355 q6 183 161 133 133 q7 909 745 618 618 q8 9322149610531053 q9 6856510451555104 q10 6870233318901890 q11 498 295 275 275 q12 359 345 225 225 q13 1 369330563056 q14 232 249 211 211 q15 544 488 488 488 q16 443 439 368 368 q17 602 874 373 373 q18 7610737071597159 q19 1808968 543 543 q20 324 331 228 228 q21 3904316223892389 q22 10721027963 963 Total cold run time: 116294 ms Total hot run time: 33874 ms - Round 2, with runtime_filter_mode=off - q1 5185505650635056 q2 239 323 242 242 q3 2253267323042304 q4 1392182815041504 q5 4542444143504350 q6 216 168 127 127 q7 1983193717791779 q8 2572261825282528 q9 7236712871087108 q10 3015315927282728 q11 577 504 479 479 q12 695 732 654 654 q13 3521395233003300 q14 278 297 263 263 q15 528 483 475 475 q16 432 505 450 450 q17 1140152213841384 q18 7818751074087408 q19 862 892 1012892 q20 2008201418121812 q21 4739435743534353 q22 10761029998 998 Total cold run time: 52307 ms Total hot run time: 50194 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
zclllyybb commented on PR #50863: URL: https://github.com/apache/doris/pull/50863#issuecomment-2890708990 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
hello-stephen commented on PR #50863: URL: https://github.com/apache/doris/pull/50863#issuecomment-2890059589 # BE UT Coverage Report Increment line coverage `18.75% (3/16)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/4e73f7a69f299c2016ec4792cf1ffb9ad473244b_4e73f7a69f299c2016ec4792cf1ffb9ad473244b/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/4e73f7a69f299c2016ec4792cf1ffb9ad473244b_4e73f7a69f299c2016ec4792cf1ffb9ad473244b/report/index.html) | Category | Coverage | |---|| | Function Coverage | 55.80% (14882/26670) | | Line Coverage | 44.60% (131894/295737) | | Region Coverage | 43.72% (66397/151856) | | Branch Coverage | 38.33% (34021/88762) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
doris-robot commented on PR #50863: URL: https://github.com/apache/doris/pull/50863#issuecomment-2889822449 ClickBench: Total hot run time: 28.77 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 4e73f7a69f299c2016ec4792cf1ffb9ad473244b, data reload: false query1 0.030.040.03 query2 0.130.100.11 query3 0.250.200.20 query4 1.590.190.20 query5 0.460.450.43 query6 1.150.660.65 query7 0.020.020.01 query8 0.040.030.04 query9 0.590.510.54 query10 0.580.570.57 query11 0.160.100.11 query12 0.150.120.12 query13 0.620.590.59 query14 0.790.800.81 query15 0.880.860.85 query16 0.390.380.39 query17 1.051.041.04 query18 0.220.210.21 query19 1.881.741.79 query20 0.010.020.01 query21 15.39 0.900.55 query22 0.751.120.71 query23 14.95 1.390.66 query24 7.271.150.58 query25 0.430.300.08 query26 0.590.180.14 query27 0.060.050.04 query28 9.820.920.44 query29 12.61 3.963.26 query30 0.260.090.06 query31 2.810.600.38 query32 3.240.550.49 query33 3.023.013.05 query34 15.54 5.084.50 query35 4.544.534.52 query36 0.660.490.49 query37 0.080.060.06 query38 0.050.040.03 query39 0.030.020.02 query40 0.160.150.13 query41 0.080.020.02 query42 0.030.030.02 query43 0.040.040.02 Total cold run time: 103.4 s Total hot run time: 28.77 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
doris-robot commented on PR #50863: URL: https://github.com/apache/doris/pull/50863#issuecomment-2889810142 TPC-DS: Total hot run time: 193181 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 4e73f7a69f299c2016ec4792cf1ffb9ad473244b, data reload: false query1 1398108010481048 query2 6496186218551855 query3 11041 448243884388 query4 55419 25502 23494 23494 query5 5151541 485 485 query6 361 206 217 206 query7 4918531 291 291 query8 317 260 238 238 query9 5600267226922672 query10 450 322 275 275 query11 15142 15036 14827 14827 query12 159 108 103 103 query13 1056508 388 388 query14 10058 628664456286 query15 208 200 183 183 query16 7078651 471 471 query17 1091705 571 571 query18 1610413 323 323 query19 195 199 172 172 query20 123 122 121 121 query21 211 132 103 103 query22 4399447742924292 query23 34156 33632 33556 33556 query24 6552246824322432 query25 495 487 418 418 query26 701 280 162 162 query27 2220512 357 357 query28 2968219222032192 query29 579 584 454 454 query30 283 222 186 186 query31 843 848 778 778 query32 76 65 64 64 query33 471 367 312 312 query34 797 880 544 544 query35 809 832 765 765 query36 981 1029911 911 query37 118 108 85 85 query38 4204421542124212 query39 1723146814491449 query40 218 128 124 124 query41 68 58 60 58 query42 135 120 114 114 query43 556 510 476 476 query44 1357879 876 876 query45 183 176 174 174 query46 871 1045679 679 query47 1833188918151815 query48 418 479 331 331 query49 699 532 453 453 query50 666 715 420 420 query51 4235424642394239 query52 118 110 103 103 query53 232 259 194 194 query54 609 585 515 515 query55 86 93 84 84 query56 333 334 316 316 query57 1222118211701170 query58 266 258 254 254 query59 2762289826532653 query60 340 331 329 329 query61 126 125 125 125 query62 721 764 682 682 query63 239 192 199 192 query64 14901036713 713 query65 4303421042414210 query66 712 404 304 304 query67 15895 15439 15315 15315 query68 6481885 538 538 query69 546 305 273 273 query70 1246112510901090 query71 456 327 299 299 query72 5722486550814865 query73 1316715 381 381 query74 8915886489928864 query75 3254321226782678 query76 38251200778 778 query77 553 368 286 286 query78 9901994792849284 query79 2232817 582 582 query80 590 536 456 456 query81 483 316 214 214 query82 422 123 98 98 query83 349 257 233 233 query84 298 116 82 82 query85 769 346 305 305 query86 385 337 276 276 query87 4418440943834383 query88 3401241223872387 query89 401 313 289 289 query90 1726212 205 205 query91 140 139 110 110 query92 70 58 59 58 query93 1825928 585 585 query94 711 409 311 311 query95 370 290 294 290 query96 501 578 290 290 query97 2762272026422642 query98 242 211 201 201 query99 1484140212611261 Total cold run time: 296944 ms Total hot run time: 193181 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
doris-robot commented on PR #50863: URL: https://github.com/apache/doris/pull/50863#issuecomment-2889784415 TPC-H: Total hot run time: 33714 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 4e73f7a69f299c2016ec4792cf1ffb9ad473244b, data reload: false -- Round 1 -- q1 26316 507450015001 q2 2070284 191 191 q3 10587 1256722 722 q4 10259 1005516 516 q5 7910237723282328 q6 185 161 133 133 q7 914 729 614 614 q8 9327130310321032 q9 6913509750765076 q10 6855233319231923 q11 490 311 272 272 q12 342 350 217 217 q13 17795 364031023102 q14 243 230 210 210 q15 526 481 481 481 q16 419 422 378 378 q17 596 861 363 363 q18 7516726071047104 q19 22521207552 552 q20 320 327 217 217 q21 3613314323412341 q22 10251004941 941 Total cold run time: 116473 ms Total hot run time: 33714 ms - Round 2, with runtime_filter_mode=off - q1 5271511351055105 q2 233 325 224 224 q3 2148269422742274 q4 1338179814071407 q5 4485439644314396 q6 214 172 127 127 q7 2028191917351735 q8 2578253526782535 q9 7276714470127012 q10 3034321027462746 q11 585 498 499 498 q12 721 747 622 622 q13 3476384333033303 q14 294 314 290 290 q15 539 493 472 472 q16 446 486 432 432 q17 1152161413611361 q18 7712748074547454 q19 801 836 857 836 q20 2016198117931793 q21 4726451143774377 q22 1096106410471047 Total cold run time: 52169 ms Total hot run time: 50046 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
HappenLee commented on code in PR #50863:
URL: https://github.com/apache/doris/pull/50863#discussion_r2094816161
##
fe/fe-core/src/main/java/org/apache/doris/dictionary/Dictionary.java:
##
@@ -297,6 +302,30 @@ public long getSrcVersion() {
return srcVersion;
}
+public void updateLatestInvalidVersion(long value) {
+latestInvalidVersion = Math.max(latestInvalidVersion, value);
Review Comment:
check why the value small than latestInvalidVersion
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
zclllyybb commented on PR #50863: URL: https://github.com/apache/doris/pull/50863#issuecomment-2889588274 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
HappenLee commented on code in PR #50863:
URL: https://github.com/apache/doris/pull/50863#discussion_r2094816371
##
fe/fe-core/src/main/java/org/apache/doris/dictionary/Dictionary.java:
##
@@ -297,6 +302,30 @@ public long getSrcVersion() {
return srcVersion;
}
+public void updateLatestInvalidVersion(long value) {
+latestInvalidVersion = Math.max(latestInvalidVersion, value);
+}
+
+/**
+ * if has latestInvalidVersion and the base table's data not changed, we
can skip update.
+ */
+public boolean baseDataMaybeValid() {
Review Comment:
checkBaseDataValid
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
HappenLee commented on code in PR #50863:
URL: https://github.com/apache/doris/pull/50863#discussion_r2094809863
##
fe/fe-core/src/main/java/org/apache/doris/dictionary/DictionaryManager.java:
##
@@ -438,6 +438,12 @@ public void dataLoad(ConnectContext ctx, Dictionary
dictionary, boolean adaptive
if (ctx.getState().getErrorCode() != null &&
ctx.getState().getErrorMessage() != null) {
dictionary.trySetStatus(oldStatus);
dictionary.setLastUpdateResult(ctx.getState().getErrorMessage());
+// for must failed refresh, we can skip it at next time. this mark
is tricky but we have to do now.
+if
(ctx.getState().getErrorMessage().contains("[INVALID_DICT_MARK]")) {
+LOG.warn("Dictionary {} load failed with src version {}, mark
it invalid", dictionary.getName(),
+
ctx.getStatementContext().getDictionaryUsedSrcVersion());
+
dictionary.updateLatestInvalidVersion(ctx.getStatementContext().getDictionaryUsedSrcVersion());
Review Comment:
the `tDictionaryUsedSrcVersion` is table `version`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
hello-stephen commented on PR #50863: URL: https://github.com/apache/doris/pull/50863#issuecomment-288733 # BE Regression && UT Coverage Report Increment line coverage `64.71% (11/17)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/50863_a7db5058eb90365289e7222b974ff00092372ea9_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/50863_a7db5058eb90365289e7222b974ff00092372ea9_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 79.45% (20840/26229) | | Line Coverage | 72.68% (214703/295404) | | Region Coverage | 70.84% (126311/178310) | | Branch Coverage | 64.57% (65438/101340) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
doris-robot commented on PR #50863: URL: https://github.com/apache/doris/pull/50863#issuecomment-2887028579 # BE UT Coverage Report Increment line coverage `18.75% (3/16)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/a7db5058eb90365289e7222b974ff00092372ea9_a7db5058eb90365289e7222b974ff00092372ea9/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/a7db5058eb90365289e7222b974ff00092372ea9_a7db5058eb90365289e7222b974ff00092372ea9/report/index.html) | Category | Coverage | |---|| | Function Coverage | 55.87% (14883/26641) | | Line Coverage | 44.65% (131887/295392) | | Region Coverage | 43.75% (66397/151750) | | Branch Coverage | 38.35% (34015/88706) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
hello-stephen commented on PR #50863: URL: https://github.com/apache/doris/pull/50863#issuecomment-2877411164 # BE Regression && UT Coverage Report Increment line coverage `45.45% (10/22)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/50863_0f613922410fcba3054987a505c3ec2c879c49a6_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/50863_0f613922410fcba3054987a505c3ec2c879c49a6_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 79.32% (20832/26264) | | Line Coverage | 72.51% (214282/295520) | | Region Coverage | 70.70% (126105/178354) | | Branch Coverage | 64.45% (65335/101372) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
doris-robot commented on PR #50863: URL: https://github.com/apache/doris/pull/50863#issuecomment-2877002475 ClickBench: Total hot run time: 29.29 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 0f613922410fcba3054987a505c3ec2c879c49a6, data reload: false query1 0.030.040.03 query2 0.130.110.12 query3 0.260.190.20 query4 1.600.190.19 query5 0.600.580.58 query6 1.170.720.72 query7 0.020.020.02 query8 0.040.040.04 query9 0.560.520.51 query10 0.560.580.56 query11 0.160.110.10 query12 0.160.110.12 query13 0.610.590.61 query14 0.790.810.80 query15 0.880.850.85 query16 0.400.380.38 query17 1.011.011.05 query18 0.220.200.21 query19 1.931.841.76 query20 0.020.010.01 query21 15.41 0.880.54 query22 0.771.010.75 query23 14.96 1.290.61 query24 6.801.970.93 query25 0.490.280.07 query26 0.680.160.14 query27 0.050.060.05 query28 9.770.830.45 query29 12.53 3.953.26 query30 0.240.080.06 query31 2.840.600.40 query32 3.230.550.46 query33 3.183.113.01 query34 15.81 5.064.52 query35 4.504.514.47 query36 0.670.500.48 query37 0.080.060.06 query38 0.060.050.04 query39 0.030.030.02 query40 0.170.130.14 query41 0.080.030.03 query42 0.030.030.02 query43 0.040.040.03 Total cold run time: 103.57 s Total hot run time: 29.29 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
doris-robot commented on PR #50863: URL: https://github.com/apache/doris/pull/50863#issuecomment-2877255829 # BE UT Coverage Report Increment line coverage `0.00% (0/22)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/0f613922410fcba3054987a505c3ec2c879c49a6_0f613922410fcba3054987a505c3ec2c879c49a6/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/0f613922410fcba3054987a505c3ec2c879c49a6_0f613922410fcba3054987a505c3ec2c879c49a6/report/index.html) | Category | Coverage | |---|| | Function Coverage | 55.78% (14892/26698) | | Line Coverage | 44.58% (131764/295584) | | Region Coverage | 43.63% (66258/151849) | | Branch Coverage | 38.26% (33956/88762) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
doris-robot commented on PR #50863: URL: https://github.com/apache/doris/pull/50863#issuecomment-2876986006 TPC-DS: Total hot run time: 186387 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 0f613922410fcba3054987a505c3ec2c879c49a6, data reload: false query1 1002475 515 475 query2 6557190318821882 query3 6766218 218 218 query4 25726 24056 22907 22907 query5 4335627 428 428 query6 291 197 194 194 query7 4621509 288 288 query8 289 248 225 225 query9 8627262326482623 query10 488 339 265 265 query11 15787 15048 14827 14827 query12 160 108 112 108 query13 1656548 423 423 query14 9024616660786078 query15 216 192 173 173 query16 7166653 487 487 query17 1214738 606 606 query18 1982436 313 313 query19 199 188 165 165 query20 120 120 122 120 query21 272 126 109 109 query22 4136454043344334 query23 34045 33186 33052 33052 query24 8445240623632363 query25 545 461 389 389 query26 1249271 154 154 query27 2739521 343 343 query28 4289210821062106 query29 756 563 421 421 query30 279 221 191 191 query31 935 835 750 750 query32 77 63 63 63 query33 562 354 325 325 query34 816 877 510 510 query35 772 795 728 728 query36 914 969 896 896 query37 116 103 80 80 query38 4185413340444044 query39 1498137814281378 query40 209 115 105 105 query41 57 56 54 54 query42 129 108 107 107 query43 503 515 488 488 query44 1349819 817 817 query45 184 169 169 169 query46 891 1056628 628 query47 1751179317311731 query48 384 436 317 317 query49 769 524 432 432 query50 686 687 395 395 query51 4086413741394137 query52 121 106 104 104 query53 232 245 182 182 query54 615 577 518 518 query55 85 81 80 80 query56 290 299 302 299 query57 1125113810821082 query58 258 249 251 249 query59 2550278426512651 query60 321 309 297 297 query61 125 131 123 123 query62 788 735 650 650 query63 229 189 190 189 query64 4462997 661 661 query65 4333419642444196 query66 1155415 311 311 query67 15809 15479 15484 15479 query68 7840912 524 524 query69 472 307 273 273 query70 1170104011051040 query71 472 318 294 294 query72 5764478749034787 query73 716 711 352 352 query74 8828906487218721 query75 3837326427622762 query76 36471245765 765 query77 778 375 282 282 query78 10021 10200 94619461 query79 2000855 576 576 query80 614 524 456 456 query81 469 256 224 224 query82 434 128 97 97 query83 255 253 239 239 query84 239 109 88 88 query85 792 361 307 307 query86 339 320 297 297 query87 4424435443644354 query88 3305231222882288 query89 405 309 299 299 query90 2139211 207 207 query91 143 158 116 116 query92 81 61 56 56 query93 1425982 589 589 query94 689 435 298 298 query95 385 290 277 277 query96 502 626 288 288 query97 3144327130833083 query98 240 210 203 203 query99 1713143412921292 Total cold run time: 273634 ms Total hot run time: 186387 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
doris-robot commented on PR #50863: URL: https://github.com/apache/doris/pull/50863#issuecomment-2876948662 TPC-H: Total hot run time: 33795 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 0f613922410fcba3054987a505c3ec2c879c49a6, data reload: false -- Round 1 -- q1 26142 502450655024 q2 2060275 192 192 q3 10390 1234682 682 q4 10221 1010525 525 q5 7548241023792379 q6 182 166 132 132 q7 908 763 602 602 q8 9334128311401140 q9 6722519951045104 q10 6830229218921892 q11 480 277 268 268 q12 338 347 210 210 q13 17756 365230703070 q14 228 231 219 219 q15 550 500 503 500 q16 427 445 368 368 q17 573 858 357 357 q18 7885714469916991 q19 1085938 564 564 q20 326 338 225 225 q21 4088268023952395 q22 10521023956 956 Total cold run time: 115125 ms Total hot run time: 33795 ms - Round 2, with runtime_filter_mode=off - q1 5067505852715058 q2 237 330 227 227 q3 2151265622842284 q4 1326173513221322 q5 4485444843984398 q6 229 173 135 135 q7 2025190317341734 q8 2578259825192519 q9 7166710171177101 q10 2993314727472747 q11 582 508 491 491 q12 696 772 612 612 q13 3443377632713271 q14 290 295 283 283 q15 540 504 499 499 q16 435 482 436 436 q17 1123154113911391 q18 7697760074187418 q19 805 865 865 q20 1949205918941894 q21 5180464346214621 q22 10241022987 987 Total cold run time: 52021 ms Total hot run time: 50293 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
zclllyybb opened a new pull request, #50863: URL: https://github.com/apache/doris/pull/50863 ### What problem does this PR solve? Issue Number: close #xxx Related PR: #xxx Problem Summary: avoid too many warn log like: ``` 2025-05-12 21:18:55,390 WARN (mysql-nio-pool-11|212) [StmtExecutor.execute():627] Analyze failed. stmt[290, 18cb1394557f4aeb-b2efd4360e8ee1c5] org.apache.doris.common.NereidsException: errCode = 2, detailMessage = (10.16.10.2)[INVALID_ARGUMENT]The CIDR has duplicate data in IpAddressDictionary at org.apache.doris.qe.StmtExecutor.executeByNereids(StmtExecutor.java:802) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:612) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.qe.StmtExecutor.queryRetry(StmtExecutor.java:574) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:559) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:349) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:249) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:233) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:261) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:444) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?] at java.lang.Thread.run(Thread.java:842) ~[?:?] Caused by: org.apache.doris.common.AnalysisException: errCode = 2, detailMessage = (10.16.10.2)[INVALID_ARGUMENT]The CIDR has duplicate data in IpAddressDictionary ... 13 more Caused by: java.lang.RuntimeException: errCode = 2, detailMessage = (10.16.10.2)[INVALID_ARGUMENT]The CIDR has duplicate data in IpAddressDictionary at org.apache.doris.dictionary.DictionaryManager.dataLoad(DictionaryManager.java:446) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.nereids.trees.plans.commands.refresh.RefreshDictionaryCommand.run(RefreshDictionaryCommand.java:49) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.qe.StmtExecutor.executeByNereids(StmtExecutor.java:771) ~[doris-fe.jar:1.2-SNAPSHOT] ... 12 more ``` ### Release note None ### Check List (For Author) - Test - [x] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason - Behavior changed: - [ ] No. - [x] Yes. If a dictionary's base table's data is known that is invalid. not trigger refreshing it. - Does this need documentation? - [x] No. - [ ] Yes. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
hello-stephen commented on PR #50863: URL: https://github.com/apache/doris/pull/50863#issuecomment-2876874458 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
zclllyybb commented on PR #50863: URL: https://github.com/apache/doris/pull/50863#issuecomment-2876874474 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
doris-robot commented on PR #50822: URL: https://github.com/apache/doris/pull/50822#issuecomment-2872934872 ClickBench: Total hot run time: 28.97 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit e8d5e13b222f483bfc57e08034884c5ae257f8c5, data reload: false query1 0.040.040.03 query2 0.130.110.11 query3 0.250.200.19 query4 1.590.180.20 query5 0.590.610.60 query6 1.200.730.71 query7 0.030.010.02 query8 0.040.040.04 query9 0.570.530.51 query10 0.560.580.55 query11 0.160.100.11 query12 0.150.120.12 query13 0.610.600.59 query14 0.780.800.80 query15 0.890.860.85 query16 0.380.370.38 query17 1.061.041.04 query18 0.230.210.21 query19 1.991.831.79 query20 0.010.010.02 query21 15.42 0.890.56 query22 0.741.270.64 query23 14.87 1.430.62 query24 7.601.450.55 query25 0.480.190.07 query26 0.500.170.15 query27 0.050.050.05 query28 8.880.900.47 query29 12.76 4.053.33 query30 0.250.090.06 query31 2.830.610.38 query32 3.230.550.49 query33 3.033.053.09 query34 15.68 5.154.45 query35 4.484.554.50 query36 0.670.500.48 query37 0.080.060.06 query38 0.050.040.03 query39 0.030.030.02 query40 0.170.140.13 query41 0.070.020.02 query42 0.030.020.02 query43 0.040.030.03 Total cold run time: 103.2 s Total hot run time: 28.97 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
doris-robot commented on PR #50822: URL: https://github.com/apache/doris/pull/50822#issuecomment-2872915707 TPC-DS: Total hot run time: 193580 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit e8d5e13b222f483bfc57e08034884c5ae257f8c5, data reload: false query1 1401108010551055 query2 6353183418181818 query3 11064 455045024502 query4 54546 25784 23018 23018 query5 5045599 466 466 query6 348 207 203 203 query7 4868515 290 290 query8 324 264 250 250 query9 5268265426342634 query10 450 330 284 284 query11 15052 15064 15388 15064 query12 176 118 105 105 query13 1057528 440 440 query14 10332 645064156415 query15 212 226 182 182 query16 7094665 487 487 query17 1055757 607 607 query18 1595407 306 306 query19 194 191 176 176 query20 133 130 124 124 query21 203 135 108 108 query22 4304441243384338 query23 34536 33587 33492 33492 query24 6521242624042404 query25 452 504 406 406 query26 701 276 148 148 query27 2230523 344 344 query28 2912216621492149 query29 585 578 436 436 query30 277 224 197 197 query31 889 832 765 765 query32 73 65 61 61 query33 449 366 321 321 query34 788 865 543 543 query35 784 849 755 755 query36 947 1016898 898 query37 122 101 79 79 query38 4197426042054205 query39 1539143414531434 query40 218 127 117 117 query41 62 58 58 58 query42 132 116 119 116 query43 518 527 486 486 query44 1346838 831 831 query45 184 181 177 177 query46 865 1038650 650 query47 1867187517911791 query48 426 439 322 322 query49 708 566 468 468 query50 677 703 413 413 query51 4264435242134213 query52 116 106 102 102 query53 241 276 191 191 query54 604 592 530 530 query55 92 86 88 86 query56 340 319 308 308 query57 1176126811891189 query58 270 260 257 257 query59 2635283226662666 query60 344 346 322 322 query61 132 133 125 125 query62 749 772 696 696 query63 234 195 204 195 query64 18951044799 799 query65 4384424842214221 query66 745 399 299 299 query67 15984 15614 15449 15449 query68 7171891 520 520 query69 546 307 278 278 query70 1222114210521052 query71 508 338 317 317 query72 5828484249374842 query73 1515698 358 358 query74 9295914187898789 query75 4247326027122712 query76 42101213770 770 query77 763 392 296 296 query78 10086 10196 93919391 query79 2236867 575 575 query80 588 490 446 446 query81 483 248 216 216 query82 445 125 97 97 query83 258 255 237 237 query84 293 100 82 82 query85 775 348 314 314 query86 359 302 275 275 query87 4485443444304430 query88 3868230322792279 query89 414 324 294 294 query90 1812212 210 210 query91 153 145 113 113 query92 75 61 58 58 query93 1793914 580 580 query94 677 419 300 300 query95 380 289 284 284 query96 496 574 280 280 query97 3154319331043104 query98 230 220 206 206 query99 1418141012891289 Total cold run time: 300067 ms Total hot run time: 193580 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
doris-robot commented on PR #50822: URL: https://github.com/apache/doris/pull/50822#issuecomment-2872865178 TPC-H: Total hot run time: 33758 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit e8d5e13b222f483bfc57e08034884c5ae257f8c5, data reload: false -- Round 1 -- q1 25913 502549984998 q2 2068335 202 202 q3 10312 1245690 690 q4 10227 988 523 523 q5 7528225123862251 q6 177 165 132 132 q7 914 768 621 621 q8 9311129210671067 q9 6927515950645064 q10 6868229119081908 q11 504 297 298 297 q12 348 345 209 209 q13 17807 368630463046 q14 232 226 210 210 q15 521 493 483 483 q16 419 424 379 379 q17 636 853 394 394 q18 7509714771007100 q19 2072959 584 584 q20 341 355 228 228 q21 3860334823962396 q22 1021990 976 976 Total cold run time: 115515 ms Total hot run time: 33758 ms - Round 2, with runtime_filter_mode=off - q1 5124524649924992 q2 240 327 228 228 q3 2166266223242324 q4 1350177913371337 q5 4451441843844384 q6 225 170 125 125 q7 2015191117901790 q8 2560252426092524 q9 7232721670257025 q10 3019322127682768 q11 594 515 507 507 q12 687 750 676 676 q13 3460390832383238 q14 309 303 293 293 q15 526 474 462 462 q16 440 469 471 469 q17 1144150114111411 q18 7770744474307430 q19 874 856 892 856 q20 2011203217931793 q21 4978471747444717 q22 1118108110181018 Total cold run time: 52293 ms Total hot run time: 50367 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
zclllyybb closed pull request #50822: [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid URL: https://github.com/apache/doris/pull/50822 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
zclllyybb commented on PR #50822: URL: https://github.com/apache/doris/pull/50822#issuecomment-2872671300 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
zclllyybb commented on PR #50822: URL: https://github.com/apache/doris/pull/50822#issuecomment-2872638971 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
zclllyybb opened a new pull request, #50822: URL: https://github.com/apache/doris/pull/50822 ### What problem does this PR solve? Issue Number: close #xxx Related PR: #xxx Problem Summary: avoid too many warn log like: ``` 2025-05-12 21:18:55,390 WARN (mysql-nio-pool-11|212) [StmtExecutor.execute():627] Analyze failed. stmt[290, 18cb1394557f4aeb-b2efd4360e8ee1c5] org.apache.doris.common.NereidsException: errCode = 2, detailMessage = (10.16.10.2)[INVALID_ARGUMENT]The CIDR has duplicate data in IpAddressDictionary at org.apache.doris.qe.StmtExecutor.executeByNereids(StmtExecutor.java:802) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:612) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.qe.StmtExecutor.queryRetry(StmtExecutor.java:574) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:559) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:349) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:249) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:233) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:261) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:444) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?] at java.lang.Thread.run(Thread.java:842) ~[?:?] Caused by: org.apache.doris.common.AnalysisException: errCode = 2, detailMessage = (10.16.10.2)[INVALID_ARGUMENT]The CIDR has duplicate data in IpAddressDictionary ... 13 more Caused by: java.lang.RuntimeException: errCode = 2, detailMessage = (10.16.10.2)[INVALID_ARGUMENT]The CIDR has duplicate data in IpAddressDictionary at org.apache.doris.dictionary.DictionaryManager.dataLoad(DictionaryManager.java:446) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.nereids.trees.plans.commands.refresh.RefreshDictionaryCommand.run(RefreshDictionaryCommand.java:49) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.qe.StmtExecutor.executeByNereids(StmtExecutor.java:771) ~[doris-fe.jar:1.2-SNAPSHOT] ... 12 more ``` ### Release note None ### Check List (For Author) - Test - [x] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason - Behavior changed: - [ ] No. - [x] Yes. If a dictionary's base table's data is known that is invalid. not trigger refreshing it. - Does this need documentation? - [x] No. - [ ] Yes. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](dictionary) Skip auto refresh dictionary if we know source data is invalid [doris]
Thearas commented on PR #50822: URL: https://github.com/apache/doris/pull/50822#issuecomment-2872638834 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
