Re: [PR] [fix](profile) Fix ExecutionProfile is missing [doris]
zhiqiang- closed pull request #35255: [fix](profile) Fix ExecutionProfile is missing URL: https://github.com/apache/doris/pull/35255 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](profile) Fix ExecutionProfile is missing [doris]
yiguolei commented on code in PR #35255:
URL: https://github.com/apache/doris/pull/35255#discussion_r1611062885
##
fe/fe-core/src/main/java/org/apache/doris/common/util/ProfileManager.java:
##
@@ -178,6 +178,11 @@ public void addExecutionProfile(ExecutionProfile
executionProfile) {
if (queryIdToExecutionProfiles.size() > 2 *
Config.max_query_profile_num) {
List finishOrExpireExecutionProfiles =
Lists.newArrayList();
for (ExecutionProfile tmpProfile :
queryIdToExecutionProfiles.values()) {
+// Query still running, skip them.
+if (tmpProfile.getQueryFinishTime() ==
ExecutionProfile.QUERY_FINISH_TIME_INIT) {
Review Comment:
There maybe many unfinished profile in manager and FE will OOM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](profile) Fix ExecutionProfile is missing [doris]
zhiqiang- commented on PR #35255: URL: https://github.com/apache/doris/pull/35255#issuecomment-2126298605 p0 失败与 pr 无关 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](profile) Fix ExecutionProfile is missing [doris]
doris-robot commented on PR #35255: URL: https://github.com/apache/doris/pull/35255#issuecomment-2126187307 ClickBench: Total hot run time: 30.16 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 4e6bd64bf8c1d33aaa1bb570a1c7dfb2ff5903eb, data reload: false query1 0.040.030.03 query2 0.080.040.04 query3 0.230.050.05 query4 1.690.090.08 query5 0.490.510.50 query6 1.120.730.72 query7 0.020.010.02 query8 0.050.030.04 query9 0.530.480.49 query10 0.550.550.52 query11 0.140.110.12 query12 0.150.120.12 query13 0.590.590.60 query14 0.790.790.78 query15 0.830.820.81 query16 0.370.340.37 query17 0.961.021.04 query18 0.240.250.24 query19 1.811.721.76 query20 0.010.010.01 query21 15.71 0.680.66 query22 4.837.381.45 query23 18.30 1.381.26 query24 1.480.340.22 query25 0.140.080.07 query26 0.260.160.17 query27 0.080.080.07 query28 13.41 1.001.01 query29 13.07 3.303.24 query30 0.250.060.06 query31 2.880.400.39 query32 3.250.480.48 query33 2.882.962.89 query34 17.17 4.474.46 query35 4.544.534.50 query36 0.650.460.46 query37 0.180.150.15 query38 0.150.140.15 query39 0.050.040.04 query40 0.170.160.13 query41 0.100.040.04 query42 0.060.050.05 query43 0.030.040.04 Total cold run time: 110.33 s Total hot run time: 30.16 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](profile) Fix ExecutionProfile is missing [doris]
doris-robot commented on PR #35255: URL: https://github.com/apache/doris/pull/35255#issuecomment-2126183465 TPC-DS: Total hot run time: 169742 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 4e6bd64bf8c1d33aaa1bb570a1c7dfb2ff5903eb, data reload: false query1 911 376 370 370 query2 6453247824432443 query3 6653214 212 212 query4 19136 17170 17365 17170 query5 4187420 431 420 query6 248 156 152 152 query7 4583305 291 291 query8 237 184 185 184 query9 8512237423772374 query10 463 288 270 270 query11 10558 10180 98789878 query12 138 98 89 89 query13 1651361 370 361 query14 10059 756278067562 query15 248 173 175 173 query16 8073283 276 276 query17 1809543 542 542 query18 1893284 275 275 query19 210 173 167 167 query20 100 89 85 85 query21 193 138 130 130 query22 4158392939053905 query23 33709 33042 33148 33042 query24 12004 289228292829 query25 630 353 353 353 query26 1710156 155 155 query27 2978321 332 321 query28 7103202620672026 query29 969 618 613 613 query30 307 177 180 177 query31 957 753 747 747 query32 90 53 53 53 query33 774 272 259 259 query34 1025468 466 466 query35 740 577 596 577 query36 1054892 920 892 query37 285 68 90 68 query38 2866277328032773 query39 875 809 792 792 query40 278 124 122 122 query41 47 43 43 43 query42 103 96 98 96 query43 568 534 548 534 query44 1278725 728 725 query45 179 163 165 163 query46 1085724 705 705 query47 1843175417911754 query48 377 306 303 303 query49 1196374 380 374 query50 773 402 389 389 query51 6950689167436743 query52 101 90 91 90 query53 351 285 286 285 query54 971 425 420 420 query55 77 74 77 74 query56 259 237 241 237 query57 1141104410351035 query58 262 210 210 210 query59 324231753175 query60 282 256 259 256 query61 88 86 84 84 query62 649 458 455 455 query63 307 285 283 283 query64 9802223617091709 query65 3171309431133094 query66 1336326 327 326 query67 15260 14814 14757 14757 query68 4612545 552 545 query69 441 269 263 263 query70 1174104510731045 query71 417 267 277 267 query72 7842563227082708 query73 700 323 322 322 query74 5960567055655565 query75 3429265126292629 query76 28711002923 923 query77 421 271 273 271 query78 10272 972696949694 query79 2569518 517 517 query80 1956439 437 437 query81 521 241 239 239 query82 751 98 95 95 query83 282 177 173 173 query84 269 88 88 88 query85 2087258 265 258 query86 516 308 306 306 query87 3282311930823082 query88 4587234923262326 query89 485 398 384 384 query90 2014189 191 189 query91 123 96 157 96 query92 59 50 49 49 query93 4628525 511 511 query94 1265191 188 188 query95 402 319 312 312 query96 603 265 268 265 query97 3190305129972997 query98 254 217 218 217 query99 1224890 868 868 Total cold run time: 279822 ms Total hot run time: 169742 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [fix](profile) Fix ExecutionProfile is missing [doris]
doris-robot commented on PR #35255: URL: https://github.com/apache/doris/pull/35255#issuecomment-2126176185 TPC-H: Total hot run time: 40083 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 4e6bd64bf8c1d33aaa1bb570a1c7dfb2ff5903eb, data reload: false -- Round 1 -- q1 17957 447044424442 q2 2575201 205 201 q3 10960 126712421242 q4 10552 765 785 765 q5 7497271127082708 q6 224 138 137 137 q7 974 621 616 616 q8 9274205520602055 q9 8810643464526434 q10 8996377237503750 q11 463 252 236 236 q12 451 228 230 228 q13 17834 303129722972 q14 261 226 234 226 q15 516 481 477 477 q16 537 399 381 381 q17 961 642 777 642 q18 8048745373477347 q19 4393152715131513 q20 648 299 313 299 q21 4983398031393139 q22 350 284 273 273 Total cold run time: 117264 ms Total hot run time: 40083 ms - Round 2, with runtime_filter_mode=off - q1 4292418341994183 q2 373 273 269 269 q3 2988274327432743 q4 1863161515811581 q5 5233526052615260 q6 212 125 126 125 q7 2129171617631716 q8 3197332133023302 q9 8305826083198260 q10 3854368836353635 q11 606 505 484 484 q12 760 570 594 570 q13 16258 300830243008 q14 304 255 268 255 q15 516 484 466 466 q16 468 427 432 427 q17 1783148314611461 q18 7587751173737373 q19 1684154315341534 q20 1979179217951792 q21 5044494647334733 q22 579 488 468 468 Total cold run time: 70014 ms Total hot run time: 53645 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](profile) Fix ExecutionProfile is missing [doris]
zhiqiang- commented on PR #35255: URL: https://github.com/apache/doris/pull/35255#issuecomment-2126157806 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](profile) Fix ExecutionProfile is missing [doris]
zhiqiang- commented on PR #35255: URL: https://github.com/apache/doris/pull/35255#issuecomment-2126157703 This fix need to be merged to 2.1 branch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](profile) Fix ExecutionProfile is missing [doris]
doris-robot commented on PR #35255: URL: https://github.com/apache/doris/pull/35255#issuecomment-2126157520 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
