Re: [PR] [improvement](parser) Assign query organization to a single owner [doris]
morrySnow merged PR #67429: URL: https://github.com/apache/doris/pull/67429 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [improvement](parser) Assign query organization to a single owner [doris]
hello-stephen commented on PR #67429: URL: https://github.com/apache/doris/pull/67429#issuecomment-5535323769 ClickBench: Total hot run time: 14.71 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 60d04ae69774cd63a9de1e9edaf6521c2ddb116e, data reload: false query1 0.010.000.00 query2 0.070.030.03 query3 0.250.120.11 query4 1.620.090.10 query5 0.190.150.16 query6 1.290.690.67 query7 0.030.010.00 query8 0.050.030.03 query9 0.290.210.21 query10 0.360.350.34 query11 0.160.120.11 query12 0.150.120.13 query13 0.320.300.31 query14 0.460.440.46 query15 0.360.350.35 query16 0.210.230.21 query17 0.690.720.68 query18 0.180.170.17 query19 1.211.181.24 query20 0.020.010.01 query21 15.44 0.160.11 query22 5.100.040.04 query23 16.17 0.250.10 query24 3.040.300.28 query25 0.110.040.03 query26 0.860.150.13 query27 0.030.030.02 query28 3.640.550.28 query29 12.49 3.232.59 query30 0.250.110.12 query31 2.750.370.17 query32 3.520.310.23 query33 1.371.411.47 query34 15.36 2.131.77 query35 1.771.731.75 query36 0.450.300.29 query37 0.060.040.04 query38 0.040.020.02 query39 0.030.030.02 query40 0.110.080.08 query41 0.090.020.03 query42 0.030.020.03 query43 0.030.020.03 Total cold run time: 90.66 s Total hot run time: 14.71 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [improvement](parser) Assign query organization to a single owner [doris]
hello-stephen commented on PR #67429: URL: https://github.com/apache/doris/pull/67429#issuecomment-5535249832 TPC-H: Total hot run time: 16941 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 60d04ae69774cd63a9de1e9edaf6521c2ddb116e, data reload: false -- Round 1 -- q1 17563 308730663066 q2 2093266 225 225 q3 10227 906 519 519 q4 4673249 205 205 q5 7675550 381 381 q6 141 125 92 92 q7 536 511 378 378 q8 9233899 955 899 q9 3422238723832383 q10 6502823 695 695 q11 384 202 181 181 q12 606 260 202 202 q13 18131 154311551155 q14 164 152 141 141 q15 q16 426 397 364 364 q17 1320896 845 845 q18 3139223122262226 q19 1259896 792 792 q20 384 288 207 207 q21 5584176218551762 q22 325 271 223 223 Total cold run time: 93787 ms Total hot run time: 16941 ms - Round 2, with runtime_filter_mode=off - q1 3408336533503350 q2 511 390 372 372 q3 2191228921412141 q4 11841165890 890 q5 2167211621062106 q6 166 120 87 87 q7 1030908 823 823 q8 1586139814451398 q9 3146311631423116 q10 1849177516051605 q11 352 268 253 253 q12 451 434 347 347 q13 1489154011571157 q14 173 175 180 175 q15 q16 397 403 356 356 q17 3618328732203220 q18 4794439247294392 q19 840 878 876 876 q20 1011959 832 832 q21 3898310033073100 q22 400 340 321 321 Total cold run time: 34661 ms Total hot run time: 30917 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [improvement](parser) Assign query organization to a single owner [doris]
hello-stephen commented on PR #67429: URL: https://github.com/apache/doris/pull/67429#issuecomment-5535294583 TPC-DS: Total hot run time: 82049 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 60d04ae69774cd63a9de1e9edaf6521c2ddb116e, data reload: false query5 4237408 324 324 query6 370 144 122 122 query7 4972403 236 236 query8 286 122 114 114 query9 8665290628852885 query10 385 209 203 203 query11 53701032899 899 query12 123 75 68 68 query13 1189462 326 326 query14 6189219720732073 query14_11963194819661948 query15 173 116 119 116 query16 913 369 371 369 query17 808 471 358 358 query18 2336318 243 243 query19 164 151 119 119 query20 74 69 68 68 query21 199 99 93 93 query22 5520531954635319 query23 6808618461506150 query23_16049610461476104 query24 73091088777 777 query24_1790 785 807 785 query25 436 309 260 260 query26 1232241 135 135 query27 2770433 266 266 query28 4687149915091499 query29 934 453 358 358 query30 251 159 131 131 query31 813 394 327 327 query32 125 79 74 74 query33 463 226 188 188 query34 1003829 478 478 query35 411 402 337 337 query36 570 581 530 530 query37 123 84 70 70 query38 999 843 811 811 query39 492 480 465 465 query39_1485 442 447 442 query40 205 94 83 83 query41 60 55 56 55 query42 74 73 76 73 query43 239 240 213 213 query44 1008551 562 551 query45 110 104 131 104 query46 755 842 542 542 query47 749 748 701 701 query48 302 288 210 210 query49 520 246 192 192 query50 718 255 198 198 query51 8093795879107910 query52 73 76 58 58 query53 198 216 153 153 query54 215 185 176 176 query55 77 57 54 54 query56 189 200 241 200 query57 685 669 665 665 query58 188 158 154 154 query59 1196121411011101 query60 219 195 183 183 query61 119 118 112 112 query62 368 202 173 173 query63 166 138 140 138 query64 2693699 573 573 query65 1693155416011554 query66 1886265 207 207 query67 9628979097209720 query68 30481217762 762 query69 355 220 195 195 query70 672 626 601 601 query71 254 179 165 165 query72 2333173014331433 query73 629 611 353 353 query74 1995122011481148 query75 11851098939 939 query76 2377734 560 560 query77 262 266 222 222 query78 3824364532063206 query79 2912828 567 567 query80 1596326 274 274 query81 515 154 131 131 query82 618 121 95 95 query83 281 209 189 189 query84 295 109 87 87 query85 814 363 307 307 query86 475 166 160 160 query87 1011966 891 891 query88 2918211520782078 query89 291 191 175 175 query90 2199131 134 131 query91 129 118 98 98 query92 94 69 68 68 query93 19191080753 753 query94 611 210 218 210 query95 516 319 221 221 query96 793 620 268 268 query97 1048105310131013 query98 174 139 130 130 query99 435 353 312 312 Total cold run time: 179114 ms Total hot run time: 82049 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific
Re: [PR] [improvement](parser) Assign query organization to a single owner [doris]
hello-stephen commented on PR #67429: URL: https://github.com/apache/doris/pull/67429#issuecomment-5526699490 # FE Regression Coverage Report Increment line coverage ` 9.09% (6/66)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/67429_60d04ae69774cd63a9de1e9edaf6521c2ddb116e_merge_fe/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/67429_60d04ae69774cd63a9de1e9edaf6521c2ddb116e_merge_fe/report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [improvement](parser) Assign query organization to a single owner [doris]
morrySnow commented on PR #67429: URL: https://github.com/apache/doris/pull/67429#issuecomment-5523852623 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [improvement](parser) Assign query organization to a single owner [doris]
hello-stephen commented on PR #67429: URL: https://github.com/apache/doris/pull/67429#issuecomment-5522870600 # FE UT Coverage Report Increment line coverage ` 100.00% (6/6)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/67429_b6c7537dd11eac7877bf7af0701a4810f4d573f9/fe_increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/67429_b6c7537dd11eac7877bf7af0701a4810f4d573f9/fe_report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [improvement](parser) Assign query organization to a single owner [doris]
morrySnow commented on PR #67429: URL: https://github.com/apache/doris/pull/67429#issuecomment-5522785166 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [improvement](parser) Assign query organization to a single owner [doris]
morrySnow commented on PR #67429: URL: https://github.com/apache/doris/pull/67429#issuecomment-5521729932 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [improvement](parser) Assign query organization to a single owner [doris]
hello-stephen commented on PR #67429: URL: https://github.com/apache/doris/pull/67429#issuecomment-5520166054 # FE Regression Coverage Report Increment line coverage ` 100.00% (6/6)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/67429_0bf9111810c61ac7584d0c865cb41287f8de9beb_merge_fe/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/67429_0bf9111810c61ac7584d0c865cb41287f8de9beb_merge_fe/report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [improvement](parser) Assign query organization to a single owner [doris]
hello-stephen commented on PR #67429: URL: https://github.com/apache/doris/pull/67429#issuecomment-5517366172 # FE Regression Coverage Report Increment line coverage ` 100.00% (6/6)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/67429_0bf9111810c61ac7584d0c865cb41287f8de9beb_merge_fe/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/67429_0bf9111810c61ac7584d0c865cb41287f8de9beb_merge_fe/report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [improvement](parser) Assign query organization to a single owner [doris]
hello-stephen commented on PR #67429: URL: https://github.com/apache/doris/pull/67429#issuecomment-5514584901 ClickBench: Total hot run time: 14.62 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 0bf9111810c61ac7584d0c865cb41287f8de9beb, data reload: false query1 0.010.010.00 query2 0.080.040.04 query3 0.250.110.11 query4 1.600.090.09 query5 0.170.170.15 query6 1.230.670.68 query7 0.030.010.00 query8 0.050.030.03 query9 0.290.210.22 query10 0.360.340.34 query11 0.160.120.11 query12 0.150.120.13 query13 0.290.300.33 query14 0.440.460.44 query15 0.360.340.36 query16 0.220.230.23 query17 0.730.680.72 query18 0.160.150.15 query19 1.201.181.14 query20 0.020.010.01 query21 15.44 0.170.13 query22 5.010.040.04 query23 16.18 0.260.11 query24 3.030.330.26 query25 0.090.050.05 query26 0.740.160.13 query27 0.040.030.04 query28 3.660.530.29 query29 12.46 3.192.54 query30 0.270.110.12 query31 2.760.370.17 query32 3.520.320.23 query33 1.341.391.37 query34 15.36 2.211.78 query35 1.751.751.71 query36 0.460.280.29 query37 0.070.040.04 query38 0.040.040.04 query39 0.030.020.02 query40 0.110.080.07 query41 0.080.020.03 query42 0.030.030.02 query43 0.030.020.03 Total cold run time: 90.3 s Total hot run time: 14.62 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [improvement](parser) Assign query organization to a single owner [doris]
hello-stephen commented on PR #67429: URL: https://github.com/apache/doris/pull/67429#issuecomment-5514531323 TPC-DS: Total hot run time: 82716 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 0bf9111810c61ac7584d0c865cb41287f8de9beb, data reload: false query5 4262409 335 335 query6 388 136 121 121 query7 4952433 230 230 query8 300 124 123 123 query9 8700290529042904 query10 398 240 185 185 query11 53861056910 910 query12 121 71 72 71 query13 1197462 337 337 query14 6025222821512151 query14_11975205220022002 query15 172 119 115 115 query16 927 381 372 372 query17 812 473 369 369 query18 2347332 245 245 query19 183 153 117 117 query20 92 73 69 69 query21 208 101 88 88 query22 5404534353235323 query23 6811625960316031 query23_16028620260736073 query24 72921104775 775 query24_1792 753 808 753 query25 449 310 287 287 query26 1245239 127 127 query27 2790416 272 272 query28 4672150814921492 query29 933 449 359 359 query30 258 159 131 131 query31 831 397 337 337 query32 162 77 79 77 query33 466 224 189 189 query34 1006849 489 489 query35 400 411 338 338 query36 562 600 512 512 query37 126 82 73 73 query38 1007858 840 840 query39 503 483 470 470 query39_1461 448 480 448 query40 211 93 81 81 query41 59 56 56 56 query42 110 70 74 70 query43 243 246 211 211 query44 1021555 548 548 query45 115 103 103 103 query46 757 827 520 520 query47 757 741 727 727 query48 300 294 228 228 query49 540 241 187 187 query50 726 264 200 200 query51 8077820381188118 query52 67 73 60 60 query53 192 211 152 152 query54 251 189 149 149 query55 76 64 63 63 query56 310 173 163 163 query57 695 670 667 667 query58 203 163 158 158 query59 1246124511491149 query60 241 181 173 173 query61 133 127 111 111 query62 344 210 167 167 query63 168 143 151 143 query64 2642673 647 647 query65 1606159415921592 query66 1834250 201 201 query67 10025 974097269726 query68 28931178745 745 query69 348 223 203 203 query70 657 638 617 617 query71 255 175 171 171 query72 2307172715741574 query73 654 602 364 364 query74 1959124611371137 query75 11851103958 958 query76 2293730 520 520 query77 254 259 206 206 query78 3994361732643264 query79 2755803 583 583 query80 1620326 292 292 query81 511 159 131 131 query82 995 126 99 99 query83 281 223 190 190 query84 300 115 90 90 query85 848 375 303 303 query86 474 175 178 175 query87 1025978 888 888 query88 3005211121142111 query89 295 194 173 173 query90 2153134 130 130 query91 131 120 102 102 query92 108 67 69 67 query93 21411019691 691 query94 653 260 213 213 query95 523 243 297 243 query96 784 558 273 273 query97 1082103210361032 query98 173 137 137 137 query99 424 344 311 311 Total cold run time: 180086 ms Total hot run time: 82716 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific
Re: [PR] [improvement](parser) Assign query organization to a single owner [doris]
hello-stephen commented on PR #67429: URL: https://github.com/apache/doris/pull/67429#issuecomment-5514450903 TPC-H: Total hot run time: 16716 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 0bf9111810c61ac7584d0c865cb41287f8de9beb, data reload: false -- Round 1 -- q1 17570 306030143014 q2 2134256 221 221 q3 10226 989 520 520 q4 4673251 206 206 q5 7675557 385 385 q6 141 124 95 95 q7 532 501 376 376 q8 9253854 883 854 q9 3474236424022364 q10 6532840 711 711 q11 393 201 181 181 q12 613 268 203 203 q13 18114 153111681168 q14 162 152 141 141 q15 q16 444 399 376 376 q17 1303826 769 769 q18 3106224322622243 q19 1291856 744 744 q20 377 296 204 204 q21 5592170818671708 q22 340 271 233 233 Total cold run time: 93945 ms Total hot run time: 16716 ms - Round 2, with runtime_filter_mode=off - q1 3380333633243324 q2 506 395 368 368 q3 2238231021652165 q4 12041184912 912 q5 2183209221142092 q6 172 122 90 90 q7 1016900 853 853 q8 1596139514051395 q9 3127310031193100 q10 1861183116351635 q11 361 275 255 255 q12 448 426 339 339 q13 1497154911651165 q14 164 175 159 159 q15 q16 388 390 364 364 q17 3604321731953195 q18 4795437247404372 q19 834 795 930 795 q20 1027969 810 810 q21 3823310132953101 q22 395 360 339 339 Total cold run time: 34619 ms Total hot run time: 30828 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [improvement](parser) Assign query organization to a single owner [doris]
morrySnow commented on PR #67429: URL: https://github.com/apache/doris/pull/67429#issuecomment-5512959505 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[PR] [improvement](parser) Assign query organization to a single owner [doris]
morrySnow opened a new pull request, #67429:
URL: https://github.com/apache/doris/pull/67429
### What problem does this PR solve?
Issue Number: None
Related PR: #67427
Problem Summary: This draft PR is stacked on #67427. The grammar allowed
both `querySpecification` and its outer `query` to consume `ORDER BY` and
`LIMIT`, so ANTLR repeatedly entered `queryOrganization` and reported
full-context ambiguities. This change gives every clause group one owner
according to ANSI mode and query level, makes the rule structurally non-empty,
preserves the existing accepted/rejected SQL matrix and first-error positions,
and updates the FE plan builder for the labeled contexts. It also adds focused
standalone-parser and FE plan tests plus a JMH harness for the affected path.
### Benchmark
Lower latency is better. The target input is `SELECT a, b, c FROM t WHERE a
> 1 ORDER BY a, b DESC LIMIT 20 OFFSET 10`; the control input has the same
SELECT without query-organization clauses.
- Host: MacBookPro17,1, Apple M1 (8 cores, 16 GB), macOS 15.0.1
- Runtime: OpenJDK 17.0.20.1, ANTLR 4.13.1, JMH 1.37, 1 thread, 1 GB heap
- JMH: 3 forks, 4 x 300 ms warmup, 7 x 400 ms measurement; C1-B1-C2
interleaving
- Baseline: `5e0eadb13e9`; parser jar SHA-256
`530f0ed45c4bac3a096373a1e932aae0025c603a6f905c3d975a2ad53bd0e7ca`; benchmark
jar SHA-256 `7543595b2a87f55b9b53538336cfedf55c13484302ffb487efcfb4fdd66b1c0b`
- Candidate: `773c4ee1027`; parser jar SHA-256
`65730a22a5262370a832bda7a04ae4b8e0de5c107be65c5ae80f95c711842f59`; benchmark
jar SHA-256 `47284ffd6779e9d534a19a682de5a4d468f83a62a98e048982807497f1f665b6`
- Harness:
`fe/fe-sql-parser-benchmark/src/main/java/org/apache/doris/sqlparser/benchmark/QueryOrganizationBenchmark.java`
Ordered SELECT target
The candidate change is calculated from the mean of C1 and C2 against B1.
Values are us/op with JMH's 99.9% error estimate.
| Path | Mode | Baseline B1 | Candidate C1 | Candidate C2 | Candidate mean
change |
|---|---|---:|---:|---:|---:|
| End-to-end | Legacy | 9.926 +/- 0.347 | 9.303 +/- 0.333 | 9.122 +/- 0.267
| -7.2% |
| End-to-end | ANSI | 9.170 +/- 0.437 | 8.709 +/- 0.646 | 8.492 +/- 0.241 |
-6.2% |
| Pre-tokenized parser | Legacy | 7.653 +/- 0.266 | 7.246 +/- 0.244 | 7.069
+/- 0.090 | -6.5% |
| Pre-tokenized parser | ANSI | 7.344 +/- 0.105 | 7.181 +/- 0.099 | 7.099
+/- 0.076 | -2.8% |
Plain SELECT control
| Path | Mode | Baseline us/op | Candidate us/op | Change |
|---|---|---:|---:|---:|
| End-to-end | Legacy | 6.580 +/- 0.710 | 6.657 +/- 0.809 | +1.2% |
| End-to-end | ANSI | 6.944 +/- 0.648 | 6.448 +/- 0.440 | -7.1% |
| Pre-tokenized parser | Legacy | 5.684 +/- 1.205 | 5.084 +/- 0.054 | -10.6%
|
| Pre-tokenized parser | ANSI | 5.313 +/- 0.125 | 5.193 +/- 0.089 | -2.3% |
No control-path latency regression exceeds the 3% threshold. A later B2 run
was affected by large machine-level outliers (`13.841 +/- 3.137` us/op for
legacy end-to-end) and is retained as raw data but excluded from the latency
claim; the target numbers above should be rerun on a quiet host before marking
the PR ready for review.
Allocation
The same artifacts were measured separately with `-prof gc`. Timing under
the profiler was noisy, so this table uses only normalized allocation.
| Path | Mode | Baseline B/op | Candidate B/op | Change |
|---|---|---:|---:|---:|
| End-to-end | Legacy | 15,473.6 | 15,719.6 | +1.59% |
| End-to-end | ANSI | 15,426.8 | 15,536.1 | +0.71% |
| Pre-tokenized parser | Legacy | 12,368.1 | 12,381.4 | +0.11% |
| Pre-tokenized parser | ANSI | 12,317.4 | 12,421.4 | +0.84% |
All allocation changes are below the 3% threshold. The profiler's legacy
end-to-end timing changed by +6.6%, but its broad confidence intervals overlap;
the non-profiler interleaved results above are used for the latency conclusion.
Profile attribution
- Before P2, tracked SQL entered `queryOrganization` 13,196 times and
examined 49,902 lookahead tokens; SSB and Trino profiling reported 40 and 20
related ambiguities.
- After P2, the 4,275 parseable tracked SQL files enter the target rule
2,794 times and examine 8,645 lookahead tokens, a reduction of about 79% and
83% respectively.
- LL_EXACT reports zero fallback and zero ambiguity for both the
`querySpecification` and `queryOrganization` target decisions.
- The gain therefore comes from eliminating duplicate ownership and adaptive
lookahead, not from lexer or token changes.
Build and benchmark commands:
```shell
mvn -Pbenchmark -pl fe-sql-parser-benchmark -am package \
-Dmaven.build.cache.enabled=false -Dskip.doc=true
java -jar
fe/fe-sql-parser-benchmark/target/doris-fe-sql-parser-benchmarks.jar \
'.*QueryOrganizationBenchmark.*' -p workload=orderedSelect \
-rf json -rff
Re: [PR] [improvement](parser) Assign query organization to a single owner [doris]
hello-stephen commented on PR #67429: URL: https://github.com/apache/doris/pull/67429#issuecomment-5505141165 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
