Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
morningman merged PR #35812: URL: https://github.com/apache/doris/pull/35812 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
AshinGau commented on PR #35812: URL: https://github.com/apache/doris/pull/35812#issuecomment-2144880582 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
doris-robot commented on PR #35812: URL: https://github.com/apache/doris/pull/35812#issuecomment-2144880377 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [feat](nereids) support Iceberg time travel syntax [doris]
AshinGau opened a new pull request, #35812: URL: https://github.com/apache/doris/pull/35812 backport: #34681 https://github.com/apache/doris/pull/15418 added Iceberg time travel in legacy parser but not added this syntax Neredis. If we enable nereids and disable fallback to original palnner, time travel won't be available. This PR added time travel syntas in Neredis. BTW, we already have nereids time travel regression-test in https://github.com/apache/doris/blob/master/regression-test/suites/external_table_p2/iceberg/test_external_catalog_icebergv2_nereids.groovy this regression-test will always fail without this PR. https://github.com/apache/doris/blob/88530bf9437e60190f198252ab82f53fa53d4c10/regression-test/suites/external_table_p2/iceberg/test_external_catalog_icebergv2_nereids.groovy#L70-L75 ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
morrySnow merged PR #34681: URL: https://github.com/apache/doris/pull/34681 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
github-actions[bot] commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2126206590 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
wuwenchi commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2126174977 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
doris-robot commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2123802307 ClickBench: Total hot run time: 30.89 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 761c29f4fd17b4c6f3d87d65c447d6cfefa67621, data reload: false query1 0.040.040.03 query2 0.080.040.04 query3 0.240.060.05 query4 1.660.080.08 query5 0.500.500.51 query6 1.120.720.73 query7 0.020.010.01 query8 0.050.040.05 query9 0.550.490.49 query10 0.540.550.53 query11 0.150.120.11 query12 0.160.120.11 query13 0.600.600.60 query14 0.780.780.77 query15 0.850.820.81 query16 0.380.360.37 query17 1.010.990.99 query18 0.210.260.21 query19 1.801.671.68 query20 0.020.010.01 query21 15.73 0.650.64 query22 4.506.322.25 query23 18.27 1.431.35 query24 1.770.230.26 query25 0.150.080.08 query26 0.260.170.17 query27 0.080.080.08 query28 13.33 1.030.99 query29 13.76 3.273.22 query30 0.240.070.06 query31 2.830.410.39 query32 3.280.460.47 query33 2.912.952.95 query34 17.24 4.354.39 query35 4.474.504.63 query36 0.660.490.45 query37 0.170.170.15 query38 0.150.150.14 query39 0.040.040.04 query40 0.160.150.14 query41 0.090.050.04 query42 0.050.050.04 query43 0.040.040.04 Total cold run time: 110.94 s Total hot run time: 30.89 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
doris-robot commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2123798531 TPC-DS: Total hot run time: 181302 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 761c29f4fd17b4c6f3d87d65c447d6cfefa67621, data reload: false query1 910 397 374 374 query2 6440252123922392 query3 6650211 203 203 query4 22935 21234 21131 21131 query5 4107422 426 422 query6 259 172 172 172 query7 4586300 295 295 query8 235 191 188 188 query9 8479239023802380 query10 452 251 258 251 query11 14855 14201 14072 14072 query12 135 94 88 88 query13 1635390 359 359 query14 10454 864877417741 query15 248 172 165 165 query16 8071271 251 251 query17 1718541 542 541 query18 2100281 265 265 query19 189 144 150 144 query20 95 86 86 86 query21 187 129 126 126 query22 5062484747654765 query23 34198 33565 33268 33268 query24 6699277928582779 query25 541 375 359 359 query26 700 154 156 154 query27 1887311 330 311 query28 3619206120482048 query29 879 610 595 595 query30 256 182 173 173 query31 978 775 727 727 query32 91 52 53 52 query33 485 239 246 239 query34 871 479 492 479 query35 750 678 659 659 query36 1062908 899 899 query37 109 70 73 70 query38 2906277327752773 query39 1622153116181531 query40 211 130 133 130 query41 46 44 45 44 query42 104 96 96 96 query43 558 551 550 550 query44 1082728 753 728 query45 263 266 237 237 query46 1060730 712 712 query47 1973186218721862 query48 391 304 288 288 query49 786 393 406 393 query50 773 391 387 387 query51 6835669167536691 query52 105 91 91 91 query53 346 293 285 285 query54 526 429 430 429 query55 75 73 73 73 query56 238 219 224 219 query57 1230114311471143 query58 220 199 199 199 query59 3523319032253190 query60 255 237 239 237 query61 90 129 88 88 query62 614 457 467 457 query63 308 277 285 277 query64 8453224517431743 query65 3204308231473082 query66 789 333 340 333 query67 15485 15202 15279 15202 query68 4566523 535 523 query69 481 313 312 312 query70 1161113311661133 query71 390 282 274 274 query72 7887260523502350 query73 713 316 316 316 query74 6704619062156190 query75 3318263026042604 query76 2328938 1007938 query77 396 268 268 268 query78 10567 10172 10107 10107 query79 1670508 532 508 query80 966 433 444 433 query81 526 249 243 243 query82 961 102 101 101 query83 254 171 172 171 query84 256 89 94 89 query85 1049334 262 262 query86 465 309 297 297 query87 3283306530803065 query88 3061232423332324 query89 475 379 388 379 query90 1976201 193 193 query91 126 97 100 97 query92 58 50 48 48 query93 1470505 495 495 query94 1094189 184 184 query95 400 306 304 304 query96 577 263 269 263 query97 3178302330543023 query98 240 229 217 217 query99 1230898 937 898 Total cold run time: 268824 ms Total hot run time: 181302 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
doris-robot commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2123790783 TPC-H: Total hot run time: 42135 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 761c29f4fd17b4c6f3d87d65c447d6cfefa67621, data reload: false -- Round 1 -- q1 17595 490642604260 q2 2017193 197 193 q3 10447 118712601187 q4 10224 913 866 866 q5 7499270527412705 q6 218 136 134 134 q7 1062598 588 588 q8 9227216321352135 q9 9112678466616661 q10 9948384639413846 q11 438 245 263 245 q12 431 228 217 217 q13 17486 321332593213 q14 256 215 209 209 q15 535 465 486 465 q16 522 411 394 394 q17 1002689 761 689 q18 8504798378887888 q19 4204154215771542 q20 650 319 333 319 q21 5309409741414097 q22 369 282 298 282 Total cold run time: 117055 ms Total hot run time: 42135 ms - Round 2, with runtime_filter_mode=off - q1 4543439744354397 q2 378 276 268 268 q3 3168293327552755 q4 1927160716581607 q5 5529550155725501 q6 221 124 125 124 q7 2354195619641956 q8 3268344334213421 q9 8705869886208620 q10 3946386338523852 q11 612 502 495 495 q12 818 632 616 616 q13 17417 318130863086 q14 285 272 257 257 q15 509 475 463 463 q16 470 409 413 409 q17 1778149014571457 q18 7678764873957395 q19 1692153216051532 q20 2055181417761776 q21 10752 498548554855 q22 598 503 490 490 Total cold run time: 78703 ms Total hot run time: 55332 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
zhangbutao commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2123727221 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
zhangbutao commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2120173604 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
zhangbutao commented on code in PR #34681: URL: https://github.com/apache/doris/pull/34681#discussion_r1606578056 ## fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HMSExternalTable.java: ## @@ -169,6 +170,7 @@ public class HMSExternalTable extends ExternalTable implements MTMVRelatedTableI // for hudi incremental read private TableScanParams scanParams = null; private IncrementalRelation incrementalRelation = null; +private TableSnapshot tableSnapshot = null; Review Comment: Thanks for you suggesstion! I refine this PR in https://github.com/apache/doris/pull/34681/commits/21081393f5ddf734cea06c52c7ed73a7079ec7e2. Now we can support all types of iceberg tables, and i set the snapshot in IcebergScanNode through method `PhysicalPlanTranslator::visitPhysicalFileScan `. Please help to check if this change is good for you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
zhangbutao commented on code in PR #34681: URL: https://github.com/apache/doris/pull/34681#discussion_r1606573735 ## fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/BindRelation.java: ## @@ -273,6 +273,7 @@ private LogicalPlan getLogicalPlan(TableIf table, UnboundRelation unboundRelatio return new LogicalSubQueryAlias<>(tableQualifier, hiveViewPlan); } hmsTable.setScanParams(unboundRelation.getScanParams()); + hmsTable.setTableSnapshotVersion(unboundRelation.getTableSnapshot().orElse(null)); Review Comment: Fixed. Now all types of iceberg tables can be supported. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
zhangbutao commented on code in PR #34681:
URL: https://github.com/apache/doris/pull/34681#discussion_r1606572386
##
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java:
##
@@ -1362,15 +1363,27 @@ public LogicalPlan visitTableName(TableNameContext ctx)
{
scanParams = new
TableScanParams(ctx.optScanParams().funcName.getText(), map);
}
+TableSnapshot tableSnapshot = null;
+if (ctx.tableSnapshot() != null) {
+if (ctx.tableSnapshot().TIME() != null) {
+tableSnapshot = new
TableSnapshot(stripQuotes(ctx.tableSnapshot().time.getText()));
+tableSnapshot.setType(TableSnapshot.VersionType.TIME);
Review Comment:
Fixed. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
zhangbutao commented on code in PR #34681: URL: https://github.com/apache/doris/pull/34681#discussion_r1606571735 ## fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4: ## @@ -966,6 +966,11 @@ sampleMethod | INTEGER_VALUE ROWS #sampleByRows ; +tableSnapshot +: FOR VERSION AS OF version=number Review Comment: Fixed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
zhangbutao commented on code in PR #34681: URL: https://github.com/apache/doris/pull/34681#discussion_r1604572540 ## fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/BindRelation.java: ## @@ -273,6 +273,7 @@ private LogicalPlan getLogicalPlan(TableIf table, UnboundRelation unboundRelatio return new LogicalSubQueryAlias<>(tableQualifier, hiveViewPlan); } hmsTable.setScanParams(unboundRelation.getScanParams()); + hmsTable.setTableSnapshotVersion(unboundRelation.getTableSnapshot().orElse(null)); Review Comment: Good catch! Let me check this later. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
wuwenchi commented on code in PR #34681: URL: https://github.com/apache/doris/pull/34681#discussion_r1604433431 ## fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/BindRelation.java: ## @@ -273,6 +273,7 @@ private LogicalPlan getLogicalPlan(TableIf table, UnboundRelation unboundRelatio return new LogicalSubQueryAlias<>(tableQualifier, hiveViewPlan); } hmsTable.setScanParams(unboundRelation.getScanParams()); + hmsTable.setTableSnapshotVersion(unboundRelation.getTableSnapshot().orElse(null)); Review Comment: Only HMS tables are supported, but can other types of iceberg tables be supported at the same time? ## fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4: ## @@ -966,6 +966,11 @@ sampleMethod | INTEGER_VALUE ROWS #sampleByRows ; +tableSnapshot +: FOR VERSION AS OF version=number Review Comment: The range of `number` is relatively wide, and we can use `INTEGER_VALUE` ## fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HMSExternalTable.java: ## @@ -169,6 +170,7 @@ public class HMSExternalTable extends ExternalTable implements MTMVRelatedTableI // for hudi incremental read private TableScanParams scanParams = null; private IncrementalRelation incrementalRelation = null; +private TableSnapshot tableSnapshot = null; Review Comment: Can we get snapshot version information directly from IcebergScanNode? 1. This naturally supports all types of iceberg tables. 2. Specify that the snapshot belongs to the query, and its valid range is related to the query, so it is not appropriate to put this information in the HMSExternalTable. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
wuwenchi commented on code in PR #34681:
URL: https://github.com/apache/doris/pull/34681#discussion_r1604421627
##
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java:
##
@@ -1362,15 +1363,27 @@ public LogicalPlan visitTableName(TableNameContext ctx)
{
scanParams = new
TableScanParams(ctx.optScanParams().funcName.getText(), map);
}
+TableSnapshot tableSnapshot = null;
+if (ctx.tableSnapshot() != null) {
+if (ctx.tableSnapshot().TIME() != null) {
+tableSnapshot = new
TableSnapshot(stripQuotes(ctx.tableSnapshot().time.getText()));
+tableSnapshot.setType(TableSnapshot.VersionType.TIME);
Review Comment:
The corresponding type has been set in the constructor of `TableSnapshot`,
why should we set it again?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
morningman commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2116595342 @wuwenchi PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
morrySnow commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2116564775 > @morrySnow Gentle ping. Can we merge this change? Or Is there anything else that needs to be done? we need 2 approve to merge pr. let's find another reviewer to review this PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
zhangbutao commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2115692662 @morrySnow Gentle ping. Can we merge this change? Or Is there anything else that needs to be done? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
doris-robot commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2113947012 ClickBench: Total hot run time: 30.16 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit f18eb36bd4f1121530cf1b910b13b85645a36674, data reload: false query1 0.040.040.03 query2 0.080.040.04 query3 0.240.040.05 query4 1.660.070.06 query5 0.490.490.52 query6 1.120.730.72 query7 0.010.020.01 query8 0.060.040.04 query9 0.530.480.48 query10 0.540.520.53 query11 0.160.120.11 query12 0.150.120.11 query13 0.590.590.60 query14 0.770.770.79 query15 0.820.800.80 query16 0.350.370.36 query17 0.970.930.92 query18 0.220.220.26 query19 1.791.691.70 query20 0.020.010.01 query21 15.73 0.650.64 query22 4.237.671.73 query23 18.31 1.451.29 query24 1.290.450.21 query25 0.140.090.08 query26 0.270.170.18 query27 0.080.080.08 query28 13.31 1.011.00 query29 13.14 3.383.33 query30 0.240.060.06 query31 2.860.390.37 query32 3.280.470.47 query33 2.812.802.81 query34 17.04 4.394.47 query35 4.484.484.47 query36 0.710.490.48 query37 0.170.150.15 query38 0.150.140.14 query39 0.040.040.03 query40 0.160.140.14 query41 0.090.050.04 query42 0.060.040.05 query43 0.040.040.04 Total cold run time: 109.24 s Total hot run time: 30.16 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
doris-robot commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2113935285 TPC-DS: Total hot run time: 186741 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit f18eb36bd4f1121530cf1b910b13b85645a36674, data reload: false query1 911 388 374 374 query2 6321245623292329 query3 6654218 216 216 query4 26214 21774 21931 21774 query5 3789426 429 426 query6 280 187 171 171 query7 4542297 296 296 query8 239 186 192 186 query9 8596240524632405 query10 426 246 261 246 query11 15191 14946 14840 14840 query12 119 94 85 85 query13 1569356 372 356 query14 10273 677076656770 query15 236 168 166 166 query16 8115255 255 255 query17 1706549 526 526 query18 2085265 267 265 query19 194 151 142 142 query20 88 81 80 80 query21 186 130 129 129 query22 5099494649184918 query23 33926 33701 33356 33356 query24 10680 284128982841 query25 597 360 355 355 query26 1169152 147 147 query27 3006310 322 310 query28 7136200619951995 query29 853 635 607 607 query30 262 171 169 169 query31 987 739 748 739 query32 94 49 57 49 query33 734 245 238 238 query34 1069490 472 472 query35 774 666 651 651 query36 1045917 922 917 query37 138 70 68 68 query38 2903276428352764 query39 1595156715581558 query40 192 125 119 119 query41 47 42 41 41 query42 100 96 93 93 query43 548 551 527 527 query44 1186725 732 725 query45 266 248 258 248 query46 1068722 700 700 query47 1981188918791879 query48 365 305 292 292 query49 895 386 381 381 query50 749 376 385 376 query51 6857678968076789 query52 98 94 92 92 query53 346 280 278 278 query54 841 416 420 416 query55 75 72 72 72 query56 236 215 221 215 query57 1209117811221122 query58 214 194 196 194 query59 3368318129522952 query60 279 230 226 226 query61 86 85 88 85 query62 637 472 454 454 query63 311 280 283 280 query64 8802736973967369 query65 3096307630553055 query66 804 325 334 325 query67 15816 15012 14865 14865 query68 4531517 520 517 query69 477 300 299 299 query70 1134108510911085 query71 423 266 261 261 query72 7095259123242324 query73 719 317 319 317 query74 6611619562216195 query75 3415264126072607 query76 2702955 975 955 query77 365 261 263 261 query78 10568 10201 10017 10017 query79 2521515 508 508 query80 1052425 417 417 query81 579 240 241 240 query82 943 94 95 94 query83 260 164 163 163 query84 232 87 84 84 query85 1179264 255 255 query86 473 318 295 295 query87 3352311832013118 query88 4189230023172300 query89 467 384 390 384 query90 1938193 185 185 query91 123 97 107 97 query92 61 48 48 48 query93 1810484 484 484 query94 1145173 172 172 query95 387 293 315 293 query96 586 260 268 260 query97 3165306630143014 query98 237 227 215 215 query99 1177916 934 916 Total cold run time: 284189 ms Total hot run time: 186741 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
doris-robot commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2109219781 TPC-DS: Total hot run time: 187388 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit f18eb36bd4f1121530cf1b910b13b85645a36674, data reload: false query1 915 365 345 345 query2 6454239523332333 query3 6638208 208 208 query4 23075 21386 21203 21203 query5 4146421 422 421 query6 256 172 177 172 query7 4579296 283 283 query8 243 191 189 189 query9 8480249124502450 query10 459 242 251 242 query11 14704 14285 14240 14240 query12 137 91 89 89 query13 1634382 379 379 query14 10510 819476417641 query15 228 167 170 167 query16 8080267 260 260 query17 1854580 539 539 query18 2059267 269 267 query19 214 155 151 151 query20 109 87 89 87 query21 202 130 134 130 query22 5027482848874828 query23 34147 33526 33475 33475 query24 11904 294229432942 query25 624 389 362 362 query26 1695155 153 153 query27 2987322 336 322 query28 7441207220612061 query29 982 626 611 611 query30 285 150 153 150 query31 975 766 742 742 query32 92 52 58 52 query33 751 249 238 238 query34 1052517 499 499 query35 796 669 681 669 query36 1087896 920 896 query37 133 67 67 67 query38 2890278127522752 query39 1653155615571556 query40 276 126 125 125 query41 41 39 39 39 query42 106 97 97 97 query43 571 549 572 549 query44 1218733 737 733 query45 261 258 259 258 query46 1085716 727 716 query47 1959189319381893 query48 370 307 305 305 query49 1128392 394 392 query50 777 392 399 392 query51 6830679668546796 query52 107 86 92 86 query53 350 287 284 284 query54 965 431 439 431 query55 80 77 81 77 query56 254 232 265 232 query57 1268119611621162 query58 219 203 191 191 query59 3511321836163218 query60 263 230 239 230 query61 92 87 85 85 query62 665 471 461 461 query63 309 295 286 286 query64 9793739673567356 query65 3154309630873087 query66 1383333 347 333 query67 15474 14923 14864 14864 query68 4555535 532 532 query69 482 299 299 299 query70 1156113810761076 query71 405 270 260 260 query72 7401267523532353 query73 698 326 329 326 query74 6544620061546154 query75 3344261326222613 query76 2611991 979 979 query77 402 269 264 264 query78 10570 10224 10001 10001 query79 2472517 519 517 query80 1041482 426 426 query81 508 223 222 222 query82 743 91 94 91 query83 247 164 173 164 query84 234 92 87 87 query85 1589271 268 268 query86 521 304 310 304 query87 3312311031083108 query88 4323244724552447 query89 471 379 377 377 query90 1959190 190 190 query91 127 97 98 97 query92 65 48 51 48 query93 1710511 495 495 query94 1192186 185 185 query95 415 303 307 303 query96 585 275 277 275 query97 3158297029632963 query98 224 220 212 212 query99 1139903 918 903 Total cold run time: 286209 ms Total hot run time: 187388 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
doris-robot commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2109209310 TPC-H: Total hot run time: 40423 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit f18eb36bd4f1121530cf1b910b13b85645a36674, data reload: false -- Round 1 -- q1 17675 433841844184 q2 2016196 190 190 q3 10575 129212181218 q4 10547 760 777 760 q5 7523266826322632 q6 211 132 132 132 q7 982 585 594 585 q8 9634209120652065 q9 9207664665976597 q10 9601371738553717 q11 444 241 241 241 q12 447 214 216 214 q13 17778 294629562946 q14 269 207 230 207 q15 514 466 477 466 q16 522 386 384 384 q17 968 663 730 663 q18 8108732574197325 q19 4221154114941494 q20 652 305 302 302 q21 5318382438673824 q22 349 277 290 277 Total cold run time: 117561 ms Total hot run time: 40423 ms - Round 2, with runtime_filter_mode=off - q1 4303420041954195 q2 375 276 273 273 q3 2988276127162716 q4 1823152115941521 q5 5238525152575251 q6 210 125 123 123 q7 2228186718931867 q8 3158328033283280 q9 8295828582938285 q10 3819366936503650 q11 580 499 494 494 q12 768 573 588 573 q13 16386 296029762960 q14 305 276 274 274 q15 516 478 471 471 q16 451 415 421 415 q17 1757146714771467 q18 7710760973317331 q19 1645159315261526 q20 1953177817611761 q21 4787476647744766 q22 551 485 475 475 Total cold run time: 69846 ms Total hot run time: 53674 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
github-actions[bot] commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2109207268 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
github-actions[bot] commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2109207241 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
zhangbutao commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2109192782 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
zhangbutao commented on code in PR #34681:
URL: https://github.com/apache/doris/pull/34681#discussion_r1599310349
##
fe/fe-core/src/main/java/org/apache/doris/nereids/analyzer/UnboundRelation.java:
##
@@ -59,41 +60,46 @@ public class UnboundRelation extends LogicalRelation
implements Unbound, BlockFu
// the start and end position of the sql substring(e.g. "t1", "db1.t1",
"ctl1.db1.t1")
private final Optional> indexInSqlString;
+private final Optional tableSnapshot;
+
public UnboundRelation(RelationId id, List nameParts) {
this(id, nameParts, Optional.empty(), Optional.empty(),
ImmutableList.of(), false, ImmutableList.of(),
-ImmutableList.of(), Optional.empty(), Optional.empty(), null,
Optional.empty());
+ImmutableList.of(), Optional.empty(), Optional.empty(), null,
Optional.empty(), null);
}
public UnboundRelation(RelationId id, List nameParts, List
partNames, boolean isTempPart) {
this(id, nameParts, Optional.empty(), Optional.empty(), partNames,
isTempPart, ImmutableList.of(),
-ImmutableList.of(), Optional.empty(), Optional.empty(), null,
Optional.empty());
+ImmutableList.of(), Optional.empty(), Optional.empty(), null,
Optional.empty(), null);
}
public UnboundRelation(RelationId id, List nameParts, List
partNames, boolean isTempPart,
List tabletIds, List hints, Optional
tableSample, Optional indexName) {
this(id, nameParts, Optional.empty(), Optional.empty(),
-partNames, isTempPart, tabletIds, hints, tableSample,
indexName, null, Optional.empty());
+partNames, isTempPart, tabletIds, hints, tableSample,
indexName, null, Optional.empty(), null);
}
public UnboundRelation(RelationId id, List nameParts, List
partNames, boolean isTempPart,
List tabletIds, List hints, Optional
tableSample, Optional indexName,
-TableScanParams scanParams) {
+TableScanParams scanParams, Optional tableSnapshot)
{
this(id, nameParts, Optional.empty(), Optional.empty(),
-partNames, isTempPart, tabletIds, hints, tableSample,
indexName, scanParams, Optional.empty());
+partNames, isTempPart, tabletIds, hints, tableSample,
indexName, scanParams, Optional.empty(),
+tableSnapshot);
}
public UnboundRelation(RelationId id, List nameParts,
Optional groupExpression,
Optional logicalProperties, List
partNames, boolean isTempPart,
List tabletIds, List hints, Optional
tableSample, Optional indexName) {
this(id, nameParts, groupExpression, logicalProperties, partNames,
-isTempPart, tabletIds, hints, tableSample, indexName, null,
Optional.empty());
+isTempPart, tabletIds, hints, tableSample, indexName, null,
Optional.empty(), null);
Review Comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
morrySnow commented on code in PR #34681:
URL: https://github.com/apache/doris/pull/34681#discussion_r1599282199
##
fe/fe-core/src/main/java/org/apache/doris/nereids/analyzer/UnboundRelation.java:
##
@@ -59,41 +60,46 @@ public class UnboundRelation extends LogicalRelation
implements Unbound, BlockFu
// the start and end position of the sql substring(e.g. "t1", "db1.t1",
"ctl1.db1.t1")
private final Optional> indexInSqlString;
+private final Optional tableSnapshot;
+
public UnboundRelation(RelationId id, List nameParts) {
this(id, nameParts, Optional.empty(), Optional.empty(),
ImmutableList.of(), false, ImmutableList.of(),
-ImmutableList.of(), Optional.empty(), Optional.empty(), null,
Optional.empty());
+ImmutableList.of(), Optional.empty(), Optional.empty(), null,
Optional.empty(), null);
}
public UnboundRelation(RelationId id, List nameParts, List
partNames, boolean isTempPart) {
this(id, nameParts, Optional.empty(), Optional.empty(), partNames,
isTempPart, ImmutableList.of(),
-ImmutableList.of(), Optional.empty(), Optional.empty(), null,
Optional.empty());
+ImmutableList.of(), Optional.empty(), Optional.empty(), null,
Optional.empty(), null);
}
public UnboundRelation(RelationId id, List nameParts, List
partNames, boolean isTempPart,
List tabletIds, List hints, Optional
tableSample, Optional indexName) {
this(id, nameParts, Optional.empty(), Optional.empty(),
-partNames, isTempPart, tabletIds, hints, tableSample,
indexName, null, Optional.empty());
+partNames, isTempPart, tabletIds, hints, tableSample,
indexName, null, Optional.empty(), null);
}
public UnboundRelation(RelationId id, List nameParts, List
partNames, boolean isTempPart,
List tabletIds, List hints, Optional
tableSample, Optional indexName,
-TableScanParams scanParams) {
+TableScanParams scanParams, Optional tableSnapshot)
{
this(id, nameParts, Optional.empty(), Optional.empty(),
-partNames, isTempPart, tabletIds, hints, tableSample,
indexName, scanParams, Optional.empty());
+partNames, isTempPart, tabletIds, hints, tableSample,
indexName, scanParams, Optional.empty(),
+tableSnapshot);
}
public UnboundRelation(RelationId id, List nameParts,
Optional groupExpression,
Optional logicalProperties, List
partNames, boolean isTempPart,
List tabletIds, List hints, Optional
tableSample, Optional indexName) {
this(id, nameParts, groupExpression, logicalProperties, partNames,
-isTempPart, tabletIds, hints, tableSample, indexName, null,
Optional.empty());
+isTempPart, tabletIds, hints, tableSample, indexName, null,
Optional.empty(), null);
Review Comment:
```suggestion
isTempPart, tabletIds, hints, tableSample, indexName, null,
Optional.empty(), Optional.empty());
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
zhangbutao commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2106114670 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
doris-robot commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2105928787 TPC-DS: Total hot run time: 186552 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit fa83cce7bede1c1f53806f37981ee6ab5f4bd6b7, data reload: false query1 918 370 341 341 query2 6445238223852382 query3 6655207 210 207 query4 23063 21131 21075 21075 query5 4155421 423 421 query6 267 173 170 170 query7 4592290 290 290 query8 258 189 187 187 query9 8738240423922392 query10 444 250 257 250 query11 14772 14110 14330 14110 query12 133 94 91 91 query13 1649371 366 366 query14 10595 838075997599 query15 262 171 172 171 query16 8174257 263 257 query17 1867563 542 542 query18 2101273 280 273 query19 328 147 147 147 query20 90 89 88 88 query21 212 127 124 124 query22 5064488348054805 query23 34218 33801 33493 33493 query24 11594 300529452945 query25 635 380 358 358 query26 1736153 152 152 query27 2815317 322 317 query28 7264205320302030 query29 1004603 591 591 query30 291 149 150 149 query31 950 745 728 728 query32 92 52 55 52 query33 751 249 246 246 query34 1022485 493 485 query35 813 683 672 672 query36 1066913 912 912 query37 175 65 65 65 query38 2881273727312731 query39 1631156915381538 query40 282 125 124 124 query41 41 40 39 39 query42 102 97 100 97 query43 584 515 542 515 query44 1201714 739 714 query45 260 250 259 250 query46 1071725 704 704 query47 1964190218961896 query48 371 297 291 291 query49 1184393 396 393 query50 785 386 392 386 query51 6895676566966696 query52 104 83 90 83 query53 348 276 274 274 query54 943 431 429 429 query55 73 71 72 71 query56 238 223 217 217 query57 1244112411391124 query58 224 194 197 194 query59 3378315831303130 query60 263 238 238 238 query61 90 88 86 86 query62 668 476 473 473 query63 309 279 277 277 query64 9766736673257325 query65 3146307630763076 query66 1382349 344 344 query67 15627 14958 15130 14958 query68 4906538 533 533 query69 525 300 306 300 query70 1154109711181097 query71 427 267 264 264 query72 7551259123482348 query73 724 322 324 322 query74 6656614259585958 query75 3630260126022601 query76 3610986 985 985 query77 639 276 263 263 query78 10574 10090 10076 10076 query79 1349519 508 508 query80 1062432 436 432 query81 493 220 220 220 query82 653 95 94 94 query83 193 163 167 163 query84 263 93 89 89 query85 1511264 258 258 query86 403 289 305 289 query87 3334312831193119 query88 3452242424102410 query89 474 389 383 383 query90 2009188 190 188 query91 121 98 97 97 query92 58 48 48 48 query93 1285510 504 504 query94 1120179 188 179 query95 384 304 309 304 query96 584 268 271 268 query97 3168299330142993 query98 245 220 214 214 query99 1302915 889 889 Total cold run time: 286122 ms Total hot run time: 186552 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
doris-robot commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2105926033 TPC-H: Total hot run time: 39892 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit fa83cce7bede1c1f53806f37981ee6ab5f4bd6b7, data reload: false -- Round 1 -- q1 17764 460843664366 q2 2618188 193 188 q3 10985 120011671167 q4 10630 807 746 746 q5 7638266127042661 q6 225 134 132 132 q7 944 528 515 515 q8 9204206320212021 q9 8583651665166516 q10 8883377636593659 q11 442 239 239 239 q12 462 216 213 213 q13 17754 293229412932 q14 253 223 220 220 q15 503 477 471 471 q16 517 384 380 380 q17 958 733 817 733 q18 8020754374597459 q19 4600151715181517 q20 641 338 311 311 q21 5083316832673168 q22 348 278 279 278 Total cold run time: 117055 ms Total hot run time: 39892 ms - Round 2, with runtime_filter_mode=off - q1 4371424242434242 q2 375 271 261 261 q3 2976279826942694 q4 1871158915411541 q5 5285525452525252 q6 216 124 125 124 q7 1736137713451345 q8 3168331533103310 q9 8262834483258325 q10 3834362036383620 q11 582 474 486 474 q12 772 593 600 593 q13 17639 298729972987 q14 287 265 267 265 q15 527 468 463 463 q16 459 400 405 400 q17 1757147514691469 q18 7546756674067406 q19 1638156715461546 q20 1945175217851752 q21 4833493348304830 q22 563 501 494 494 Total cold run time: 70642 ms Total hot run time: 53393 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
zhangbutao commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2105900079 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
zhangbutao commented on code in PR #34681: URL: https://github.com/apache/doris/pull/34681#discussion_r1597458343 ## fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4: ## @@ -966,6 +966,10 @@ sampleMethod | INTEGER_VALUE ROWS #sampleByRows ; +tableSnapshot +: FOR tableSnapshotType=(TIME | VERSION) AS OF valueExpression Review Comment: Good suggesstion! Fixed. Thanks. ## fe/fe-core/src/main/java/org/apache/doris/nereids/analyzer/UnboundRelation.java: ## @@ -59,41 +60,46 @@ public class UnboundRelation extends LogicalRelation implements Unbound, BlockFu // the start and end position of the sql substring(e.g. "t1", "db1.t1", "ctl1.db1.t1") private final Optional> indexInSqlString; +private final TableSnapshot tableSnapshot; Review Comment: Fixed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
morrySnow commented on code in PR #34681: URL: https://github.com/apache/doris/pull/34681#discussion_r1597396449 ## fe/fe-core/src/main/java/org/apache/doris/nereids/analyzer/UnboundRelation.java: ## @@ -59,41 +60,46 @@ public class UnboundRelation extends LogicalRelation implements Unbound, BlockFu // the start and end position of the sql substring(e.g. "t1", "db1.t1", "ctl1.db1.t1") private final Optional> indexInSqlString; +private final TableSnapshot tableSnapshot; Review Comment: ```suggestion private final Optional tableSnapshot; ``` ## fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4: ## @@ -966,6 +966,10 @@ sampleMethod | INTEGER_VALUE ROWS #sampleByRows ; +tableSnapshot +: FOR tableSnapshotType=(TIME | VERSION) AS OF valueExpression Review Comment: valueExpression is a big scope, such as a + b. If only need number literal, please use `number`. if also need string literal, please use `number | STRING_LITERAL` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
zhangbutao commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2105432896 @morningman @morrySnow kindly request your review. Thx. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
zhangbutao commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2105430999 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support Iceberg time travel syntax [doris]
doris-robot commented on PR #34681: URL: https://github.com/apache/doris/pull/34681#issuecomment-2105430787 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [feat](nereids) support Iceberg time travel syntax [doris]
zhangbutao opened a new pull request, #34681: URL: https://github.com/apache/doris/pull/34681 ## Proposed changes https://github.com/apache/doris/pull/15418 added Iceberg time travel in legacy parser but not added this syntax Neredis. If we enable nereids and disable fallback to original palnner, time travel won't be available. This PR added time travel syntas in Neredis. ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
