[Impala-ASF-CR] IMPALA-13055: Some Iceberg metadata table tests don't assert
Gabor Kaszab has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21394 Change subject: IMPALA-13055: Some Iceberg metadata table tests don't assert .. IMPALA-13055: Some Iceberg metadata table tests don't assert Some tests in the Iceberg metadata table suite use the following regex to verify numbers in the output: [1-9]\d*|0 However, if this format is given, the test unconditionally passes. This patch changes this format to \d+ and fixes the test results that incorrectly passed before due to the test not asserting. Change-Id: Ie47093f25a70253b3e6faca27d466d7cf6999fad --- M testdata/workloads/functional-query/queries/QueryTest/iceberg-metadata-tables.test 1 file changed, 117 insertions(+), 117 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/94/21394/1 -- To view, visit http://gerrit.cloudera.org:8080/21394 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ie47093f25a70253b3e6faca27d466d7cf6999fad Gerrit-Change-Number: 21394 Gerrit-PatchSet: 1 Gerrit-Owner: Gabor Kaszab
[Impala-ASF-CR] IMPALA-13055: Some Iceberg metadata table tests don't assert
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21394 ) Change subject: IMPALA-13055: Some Iceberg metadata table tests don't assert .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/16092/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21394 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie47093f25a70253b3e6faca27d466d7cf6999fad Gerrit-Change-Number: 21394 Gerrit-PatchSet: 1 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 03 May 2024 09:37:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-13036: Document Iceberg metadata tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21387 ) Change subject: IMPALA-13036: Document Iceberg metadata tables .. Patch Set 2: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/766/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/21387 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic453f567b814cb4363a155e2008029e94efb6ed1 Gerrit-Change-Number: 21387 Gerrit-PatchSet: 2 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Noemi Pap-Takacs Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Fri, 03 May 2024 09:45:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-13036: Document Iceberg metadata tables
Daniel Becker has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/21387 ) Change subject: IMPALA-13036: Document Iceberg metadata tables .. IMPALA-13036: Document Iceberg metadata tables This change adds documentation on how Iceberg metadata tables can be used. Testing: - built docs locally Change-Id: Ic453f567b814cb4363a155e2008029e94efb6ed1 --- M docs/topics/impala_iceberg.xml 1 file changed, 72 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/87/21387/2 -- To view, visit http://gerrit.cloudera.org:8080/21387 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic453f567b814cb4363a155e2008029e94efb6ed1 Gerrit-Change-Number: 21387 Gerrit-PatchSet: 2 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Noemi Pap-Takacs Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-13036: Document Iceberg metadata tables
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/21387 ) Change subject: IMPALA-13036: Document Iceberg metadata tables .. Patch Set 2: (5 comments) http://gerrit.cloudera.org:8080/#/c/21387/1/docs/topics/impala_iceberg.xml File docs/topics/impala_iceberg.xml: http://gerrit.cloudera.org:8080/#/c/21387/1/docs/topics/impala_iceberg.xml@729 PS1, Line 729: > nit:, I don't think a comma is needed before "and". http://gerrit.cloudera.org:8080/#/c/21387/1/docs/topics/impala_iceberg.xml@730 PS1, Line 730: fr > I'm not sure: from? Yes, neither is perfect, I changed it to "from". http://gerrit.cloudera.org:8080/#/c/21387/1/docs/topics/impala_iceberg.xml@730 PS1, Line 730: > nit:, I don't think a comma is needed here. http://gerrit.cloudera.org:8080/#/c/21387/1/docs/topics/impala_iceberg.xml@730 PS1, Line 730: > *direclty Added a sentence at the end to make it clearer. http://gerrit.cloudera.org:8080/#/c/21387/1/docs/topics/impala_iceberg.xml@770 PS1, Line 770: > *are The subject is "unnesting", which is singular, that's why it's "is". -- To view, visit http://gerrit.cloudera.org:8080/21387 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic453f567b814cb4363a155e2008029e94efb6ed1 Gerrit-Change-Number: 21387 Gerrit-PatchSet: 2 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Noemi Pap-Takacs Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Fri, 03 May 2024 09:45:39 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-13036: Document Iceberg metadata tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21387 ) Change subject: IMPALA-13036: Document Iceberg metadata tables .. Patch Set 2: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/766/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/21387 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic453f567b814cb4363a155e2008029e94efb6ed1 Gerrit-Change-Number: 21387 Gerrit-PatchSet: 2 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Noemi Pap-Takacs Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Fri, 03 May 2024 09:53:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-13035: Querying metadata tables from non-Iceberg tables throws IllegalArgumentException
Daniel Becker has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/21361 ) Change subject: IMPALA-13035: Querying metadata tables from non-Iceberg tables throws IllegalArgumentException .. IMPALA-13035: Querying metadata tables from non-Iceberg tables throws IllegalArgumentException When attempting to query a metadata table of a non-Iceberg table the analyzer throws 'IllegalArgumentException'. The problem is that 'IcebergMetadataTable.isIcebergMetadataTable()' doesn't actually check whether the given path belongs to a valid metadata table, it only checks whether the path could syntactically refer to one. This is because it is called in 'Path.getCandidateTables()', at which point analysis has not been done yet. However, 'IcebergMetadataTable.isIcebergMetadataTable()' is also called in 'Analyzer.getTable()'. If 'isIcebergMetadataTable()' returns true, 'getTable()' tries to instantiate an 'IcebergMetadataTable' object with the table ref of the base table. If that table is not an Iceberg table, a precondition check fails. This change renames 'isIcebergMetadataTable()' to 'canBeIcebergMetadataTable()' and adds a new 'isIcebergMetadataTable()' function, which also takes an 'Analyzer' as a parameter. With the help of the 'Analyzer' it is possible to determine whether the base table is an Iceberg table. 'Analyzer.getTable()' then uses this new 'isIcebergMetadataTable()' function instead of canBeIcebergMetadataTable(). The constructor of 'IcebergMetadataTable' is also modified to take an 'FeIcebergTable' as the parameter for the base table instead of a general 'FeTable'. Testing: - Added a test query in iceberg-metadata-tables.test. Change-Id: Ia7c25ed85a8813011537c73f0aaf72db1501f9ef Reviewed-on: http://gerrit.cloudera.org:8080/21361 Tested-by: Impala Public Jenkins Reviewed-by: Peter Rozsa --- M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/Path.java M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergMetadataTable.java M fe/src/main/java/org/apache/impala/service/DescribeResultFactory.java M fe/src/main/java/org/apache/impala/service/Frontend.java M testdata/workloads/functional-query/queries/QueryTest/iceberg-metadata-tables.test 6 files changed, 52 insertions(+), 16 deletions(-) Approvals: Impala Public Jenkins: Verified Peter Rozsa: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/21361 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia7c25ed85a8813011537c73f0aaf72db1501f9ef Gerrit-Change-Number: 21361 Gerrit-PatchSet: 5 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-13053: Update test to use ORC files
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/21391 ) Change subject: IMPALA-13053: Update test to use ORC files .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/21391 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2f5bdbb86af0e651d189217a18882d5eda1098d5 Gerrit-Change-Number: 21391 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Fri, 03 May 2024 17:54:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/21358 ) Change subject: IMPALA-13051: Speed up, refactor query log tests .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/21358/5/tests/common/custom_cluster_test_suite.py File tests/common/custom_cluster_test_suite.py: http://gerrit.cloudera.org:8080/#/c/21358/5/tests/common/custom_cluster_test_suite.py@236 PS5, Line 236: super(CustomClusterTestSuite, self).teardown_class() > Should it be called the very last? I see couple custom cluster tests overri I'll probably back this change out, it's weird but doesn't cause issues right now. -- To view, visit http://gerrit.cloudera.org:8080/21358 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855 Gerrit-Change-Number: 21358 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jason Fehr Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Fri, 03 May 2024 20:29:03 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-13054: Avoid revisiting children in QueryStateExpanded
Hello Andrew Sherman, Riza Suminto, Jason Fehr, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21392 to look at the new patch set (#2). Change subject: IMPALA-13054: Avoid revisiting children in QueryStateExpanded .. IMPALA-13054: Avoid revisiting children in QueryStateExpanded Avoids revisiting children in QueryStateExpanded due to duplicate recursion. Since process_exec_profile visits children recursively, we only want immediate children from GetChildren, not all children from GetAllChildren. Adjusts HDFS_SCAN_NODE test to look back at top of stack, rather than depend on a specific depth that was caused by calling GetAllChildren from the root. This was identified by running the Impala E2E test suite with workload management enabled, where test_nested_types.py::TestMaxNestingDepth::test_max_nesting_depth took multiple hours to run and blocked the FinishUnregisterQuery for other tests. Testing: - Manual test with enable_workload_mgmt=true running test_max_nesting_depth. Change-Id: Ia36d53e52095d487ffb3a162fc723f1e815bec11 --- M be/src/service/query-state-record.cc 1 file changed, 23 insertions(+), 11 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/21392/2 -- To view, visit http://gerrit.cloudera.org:8080/21392 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia36d53e52095d487ffb3a162fc723f1e815bec11 Gerrit-Change-Number: 21392 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jason Fehr Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Riza Suminto
[Impala-ASF-CR] IMPALA-12977: add search and pagination to /hadoop-varz
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/21329 ) Change subject: IMPALA-12977: add search and pagination to /hadoop-varz .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/21329/2/www/hadoop-varz.tmpl File www/hadoop-varz.tmpl: http://gerrit.cloudera.org:8080/#/c/21329/2/www/hadoop-varz.tmpl@42 PS2, Line 42: "aaSorting" We use "order" in other places. What's the difference between them? -- To view, visit http://gerrit.cloudera.org:8080/21329 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic8cac23b655fa58ce12d9857649705574614a5f0 Gerrit-Change-Number: 21329 Gerrit-PatchSet: 2 Gerrit-Owner: Saurabh Katiyal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Sat, 04 May 2024 06:12:02 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-13053: Update test to use ORC files
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21391 ) Change subject: IMPALA-13053: Update test to use ORC files .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10603/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/21391 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2f5bdbb86af0e651d189217a18882d5eda1098d5 Gerrit-Change-Number: 21391 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Sat, 04 May 2024 06:20:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-13053: Update test to use ORC files
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21391 ) Change subject: IMPALA-13053: Update test to use ORC files .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/21391 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2f5bdbb86af0e651d189217a18882d5eda1098d5 Gerrit-Change-Number: 21391 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Sat, 04 May 2024 06:20:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-13053: Update test to use ORC files
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/21391 ) Change subject: IMPALA-13053: Update test to use ORC files .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/21391 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2f5bdbb86af0e651d189217a18882d5eda1098d5 Gerrit-Change-Number: 21391 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Sat, 04 May 2024 06:19:48 + Gerrit-HasComments: No
