Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17038 )
Change subject: IMPALA-10482, IMPALA-10493: Fix bugs in full ACID collection query rewrites ...................................................................... IMPALA-10482, IMPALA-10493: Fix bugs in full ACID collection query rewrites IMPALA-10482: SELECT * query on unrelative collection column of transactional ORC table will hit IllegalStateException. The AcidRewriter will rewrite queries like "select item from my_complex_orc.int_array" to "select item from my_complex_orc t, t.int_array" This cause troubles in star expansion. Because the original query "select * from my_complex_orc.int_array" is analyzed as "select item from my_complex_orc.int_array" But the rewritten query "select * from my_complex_orc t, t.int_array" is analyzed as "select id, item from my_complex_orc t, t.int_array". Hidden table refs can also cause issues during regular column resolution. E.g. when the table has top-level 'pos'/'item'/'key'/'value' columns. The workaround is to keep track of the automatically added table refs during query rewrite. So when we analyze the rewritten query we can ignore these auxiliary table refs. IMPALA-10493: Using JOIN ON syntax to join two full ACID collections produces wrong results. When AcidRewriter.splitCollectionRef() creates a new collection ref it doesn't copy every information needed to correctly execute the query. E.g. it dropped the ON clause, turning INNER joins to CROSS joins. Testing: * added e2e tests Change-Id: I8fc758d3c1e75c7066936d590aec8bff8d2b00b0 Reviewed-on: http://gerrit.cloudera.org:8080/17038 Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> --- M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java M fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java M fe/src/main/java/org/apache/impala/analysis/TableRef.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/nested-types-scanner-basic.test 8 files changed, 233 insertions(+), 2 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17038 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I8fc758d3c1e75c7066936d590aec8bff8d2b00b0 Gerrit-Change-Number: 17038 Gerrit-PatchSet: 8 Gerrit-Owner: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com>
