Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/17038 )
Change subject: IMPALA-10482, IMPALA-10493: Fix bugs in full ACID collection query rewrites ...................................................................... Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/17038/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17038/3//COMMIT_MSG@34 PS3, Line 34: When AcidRewriter.splitCollectionRef() creates a new collection ref : it doesn't copy every information needed to correctly execute the : query. E.g. it dropped the ON clause, turning INNER joins to CROSS : joins. We faced the same problem in column masking, and made a tricky way to resolve it: https://gerrit.cloudera.org/c/15108/ So yes, the rewritten query is illegal in syntax. But we don't need to expose it to users so it's ok. So we made some hacks to let the analyzer be able to resolve collection ref through the view. Basically in 3 steps: 1. Add the collection columns in the view's output structure: https://github.com/apache/impala/blob/94f67a343294430b53da58d1af60e659af18545a/fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java#L322-L349 2. Then Analyzer can resolve the collection paths, but we need to further resolve them into the inline view: https://github.com/apache/impala/blob/94f67a343294430b53da58d1af60e659af18545a/fe/src/main/java/org/apache/impala/analysis/Analyzer.java#L1030-L1054 3*. Finally, need to use the inline view's tuple id in resolving correlated refs: https://github.com/apache/impala/blob/94f67a343294430b53da58d1af60e659af18545a/fe/src/main/java/org/apache/impala/analysis/CollectionTableRef.java#L90-L96 We might be able to reuse these logics. I think once we have IMPALA-9494, we can get rid of these hacks. > Anyway, this patch already solves a few issues while not introducing new > ones, so how about pushing this to master and opening a new Jira to deal with > the above problem as that seems to be the toughest one? Yeah, agree on this. -- To view, visit http://gerrit.cloudera.org:8080/17038 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8fc758d3c1e75c7066936d590aec8bff8d2b00b0 Gerrit-Change-Number: 17038 Gerrit-PatchSet: 3 Gerrit-Owner: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Comment-Date: Wed, 28 Apr 2021 07:13:57 +0000 Gerrit-HasComments: Yes
