Great, glad you found a fix. Looking through the code I noticed that we will interpret any unknown select hint as 'straigh_join'. IMPALA-7484 <https://issues.apache.org/jira/browse/IMPALA-7484> has the details, you might want to keep an eye on this when using this annotation technique with joins.
Cheers, Lars On Thu, Aug 23, 2018 at 9:46 PM Quanlong Huang <[email protected]> wrote: > Thank you, Todd and Lars! We plan to hotfix this by quotes the vars in > PlanHint: > > *diff --git a/fe/src/main/java/org/apache/impala/analysis/PlanHint.java > b/fe/src/main/java/org/apache/impala/analysis/PlanHint.java* > > *index d16919f09..4da0e91c2 100644* > > *--- a/fe/src/main/java/org/apache/impala/analysis/PlanHint.java* > > *+++ b/fe/src/main/java/org/apache/impala/analysis/PlanHint.java* > > @@ -61,8 +61,10 @@ public class PlanHint { > > @Override > > public String toString() { > > StringBuilder sb = new StringBuilder(); > > - sb.append(name_); > > - if (!args_.isEmpty()) { > > + if (args_.isEmpty()) { > > + sb.append('`').append(name_).append('`'); > > + } else { > > + sb.append(name_); > > sb.append("("); > > sb.append(Joiner.on(",").join(args_)); > > sb.append(")"); > > > > Quanlong Huang <[email protected]> 于2018年8月24日周五 下午12:09写道： > >> Unfortunately, the queries come from a system that we don't have control. >> I think we should change the behavior of the CreateView statement to save >> everything in the comments. >> >> Lars Volker <[email protected]> 于2018年8月24日周五 上午11:37写道： >> >>> We changed query hints from being interpreted as strings to being parsed >>> properly in commit ce9b332e >>> <https://github.com/apache/impala/commit/ce9b332ee9e640d79c8ae35e7abb8c7d787ddf78#diff-27ff375b57f7fc982be98c4f503cb04dR2385> >>> for IMPALA-4163 <https://issues.apache.org/jira/browse/IMPALA-4163>. >>> Before that change we called String.trim() on the hint which I suspect >>> preserved the ``. After this change we parse it as an IDENT, which likely >>> strips the ``. >>> >>> I feel that hints are not the right solution to embed a query id into >>> the view definition, but I cannot think of a better one, either. With the >>> current syntax you can write the hint as queryId_abc or queryId(abc) and >>> while you'll get a bunch of warning it seems to work on my end. >>> >>> I also think there's a (unrelated) bug in SelectList.java:87 >>> <https://github.com/apache/impala/blob/64e6719870db5602a6fa85014bc6c264080b9414/fe/src/main/java/org/apache/impala/analysis/SelectList.java#L87>: >>> It should have a continue statement inside the check, or wrap the call to >>> setIsStraightJoin() in an else clause. >>> >>> On Thu, Aug 23, 2018 at 8:19 PM Quanlong Huang <[email protected]> >>> wrote: >>> >>>> Yeah, the results of CreateView statement is different. We created the >>>> same view by impala-2.5. The difference is that impala-2.12 delete the >>>> quotes in the comment. >>>> >>>> Found the difference in beeline: >>>> >>>> > show create table view_2_12; >>>> >>>> createtab_stmt >>>> >>>> CREATE VIEW `view_2_12` AS SELECT >>>> >>>> -- +queryId=abc >>>> >>>> * FROM test.video_parq >>>> >>>> 3 rows selected (4.518 seconds) >>>> >>>> >>>> > show create table view_2_5; >>>> >>>> createtab_stmt >>>> >>>> CREATE VIEW `view_2_5` AS SELECT >>>> >>>> -- +`queryId=abc` >>>> >>>> * FROM test.video_parq >>>> >>>> 3 rows selected (1.202 seconds) >>>> >>>> view_2_12 is created by impala-2.12, while view_2_5 is created by >>>> impala-2.5. >>>> >>>> Todd Lipcon <[email protected]> 于2018年8月24日周五 上午10:53写道： >>>> >>>>> I'm curious: can you describe the view using Hive to see what the >>>>> stored query consists of? >>>>> >>>>> On Thu, Aug 23, 2018, 7:44 PM Quanlong Huang <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> After we upgrade Impala from 2.5 to 2.12, we found some queries on >>>>>> views >>>>>> failed. The views contain a query hint which I think is the cause. I >>>>>> can't >>>>>> find anything about such kind of incompatibility was mentioned in >>>>>> >>>>>> https://www.cloudera.com/documentation/enterprise/release-notes/topics/impala_new_features.html >>>>>> or >>>>>> >>>>>> https://www.cloudera.com/documentation/enterprise/release-notes/topics/impala_incompatible_changes.html >>>>>> >>>>>> The error can be reproduced by >>>>>> >>>>>> [impala-shell:21000] > use test; >>>>>> [impala-shell:21000] > create view view_2_12 as select /* >>>>>> +`queryId=abc` */ * from video_parq; >>>>>> Query: create view view_2_12 as select /* +`queryId=abc` */ * from >>>>>> video_parq >>>>>> +------------------------+ >>>>>> | summary | >>>>>> +------------------------+ >>>>>> | View has been created. | >>>>>> +------------------------+ >>>>>> WARNINGS: PLAN hint not recognized: queryId=abc >>>>>> >>>>>> Fetched 1 row(s) in 24.00s >>>>>> [impala-shell:21000] > desc formatted view_2_12; >>>>>> Query: describe formatted view_2_12 >>>>>> ERROR: AnalysisException: Failed to parse view-definition statement of >>>>>> view: test.view_2_12 >>>>>> CAUSED BY: TableLoadingException: Failed to parse view-definition >>>>>> statement of view: test.view_2_12 >>>>>> >>>>>> >>>>>> Since this's an emergency for us, I post an email here for help. I'm >>>>>> also >>>>>> looking into the history of the cup file and looking for a hotfix for >>>>>> this. >>>>>> Thanks if anyone can give some pointers! >>>>>> >>>>>> Thanks, >>>>>> Quanlong >>>>>> >>>>>
