Great, glad you found a fix. Looking through the code I noticed that we
will interpret any unknown select hint as 'straigh_join'. IMPALA-7484
<https://issues.apache.org/jira/browse/IMPALA-7484> has the details, you
might want to keep an eye on this when using this annotation technique with
joins.
Cheers, Lars
On Thu, Aug 23, 2018 at 9:46 PM Quanlong Huang <huangquanl...@gmail.com>
wrote:
> Thank you, Todd and Lars! We plan to hotfix this by quotes the vars in
> PlanHint:
>
> *diff --git a/fe/src/main/java/org/apache/impala/analysis/PlanHint.java
> b/fe/src/main/java/org/apache/impala/analysis/PlanHint.java*
>
> *index d16919f09..4da0e91c2 100644*
>
> *--- a/fe/src/main/java/org/apache/impala/analysis/PlanHint.java*
>
> *+++ b/fe/src/main/java/org/apache/impala/analysis/PlanHint.java*
>
> @@ -61,8 +61,10 @@ public class PlanHint {
>
> @Override
>
> public String toString() {
>
> StringBuilder sb = new StringBuilder();
>
> - sb.append(name_);
>
> - if (!args_.isEmpty()) {
>
> + if (args_.isEmpty()) {
>
> + sb.append('`').append(name_).append('`');
>
> + } else {
>
> + sb.append(name_);
>
> sb.append("(");
>
> sb.append(Joiner.on(",").join(args_));
>
> sb.append(")");
>
>
>
> Quanlong Huang <huangquanl...@gmail.com> 于2018年8月24日周五 下午12:09写道：
>
>> Unfortunately, the queries come from a system that we don't have control.
>> I think we should change the behavior of the CreateView statement to save
>> everything in the comments.
>>
>> Lars Volker <l...@cloudera.com> 于2018年8月24日周五 上午11:37写道：
>>
>>> We changed query hints from being interpreted as strings to being parsed
>>> properly in commit ce9b332e
>>> <https://github.com/apache/impala/commit/ce9b332ee9e640d79c8ae35e7abb8c7d787ddf78#diff-27ff375b57f7fc982be98c4f503cb04dR2385>
>>> for IMPALA-4163 <https://issues.apache.org/jira/browse/IMPALA-4163>.
>>> Before that change we called String.trim() on the hint which I suspect
>>> preserved the ``. After this change we parse it as an IDENT, which likely
>>> strips the ``.
>>>
>>> I feel that hints are not the right solution to embed a query id into
>>> the view definition, but I cannot think of a better one, either. With the
>>> current syntax you can write the hint as queryId_abc or queryId(abc) and
>>> while you'll get a bunch of warning it seems to work on my end.
>>>
>>> I also think there's a (unrelated) bug in SelectList.java:87
>>> <https://github.com/apache/impala/blob/64e6719870db5602a6fa85014bc6c264080b9414/fe/src/main/java/org/apache/impala/analysis/SelectList.java#L87>:
>>> It should have a continue statement inside the check, or wrap the call to
>>> setIsStraightJoin() in an else clause.
>>>
>>> On Thu, Aug 23, 2018 at 8:19 PM Quanlong Huang <huangquanl...@gmail.com>
>>> wrote:
>>>
>>>> Yeah, the results of CreateView statement is different. We created the
>>>> same view by impala-2.5. The difference is that impala-2.12 delete the
>>>> quotes in the comment.
>>>>
>>>> Found the difference in beeline:
>>>>
>>>> > show create table view_2_12;
>>>>
>>>> createtab_stmt
>>>>
>>>> CREATE VIEW `view_2_12` AS SELECT
>>>>
>>>> -- +queryId=abc
>>>>
>>>> * FROM test.video_parq
>>>>
>>>> 3 rows selected (4.518 seconds)
>>>>
>>>>
>>>> > show create table view_2_5;
>>>>
>>>> createtab_stmt
>>>>
>>>> CREATE VIEW `view_2_5` AS SELECT
>>>>
>>>> -- +`queryId=abc`
>>>>
>>>> * FROM test.video_parq
>>>>
>>>> 3 rows selected (1.202 seconds)
>>>>
>>>> view_2_12 is created by impala-2.12, while view_2_5 is created by
>>>> impala-2.5.
>>>>
>>>> Todd Lipcon <t...@cloudera.com> 于2018年8月24日周五 上午10:53写道：
>>>>
>>>>> I'm curious: can you describe the view using Hive to see what the
>>>>> stored query consists of?
>>>>>
>>>>> On Thu, Aug 23, 2018, 7:44 PM Quanlong Huang <huangquanl...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> After we upgrade Impala from 2.5 to 2.12, we found some queries on
>>>>>> views
>>>>>> failed. The views contain a query hint which I think is the cause. I
>>>>>> can't
>>>>>> find anything about such kind of incompatibility was mentioned in
>>>>>>
>>>>>> https://www.cloudera.com/documentation/enterprise/release-notes/topics/impala_new_features.html
>>>>>> or
>>>>>>
>>>>>> https://www.cloudera.com/documentation/enterprise/release-notes/topics/impala_incompatible_changes.html
>>>>>>
>>>>>> The error can be reproduced by
>>>>>>
>>>>>> [impala-shell:21000] > use test;
>>>>>> [impala-shell:21000] > create view view_2_12 as select /*
>>>>>> +`queryId=abc` */ * from video_parq;
>>>>>> Query: create view view_2_12 as select /* +`queryId=abc` */ * from
>>>>>> video_parq
>>>>>> +------------------------+
>>>>>> | summary |
>>>>>> +------------------------+
>>>>>> | View has been created. |
>>>>>> +------------------------+
>>>>>> WARNINGS: PLAN hint not recognized: queryId=abc
>>>>>>
>>>>>> Fetched 1 row(s) in 24.00s
>>>>>> [impala-shell:21000] > desc formatted view_2_12;
>>>>>> Query: describe formatted view_2_12
>>>>>> ERROR: AnalysisException: Failed to parse view-definition statement of
>>>>>> view: test.view_2_12
>>>>>> CAUSED BY: TableLoadingException: Failed to parse view-definition
>>>>>> statement of view: test.view_2_12
>>>>>>
>>>>>>
>>>>>> Since this's an emergency for us, I post an email here for help. I'm
>>>>>> also
>>>>>> looking into the history of the cup file and looking for a hotfix for
>>>>>> this.
>>>>>> Thanks if anyone can give some pointers!
>>>>>>
>>>>>> Thanks,
>>>>>> Quanlong
>>>>>>
>>>>>
